. "$BST_LIB" # elfutils 0.191 -- and, like 15-bash.test and 77-cunit.test, a suite whose # evidence is UPSTREAM'S OWN TEST SUITE rather than one we wrote. # # WHY THAT IS WORTH MORE HERE THAN A SUITE WE COULD HAVE WRITTEN. elfutils is a # parser for two large, versioned, deeply-nested binary formats -- ELF and DWARF # -- across a dozen architectures and both endiannesses. Its correctness is not # a handful of behaviours anyone could sample; it is thousands of field decodings # that are either byte-exact or wrong. Upstream ships 240 run-*.sh drivers and # 356 compressed reference object files (i386, x86_64, aarch64, ppc, ppc64, # s390, arm, riscv), and each driver runs a eu-* tool over a fixture and DIFFS # ITS OUTPUT against a recorded expectation. Nothing we wrote would reach that # resolution, and -- unlike the native x86_64 controls the library ports here # needed -- the oracle ships in the tarball, so there is no second build to keep # in step. # # THE FIXTURES ARE THE REASON THIS PORT MEANS ANYTHING AT ALL. Every binary on # this image is a wasm module, so there is no native ELF here for eu-readelf to # read: pointed at /usr/bin/eu-nm it correctly reports that the file is not ELF. # The tools are here for the cross-architecture object files a developer brings # to the guest, and upstream's fixtures are precisely those files. The direct # cases below therefore read a shipped fixture rather than anything the image # built, which is also what makes them a real test of the DWARF reader and not # of the wasm loader. # # HOW THE SUITE IS DRIVEN, AND WHY IT IS NOT run-ptest DIRECTLY. This ticket # closes on the suite run UNDER ptest-runner, so the full path execs the harness # once and asserts its discovery, its child plumbing and its tally alongside # elfutils' own verdicts. An image with no ptest-runner skips those cases and # stops -- there is no run-ptest-directly fallback, because the harness is half # of what the ticket closes on and a silently different driver would make the # record mean something else. # # SPAWN BUDGET. exec on this arm is a fresh wasm module instantiation, ~2-3 s # apiece, and the suite is already several thousand of them. So this file spends # its own spawns sparingly: one decompression, four tool runs against the # fixture it produces, and presence checks done with check_rc + `test`, which is # an ash builtin and forks nothing. require eu-readelf elfutils/present ptestdir=/usr/lib/elfutils/ptest runner=/usr/bin/ptest-runner tmp=${TMPDIR:-/tmp}/bst-elfutils.$$ mkdir -p "$tmp" 2>/dev/null || tmp=/tmp # ------------------------------------------------------------ what shipped --- # All seventeen programs, by name. eu-ar, eu-ranlib, eu-readelf and eu-unstrip # are called out because they are the four the coroutinizing pass refused to # link until their dynamic stack buffers moved to the heap -- if a toolchain # change ever regresses that, these are the cases that go red first. for t in addr2line ar elfclassify elfcmp elfcompress elflint findtextrel nm \ objdump ranlib readelf size srcfiles stack strings strip unstrip; do check_rc "elfutils/tool-$t" 0 test -x "/usr/bin/eu-$t" done check_out elfutils/version "0.191" eu-readelf --version # The static archives and headers, which are what the four never-emitted # DSO-only packages were replaced by (see the image recipe). check_rc elfutils/libelf-archive 0 test -s /usr/lib/libelf.a check_rc elfutils/libdw-archive 0 test -s /usr/lib/libdw.a check_rc elfutils/libasm-archive 0 test -s /usr/lib/libasm.a check_rc elfutils/libelf-header 0 test -s /usr/include/libelf.h check_rc elfutils/libdw-header 0 test -s /usr/include/elfutils/libdw.h # ...and no DSO, which is the invariant the whole port rests on. A libelf.so # appearing here would mean the static-only patch stopped applying. check_rc elfutils/no-dso 1 sh -c 'ls /usr/lib/libelf*.so* /usr/lib/libdw*.so* 2>/dev/null | grep -q .' # ------------------------------------------------------------ ptest payload --- check_rc elfutils/ptest-dir 0 test -d "$ptestdir" check_rc elfutils/ptest-runptest 0 test -x "$ptestdir/run-ptest" check_rc elfutils/ptest-makefile 0 test -s "$ptestdir/tests/Makefile" check_rc elfutils/ptest-src 0 test -x "$ptestdir/src/readelf" # The drivers and their fixtures, both of which the suite is worthless without. # # COUNTED WITHOUT GLOBBING, AND THAT IS NOT STYLE. `ls "$ptestdir"/tests/run-*.sh` # expands to 240 absolute paths -- a 13,897-byte argv block -- and an execve whose # argv exceeds __wasm32_args_buf (8192 B) never returns on this arm: the child is # lost and the parent waits forever (brintos/hardwarejs#15, filed by BRINTOS-506 # against bash's suite, which wedges the same way). The first cut of this file did # glob, and hung here until the runner's own wall-clock bound killed it. Listing # the directory once and counting with grep passes one argument instead of 240. # Nothing here needs undoing when the defect closes -- this form is correct # everywhere -- so it carries no removal marker, only this explanation. ndrv=$(ls "$ptestdir/tests" 2>/dev/null | grep -c '^run-.*\.sh$') check_rc elfutils/ptest-drivers 0 test "$ndrv" -ge 200 nfix=$(ls "$ptestdir/tests" 2>/dev/null | grep -c '\.bz2$') check_rc elfutils/ptest-fixtures 0 test "$nfix" -ge 300 # ------------------------------------------------ direct evidence, on a fixture # One decompression, then four readers over the same file. testfile is the # suite's canonical i386 fixture: a small C program compiled with debug info, so # it exercises the ELF header, the section and symbol tables, and DWARF line and # DIE parsing in one object. fixture=$ptestdir/tests/testfile.bz2 if [ -s "$fixture" ] && command -v bunzip2 >/dev/null 2>&1; then bunzip2 -c "$fixture" > "$tmp/testfile" 2>/dev/null check_rc elfutils/fixture-extracted 0 test -s "$tmp/testfile" # ELF header: 32-bit little-endian i386 REL/EXEC, read on a 32-bit # little-endian wasm host -- the same width, so a byte-order or width bug # would not show here; the cross-width case is the suite's job, below. check_out elfutils/readelf-header "Intel 80386" eu-readelf -h "$tmp/testfile" # Section table, symbol table, and DWARF: three different readers. check_out elfutils/readelf-sections ".debug_info" eu-readelf -S "$tmp/testfile" check_out elfutils/nm-symbols "main" eu-nm "$tmp/testfile" # eu-readelf prints DIE tags with the DW_TAG_ prefix stripped -- "compile_unit", # not "DW_TAG_compile_unit". That is upstream's own spelling, as its recorded # .right expectations show, and it is what this needle has to match. check_out elfutils/readelf-debug "compile_unit" eu-readelf --debug-dump=info "$tmp/testfile" # elflint is the one that says the file is WELL-FORMED rather than merely # readable, which is a stronger statement about the parser. check elfutils/elflint eu-elflint --gnu "$tmp/testfile" else for c in fixture-extracted readelf-header readelf-sections nm-symbols \ readelf-debug elflint; do bst_skip "elfutils/$c" "no $fixture, or no bunzip2 to expand it" done fi # A wasm module is not an ELF file, and the tools must say so rather than # mis-parse it. This is the negative half of the pair above and costs one spawn. check_rc elfutils/rejects-non-elf 1 eu-readelf -h /usr/bin/eu-size # --------------------------------------------------------- the upstream suite -- harness=0 if [ ! -x "$runner" ]; then bst_skip elfutils/ptest-runner-present "this image ships no ptest-runner" for c in ptest-runner-rc runner-begin runner-end runner-tally runner-no-error \ suite-drivers-run tally-consistent ptest-suite; do bst_skip "elfutils/$c" "no ptest-runner to drive the suite" done rm -rf "$tmp" return 2>/dev/null || exit 0 fi bst_ok elfutils/ptest-runner-present if [ -z "${BST_ELFUTILS_PTEST:-}" ]; then # WHY THE FULL RUN IS BEHIND AN ENV VAR. 244 TESTS, each a fresh wasm module # instantiation of a eu-* tool plus the shell that drives it, is far past what # a whole-image sweep can absorb -- the measured run is 2 h 53 m -- and the # sweep's per-suite budget would kill it partway, reporting a timeout that says # less than this line does. The real run is one variable away and the test plan # states the invocation: # # BST_ELFUTILS_PTEST=1 BST_SUITE_TIMEOUT=20000 brintos-selftest 90-elfutils.test # # This is a SKIP and not a red: unlike bash's suite, which is present and # blocked on a platform defect, elfutils' suite runs -- it is simply not run # HERE, by choice, and the Outcome of the design doc records what it did when # it was run. A red would claim a defect that the measurement does not support. bst_skip elfutils/ptest-suite \ "full ptest not run by default (244 tests, ~3 h on this arm). Run it with BST_ELFUTILS_PTEST=1 BST_SUITE_TIMEOUT=20000; see the design doc for BRINTOS-507." for c in ptest-runner-rc runner-begin runner-end runner-tally runner-no-error \ suite-drivers-run tally-consistent; do bst_skip "elfutils/$c" "full ptest not run (BST_ELFUTILS_PTEST unset); see elfutils/ptest-suite" done rm -rf "$tmp" return 2>/dev/null || exit 0 fi harness=1 out=$tmp/runner.out "$runner" -t 40000 elfutils > "$out" 2>&1 rc=$? # WHAT THIS CASE IS FOR, AND WHY IT IS NOT `-eq 0`. ptest-runner returns the # number of failing ptests clamped to 1 (`if (rc > 0) rc = 1`), and -1 -- exit # 255 -- when its own plumbing fails (pipe2, openpty, fork). So 0 and 1 both # mean "the harness ran the ptest and got a verdict out of it", and only that is # this case's business: the suite's verdict is the suite-* cases below, one per # driver, which is where an attributed red belongs. Asserting 0 here would score # the whole suite a second time in a single case that no defect can be pinned # to, and would go GREEN->RED on any red at all rather than on a harness fault. check_rc elfutils/ptest-runner-rc 0 test "$rc" -le 1 # ------------------------------------------------------- the single join ------ # One awk pass over the transcript, emitting "\t\t" per case. # This layer's run-ptest keeps automake's serial-tests reporting -- one # "PASS: " / "FAIL: " / "SKIP: " line per test, scored by the # same exit-status contract -- so the join is over those rather than over # anything elfutils prints itself. awk -v OFS='\t' ' function emit(id, want, got) { print id, want, got } $0 ~ /^PASS: / { name = $2; verdict[name] = "PASS"; order[++n] = name; next } $0 ~ /^FAIL: / { name = $2; verdict[name] = "FAIL"; order[++n] = name; next } $0 ~ /^SKIP: / { name = $2; verdict[name] = "SKIP"; order[++n] = name; next } $0 ~ /^XFAIL: / { name = $2; verdict[name] = "XFAIL"; order[++n] = name; next } $0 ~ /^BEGIN: / { begin_line = $2; next } $0 ~ /^END: / { end_line = $2; next } # Two different programs print a "TOTAL: " line into one transcript: run-ptest # its own "TOTAL: n PASS: n FAIL: n SKIP: n" (8 fields) and ptest-runner its # "TOTAL: n FAIL: n" (4). They are told apart by field count and never by # position -- a transcript cut short can end on either. $0 ~ /^TOTAL: / && NF == 8 { suite_total = $2; next } $0 ~ /^TOTAL: / && NF == 4 { runner_tally = $1 " " $2; next } $0 ~ /^TIMEOUT: / { runner_error = runner_error $0 " "; next } # "ERROR: Exit status is N" is ptest-runner echoing back the OWN exit status # of the ptest, i.e. the suite verdict -- which the suite-* cases already carry, # driver by driver, where each red can be attributed. Every other ERROR: is # the harness failing at its own job (pipe2, openpty, fork, setsid, chdir), # which is what runner-no-error exists to catch; folding the two together # would leave a red that names no defect and hides the ones that do. $0 ~ /^ERROR: Exit status is / { next } $0 ~ /^ERROR: / { runner_error = runner_error $0 " "; next } END { # A SKIP is upstream declining a test on this environment (no gcc, no # locale, no ptrace) and is reported as a skip, not scored as a pass. for (i = 1; i <= n; i++) { if (verdict[order[i]] == "SKIP" || verdict[order[i]] == "XFAIL") continue emit("elfutils/suite-" order[i], "PASS", verdict[order[i]]) } # A run that stopped a third of the way through leaves every test it DID # reach green, so the count is asserted separately from the verdicts. emit("elfutils/suite-drivers-run", "ge:" MINTESTS, n "") # ...and the count run-ptest reported against the verdicts actually seen. A # transcript that ends mid-run leaves these unequal, which is not visible # from any per-driver case: every driver it DID reach still reported. The id # is deliberately outside the suite-* namespace, which holds exactly one # case per driver plus the count above. emit("elfutils/tally-consistent", n "", suite_total "") if (HARNESS == "1") { emit("elfutils/runner-begin", PTESTDIR, begin_line "") emit("elfutils/runner-end", PTESTDIR, end_line "") # ptest-runner counts PTESTS, not cases -- elfutils is one ptest however # many drivers it contains -- so this is "the harness ran the one ptest it # was given" and NOT the suite verdict. The suite-* cases above are that. emit("elfutils/runner-tally", "TOTAL: 1", runner_tally "") # ptest-runner reports its own fork/execv/openpty/pipe2 failures, and its # -t expiry, as ERROR/TIMEOUT lines while still exiting 0 for some of # them, so absence is asserted directly. emit("elfutils/runner-no-error", "", runner_error "") } } ' MINTESTS=200 PTESTDIR="$ptestdir" HARNESS="$harness" \ "$out" > "$tmp/verdicts" 2>/dev/null # ------------------------------------------------------ report, fork-free ----- n=0 while IFS=' ' read -r id want got; do [ -n "$id" ] || continue case "$want" in ge:*) check_rc "$id" 0 test "$got" -ge "${want#ge:}" ;; *) check_eq "$id" "$want" "$got" ;; esac n=$((n + 1)) done < "$tmp/verdicts" # A join that emitted nothing at all is itself a failure, and one that emitted # half its cases would otherwise read as a short but green suite. check_rc elfutils/join-nonempty 0 test "$n" -gt 1 rm -rf "$tmp"