. "$BST_LIB" # cmocka (1.1.7+git) -- the C unit-testing framework with mock support, closed on # UPSTREAM'S OWN TEST SUITE run under ptest-runner. # # WHAT MAKES THIS PORT'S EVIDENCE DIFFERENT. There is no C compiler on this image, # so a library port normally has to invent its evidence: 76-gmp.test and # 76-libatomic-ops.test can assert an archive symbol index and little else, and # 78-safec.test / 73-cyaml.test ship a probe cross-linked at recipe time to have # anything that RUNS. cmocka needs neither -- OpenEmbedded packages upstream's own # tests/test_* as a ptest, 23 binaries cross-compiled here and installed under # /usr/lib/cmocka/ptest -- so what executes on this target is the suite cmocka's # authors wrote to test cmocka. # # AND THE SUITE IS UNUSUALLY WELL AIMED AT THIS ARM, which is why it is worth more # here than its 23 green lines would suggest elsewhere: # - cmocka does not report a failed assertion by returning a code. It LONGJMPS # out of the test through a per-test jmp_buf (cmocka.c, _fail -> longjmp). # That is the coroutinize hazard the Lua port was chosen to probe, reached # from a second direction: here the longjmp is the library's normal, hot path, # taken by every one of the seven by-design failures below. # - test_exception_handler raises three REAL SIGSEGVs and requires the handler to # recover from each. No other suite on this image asks whether a genuine # memory fault is delivered as a signal rather than killing the module. It is # also the one case this arm cannot answer -- see the mask below. # - test_alloc is cmocka's own leak/overflow tracker: it allocates, writes past # both ends of a block on purpose, and requires the guard bytes to be caught. # # THE ORACLE IS A NATIVE x86_64 BUILD OF THE IDENTICAL SOURCE (SRCREV # a01cc69ee9536f90e57c61a198f2d1944d3d4313), configured with the recipe's own # flags, and it ships beside this file as data/cmocka-native.txt. Every expected # number below is read off that run rather than off the source. cmocka's exit # status is its COUNT OF FAILED TESTS, and those counts are fixed by the suite's # control flow -- no pointer width, no sizeof, no address, no timing -- so a # single disagreement is a defect on this arm, not a platform difference. # # SEVEN OF THE 23 FAIL ON PURPOSE. They are cmocka's tests of its own failure # REPORTING (upstream marks them WILL_FAIL / PASS_REGULAR_EXPRESSION in # tests/CMakeLists.txt), so "the suite passed" means those seven failed in exactly # the expected way. That scoring lives in run-ptest, which this port had to # replace outright: meta-oe's ships `if [ ./$i ]`, a one-argument test on a # non-empty string, which never executes a binary and prints PASS for every file # in the directory. The ptest-content case below is what keeps that regression # from ever passing silently again. # # SPAWN BUDGET. exec is a fresh wasm module instantiation here (~2-3 s), not a # fork, and the suite already pays 23 of them inside run-ptest. So this file runs # the harness ONCE, joins everything in ONE awk pass that emits a pre-computed # " " triple per case, and the reporting loop spawns nothing. # Presence checks use check_rc with `test`, an ash builtin. ptestdir=/usr/lib/cmocka/ptest runner=/usr/bin/ptest-runner hdr=/usr/include/cmocka.h lib=/usr/lib/libcmocka.a oracle=/usr/lib/brintos-tests/data/cmocka-native.txt tmp=${TMPDIR:-/tmp}/bst-cmocka.$$ mkdir -p "$tmp" 2>/dev/null || tmp=/tmp # ------------------------------------------------------------------ the mask -- # BRINTOS-WORKAROUND: linux#21 remove — empty $masked and drop this block once # signal delivery works. The binary's own case is SKIPPED and the four totals it # feeds are re-based off the mask, so none of them will flip green by itself. # # test_exception_handler is the single binary this arm cannot score. Its three # tests each call raise(SIGSEGV) (tests/test_exception_handler.c) and require # cmocka's handler to catch the signal and mark the test failed; natively it # exits 3. Here every raise() is a no-op — all three report [ OK ] and the binary # exits 0. cmocka's exception machinery is present in the archive # (exception_handler, exception_signals, default_signal_functions), so it was # built and installed; it is simply never invoked. Nor is the default # disposition, which for SIGSEGV would have killed the process. # # Not cmocka's doing and not this port's. On this same image, in plain shell: # sh -c 'trap "echo TRAPPED; exit 0" TERM; kill -TERM $$; sleep 2; echo NOT-TRAPPED' # -> NOT-TRAPPED # sleep 300 & kill -KILL $! ; sleep 2 ; kill -0 $! -> still alive # That is brintos/linux#21 ("signal delivery does not work at all — kill(2) # reports success and nothing is delivered, including SIGKILL and a process # signalling itself"), reached from C rather than from the shell. masked='test_exception_handler' mask_reason='linux#21 — raise() delivers nothing on this arm, so cmocka never sees the SIGSEGV' # run-ptest exits non-zero exactly when a binary missed its expectation, and the # masked ones are precisely those. ptest-runner propagates it. if [ -n "$masked" ]; then want_ptest_rc=1; else want_ptest_rc=0; fi # ----------------------------------------------------------- what shipped ---- check_rc cmocka/ptest-dir 0 test -d "$ptestdir" check_rc cmocka/ptest-runptest 0 test -x "$ptestdir/run-ptest" check_rc cmocka/ptest-testsdir 0 test -d "$ptestdir/tests" check_rc cmocka/ar-present 0 test -s "$lib" check_rc cmocka/hdr-present 0 test -s "$hdr" check_rc cmocka/oracle-present 0 test -s "$oracle" # cmocka_pbc.h is the design-by-contract header; it ships from -dev beside # cmocka.h, so its absence means the install was partial rather than the build # having failed. check_rc cmocka/hdr-pbc 0 test -s /usr/include/cmocka_pbc.h # THE REPLACED run-ptest, asserted by CONTENT. meta-oe's stock script is the # reason this case exists: it reports PASS for every file without executing any of # them, so an image that silently picked the stock one up again would show 23 # green lines here and mean nothing at all. The needle is a string only the # replacement carries, and it is checked before the suite is believed. check cmocka/ptest-scores-exit grep -q 'no expectation recorded' "$ptestdir/run-ptest" # ------------------------------------------------------- run it, once --------- # ptest-runner resolves /usr/lib itself (the recipe compiles DEFAULT_DIRECTORY # in), discovers cmocka/ptest, chdirs into it and execs run-ptest under a pty; # naming the ptest confines the run to this one suite. That is the arrangement # this ticket closes on, so the harness cases are asserted rather than assumed. # # If the harness is absent the suite still runs run-ptest directly -- byte for # byte the script ptest-runner would exec, so every assertion about CMOCKA is # unaffected -- and skips exactly the cases that are about the HARNESS. out=$tmp/runner.out harness=0 if [ -x "$runner" ]; then harness=1 bst_ok cmocka/ptest-runner-present "$runner" cmocka > "$out" 2>&1 check_eq cmocka/ptest-runner-rc "$want_ptest_rc" "$?" else bst_skip cmocka/ptest-runner-present "no ptest-runner on this image" bst_skip cmocka/ptest-runner-rc "no ptest-runner to run" for c in runner-begin runner-end runner-tally runner-no-error; do bst_skip "cmocka/$c" "harness cases need ptest-runner" done sh "$ptestdir/run-ptest" > "$out" 2>&1 fi # --------------------------------------------------- the archive, flattened --- idx=$tmp/ar.idx tr -c '\40-\176\n' ' ' < "$lib" > "$idx" 2>/dev/null # ------------------------------------------------------- the single join ------ awk -v OFS='\t' ' # The reporting loop below reads these back with IFS=tab, and tab is IFS # WHITESPACE: consecutive tabs collapse to one delimiter, so an empty field # would shift every later field left and report a case with want and got # swapped. Nothing is ever emitted empty. function emit(id, want, got) { print id, (want == "" ? "(none)" : want), (got == "" ? "(none)" : got) } function is_masked(t) { return index(" " MASKED " ", " " t " ") > 0 } # ---- file 1: the native oracle. Its binary list drives everything. ---- FILENAME == ORACLE { if ($0 ~ /^#/ || NF < 2) next want_rc[$1] = $2 order[++nwant] = $1 next } # ---- file 2: the run output ---- FILENAME == RUNOUT { if ($0 ~ /^PASS: test_/) { split($0, a, " "); got[a[2]] = "PASS" } if ($0 ~ /^FAIL: test_/) { split($0, a, " "); got[a[2]] = "FAIL" } if ($0 ~ /^CMOCKA TOTAL: /) suite_tally = $0 # ptest-runner framing, which is about the HARNESS and not the suite. Its # tally line is "TOTAL: n FAIL: n"; run-ptest deliberately labels its own # "CMOCKA TOTAL:" so the two cannot be confused for each other here. if ($0 ~ /^BEGIN: /) begin_line = $2 if ($0 ~ /^END: /) end_line = $2 if ($0 ~ /^TOTAL: [0-9]+ FAIL: [0-9]+$/) runner_tally = $0 if ($0 ~ /^ERROR: /) runner_error = (runner_error == "" ? $0 : runner_error "; " $0) next } # ---- file 3: the flattened archive ---- FILENAME == IDX { idx = idx " " $0; next } function has(tok) { return index(" " idx " ", " " tok " ") > 0 } END { # --- the upstream suite, one case per binary. THIS is the port evidence. --- # Scored against the oracle list, so a binary that upstream ships and this # image did not build reports MISSING rather than quietly not being asked # about. # A masked binary is one the platform cannot answer (see $masked): its own # case is skipped, and it is counted so the four totals it feeds can be # re-based rather than skipped too -- a SECOND red still turns them. for (i = 1; i <= nwant; i++) { t = order[i] if (is_masked(t)) { nmask++; emit("cmocka/suite-" t, "SKIP", MASKREASON); continue } emit("cmocka/suite-" t, "PASS", (t in got ? got[t] : "MISSING")) } # All 23 ran: a truncated run whose every reported case passed is not a pass. # A masked binary still runs and is still reported, so it counts here. emit("cmocka/suite-count", nwant "", ncount(order, nwant, got) "") # The tally run-ptest computed for itself, rather than one this join added up, # so a disagreement between the two is caught as well. # nmask+0, not nmask: with an empty mask it is never assigned, and awk # concatenates an unset variable as "" -- which would ask for "FAIL: ". emit("cmocka/suite-tally", "CMOCKA TOTAL: " nwant " FAIL: " nmask+0, suite_tally "") # --- the harness --- if (HARNESS == "1") { emit("cmocka/runner-begin", PTESTDIR, begin_line "") emit("cmocka/runner-end", PTESTDIR, end_line "") emit("cmocka/runner-tally", "TOTAL: 1 FAIL: " (nmask > 0 ? 1 : 0), runner_tally "") # ptest-runner reports its own fork/execv/openpty/pipe2 failures as ERROR # lines and still exits 0 for some of them, so the set is asserted directly # rather than only through the exit status. With a mask in force it also # reports the non-zero exit of the suite, which is that mask and nothing else. emit("cmocka/runner-no-error", (nmask > 0 ? "ERROR: Exit status is 1" : ""), runner_error) } # --- the payload --- # One exported entry point per area a consumer actually calls: the runner, # the mock/will_return machinery, the allocator tracker and the fixtures. ns = split("_run_group_tests _run_tests _assert_true _assert_int_equal " \ "_assert_string_equal _assert_memory_equal _will_return _mock " \ "_check_expected _expect_check _test_malloc _test_free " \ "_test_calloc _test_realloc _fail _skip cmocka_set_message_output", want_s, " ") for (i = 1; i <= ns; i++) emit("cmocka/ar-sym-" want_s[i], want_s[i], (has(want_s[i]) ? want_s[i] : "MISSING")) # The single compilation unit, by the name cmake gives it (.o, not # .o) and by the name ar stores it under -- slash-terminated, which is # the token actually present in the file. emit("cmocka/ar-member", "cmocka.c.o", (has("cmocka.c.o/") ? "cmocka.c.o" : "MISSING")) } function ncount(ord, n, g, i, c) { c = 0 for (i = 1; i <= n; i++) if (ord[i] in g) c++ return c } ' ORACLE="$oracle" RUNOUT="$out" IDX="$idx" PTESTDIR="$ptestdir" HARNESS="$harness" \ MASKED="$masked" MASKREASON="$mask_reason" \ "$oracle" "$out" "$idx" > "$tmp/verdicts" 2>/dev/null # ------------------------------------------------------ report, fork-free ----- n=0 while IFS=' ' read -r id want got; do [ -n "$id" ] || continue if [ "$want" = SKIP ]; then bst_skip "$id" "$got" else check_eq "$id" "$want" "$got"; fi n=$((n + 1)) done < "$tmp/verdicts" # A join that produced nothing is itself a failure, and one that emitted half its # cases would otherwise read as a short but green suite. if [ "$harness" = 1 ]; then check_eq cmocka/join-complete 47 "$n" else check_eq cmocka/join-complete 43 "$n"; fi rm -rf "$tmp"