# BST-TIMEOUT: 1500 # 71 cases, 491s measured locally -- the one suite here already over 300s unloaded . "$BST_LIB" # cunit (CUnit 2.1-3) -- the C xUnit framework, and the first port on this image # whose evidence is UPSTREAM'S OWN TEST SUITE rather than one we wrote. # # WHY THAT IS DIFFERENT FROM EVERY LIBRARY PORT BEFORE IT. There is no C compiler # on this image, so 76-gmp.test and 76-libatomic-ops.test can only show that an # archive and its headers exist, and 78-safec.test / 79-cyaml.test had to ship a # probe cross-linked at recipe time to have anything that RUNS. CUnit needs # neither: OpenEmbedded packages upstream's own CUnit/Sources/Test/test_cunit as a # ptest, cross-compiled here and installed under /usr/lib/cunit/ptest, so what # executes on target is the binary CUnit's authors wrote to test CUnit. # # THE ORACLE IS A NATIVE x86_64 BUILD OF THE IDENTICAL 2.1-3 SOURCE (sha256 # f5b29137f845bb08b77ec60584fdb728b4e58f1023e6f249a464efa49a40f214, the sum the # recipe pins) configured with the recipe's own flags -- --enable-memtrace # --enable-automated --enable-basic --enable-console --enable-test, plus the # distro-wide --disable-shared. Every number below was read off that run, never # off the source. # # AND THE COUNTS ARE A TRUE ORACLE, not a convenience. test_cunit's per-module # totals are the number of CU_TEST assertions its own code executes: fixed by the # control flow, carrying no pointer width, no sizeof, no address and no timing. # Nothing in them is ALLOWED to differ between x86_64 and wasm32, so a single # disagreement is a defect on this arm rather than a platform difference to reason # about -- which is exactly what makes a package with an upstream suite worth # porting ahead of one without. # # WHAT A RED HERE WOULD MEAN, by module: # CUError.c the error-code/message plumbing every other module reports through # MyMem.c CUnit's allocation tracker (--enable-memtrace) -- malloc/free # accounting, the one module whose subject is the heap # TestDB.c the registry: suites, tests, add/remove, lookup by name and index # TestRun.c the runner and the assertion machinery itself -- 2836 of the 3915 # Util.c string trim/compare/copy helpers, incl. the XML escaping the # automated interface writes its report with # # THE HARNESS HALF IS THE OTHER DELIVERABLE. This ticket closes on the suite run # UNDER ptest-runner, not on a hand-invoked script, so ptest-runner's own discovery # (walk /usr/lib/*/ptest), its fork+execv+openpty child plumbing and its # TOTAL/FAIL accounting are asserted here too. Their oracle is the same # ptest-runner SRCREV built and run natively against the same layout. # # SPAWN BUDGET IS A REAL CONSTRAINT ON THIS ARM, not a style preference: exec is a # fresh wasm module instantiation rather than a fork, ~2-3 s apiece, and a suite # that overruns BST_SUITE_TIMEOUT reports a FAILING suite-timeout case (78-safec # went from 30 s to over seven minutes on ~150 awk calls). So this file runs # ptest-runner once, flattens the archive index once, joins everything in ONE awk # pass that emits a pre-computed triple per case, and the shell # loop that reports them spawns nothing at all. Presence checks use check_rc with # `test`, which is an ash builtin and forks no subshell -- unlike check/check_out, # whose command substitution does. ptestdir=/usr/lib/cunit/ptest runner=/usr/bin/ptest-runner hdr=/usr/include/CUnit/CUnit.h lib=/usr/lib/libcunit.a pc=/usr/lib/pkgconfig/cunit.pc tmp=${TMPDIR:-/tmp}/bst-cunit.$$ mkdir -p "$tmp" 2>/dev/null || tmp=/tmp # ----------------------------------------------------------- what shipped ---- # The ptest package's two files, then the -dev/-staticdev payload. All builtin # `test`, so this whole block costs no process. check_rc cunit/ptest-dir 0 test -d "$ptestdir" check_rc cunit/ptest-runptest 0 test -x "$ptestdir/run-ptest" check_rc cunit/ptest-testbin 0 test -x "$ptestdir/tests/test_cunit" check_rc cunit/ar-present 0 test -s "$lib" check_rc cunit/pc-present 0 test -f "$pc" # The headers CUnit installs. CUnit.h is the umbrella a consumer includes; the # other three interface headers are one per --enable-* the recipe asked for, so a # missing one means a configure flag did not take rather than a lost file. for h in CUnit TestDB TestRun CUError Util MyMem Automated Basic Console; do check_rc "cunit/hdr-$h" 0 test -s "/usr/include/CUnit/$h.h" done # The XML report contract: the automated interface writes a report that names # these, and they ship in -dev rather than being generated, so a consumer that # renders a CUnit run has them locally. for d in CUnit-Run.dtd CUnit-Run.xsl CUnit-List.dtd CUnit-List.xsl Memory-Dump.dtd; do check_rc "cunit/data-$d" 0 test -s "/usr/share/CUnit/$d" done # ------------------------------------------------------- run it, once --------- # TWO WAYS IN, AND THE SUITE PREFERS THE HARNESS WHENEVER IT IS THERE. # # ptest-runner resolves /usr/lib itself (the recipe compiles DEFAULT_DIRECTORY in), # discovers cunit/ptest, chdirs into it and execs run-ptest under a pty; naming the # ptest confines the run to this one suite. That is the arrangement this ticket is # meant to close on, and the four runner/* cases below assert it. # # IT IS ON THIS IMAGE, and getting it there was one recipe removed from CUnit: # oe-core's ptest-runner inherits useradd.bbclass to create a "ptest" system user, # which puts TARGET shadow into DEPENDS and RDEPENDS, and shadow does not build on # this arm (`configure: error: crypt() not found` -- no crypt provider is wired # into the target sysroot). The bbappend drops the dependency both halves at once, # the way cronie's already documents; the harness never links crypt. So the # harness branch below is the one this image takes, and the four runner/* cases # assert rather than skip. # # The fallback is kept anyway, for an image that does not ship the harness: it # runs /usr/lib/cunit/ptest/run-ptest directly -- byte-for-byte the script the # harness would exec, so every assertion about CUNIT is unaffected -- and SKIPS # exactly the four cases that are about the HARNESS. A skip, not a red: those # cases have no subject on such an image. out=$tmp/runner.out harness=0 if [ -x "$runner" ]; then harness=1 bst_ok cunit/ptest-runner-present "$runner" cunit > "$out" 2>&1 rc=$? check_eq cunit/ptest-runner-rc 0 "$rc" else bst_skip cunit/ptest-runner-present "this image ships no ptest-runner (useradd.bbclass pulls target shadow; see the bbappend)" bst_skip cunit/ptest-runner-rc "no ptest-runner to run" for c in runner-begin runner-end runner-tally runner-no-error; do bst_skip "cunit/$c" "harness cases have no subject without ptest-runner" done sh "$ptestdir/run-ptest" > "$out" 2>&1 fi # --------------------------------------------------- the archive, flattened --- # The 76-gmp.test idiom, but over the WHOLE archive rather than its head: the # exported-symbol map is at the front, so a head would have sufficed for symbols, # while each MEMBER's name lives in that member's own header -- Console.o's is # past the other seven members' data. One `tr`, once; the archive is ~130 KB. idx=$tmp/ar.idx tr -c '\40-\176\n' ' ' < "$lib" > "$idx" 2>/dev/null # ------------------------------------------------------- the single join ------ # One awk over four files, emitting "\t\t" per case. Everything # after this is pure shell. awk -v OFS='\t' ' function emit(id, want, got) { print id, want, got } # ---- file 1: ptest-runner output ---- FILENAME == RUNOUT { # test_cunit prints " testing .c ... N assertions, M failures" if ($0 ~ /^ *testing .*\.c \.\.\./) { mod = $2; sub(/\.c$/, "", mod) for (i = 1; i <= NF; i++) if ($i == "assertions,") { asserts[mod] = $(i-1); fails[mod] = $(i+1) } nmod++ } if ($0 ~ /Total Number of Assertions:/) total = $NF if ($0 ~ /^ *Successes:/) succ = $NF if ($0 ~ /^ *Failures:/ && $0 !~ /assertions/) failed = $NF if ($0 ~ /^PASS: cunit/) verdict = "PASS: cunit" if ($0 ~ /^FAIL: cunit/) verdict = "FAIL: cunit" if ($0 ~ /^BEGIN: /) begin_line = $2 if ($0 ~ /^END: /) end_line = $2 if ($0 ~ /^TOTAL: /) tally = $0 if ($0 ~ /^ERROR: /) runner_error = runner_error $0 " " next } # ---- file 2: CUnit.h ---- FILENAME == HDR { if ($0 ~ /^#define CU_VERSION/) { v = $3; gsub(/"/, "", v); hdrver = v } next } # ---- file 3: cunit.pc ---- FILENAME == PC { if ($0 ~ /^Name: /) { sub(/^Name: */, ""); pcname = $0 } if ($0 ~ /^Version: /) { sub(/^Version: */, ""); pcver = $0 } if ($0 ~ /^Libs: /) { sub(/^Libs: */, ""); pclibs = $0 } if ($0 ~ /^prefix=/) { sub(/^prefix=/, ""); pcprefix = $0 } next } # ---- file 4: the flattened archive index ---- FILENAME == IDX { idx = idx " " $0; next } # Whole-token lookup in the flattened index. A bare index() would report # CU_get_error present on an archive carrying only CU_get_error_msg, which is # the difference between "CUError.o compiled" and "it did not". function has(tok) { return index(" " idx " ", " " tok " ") > 0 } END { # --- the upstream suite, per module. These five ARE the port evidence. --- emit("cunit/suite-CUError-asserts", "15", asserts["CUError"] "") emit("cunit/suite-CUError-fails", "0", fails["CUError"] "") emit("cunit/suite-MyMem-asserts", "34", asserts["MyMem"] "") emit("cunit/suite-MyMem-fails", "0", fails["MyMem"] "") emit("cunit/suite-TestDB-asserts", "893", asserts["TestDB"] "") emit("cunit/suite-TestDB-fails", "0", fails["TestDB"] "") emit("cunit/suite-TestRun-asserts", "2836", asserts["TestRun"] "") emit("cunit/suite-TestRun-fails", "0", fails["TestRun"] "") emit("cunit/suite-Util-asserts", "137", asserts["Util"] "") emit("cunit/suite-Util-fails", "0", fails["Util"] "") # All five ran: a truncated run whose modules each pass is not a pass. emit("cunit/suite-modules-run", "5", nmod "") # ...and the totals, which are the sum the suite computes for itself rather # than one we add up here -- so a disagreement between them is also caught. emit("cunit/suite-total-assertions", "3915", total "") emit("cunit/suite-successes", "3915", succ "") emit("cunit/suite-failures", "0", failed "") # --- the ptest contract: run-ptest greps "Failures: 0" and says so. --- # # THIS IS THE CASE THAT CATCHES A FAILING RUN, not ptest-runner-rc above. # The upstream run-ptest ends in "rm -f", so it exits 0 whatever test_cunit # did, and ptest-runner faithfully reports TOTAL: 1 FAIL: 0 for it. Measured, # not assumed: feeding the join a run truncated after two modules leaves # ptest-runner-rc and runner-tally GREEN and reds this case along with the # module and total cases. An exit status is not the verdict of this ptest. emit("cunit/ptest-verdict", "PASS: cunit", verdict "") # --- the harness: discovery, the child plumbing, and the tally. --- # Only when ptest-runner actually ran; the direct run-ptest fallback emits # none of these lines, and the shell has already reported them as skips with # BRINTOS-394 as the cause. if (HARNESS == "1") { emit("cunit/runner-begin", PTESTDIR, begin_line "") emit("cunit/runner-end", PTESTDIR, end_line "") emit("cunit/runner-tally", "TOTAL: 1 FAIL: 0", tally "") # ptest-runner reports its own fork/execv/openpty/pipe2 failures as ERROR # lines and still exits 0 for some of them, so absence is asserted directly. emit("cunit/runner-no-error", "", runner_error "") } # --- the payload --- emit("cunit/hdr-version", "2.1-3", hdrver "") emit("cunit/pc-name", "CUnit", pcname "") emit("cunit/pc-version", "2.1-3", pcver "") emit("cunit/pc-libs", "-L${libdir} -lcunit", pclibs "") emit("cunit/pc-prefix", "/usr", pcprefix "") # The archive members, one per compilation unit. The three interface objects # are the --enable-automated/basic/console flags made visible in the artifact. # ar stores a short member name in its header as "/" (slash-terminated, # space-padded), which is the token to look for -- a bare "Console.o" is not # in the file at all and would report every member missing. no = split("CUError.o MyMem.o TestDB.o TestRun.o Util.o Automated.o Basic.o Console.o", want_o, " ") for (i = 1; i <= no; i++) emit("cunit/ar-member-" want_o[i], want_o[i], (has(want_o[i] "/") ? want_o[i] : "MISSING")) # ...and Curses.o must NOT be there: --enable-curses was deliberately not # asked for, and its presence would mean a curses dependency crept in. emit("cunit/ar-no-curses", "absent", (has("Curses.o/") ? "PRESENT" : "absent")) # One exported entry point per module a consumer actually calls -- every one # of the eight members is covered by at least one of these, so a partial # compile that produced TestDB.o but not TestRun.o fails here and not at the # first consumer. The pairing, member by member: CUError.o CU_get_error, # MyMem.o CU_calloc, TestDB.o CU_add_suite, TestRun.o CU_assertImplementation, # Util.o CU_trim, and one run_tests entry point per interface. ns = split("CU_initialize_registry CU_cleanup_registry CU_add_suite CU_add_test " \ "CU_assertImplementation CU_basic_run_tests CU_basic_set_mode " \ "CU_automated_run_tests CU_console_run_tests CU_get_number_of_failures " \ "CU_get_number_of_asserts CU_get_error CU_get_error_msg " \ "CU_get_run_summary CU_calloc CU_trim", want_s, " ") for (i = 1; i <= ns; i++) emit("cunit/ar-sym-" want_s[i], want_s[i], (has(want_s[i]) ? want_s[i] : "MISSING")) } ' RUNOUT="$out" HDR="$hdr" PC="$pc" IDX="$idx" PTESTDIR="$ptestdir" HARNESS="$harness" \ "$out" "$hdr" "$pc" "$idx" > "$tmp/verdicts" 2>/dev/null # ------------------------------------------------------ report, fork-free ----- # A join that produced nothing at all is itself a failure, and one that silently # emitted half its cases would otherwise read as a short but green suite. n=0 while IFS=' ' read -r id want got; do [ -n "$id" ] || continue check_eq "$id" "$want" "$got" n=$((n + 1)) done < "$tmp/verdicts" if [ "$harness" = 1 ]; then check_eq cunit/join-complete 49 "$n" else check_eq cunit/join-complete 45 "$n"; fi rm -rf "$tmp"