# BST-TIMEOUT: 300 . "$BST_LIB" # CppUTest 4.0 (meta-oe recipes-test/cpputest with this layer's bbappend) on # wasm32-hwjs. Numbered with the other test frameworks on this image. # # WHY THIS SUITE IS ABOUT THE FAILURE PATH AND THE LEAK DETECTOR. That a # framework's passing case works is nearly free. CppUTest earns its place beside # the others on this image for one feature none of them has -- a memory-leak # detector wired into operator new/delete and malloc, checked per TEST -- and # both of its distinguishing mechanisms are exactly the ones an arm with no # unwinder puts in doubt: # # * The ABORT path. CppUTest does not unwind out of a failing assertion; it # longjmps. UtestShell::failWith reaches a TestTerminator, and # Utest::run wraps setup/body/teardown in PlatformSpecificSetJmp -- so a # failing CHECK transfers control by setjmp/longjmp, which on this target is # the hwjs-user-rt's own implementation over a coroutinized stack, not the # host's. Section 4 is what pins that it lands in the right frame. # * The LEAK DETECTOR. It replaces global operator new/new[]/delete and the # malloc family, records every allocation with its file and line, and # compares the count across a TEST. That is a global-allocator interposition # on a target whose allocator is reached through the runtime, so it is worth # asserting rather than assuming. # # BUILT WITH UPSTREAM'S OWN EXCEPTIONS-OFF SWITCH, which is why no patch ships # with this port. CppUTest's `STD_CPP` cmake option exists for compilers without # a C++ standard library or exceptions: it defines CPPUTEST_STD_CPP_LIB_DISABLED, # which turns NormalTestTerminator's `throw CppUTestFailedException()` into a # plain longjmp and drops MemoryLeakWarningPlugin's `throw std::bad_alloc()`. # Those two sites are the ONLY things in CppUTest that need exceptions -- measured # natively with `g++ -fno-exceptions`, both arms, before the recipe was written -- # and MEMORY_LEAK_DETECTION stays ON with STD_CPP off, so the port loses nothing # it came for. cpputest/inc-config-std-cpp-disabled below is what pins that the # option really took. # # THE CONFIG PROPAGATES TO CONSUMERS BY ITSELF, unlike googletest's gtest-port # macros: CppUTestConfig.h includes the INSTALLED CppUTestGeneratedConfig.h # unconditionally (guarded only by CPPUTEST_USE_OWN_CONFIGURATION), so a consumer # compiling against these headers sees the same CPPUTEST_STD_CPP_LIB_DISABLED the # archive was built with. There is no ODR hazard to warn about here, and the # case below asserts the header is actually in the -dev package rather than # trusting it. # # THE TIMING FIELD IS NORMALISED, AND NOTHING ELSE IS. Every arm compared # byte-for-byte against a native x86-64 reference is passed through # `sed -E 's/[0-9]+ ms/ ms/g'` first, on BOTH sides -- the references in # data/ were generated through the same filter. CppUTest prints a per-test and a # summary duration, and a wall-clock reading is not a property of the port. Every # other character -- the progress dots, the counts, the failure diagnostic's # rendering of both operands and its caret, the registration order -- is compared # exactly. # # THE LEAK ARM IS ASSERTED BY CONTENT, NOT BY cmp, and that is a fact about the # output rather than a weakening. The leak report prints the heap ADDRESS of the # leaked block and a hexdump of its uninitialised bytes, and numbers the # allocation by a process-wide counter -- three things that cannot be equal to a # 64-bit native run's and need not be equal between two target runs. What IS # invariant is asserted individually below: the size, the file and line the # allocation came from, the allocation type, and the total. # # NO COMPILE HAPPENS ON TARGET. This image ships no compiler, so nothing here can # write and build a CppUTest suite; the probes were cross-linked in the same # build that installed the headers and the archive, which makes do_compile the # port's first consumer. PT=/usr/libexec/cpputest-tests INC=/usr/include/CppUTest work=${TMPDIR:-/tmp}/bst-cpputest.$$ rm -rf "$work"; mkdir -p "$work" norm() { sed -E 's/[0-9]+ ms/ ms/g'; } # ------------------------------------------------------------ what shipped --- check_rc cpputest/inc-dir 0 test -d "$INC" check_rc cpputest/hdr-testharness 0 test -s "$INC/TestHarness.h" check_rc cpputest/hdr-runner 0 test -s "$INC/CommandLineTestRunner.h" check_rc cpputest/hdr-config 0 test -s "$INC/CppUTestConfig.h" check_rc cpputest/hdr-leakdetector 0 test -s "$INC/MemoryLeakDetector.h" check_rc cpputest/hdr-newmacros 0 test -s "$INC/MemoryLeakDetectorNewMacros.h" check_rc cpputest/hdr-platform 0 test -s "$INC/PlatformSpecificFunctions.h" # THE ARCHIVE IS THE DELIVERABLE. CppUTest is a compiled library, so a -dev # package without libCppUTest.a is unusable; cpputest-staticdev is a separate # package, and this case is what notices if the image ever installs -dev alone. check_rc cpputest/lib-archive 0 test -s /usr/lib/libCppUTest.a # The generated config header, and the two settings this port turns on. It is # installed (not merely present in the build tree), so a consumer's # CppUTestConfig.h picks it up and cannot disagree with the archive. check_rc cpputest/inc-generated-config 0 test -s "$INC/CppUTestGeneratedConfig.h" check_out cpputest/inc-config-std-cpp-disabled \ "#define CPPUTEST_STD_CPP_LIB_DISABLED" cat "$INC/CppUTestGeneratedConfig.h" check_out cpputest/inc-config-long-long \ "#define CPPUTEST_USE_LONG_LONG" cat "$INC/CppUTestGeneratedConfig.h" # ...and the leak detector is NOT disabled, which is the whole reason this # framework is worth having here. The generated header spells that as a # commented-out #undef, so the assertion is on the absence of the define. check_rc cpputest/inc-config-leak-detection-on 1 \ grep -q '^#define CPPUTEST_MEM_LEAK_DETECTION_DISABLED' "$INC/CppUTestGeneratedConfig.h" # ------------------------------------------------------- the probes exist ---- if [ ! -x "$PT/cpputest-probe-pass" ]; then bst_skip cpputest/probe-present "cpputest-probe-pass not installed on this image" exit 0 fi bst_ok cpputest/probe-present for p in cpputest-probe-pass cpputest-probe-fail cpputest-probe-leak; do check_rc "cpputest/exec-$p" 0 test -x "$PT/$p" done # ------------------------------------- 3. the control: five green tests ------ # Two groups, ten checks, across CppUTest's integer, long-long, double, string, # pointer and boolean assertion families. LONGLONGS_EQUAL is in here on purpose: # -DLONGLONG=ON is what the recipe asks for and this is a 32-bit target, so a # long-long comparison that silently truncated would show up as a red here and # nowhere else. If this arm is not clean, nothing below it means anything. "$PT/cpputest-probe-pass" > "$work/pass.raw" 2>&1; rc=$? norm < "$work/pass.raw" > "$work/pass.out" check_eq cpputest/pass-rc 0 "$rc" check_rc cpputest/pass-matches-native 0 \ cmp -s "$work/pass.out" "$BST_DATA/cpputest-expected-pass.out" check_out cpputest/pass-summary \ "OK (5 tests, 5 ran, 10 checks, 0 ignored, 0 filtered out," cat "$work/pass.out" # ------------------------------ 4. the finding: a failure REPORTS and lives --- # rc 1 here is the runner's own main() returning the failure count, NOT a signal. # The fixture has three tests and the failing one is the MIDDLE, so the two # progress dots after the diagnostic and the "3 ran" in the summary are what say # the process survived its own failure -- i.e. that the longjmp out of the failing # CHECK landed in Utest::run's setjmp frame and the run continued, on a target # whose setjmp is the hwjs-user-rt's over a coroutinized stack. A run that had # taken the process down could not have reached the summary line at all. "$PT/cpputest-probe-fail" > "$work/fail.raw" 2>&1; rc=$? norm < "$work/fail.raw" > "$work/fail.out" check_eq cpputest/fail-rc 1 "$rc" check_rc cpputest/fail-matches-native 0 \ cmp -s "$work/fail.out" "$BST_DATA/cpputest-expected-fail.out" # The parts, so a cmp red names the half that moved. check_out cpputest/fail-diagnostic-line \ "cpputest_fail.cpp:12: error: Failure in TEST(FailGroup, DeliberateFailure)" \ cat "$work/fail.out" check_out cpputest/fail-expected-operand "expected <3>" cat "$work/fail.out" check_out cpputest/fail-actual-operand "but was <2>" cat "$work/fail.out" # THE case that says the run continued past its own failure. check_out cpputest/fail-run-continued \ "Errors (1 failures, 3 tests, 3 ran, 3 checks, 0 ignored, 0 filtered out," \ cat "$work/fail.out" # CppUTest's macros expand __FILE__ into a data-section string literal that no # strip removes, so a recipe that stopped compiling from a bare filename in its # own directory would leak ${WORKDIR} into the diagnostic -- which the cmp above # would report only as "differs". Asserted separately so the cause is named. check_rc cpputest/fail-no-buildpath 1 \ grep -q '/tmp-glibc/\|/work/wasm32' "$work/fail.out" # ---------------------- 5. the memory-leak detector, which is why this port --- # CppUTest replaces global operator new/new[]/delete and the malloc family and # compares the live-allocation count across each TEST. The fixture leaks exactly # one 17-byte `new []` block in its first test and frees an identical block in its # second, so this arm asserts BOTH directions: the detector fires on the leak and # stays silent on the matched pair. A detector that had been compiled out, or # whose interposition did not take on this arm, would report a clean run -- which # is a green-looking red, and is the reason the counts below are asserted exactly. # # Not compared byte-for-byte: the report carries the block's address and a hexdump # of its uninitialised contents. What is invariant is named case by case. "$PT/cpputest-probe-leak" > "$work/leak.raw" 2>&1; rc=$? norm < "$work/leak.raw" > "$work/leak.out" check_eq cpputest/leak-rc 1 "$rc" check_out cpputest/leak-detected "Memory leak(s) found." cat "$work/leak.out" # The leak is attributed to the TEST that made it, not merely to the process. check_out cpputest/leak-names-case \ "cpputest_leak.cpp:5: error: Failure in TEST(LeakGroup, LeaksOneBlock)" \ cat "$work/leak.out" # Size, allocating line and allocation type -- the three facts a leak report # exists to carry, and the ones that would degrade quietly if the detector's # bookkeeping were wrong on a 32-bit target. check_out cpputest/leak-size "Leak size: 17" cat "$work/leak.out" check_out cpputest/leak-allocation-site "cpputest_leak.cpp and line: 7" cat "$work/leak.out" check_out cpputest/leak-alloc-type 'Type: "new []"' cat "$work/leak.out" check_out cpputest/leak-total "Total number of leaks: 1" cat "$work/leak.out" # The other direction: the second test allocates and frees the same shape and is # NOT reported. One failure out of two tests is the whole assertion. check_out cpputest/leak-clean-test-not-reported \ "Errors (1 failures, 2 tests, 2 ran, 2 checks, 0 ignored, 0 filtered out," \ cat "$work/leak.out" check_rc cpputest/leak-does-not-name-clean-test 1 \ grep -q 'TEST(LeakGroup, DoesNotLeak)' "$work/leak.out" # The address the report prints must be a real heap pointer rendered by CppUTest's # own formatter, not a truncation artefact -- a 32-bit target is where that would # show. Asserted as shape only, since the value cannot be pinned. check_rc cpputest/leak-address-rendered 0 \ grep -q 'Memory: <0x[0-9a-f][0-9a-f]*>' "$work/leak.out" # ------------------------------------- 6. -g and -n select a subset ---------- # The two filters a consumer uses most, and the arms that prove CppUTest's test # REGISTRY is populated and queryable rather than merely running everything it # linked. Both are asserted through the "filtered out" count, which no amount of # output-matching could fake: -g takes one of the two groups (3 ran, 2 filtered), # -n takes one test by name (1 ran, 4 filtered). "$PT/cpputest-probe-pass" -g Arith > "$work/group.raw" 2>&1; rc=$? norm < "$work/group.raw" > "$work/group.out" check_eq cpputest/filter-group-rc 0 "$rc" check_rc cpputest/filter-group-matches-native 0 \ cmp -s "$work/group.out" "$BST_DATA/cpputest-expected-group.out" check_out cpputest/filter-group-counts \ "OK (5 tests, 3 ran, 5 checks, 0 ignored, 2 filtered out," cat "$work/group.out" "$PT/cpputest-probe-pass" -n ComparesStrings > "$work/name.raw" 2>&1; rc=$? norm < "$work/name.raw" > "$work/name.out" check_eq cpputest/filter-name-rc 0 "$rc" check_rc cpputest/filter-name-matches-native 0 \ cmp -s "$work/name.out" "$BST_DATA/cpputest-expected-name.out" check_out cpputest/filter-name-counts \ "OK (5 tests, 1 ran, 2 checks, 0 ignored, 4 filtered out," cat "$work/name.out" # ...and the NEGATIVE form is load-bearing rather than cosmetic: excluding the one # bad case turns the failing fixture's run green. "$PT/cpputest-probe-fail" -n RunsBeforeTheFailure > "$work/nfilter.raw" 2>&1; rc=$? norm < "$work/nfilter.raw" > "$work/nfilter.out" check_eq cpputest/filter-excludes-failure-rc 0 "$rc" check_out cpputest/filter-excludes-failure-counts \ "OK (3 tests, 1 ran, 1 checks, 0 ignored, 2 filtered out," cat "$work/nfilter.out" # --------------------------------- 7. -ln reads the REGISTRY, against native -- # The one arm that reads the test registry directly rather than the result of # running it: group.name pairs in REGISTRATION ORDER, with nothing else in the # output. CppUTest registers each TEST from a load-time static initialiser onto a # singly-linked list by prepending, so this string is the exact reverse of the # link order -- and a cross-compiled build whose static initialisers ran in a # different sequence, which is the failure a wasm target with its own init order # could plausibly have, shows up here and nowhere else. "$PT/cpputest-probe-pass" -ln > "$work/list.raw" 2>&1; rc=$? norm < "$work/list.raw" > "$work/list.out" check_eq cpputest/list-rc 0 "$rc" check_rc cpputest/list-matches-native 0 \ cmp -s "$work/list.out" "$BST_DATA/cpputest-expected-list.out" check_out cpputest/list-names-a-test "Arith.AddsIntegers" cat "$work/list.out" # -ln must LIST and not RUN; a build that ran them anyway would still contain # every name above, so the tell is the absence of a summary line. check_rc cpputest/list-did-not-run 1 grep -q 'checks, 0 ignored' "$work/list.out" # --------------------------------- 8. -v names each test as it runs ---------- # The verbose arm, compared byte-for-byte: it prints one line per TEST in # execution order plus the summary, so it is the second independent statement # about ordering and the only one that shows the per-test duration field the # normalisation above is there for. "$PT/cpputest-probe-pass" -v > "$work/verbose.raw" 2>&1; rc=$? norm < "$work/verbose.raw" > "$work/verbose.out" check_eq cpputest/verbose-rc 0 "$rc" check_rc cpputest/verbose-matches-native 0 \ cmp -s "$work/verbose.out" "$BST_DATA/cpputest-expected-verbose.out" check_out cpputest/verbose-names-test "TEST(Arith, AddsIntegers)" cat "$work/verbose.out" rm -rf "$work"