. "$BST_LIB" # libcheck (Check 0.15.2) -- the unit-testing framework for C. The port has two # halves and they need completely different kinds of evidence, so this file is # written in two halves to match. # # THE RUNNABLE HALF IS checkmk, AND IT IS THE WHOLE REASON THIS SUITE CAN DO # MORE THAN INSPECT A PAYLOAD. Check's library is consumed by COMPILING against # it, and there is no C compiler on this image -- the position 76-gmp.test and # 76-libatomic-ops.test are in, where the archive can only be shown to be real, # never to work. libcheck escapes that because poky splits ${bindir}/checkmk # into its own package: a code generator that turns a terse .ts test-suite file # into the C source (START_TEST bodies + a main() that builds the Suite/TCase # tree and drives an SRunner) a developer would otherwise hand-write. It is an # awk program, so it is architecture-independent -- which sounds like it makes # the test worthless on wasm32 and does the opposite: it means every red here is # a PLATFORM red. The script cannot miscompile. If checkmk misbehaves on target # it is because the shipped shebang does not resolve, or gawk on this arm # mis-executes a real awk program (dynamic regexes, sprintf, multi-dimensional # arrays, getline, print > file -- checkmk uses all of them), or /dev/stdout is # not there. Those are exactly the platform properties worth a regression guard. # # TWO SHAPE FACTS decide how the checkmk half is written, both measured against # a native x86_64 build of the same 0.15.2 source before anything ran on target: # # 1. checkmk TAKES ITS OPTIONS AS awk VARIABLE ASSIGNMENTS, not as flags. # `checkmk clean_mode=1 input.ts` is the documented spelling (checkmk.1 # SYNOPSIS), and there is no --help and no -o. A `-clean`-shaped assumption # would fail as "file not found" and look like a broken port. # # 2. #main-pre / #main-post MUST FOLLOW EVERY #test. Placed before one, # checkmk exits 1 with "Cannot specify tests after main-pre or main-post." # -- correctly. The ordering in the fixture below is therefore load-bearing; # the natural-looking order is the error case. # # THE PAYLOAD HALF follows the 76-gmp.test model: headers, the archive's symbol # index, and the pkg-config contract, because that is all that can be checked # without a compiler. It carries one wasm32-specific witness the native control # cannot pass -- check_stdint.h is GENERATED by configure from the compiling # toolchain, so the fact that it exists and names check 0.15.2 is evidence the # cross-configure ran to completion on this arm rather than being copied in. # # WHAT IS DELIBERATELY NOT ASSERTED: Check's fork mode -- and it is not asserted # because it is BROKEN ON THIS PLATFORM, which was measured rather than assumed. # Upstream's own test binaries cross-build cleanly for this arm (all nine link, # refused=0), so BRINTOS-221 ran them on target as an investigation step. fork() # works; the per-test timeout Check arms right after forking does not: # # ../../check-0.15.2/src/check_run.c:536: # Error in call to timer_create: Function not implemented # # timer_create(2) and its four siblings are unwired in the wasm32 syscall table # -- brintos.io/brintos/linux/issues/40, filed by this ticket with a candidate # fix. So EVERY Check test binary in the default CK_FORK mode dies at its first # test, and CK_FORK=no is the workaround (at the cost of the crash isolation and # the timeouts that are most of Check's value). # # None of that can be a case in this file: the image ships no consumer of the # library and there is no on-target compiler to build one, so there is nothing # here for a red to attach to. The suite asserts instead that CK_NOFORK is # present in the API, which is the workaround's precondition, and leaves the # defect to issue 40. When 40 closes, the honest coverage upgrade is a shipped # Check-linked probe binary, not another header assertion. bin=/usr/bin/checkmk hdr=/usr/include/check.h sthdr=/usr/include/check_stdint.h lib=/usr/lib/libcheck.a pc=/usr/lib/pkgconfig/check.pc m4=/usr/share/aclocal/check.m4 require checkmk libcheck/checkmk-present tmp=${TMPDIR:-/tmp}/bst-libcheck.$$ mkdir -p "$tmp" 2>/dev/null || tmp=/tmp # ------------------------------------------------------------- the fixture --- # One .ts exercising the directives a real test suite uses: pass-through text # before the first directive, one suite with two test cases, two tests in the # first and one in the second, then the main-pre/main-post pair last (shape # fact 2). Everything the generator does with it is asserted below against the # native control's output, literal for literal. cat > "$tmp/basic.ts" <<'TSEOF' /* passthru header */ #include #suite Arithmetic #tcase Addition #test add_two ck_assert_int_eq(2 + 2, 4); #test add_zero ck_assert_int_eq(7 + 0, 7); #tcase Strings #test str_eq ck_assert_str_eq("ab", "ab"); #main-pre int rc_pre = 0; #main-post (void)rc_pre; TSEOF # The shebang is baked at configure time from ac_cv_path_AWK_PATH, so a checkmk # that cannot run is nearly always a shebang pointing at an absent interpreter. # Assert the interpreter the shipped script actually names, then that it runs -- # in that order, so a failure says which of the two broke. check_out libcheck/checkmk-shebang "#! /usr/bin/gawk -f" head -1 "$bin" check libcheck/checkmk-interp test -x /usr/bin/gawk out=$tmp/basic.c check libcheck/checkmk-runs sh -c "checkmk '$tmp/basic.ts' > '$out' 2>'$tmp/basic.err'" # Every assertion from here reads the one generated file rather than re-running # the generator, so a single failure to run reports once instead of thirty times. check libcheck/gen-nonempty test -s "$out" check_out libcheck/gen-banner "DO NOT EDIT THIS FILE. Generated by checkmk." cat "$out" check_out libcheck/gen-srcref 'Edit the original source file "'"$tmp/basic.ts"'" instead.' cat "$out" check_out libcheck/gen-include "#include " cat "$out" # Text before the first directive is copied through verbatim: that is what lets # a .ts carry the includes and helpers its tests need. check_out libcheck/gen-passthru-comment "/* passthru header */" cat "$out" check_out libcheck/gen-passthru-include "#include " cat "$out" # #line directives map the generated C back onto the .ts. They are the reason a # ck_assert failure reports a line in the file the developer edits, so their # presence is a contract and not cosmetic. check_out libcheck/gen-line-file "#line 1 \"$tmp/basic.ts\"" cat "$out" check libcheck/gen-line-body grep -q '^#line 9$' "$out" # The test bodies. check_out libcheck/gen-start-add-two "START_TEST(add_two)" cat "$out" check_out libcheck/gen-start-add-zero "START_TEST(add_zero)" cat "$out" check_out libcheck/gen-start-str-eq "START_TEST(str_eq)" cat "$out" check_out libcheck/gen-end-test "END_TEST" cat "$out" check_out libcheck/gen-body-verbatim "ck_assert_int_eq(2 + 2, 4);" cat "$out" check_eq libcheck/gen-end-test-count 3 "$(grep -c '^END_TEST$' "$out")" # The generated main(): the Suite/TCase tree, the registrations, and the runner. # The variable names (s1, tc1_1, tc1_2) are checkmk's own numbering scheme and # are asserted as such -- a consumer's Makefile never sees them, but a change in # them means the generator's structure changed. check_out libcheck/gen-main "int main(void)" cat "$out" check_out libcheck/gen-suite-create 'Suite *s1 = suite_create("Arithmetic");' cat "$out" check_out libcheck/gen-tcase-1 'TCase *tc1_1 = tcase_create("Addition");' cat "$out" check_out libcheck/gen-tcase-2 'TCase *tc1_2 = tcase_create("Strings");' cat "$out" check_out libcheck/gen-srunner-new "SRunner *sr = srunner_create(s1);" cat "$out" check_out libcheck/gen-add-tcase "suite_add_tcase(s1, tc1_1);" cat "$out" check_out libcheck/gen-add-test "tcase_add_test(tc1_1, add_two);" cat "$out" check_out libcheck/gen-add-test-2 "tcase_add_test(tc1_2, str_eq);" cat "$out" check_out libcheck/gen-run-all "srunner_run_all(sr, CK_ENV);" cat "$out" check_out libcheck/gen-nfailed "nf = srunner_ntests_failed(sr);" cat "$out" check_out libcheck/gen-free "srunner_free(sr);" cat "$out" # #main-pre / #main-post land inside main(), on either side of the run. check_out libcheck/gen-main-pre "/* User-specified pre-run code */" cat "$out" check_out libcheck/gen-main-pre-body "int rc_pre = 0;" cat "$out" check_out libcheck/gen-main-post "/* User-specified post-run code */" cat "$out" check_out libcheck/gen-main-post-body "(void)rc_pre;" cat "$out" # AND #main-post SUPPRESSES THE GENERATED RETURN. checkmk emits # `return nf == 0 ? 0 : 1;` only when the user supplied no post-run code; with # #main-post the user's block is the last thing in main() and owns the exit # status. Asserting the return unconditionally is the natural mistake -- it # reports a red on a correct generator -- so both halves are pinned here: absent # from the main-post fixture, present in the plain one (gen-exit-status below). check libcheck/gen-main-post-no-return sh -c "! grep -q 'return nf == 0' '$out'" # Nothing went to stderr on the good path. check libcheck/gen-clean-stderr test ! -s "$tmp/basic.err" # ------------------------------------------------------------- clean mode --- # clean_mode=1 suppresses the #line directives (for a generated file that will # be committed rather than built in place) and changes nothing else. Both halves # are asserted so they cannot drift apart. cleanout=$tmp/clean.c check libcheck/clean-runs sh -c "checkmk clean_mode=1 '$tmp/basic.ts' > '$cleanout' 2>&1" check libcheck/clean-no-line-decl sh -c "! grep -q '^#line' '$cleanout'" check_out libcheck/clean-still-generates "START_TEST(add_two)" cat "$cleanout" check_out libcheck/clean-still-has-main "int main(void)" cat "$cleanout" # ------------------------------------------------------------------ stdin --- # Reading from stdin implies clean mode (there is no file to #line back to) and # renames the source in the banner. Both are checkmk behaviour a build system # that pipes into it depends on. stdinout=$tmp/stdin.c check libcheck/stdin-runs sh -c "checkmk < '$tmp/basic.ts' > '$stdinout' 2>&1" check_out libcheck/stdin-srcref "Edit the original source file (standard input) instead." cat "$stdinout" check libcheck/stdin-implies-clean sh -c "! grep -q '^#line' '$stdinout'" check_out libcheck/stdin-generates "START_TEST(add_two)" cat "$stdinout" # ------------------------------------------------------------ two suites ---- # A second #suite starts a new Suite and must be registered on the runner with # srunner_add_suite -- the one line that distinguishes a multi-suite generation # from a single-suite one, and the thing a naive generator forgets. cat > "$tmp/two.ts" <<'TSEOF' #suite Alpha #tcase A1 #test t_a ck_assert(1); #suite Beta #tcase B1 #test t_b ck_assert(1); TSEOF twoout=$tmp/two.c check libcheck/two-runs sh -c "checkmk clean_mode=1 '$tmp/two.ts' > '$twoout' 2>&1" check_out libcheck/two-suite-1 'Suite *s1 = suite_create("Alpha");' cat "$twoout" check_out libcheck/two-suite-2 'Suite *s2 = suite_create("Beta");' cat "$twoout" check_out libcheck/two-tcase-2 'TCase *tc2_1 = tcase_create("B1");' cat "$twoout" check_out libcheck/two-add-suite "srunner_add_suite(sr, s2);" cat "$twoout" # The plain main() -- no #main-post -- does end in the generated return, which # is the other half of the gen-main-post-no-return pair above. check_out libcheck/gen-exit-status "return nf == 0 ? 0 : 1;" cat "$twoout" # ---------------------------------------------------- parameterised tests --- # The four directive forms that carry arguments each map to a DIFFERENT Check # registration call, and getting one wrong silently registers a plain test that # never asserts what was asked. Numbers are passed straight through. cat > "$tmp/adv.ts" <<'TSEOF' #suite Adv #tcase C #test-exit(3) t_exit exit(3); #test-signal(11) t_sig ck_abort(); #test-loop(0, 4) t_loop ck_assert_int_lt(_i, 4); TSEOF advout=$tmp/adv.c check libcheck/adv-runs sh -c "checkmk clean_mode=1 '$tmp/adv.ts' > '$advout' 2>&1" check_out libcheck/adv-exit "tcase_add_exit_test(tc1_1, t_exit, 3);" cat "$advout" check_out libcheck/adv-signal "tcase_add_test_raise_signal(tc1_1, t_sig, 11);" cat "$advout" check_out libcheck/adv-loop "tcase_add_loop_test(tc1_1, t_loop, 0, 4);" cat "$advout" # ------------------------------------------------------ case insensitivity --- # The directive tags are matched case-insensitively by construction (the regexes # are spelled out character class by character class in the BEGIN block), so # #SUITE is #suite. Asserted because it is the kind of thing a locale or a # different awk would break, and this suite's whole job is to catch that. cat > "$tmp/upper.ts" <<'TSEOF' #SUITE Upper #TCASE UC #TEST t_up ck_assert(1); TSEOF upout=$tmp/upper.c check libcheck/upper-runs sh -c "checkmk clean_mode=1 '$tmp/upper.ts' > '$upout' 2>&1" check_out libcheck/upper-suite 'suite_create("Upper");' cat "$upout" check_out libcheck/upper-test "START_TEST(t_up)" cat "$upout" # --------------------------------------------------------- rejection cases --- # A generator that accepts everything is not a generator. Both cases assert the # MESSAGE as well as the exit status, because a bare rc=1 would also be produced # by an interpreter that failed to start -- which is the failure this suite most # needs to distinguish from a real rejection. : > "$tmp/empty.ts" check_rc libcheck/reject-empty-rc 1 sh -c "checkmk '$tmp/empty.ts' >/dev/null 2>&1" # The message is read off stderr with the failing status swallowed, because # check_out also requires rc=0 from the command it runs. check_out libcheck/reject-empty-msg "checkmk: Expected at least one #test line." \ sh -c "checkmk '$tmp/empty.ts' 2>&1 >/dev/null; true" cat > "$tmp/bad.ts" <<'TSEOF' #suite S #tcase C #test 9bad ck_assert(1); TSEOF check_rc libcheck/reject-badname-rc 1 sh -c "checkmk '$tmp/bad.ts' >/dev/null 2>&1" check_out libcheck/reject-badname-msg 'Malformed test name "9bad" (must be a C identifier).' \ sh -c "checkmk '$tmp/bad.ts' 2>&1 >/dev/null; true" # ================================================================ payload ==== # The library half. No compiler on target, so this is the 76-gmp.test register: # the API the header declares, the entry points the archive exports, and the # pkg-config contract a future recipe's PKG_CHECK_MODULES will consume. check libcheck/hdr-present test -f "$hdr" check libcheck/hdr-nonempty test -s "$hdr" # Version, extracted rather than substring-matched: check.h writes the numbers # parenthesised, so a literal match would encode the punctuation too. check_eq libcheck/hdr-version-major "(0)" "$(awk '/^#define CHECK_MAJOR_VERSION/{print $3}' "$hdr")" check_eq libcheck/hdr-version-minor "(15)" "$(awk '/^#define CHECK_MINOR_VERSION/{print $3}' "$hdr")" check_eq libcheck/hdr-version-micro "(2)" "$(awk '/^#define CHECK_MICRO_VERSION/{print $3}' "$hdr")" # The two macros that ARE the framework's surface syntax. START_TEST expands to # a static fn plus a TTest descriptor holding __FILE__/__LINE__ -- assert the # descriptor, not just the macro name, because an empty macro would satisfy the # name alone. check_out libcheck/hdr-start-test "#define START_TEST(__testname)" cat "$hdr" check_out libcheck/hdr-ttest-desc "static const TTest __testname ## _ttest" cat "$hdr" check_out libcheck/hdr-end-test "#define END_TEST" cat "$hdr" # The lifecycle API a generated main() calls, one assertion per call so a # partially-declared header cannot pass as a whole one. for f in suite_create suite_add_tcase tcase_create tcase_add_test \ tcase_add_exit_test tcase_add_loop_test tcase_add_test_raise_signal \ tcase_set_timeout tcase_add_checked_fixture \ srunner_create srunner_add_suite srunner_run_all \ srunner_ntests_failed srunner_free srunner_set_fork_status \ srunner_set_log srunner_set_xml srunner_set_tap; do check "libcheck/hdr-decl-$f" grep -q "$f" "$hdr" done # The assertion macros -- the part of Check a test author actually types. for a in ck_assert ck_assert_msg ck_abort ck_abort_msg \ ck_assert_int_eq ck_assert_int_ne ck_assert_int_lt ck_assert_int_gt \ ck_assert_uint_eq ck_assert_str_eq ck_assert_str_ne \ ck_assert_ptr_eq ck_assert_ptr_null ck_assert_float_eq \ ck_assert_mem_eq; do check "libcheck/hdr-assert-$a" grep -q "$a" "$hdr" done # Fork status. Not exercised (see the header comment) but its presence in the # API is what tells a future consumer on this image that CK_NOFORK is available # to opt out of the fork() path brintos/linux#14 covers. check_out libcheck/hdr-ck-fork "CK_FORK" cat "$hdr" check_out libcheck/hdr-ck-nofork "CK_NOFORK" cat "$hdr" check_out libcheck/hdr-ck-env "CK_ENV" cat "$hdr" # check_stdint.h is GENERATED by configure for the compiling toolchain, so it is # the one payload file whose mere existence is wasm32 evidence: it cannot have # been copied out of the tarball. check libcheck/stdint-present test -f "$sthdr" check_out libcheck/stdint-guard "_CHECK_CHECK_STDINT_H" cat "$sthdr" check_out libcheck/stdint-stamp '_GENERATED_STDINT_H "check 0.15.2"' cat "$sthdr" # ---------------------------------------------------------------- archive --- check libcheck/ar-present test -f "$lib" # ar magic, so a truncated or wrong-format file is reported as that rather than # as a pile of missing symbols. check_out libcheck/ar-magic "!" head -c 8 "$lib" # Flatten only the archive's symbol index (member "/", the exported-symbol map # at the head of the file) -- the 76-gmp.test idiom. 16 KB is ample here: the # whole archive is ~400 KB natively and the index is a few KB. idx=$tmp/ar.idx head -c 16384 "$lib" | tr -c '\40-\176\n' ' ' > "$idx" 2>/dev/null # One exported entry point per compilation unit that a generated main() reaches: # the suite tree, the runner, and the assertion failure path. A partial compile # that produced check.o but not check_run.o fails here rather than at the first # consumer. # # THE REGISTRATION CALL IS SPELLED _tcase_add_test IN THE ARCHIVE, not # tcase_add_test. check.h defines tcase_add_test as a macro over # tcase_add_test_raise_signal, which is itself a macro over the real entry # point; only the underscore form is a compiled symbol. Asserting the name the # generated C contains would report a red on a complete archive. for s in suite_create suite_add_tcase tcase_create _tcase_add_test \ tcase_add_checked_fixture tcase_set_timeout \ srunner_create srunner_run_all srunner_ntests_failed srunner_free \ srunner_set_fork_status _ck_assert_failed; do check "libcheck/ar-sym-$s" grep -qw "$s" "$idx" done rm -f "$idx" # ------------------------------------------------------------- pkg-config --- check libcheck/pc-present test -f "$pc" check_eq libcheck/pc-name "Check" "$(sed -n 's/^Name: *//p' "$pc" 2>/dev/null)" check_eq libcheck/pc-version "0.15.2" "$(sed -n 's/^Version: *//p' "$pc" 2>/dev/null)" check_eq libcheck/pc-prefix "/usr" "$(sed -n 's/^prefix=//p' "$pc" 2>/dev/null)" check libcheck/pc-libs grep -q '^Libs: -L${libdir} -lcheck$' "$pc" # No unsubstituted autoconf placeholder survived into the shipped file. check libcheck/pc-no-at-vars sh -c "! grep -q '@[A-Za-z_]*@' '$pc'" if command -v pkgconf >/dev/null 2>&1; then check_eq libcheck/pkgconf-exists 0 "$(pkgconf --exists check >/dev/null 2>&1; echo $?)" check_eq libcheck/pkgconf-modversion "0.15.2" "$(pkgconf --modversion check 2>/dev/null)" check_eq libcheck/pkgconf-libs "-lcheck" "$(pkgconf --libs check 2>/dev/null)" check_eq libcheck/pkgconf-libdir "/usr/lib" "$(pkgconf --variable=libdir check 2>/dev/null)" else for c in exists modversion libs libdir; do bst_skip "libcheck/pkgconf-$c" "pkgconf not installed" done fi # The autoconf macro a consumer's configure.ac calls to find us. It ships in # libcheck-dev and is the documented pre-pkg-config discovery path, so a port # that drops it breaks an AM_PATH_CHECK-based configure silently. check libcheck/m4-present test -f "$m4" check_out libcheck/m4-macro "AC_DEFUN([AM_PATH_CHECK]" cat "$m4" rm -rf "$tmp"