# BST-TIMEOUT: 120 # measured 18 s (25 execs + 2 re-runs) on a quiet host, so # # ~6x headroom. Deliberately NOT the 600 this file first # # carried: that was extrapolated from 80-libmpc's ~4.9 s per # # exec, which is the cost of ITS 33 MB binaries, not of # # libbsd's small ones (~0.7 s apiece here). A bound 33x the # # measured time hands back the protection the bound exists # # for -- see the packager README and BRINTOS-644's §134.2. . "$BST_LIB" # libbsd 0.12.1 -- the BSD compatibility library: strlcpy/strlcat, arc4random, # the BSD getopt, and the sys/queue.h and sys/tree.h macro sets glibc does not # provide. Layered on libmd (BRINTOS-40) for the MD5/SHA entry points. # See libbsd_%.bbappend for what the port adds and why. # # --------------------------------------------------------------------------- # WHAT THIS SUITE IS, AND WHY IT IS NOT A PROBE # # It closes on the PACKAGE'S OWN UPSTREAM SUITE. libbsd ships one under test/ -- # one programme per compatibility function, each asserting that function's BSD # contract in-process -- and the bbappend stages exactly the set upstream's own # test/Makefile declares in check_PROGRAMS, so what runs here is upstream's suite # and not our reading of it. # # That is a materially stronger statement than a probe. A probe asserts the # values its author thought to assert; this asserts what upstream considered # worth shipping, including the contracts nobody here would have thought of -- # strlcpy's return value on truncation (the whole reason it exists), fparseln's # escape and continuation handling, funopen's four-callback stream protocol, # vis/unvis round-tripping over every byte, and strtonum's errstr strings. # # --------------------------------------------------------------------------- # THE PROGRAMME SET IS DECIDED BY configure ON THIS ARM, WHICH IS WHY IT IS # ASSERTED # # Unlike libmpc, whose check_PROGRAMS is one unconditional list, libbsd's is # assembled from twenty-odd automake conditionals (ABI_STRL, ABI_FUNOPEN, # ABI_NLIST, LIBBSD_SYS_HAS_TIME64, ...) that configure decides per target. So # "the upstream suite" here legitimately means a DIFFERENT set on a different # arm, and the count below is this arm's resolved set, taken from the build # rather than from upstream's .am file. Asserting it is what makes a toolchain # change that silently drops a conditional a red case instead of a shorter suite # that still reads green. # # 23 here against 24 on an x86-64 reference build of the same tarball, and the # single difference is nlist -- gated off by this port because nlist(3) reads an # ELF symbol table and a wasm32 executable is a WebAssembly module. Its header, # its implementation and its check programme all drop from the one ABI_NLIST # conditional, so the suite matches the library by construction. # # Two further programmes upstream can build are absent on BOTH arms by upstream's # own gating, not by ours: arc4random needs TestU01 (a statistical test library # nothing here provides) and timeconv32 is for 32-bit time_t systems, where this # arm reports 64. Neither is a port defect. # # THE TWO check_SCRIPTS CANNOT RUN HERE AT ALL. Upstream declares # TESTS = $(check_SCRIPTS) $(check_PROGRAMS); headers-overlay.sh and # headers-system.sh compile a generated .c against each installed header using # $CC, which makes them build-host checks of the header set rather than runtime # properties of the port. The image ships no C compiler. They are reported skip # BY NAME (the names are staged from make, not typed) so this report stays # faithful to upstream's TESTS list instead of being two entries shorter with # nothing saying so. # # --------------------------------------------------------------------------- # THE ld SCRIPT IS ASSERTED ABSENT, AND THAT IS A DELIBERATE PORT DECISION # # libbsd's install-exec-hook replaces ${libdir}/libbsd.so with a GNU ld script # that GROUPs the shared library with libmd. Under --disable-shared there is no # shared library and no symlink for its `readlink` to read, so upstream's # substitution yields GROUP(/usr/lib/ AS_NEEDED(-lmd)) -- a directory where a # library belongs. The port gates that whole mechanism on --enable-shared, so # nothing is written; the bbappend keeps a `rm -f` as a backstop. A future recipe # linking -lbsd would otherwise find that file ahead of libbsd.a and fail inside # the linker rather than at a missing library, so its absence is asserted here: # putting it back has to be a visible decision. # # --------------------------------------------------------------------------- # SPAWN BUDGET IS THE CONSTRAINT, WHICH IS WHY THE HARNESS RUNS ONCE # # The whole suite costs 0.2 s of compute natively; every programme here is a # fresh wasm module instantiation and those dominate it by orders of magnitude. # So the file runs ptest-runner ONCE, reads back the PASS:/FAIL:/SKIP: lines it # collected, and reports from a shell loop that spawns nothing -- printf and # `test` are ash builtins, and the parse below is a `while read` rather than a # per-case command substitution. # # The two exceptions are the attributed reds below, which are re-run once each so # their failure can be matched against the diagnostic their issue describes # rather than waved through by name. Two extra instantiations is the price of not # absorbing a different failure in the same programme. ptestdir=/usr/lib/libbsd/ptest runner=/usr/bin/ptest-runner lib=/usr/lib/libbsd.a ctor=/usr/lib/libbsd-ctor.a # This arm's resolved set, measured from the target build -- see the header for # why it is not upstream's full list and why it is asserted at all. want_progs=23 want_scripts=2 # Below this file's BST-TIMEOUT on purpose, so an overrun is reported by # ptest-runner in its own voice instead of this whole file being killed around # it -- the arrangement 77-libpcre.test uses. runner_t=90 tmp=${TMPDIR:-/tmp}/bst-libbsd.$$ mkdir -p "$tmp" 2>/dev/null || tmp=/tmp # ----------------------------------------------------------- what shipped ---- # The -dev/-staticdev half. libbsd installs no binary and, on this arm, no # shared object: the distro is static-only, so libbsd.a IS the library. check_rc libbsd/ar 0 test -s "$lib" # libbsd-ctor.a is a SEPARATE archive holding only the ELF constructor that # arranges setproctitle_init to run before main. It is asserted because it is a # distinct deliverable a packaging change could silently drop, and because the # proctitle programme below is the thing that links it. check_rc libbsd/ctor-ar 0 test -s "$ctor" # The ld script upstream's install hook writes -- removed by the bbappend # because --disable-shared makes it malformed. See the header. check_rc libbsd/no-so 1 test -e /usr/lib/libbsd.so # The headers this port exists to provide, one per family named on the ticket. # They live under include/bsd/ rather than the libc namespace: libbsd's default # install is the non-overlay one, so a caller writes and nothing # shadows glibc's own headers. check_rc libbsd/hdr-string 0 test -s /usr/include/bsd/string.h check_rc libbsd/hdr-stdlib 0 test -s /usr/include/bsd/stdlib.h check_rc libbsd/hdr-getopt 0 test -s /usr/include/bsd/getopt.h check_rc libbsd/hdr-queue 0 test -s /usr/include/bsd/sys/queue.h check_rc libbsd/hdr-tree 0 test -s /usr/include/bsd/sys/tree.h # UNLIKE mpc, libbsd DOES ship pkg-config files, and three of them: the plain # one, the overlay variant (which adds -isystem for the header-shadowing mode) # and the ctor variant. Asserted positively for the same reason 80-libmpc.test # asserts mpc.pc's absence -- so the shape is stated rather than assumed. check_rc libbsd/pc 0 test -s /usr/lib/pkgconfig/libbsd.pc check_rc libbsd/pc-overlay 0 test -s /usr/lib/pkgconfig/libbsd-overlay.pc check_rc libbsd/pc-ctor 0 test -s /usr/lib/pkgconfig/libbsd-ctor.pc # The ptest payload the bbappend stages. check_rc libbsd/ptest-dir 0 test -d "$ptestdir" check_rc libbsd/ptest-script 0 test -x "$ptestdir/run-ptest" check_rc libbsd/ptest-progs 0 test -d "$ptestdir/progs" check_rc libbsd/ptest-skiplist 0 test -s "$ptestdir/skipped-scripts" # Counted by glob expansion rather than `ls | wc -l`, which would fork twice per # count. A short count is the failure this asserts against: the bbappend derives # its list from the check_PROGRAMS make resolved and bbfatals on a missing # binary, so a disagreement here means something was lost after do_install_ptest. set -- "$ptestdir"/progs/* check_eq libbsd/prog-count "$want_progs" "$#" # ------------------------------------------------------- run it, once --------- # The harness is preferred wherever it exists, because this ticket closes on the # suite run UNDER ptest-runner -- its discovery and its tally are part of what is # asserted. The fallback runs the identical script directly and SKIPS only the # cases that are about the harness itself; they have no subject on an image that # ships no ptest-runner. out=$tmp/runner.out if [ -x "$runner" ]; then bst_ok libbsd/ptest-runner-present "$runner" -t "$runner_t" libbsd > "$out" 2>&1 rc=$? check_eq libbsd/ptest-runner-rc 0 "$rc" # ptest-runner's own framing, read back from the same capture. begin=; end=; rerr= while read -r k v; do case "$k" in BEGIN:) begin=$v ;; END:) end=$v ;; ERROR:) rerr=$v ;; esac done < "$out" check_eq libbsd/runner-begin "$ptestdir" "$begin" check_eq libbsd/runner-end "$ptestdir" "$end" check_eq libbsd/runner-no-error "" "$rerr" else bst_skip libbsd/ptest-runner-present "this image ships no ptest-runner" bst_skip libbsd/ptest-runner-rc "no ptest-runner to run" for c in runner-begin runner-end runner-no-error; do bst_skip "libbsd/$c" "harness cases have no subject without ptest-runner" done ( cd "$ptestdir" && sh ./run-ptest ) > "$out" 2>&1 fi # --------------------------------------------- the two attributed reds -------- # TWO upstream programmes fail here for reasons that are the PLATFORM's, not # libbsd's, and each is held by a filed brintos.io issue. They are not reported # red -- but neither are they waved through by name, because a name match would # also absorb a DIFFERENT failure in the same programme and we would never hear # about it. Each is re-run and classified against the exact diagnostic its issue # describes: # # explicit_bzero sigaltstack(2) refuses a well-formed alternate signal stack, # so setup_stack() dies before the test tests anything -- # brintos/linux#66. The scrubbing itself is never reached. # # proctitle the .init_array constructor in libbsd-ctor.a, demanded with # -Wl,-u,libbsd_init_func, does not reach the linked module, # so setproctitle_init() never runs -- brintos/hwjs-cc#49. # libbsd prints its own "not initialized" diagnostic for this. # Note t-proctitle-init, the same source calling # setproctitle_init() directly, PASSES: setproctitle() works # on this arm and only the constructor path is broken. # # A pass is reported as a pass. That is the point of writing it this way rather # than hardcoding a skip: the day either issue lands, the case turns green on its # own and the suite gains real coverage without anyone editing this file. A # failure that does NOT match the signature is red, as it should be. bst_classify_attributed() { # $1 prog $2 signature $3 issue _o=$("$ptestdir/progs/$1" 2>&1); _rc=$? if [ "$_rc" -eq 0 ]; then bst_ok "libbsd/t-$1" else case "$_o" in *"$2"*) bst_skip "libbsd/t-$1" "$3" ;; *) bst_notok "libbsd/t-$1" "rc=$_rc $(_bst_1line "$_o")" ;; esac fi } # ------------------------------------- one case per upstream TESTS entry ------ # A single `while read` over the capture: no fork per case, and every entry # upstream declared gets its own BST case so a red names the function that broke # rather than just the suite. The SKIP: lines are the two check_SCRIPTS, emitted # by run-ptest from the staged name list. The two attributed programmes are # counted here but reported by the classifier above, so the tallies stay whole. attributed=" explicit_bzero proctitle " npass=0 nfail=0 nskip=0 nattr=0 while read -r verdict name _rest; do case "$verdict" in PASS:|FAIL:) case "$attributed" in *" $name "*) nattr=$((nattr + 1)); continue ;; esac ;; esac case "$verdict" in PASS:) bst_ok "libbsd/t-$name"; npass=$((npass + 1)) ;; FAIL:) bst_notok "libbsd/t-$name" "upstream check programme failed"; nfail=$((nfail + 1)) ;; SKIP:) bst_skip "libbsd/t-$name" "compile-time header check; no C compiler on target" nskip=$((nskip + 1)) ;; esac done < "$out" bst_classify_attributed explicit_bzero "sigaltstack" \ "sigaltstack(2) refuses the alt stack — brintos/linux#66" bst_classify_attributed proctitle "setproctitle not initialized" \ "libbsd-ctor .init_array ctor not linked in — brintos/hwjs-cc#49" # The tallies are asserted separately from the individual cases, because they # fail differently: a red above is one broken function, whereas a short total # means programmes never ran at all -- a truncated capture, a harness timeout, or # a scratch directory the run could not create. Without this, a run that produced # three PASS lines and died would read as three green cases and nothing else. # suite-failures counts UNATTRIBUTED failures only; the two above are reported by # the classifier and are held by their issues. check_eq libbsd/suite-total "$want_progs" "$((npass + nfail + nattr))" check_eq libbsd/suite-attributed 2 "$nattr" check_eq libbsd/suite-skipped "$want_scripts" "$nskip" check_eq libbsd/suite-failures 0 "$nfail" rm -rf "$tmp"