. "$BST_LIB" require perl libtest-needs-perl/present # BST-TIMEOUT: 2400 # # Set from a measurement, not guessed -- see the elapsed figure this suite's own # `==BST-SUITE` line reports. This suite is dominated by section 5, whose five # .t files spawn roughly 250 CHILD PERLS between them (t/lib/TestAPI.pm drives # every assertion through IPC::Open3), and a perl process costs seconds rather # than milliseconds on this arm (linux-in-the-browser#117, tcl measured at # 46-68x native). The x86-64 control runs the same five files in 2 wallclock # seconds; 2400 leaves room for the platform factor without reaching the # runner's 3600 s BST_SUITE_TIMEOUT_MAX, which would fire on load rather than # on breakage. # # Test::Needs 0.002010 (POKY meta/recipes-devtools/perl/libtest-needs-perl_0.002010.bb # -- NOT meta-oe, which the ticket names; see the design doc) -- skip a test file # cleanly when an optional module it wants is not installed. Eighteenth Perl # module on this image under BRINTOS-999. Pure Perl with no runtime dependency # at all, so BRINTOS-881 -- the -Uusedl -Uuseshrplib arm that stops every XS # module here -- does not apply. # # THIS SUITE'S PASS CRITERION IS THE DISTRIBUTION'S OWN t/, NOT A HAND-WRITTEN # BATTERY -- the ticket's own wording. The port therefore installs the suite # (libtest-needs-perl-ptest, from PTEST_ENABLED = "1" for this recipe alone) and # section 5 runs `prove -r t/` over it. # # THE PORT'S ONE REAL RISK, STATED UP FRONT, BECAUSE IT IS STRUCTURAL AND NOT # INCIDENTAL. Test::Needs' whole purpose is to SKIP a file, and on a perl where # Test2 is loaded it does that through `$ctx->plan(0, SKIP => ...)` -- which is # the same Test2 plan-0-SKIP path that `plan skip_all` takes. # 36-libdev-checklib-perl.test isolated linux-in-the-browser#93 (perl's # magic_killbackrefs use-after-free) to exactly that form: `plan tests => N` # survives on this arm, plan-0-SKIP panics in Test2::Hub's teardown. So unlike # libtest-fatal-perl -- whose four files avoid the form entirely and which # therefore passed its criterion outright -- this port drives the panicking form # ON PURPOSE, because that form IS the deliverable. # # The consequence for the criterion is specific and worth naming rather than # discovering: TestAPI.pm asserts each child's EXIT STATUS (`is $exit == 0, # !$want_exit`) before it looks at the child's output. A child that prints its # `1..0 # SKIP` correctly and then panics in global destruction exits non-zero, # so #93 turns a correct skip into a red -- and the red lands on the exit-status # case, which reads like a functional failure of Test::Needs and is not one. # Sections 4b and 4c therefore measure the skip path's OUTPUT and its EXIT # STATUS as two separate cases, on this boot, in this interpreter, so a red in # section 5 is attributed from evidence rather than from this comment. # # EVERY REFERENCE BELOW WAS MEASURED, NOT READ OFF THE TARBALL. The control is # the same 0.002010 tree on x86-64 (perl 5.38.2), driven exactly as sections 4 # and 5 drive it: `prove -r t/` gives `All tests successful.`, Files=5, # Tests=227, Result: PASS, exit 0, and the per-file plans are 24 / 36 / 25 / 71 # / 71 with no skips anywhere. # # Six layers: # 1. filesystem facts -- what the port SHIPS, asserted before anything starts # perl, so they hold on a boot where the interpreter cannot load a module. # 2. the harness the criterion names, and the modules the suite loads. # 3. can perl load the module on this boot, classified on the diagnostic. # 4. the SEMANTICS, at values, against the x86-64 control -- including the # ticket's two branches -- then (4a) the fork/exec the harness is built on, # (4b) the TAP the skip actually emits, and (4c) the plan forms measured. # 5. the criterion: prove -r t/ over the installed suite. # perl prints a four-line locale-warning preamble on EVERY invocation on this # image, on stderr. Sections 3-5 capture 2>&1 (they must: a prove that died has # said so on stderr), so without this strip every red would report "Setting # locale failed" in the 100-char detail field -- the wrong diagnosis in the one # place a later reader attributes by. Same strip as 36-libtest-fatal-perl.test. _tn_clean() { printf '%s\n' "$1" | awk '/^perl: warning/ { skip=1; next } /^[ \t]/ && skip { next } { skip=0; print }' } V=$(ls -d /usr/lib/perl5/vendor_perl/* 2>/dev/null | head -1) CORE=$(ls -d /usr/lib/perl5/5.* 2>/dev/null | head -1) PT=/usr/lib/libtest-needs-perl/ptest # A module name that cannot exist. Fixed rather than time-derived: it is # compared against a literal in several `Need ...` strings below, and a name # that changed between the payload and this file would redden every one of them # for the wrong reason. ABSENT=Module::Does::Not::Exist::BRINTOS1323 # ------------------------- 1. what the port SHIPS: filesystem facts ---------- # The VERSION dir, not the arch dir: Test::Needs is pure Perl, so cpan installs # it to vendor_perl// and NOT to the vendor_perl//-linux/ that # 36-libdbi-perl.test globs for its XS module. That placement IS the packaging's # statement that no .so is wanted, which is why BRINTOS-881 cannot reach here. check libtest-needs-perl/vendor-dir test -n "$V" check libtest-needs-perl/module-installed test -f "$V/Test/Needs.pm" # ${PN} is ONE file and nothing else -- read off the built .deb, not assumed. # Asserted because a recipe that started shipping an auto/ tree here would mean # the module had acquired an XS half, which is the one change that would put # this port back under BRINTOS-881. check libtest-needs-perl/no-xs-object test ! -d "$V/auto/Test/Needs" # THE INSTALLED SUITE -- the deliverable the ticket's criterion runs on. poky's # recipe already inherits ptest-perl, but the class deletes its own task unless # PTEST_ENABLED is 1, and this distro's DISTRO_FEATURES has no `ptest`. These # cases are what says this layer's one-line bbappend took effect. check libtest-needs-perl/ptest-dir test -d "$PT/t" check libtest-needs-perl/ptest-runptest test -f "$PT/run-ptest" for t in find_missing numify_version standalone test-builder test2; do check "libtest-needs-perl/ptest-$t" test -f "$PT/t/$t.t" done # 5 -- the whole of the distribution's t/. This recipe prunes none of them, so # the count is the tarball's and a recipe that started dropping files reds here. # The x86-64 control ran exactly these 5. check_eq libtest-needs-perl/ptest-count 5 "$(ls "$PT"/t/*.t 2>/dev/null | wc -l | tr -d ' ')" # t/lib/ IS THE SUITE, not a fixture directory that could be dropped. Three of # the five .t files are six lines long and delegate everything to # t/lib/TestAPI.pm; the other two load t/lib modules as the subjects they probe # for. ptest-perl installs the tree with a bare `cp -r ${B}/t`, so t/lib rides # along only because it happens to sit under t/ -- there is no FILES entry # naming it. Each of the four is asserted by name: without them the criterion # would fail with `Can't locate TestAPI.pm`, which reads as a broken port rather # than as a packaging omission. for m in BrokenModule ModuleWithVersion TestAPI TestScript; do check "libtest-needs-perl/ptest-lib-$m" test -f "$PT/t/lib/$m.pm" done # BrokenModule.pm is DELIBERATELY UNCOMPILABLE -- it is the fixture behind the # module's own documented distinction between "missing" (skip) and "found but # broken" (die). A packaging step that helpfully validated or reformatted the # installed .pm files would repair it and silently delete that half of the # suite's coverage, so its brokenness is asserted rather than its presence # alone. perl -c on it must FAIL; the payload's R18 case makes the same # statement against its own generated fixture. if perl -c "$PT/t/lib/BrokenModule.pm" >/dev/null 2>&1; then bst_notok libtest-needs-perl/broken-fixture-still-broken \ "t/lib/BrokenModule.pm COMPILES -- the fixture the found-but-broken branch needs has been repaired in packaging" else bst_ok libtest-needs-perl/broken-fixture-still-broken fi # ------------------- 2. the harness the criterion names ---------------------- # `prove` reached this image under libdbi-perl (perl-misc + perl-module-app-prove) # and WORKS only because libdev-checklib-perl added the 43 perl-module-tap-* # splits. Asserted as ordinary reds, not as a `require`: their absence is a # regression to report, not a reason to skip this port's criterion. check libtest-needs-perl/prove-installed test -x /usr/bin/prove check libtest-needs-perl/prove-lib test -f "$CORE/App/Prove.pm" # The four names poky puts in RDEPENDS:${PN}-ptest, each on its own case, plus # the two the suite reaches THROUGH them. Every one is a perl-module-* split of # this image's 1146-way perl. The distinction matters for attribution: a missing # IPC::Open3 kills three of the five files outright (TestAPI.pm's entire harness # is open3), a missing Test::More kills four, and a missing Test2::API or # Test::Builder does something worse than kill a file -- it makes TestAPI take # its `plan skip_all` branch, which is the one form #93 panics on. So their # presence is what keeps the criterion on the path this suite reasoned about. check libtest-needs-perl/testmore-installed test -f "$CORE/Test/More.pm" check libtest-needs-perl/open3-installed test -f "$CORE/IPC/Open3.pm" check libtest-needs-perl/lib-installed test -f "$CORE/lib.pm" check libtest-needs-perl/version-installed test -f "$CORE/version.pm" check libtest-needs-perl/tb-installed test -f "$CORE/Test/Builder.pm" check libtest-needs-perl/test2api-installed test -f "$CORE/Test2/API.pm" # Config.pm is loaded unconditionally by t/lib/TestAPI.pm and is NOT a # perl-module-* split on this image -- it ships inside the base perl package. # Asserted because that is the one name in the measured closure whose absence # would not be caught by any RDEPENDS at all. check libtest-needs-perl/config-installed test -f "$CORE/Config.pm" # ------------------ 3. can perl LOAD this module on this boot? --------------- # Pure Perl with no runtime dependency, so a clean import is the expected -- and # the only passing -- outcome. Failure is classified on the diagnostic so a red # says WHOSE defect it is rather than quoting 100 characters of transcript. pout=$(perl -e ' eval { require Test::Needs }; if ($@) { print "LOAD-FAILED: $@"; exit 1 } print "LOAD-OK $Test::Needs::VERSION\n";' 2>&1) pout=$(_tn_clean "$pout") case "$pout" in *"LOAD-OK"*) bst_ok libtest-needs-perl/require-loads ;; *"Can't locate"*) bst_notok libtest-needs-perl/require-loads \ "a module in the closure is not on the image: $(_bst_1line "$pout")" ;; *"panic:"*) bst_notok libtest-needs-perl/require-loads \ "perl PANICKED loading a pure-Perl module -- a platform defect, not this port's: $(_bst_1line "$pout")" ;; *) bst_notok libtest-needs-perl/require-loads "$(_bst_1line "$pout")" ;; esac # `use Test::Needs;` with no import list must NOT skip the caller. The import # hook calls test_needs(@_) whenever it is given arguments, so a module that # mishandled the empty list would terminate every file that merely loads it. check_out libtest-needs-perl/import-load "TN-OK" perl -MTest::Needs -e 'print "TN-OK\n"' # ------------------ 4. THE SEMANTICS, at values, vs the x86-64 control ------- # The payload measures; this file names and compares, so a single wrong answer # is one red case rather than one red suite. Everything here runs through # _find_missing -- the predicate test_needs() is built on, which RETURNS its # verdict instead of terminating the file. The TAP that test_needs itself emits # is section 4b's business. out=$(perl "$BST_DATA/libtest-needs-perl-selftest.pl" 2>&1); rc=$? out=$(_tn_clean "$out") case "$out" in *ALL-TEST-NEEDS-OK*) all=1 ;; *) all=0 ;; esac if [ "$rc" -eq 0 ] && [ "$all" -eq 1 ]; then bst_ok libtest-needs-perl/all-ok else bst_notok libtest-needs-perl/all-ok "rc=$rc $(_bst_1line "$out")"; fi val() { printf '%s\n' "$out" | awk -v t="$1" '$1==t{ $1=""; sub(/^ /,""); print; exit }'; } # --- identity, and that it came from the image ------------------------------ check_eq libtest-needs-perl/version 0.002010 "$(val R1-version)" case "$(val R2-inc-path)" in /usr/*) bst_ok libtest-needs-perl/installed-path ;; *) bst_notok libtest-needs-perl/installed-path "Test/Needs.pm resolved to $(val R2-inc-path)" ;; esac check_eq libtest-needs-perl/export-default test_needs "$(val R3-export-default)" check_eq libtest-needs-perl/can-test-needs yes "$(val R4-can-test-needs)" # unprototyped, deliberately: `test_needs 'A', 'B'` and `test_needs { A => 1 }` # are both list calls, and a prototype would break one of them. check_eq libtest-needs-perl/proto-test-needs '' "$(val R5-proto-test-needs)" # --- _numify_version, the pure function t/numify_version.t covers ------------ # One case per branch of its dispatch. A v-string with two parts must # zero-extend to three, which is the arithmetic a naive sprintf gets wrong. check_eq libtest-needs-perl/numify-int 5.000000 "$(val R6-numify-int)" check_eq libtest-needs-perl/numify-dec 5.008000 "$(val R7-numify-dec)" check_eq libtest-needs-perl/numify-vstr3 5.008000 "$(val R8-numify-vstr3)" check_eq libtest-needs-perl/numify-vstr2 5.010000 "$(val R9-numify-vstr2)" # the failure direction of the same function, without which all four above pass # against something that returns a number for any input at all. check_eq libtest-needs-perl/numify-refuses croaks "$(val R10-numify-refuses)" # --- THE TICKET'S TWO BRANCHES ---------------------------------------------- # "a module that is present and one that is deliberately absent". Present must # be undef -- nothing to report -- and absent must name the module, because that # string becomes the skip message the caller's test file carries. check_eq libtest-needs-perl/present-module '' "$(val R11-present-module)" check_eq libtest-needs-perl/absent-module "Need $ABSENT" "$(val R12-absent-module)" # and mixed, which is the shape a real caller writes: one absent among several # present must still report exactly the absent one. check_eq libtest-needs-perl/present-and-absent "Need $ABSENT" "$(val R13-present-and-absent)" # --- versions, both directions ---------------------------------------------- check_eq libtest-needs-perl/version-satisfied '' "$(val R14-version-satisfied)" check_eq libtest-needs-perl/version-unsatisfied reports-need-and-have "$(val R15-version-unsatisfied)" # 'perl' is a module name with a SPECIAL CASE -- compared against $] instead of # being require'd -- and `test_needs perl => '5.010'` is one of the module's two # documented uses, so both directions are asserted. check_eq libtest-needs-perl/perl-satisfied '' "$(val R16-perl-satisfied)" check_eq libtest-needs-perl/perl-unsatisfied reports-need-and-have "$(val R17-perl-unsatisfied)" # --- found-but-BROKEN dies; it does not skip -------------------------------- # The module's own DESCRIPTION draws this line: "Modules that are found but fail # to compile will exit with an error rather than skip." It is the case that # separates Test::Needs from `eval { require X } or skip`, and without it a # module that swallowed every require error would look correct on both branches # above while turning a real compile error in a dependency into a green skip. check_eq libtest-needs-perl/broken-dies dies-with-syntax-error "$(val R18-broken-dies)" # and a name that is not a module name at all is refused rather than probed. check_eq libtest-needs-perl/bad-name-refused croaks "$(val R19-bad-name-refused)" # --- the three argument SHAPES a caller writes ------------------------------ check_eq libtest-needs-perl/shape-list "Need $ABSENT" "$(val R20-shape-list)" check_eq libtest-needs-perl/shape-hashref "Need $ABSENT" "$(val R21-shape-hashref)" check_eq libtest-needs-perl/shape-arrayref "Need $ABSENT" "$(val R22-shape-arrayref)" # --- $^X, which the whole of section 5 rides on ----------------------------- case "$(val R23-perl-path)" in /*) bst_ok libtest-needs-perl/perl-path-absolute ;; *) bst_notok libtest-needs-perl/perl-path-absolute \ "\$^X is '$(val R23-perl-path)' -- brintos/linux#69: /proc/self/exe is unreadable, so perl's d_procselfexe path falls back to argv[0]. The criterion survives it because TestAPI re-execs through PATH, but -x on a bare name is false" ;; esac case "$(val R24-perl-path-x)" in executable) bst_ok libtest-needs-perl/perl-path-executable ;; *) bst_notok libtest-needs-perl/perl-path-executable \ "want 'executable', got '$(val R24-perl-path-x)' -- the same brintos/linux#69 fallback: \$^X is a bare name, so -x resolves it against CWD" ;; esac # ---------- 4a. THE FORK/EXEC THE HARNESS IS BUILT ON ------------------------ # t/lib/TestAPI.pm runs every one of its assertions by spawning a child perl # through IPC::Open3, so three of the five .t files are a fork+exec+pipe harness # rather than an in-process test. That shape is established on this arm by the # ipc-run port (fork(2) + pipe(2) + exec(2) + select(2)); what is NOT expressible # here is posix_spawn's clone(CLONE_VM|CLONE_VFORK), and Open3 never uses it. # Measured on THIS boot rather than cited, because if it is false the criterion # cannot pass and the reason is the platform's, not this port's. o3=$(perl -MIPC::Open3 -e ' use Symbol qw(gensym); my $err = gensym; my $pid = open3(my $in, my $out, $err, $^X, "-e", "print qq{CHILD-OK\n}"); my $got = do { local $/; <$out> }; waitpid $pid, 0; my $st = $?; print $got; printf "CHILD-STATUS %d\n", $st;' 2>&1) o3=$(_tn_clean "$o3") case "$o3" in *CHILD-OK*) bst_ok libtest-needs-perl/open3-forks ;; *) bst_notok libtest-needs-perl/open3-forks \ "IPC::Open3 could not run a child perl, so three of the five .t files cannot run at all -- $(_bst_1line "$o3")" ;; esac # and that the child was REAPED with status 0. TestAPI.pm asserts each child's # exit status before it looks at the child's output, so a fork/exec that works # but reports the wrong status would redden the criterion everywhere while every # child printed correctly. case "$o3" in *"CHILD-STATUS 0"*) bst_ok libtest-needs-perl/open3-child-status ;; *) bst_notok libtest-needs-perl/open3-child-status \ "a trivial child perl did not exit 0 under open3 -- every TestAPI assertion checks this first: $(_bst_1line "$o3")" ;; esac # backticks too: TestAPI.pm reads the child's @INC through a backtick pipe at # load time, BEFORE any assertion runs, so this one gates the whole file rather # than one case in it. bt=$(perl -e 'my $o = `$^X -le "print 42"`; chomp $o; print "BT[$o]\n";' 2>&1) bt=$(_tn_clean "$bt") check_eq libtest-needs-perl/backtick-child 'BT[42]' "$(_bst_1line "$bt")" # ---------- 4b. WHAT THE SKIP ACTUALLY EMITS, and what it EXITS with -------- # This is the port's deliverable driven end to end, in the two contexts the # ticket names, each in its own perl -- test_needs() terminates the file it is # called in, so it cannot be driven twice in one interpreter. # # OUTPUT and EXIT STATUS are separate cases on purpose. On a normal perl both # are correct together; on this arm the skip path goes through Test2's # plan-0-SKIP, and if linux-in-the-browser#93 fires in teardown the file prints # a perfectly correct `1..0 # SKIP` and then exits non-zero. Splitting them is # what makes that distinguishable from Test::Needs failing to skip at all. # (i) absent module, under Test::More -- the module's headline behaviour. sk=$(perl -MTest::More -e "use Test::Needs; test_needs '$ABSENT'; plan tests=>1; ok(1,'x');" 2>&1); skrc=$? sk=$(_tn_clean "$sk") case "$sk" in "1..0 # SKIP Need $ABSENT"*) bst_ok libtest-needs-perl/skip-absent-tap ;; *) bst_notok libtest-needs-perl/skip-absent-tap \ "want '1..0 # SKIP Need $ABSENT', got '$(_bst_1line "$sk")'" ;; esac if [ "$skrc" -eq 0 ]; then bst_ok libtest-needs-perl/skip-absent-rc elif printf '%s\n' "$sk" | grep -q 'panic: magic_killbackrefs'; then bst_notok libtest-needs-perl/skip-absent-rc \ "rc=$skrc -- the SKIP printed correctly and perl then panicked in teardown: linux-in-the-browser#93, the plan-0-SKIP form; rc is 0 on x86-64" else bst_notok libtest-needs-perl/skip-absent-rc "rc=$skrc want 0 -- $(_bst_1line "$sk")" fi # (ii) present module, under Test::More -- the OTHER branch the ticket names. # Without this the skip case above passes against a Test::Needs that skips # unconditionally, which is the single most likely way for this module to be # broken and still look right. pr=$(perl -MTest::More -e "use Test::Needs; test_needs 'strict'; plan tests=>1; ok(1,'x');" 2>&1); prrc=$? pr=$(_tn_clean "$pr") case "$pr" in *"ok 1 - x"*) case "$pr" in *SKIP*) bst_notok libtest-needs-perl/present-runs "a PRESENT module was skipped: $(_bst_1line "$pr")" ;; *) bst_ok libtest-needs-perl/present-runs ;; esac ;; *) bst_notok libtest-needs-perl/present-runs "want 'ok 1 - x', got '$(_bst_1line "$pr")'" ;; esac check_eq libtest-needs-perl/present-runs-rc 0 "$prrc" # (iii) STANDALONE -- no Test::More loaded at all. t/standalone.t's whole # subject, and the one path that does NOT go through Test2, so it is also the # control that says whether a red in (i) is Test::Needs' or Test2's. st=$(perl -e "use Test::Needs; test_needs '$ABSENT'; print \"1..1\nok 1\n\";" 2>&1); strc=$? st=$(_tn_clean "$st") case "$st" in "1..0 # SKIP Need $ABSENT"*) bst_ok libtest-needs-perl/skip-standalone-tap ;; *) bst_notok libtest-needs-perl/skip-standalone-tap "want '1..0 # SKIP Need $ABSENT', got '$(_bst_1line "$st")'" ;; esac check_eq libtest-needs-perl/skip-standalone-rc 0 "$strc" # (iv) RELEASE_TESTING=1 flips skip to FAIL. The env var is how a release # manager turns "quietly skipped" into "this build is missing something", and it # is the third of the module's documented behaviours. Exit status is expected # NON-zero here, so this case also proves the (i) rc assertion is not vacuous. rt=$(RELEASE_TESTING=1 perl -MTest::More -e "use Test::Needs; test_needs '$ABSENT'; plan tests=>1; ok(1,'x');" 2>&1); rtrc=$? rt=$(_tn_clean "$rt") case "$rt" in *"not ok 1 - Test::Needs modules available"*) bst_ok libtest-needs-perl/release-testing-fails ;; *) bst_notok libtest-needs-perl/release-testing-fails "want 'not ok 1 - Test::Needs modules available', got '$(_bst_1line "$rt")'" ;; esac case "$rt" in *"due to RELEASE_TESTING"*) bst_ok libtest-needs-perl/release-testing-diag ;; *) bst_notok libtest-needs-perl/release-testing-diag "the diagnostic does not name RELEASE_TESTING -- $(_bst_1line "$rt")" ;; esac if [ "$rtrc" -ne 0 ]; then bst_ok libtest-needs-perl/release-testing-rc else bst_notok libtest-needs-perl/release-testing-rc "rc=0 -- a RELEASE_TESTING failure must exit non-zero"; fi # (v) inside a SUBTEST -- an indented plan-0-SKIP, which is a different Test2 # code path from (i) and is 9 of TestAPI.pm's assertions per framework. su=$(perl -MTest::More -e "plan tests=>1; subtest s => sub { require Test::Needs; Test::Needs::test_needs('$ABSENT'); ok(1); };" 2>&1); surc=$? su=$(_tn_clean "$su") case "$su" in *" 1..0 # SKIP Need $ABSENT"*) bst_ok libtest-needs-perl/skip-subtest-tap ;; *) bst_notok libtest-needs-perl/skip-subtest-tap "want an indented '1..0 # SKIP', got '$(_bst_1line "$su")'" ;; esac case "$su" in *"ok 1 # skip Need $ABSENT"*) bst_ok libtest-needs-perl/skip-subtest-parent ;; *) bst_notok libtest-needs-perl/skip-subtest-parent "the parent did not record the subtest as skipped -- $(_bst_1line "$su")" ;; esac check_eq libtest-needs-perl/skip-subtest-rc 0 "$surc" # ---------- 4b'. THE LOCALE PREAMBLE, and the two defects it rides on ------- # Both of these are PLATFORM statements written in the direction that is true on # a normal perl, so they self-flip green when the defect lands -- the same shape # libmodule-build-perl's two reds have. # linux-in-the-browser#179: on this arm perl prints a five-line # `Setting locale failed` preamble to stderr on EVERY invocation, although # setlocale() itself succeeds (POSIX::setlocale(LC_ALL,"") returns C, and so # does "C"). It is asserted here because it is not cosmetic: t/lib/TestAPI.pm # merges each child's stderr into its stdout through open3 and then matches with # `qr/^1\.\.0 # SKIP/i` -- no /m, so ^ anchors at the START OF THE STRING, and # five lines of warning are enough to fail a child that skipped perfectly. That # is 2 of the distribution's 227 assertions today and 6 once #93 stops masking # the rest. This suite's own _tn_clean strips the same preamble; the case exists # so the defect is REPORTED rather than only worked around. lw=$(perl -e 'print "x\n"' 2>&1) case "$lw" in *"Setting locale failed"*) bst_notok libtest-needs-perl/no-locale-preamble \ "linux-in-the-browser#179 -- perl prints the locale preamble on stderr although setlocale() succeeds; it precedes any child TAP an open3 harness reads" ;; *) bst_ok libtest-needs-perl/no-locale-preamble ;; esac # and that it is NOT linux-in-the-browser#160 (no locale data), which is the # obvious wrong attribution: #160 predicts that `C` works and only UTF-8 fails. # Measured here so the distinction survives in the record. sl=$(perl -MPOSIX -e 'print POSIX::setlocale(POSIX::LC_ALL(), "C"), "\n"' 2>/dev/null) check_eq libtest-needs-perl/setlocale-c-succeeds C "$(_bst_1line "$sl")" # brintos/linux#69: /proc//exe is present but unreadable for every process # but pid 1, so perl's d_procselfexe path finds nothing and $^X falls back to # argv[0]. Asserted directly, because the two $^X cases in section 4 report the # CONSEQUENCE and this reports the cause -- and because it is a whole-image fact # that any program locating its own binary depends on, not a perl one. ex=$(readlink /proc/self/exe 2>/dev/null) if [ -n "$ex" ]; then bst_ok libtest-needs-perl/procselfexe-resolves else bst_notok libtest-needs-perl/procselfexe-resolves \ "brintos/linux#69 -- readlink(/proc/self/exe) yields nothing, so perl's \$^X is argv[0] rather than a path"; fi # ---------- 4c. THE TWO PLAN FORMS, measured on this boot ------------------- # 36-libdev-checklib-perl.test isolated linux-in-the-browser#93 to ONE of # Test::More's two plan forms. Both are measured here, separately, so section 5 # is attributed from evidence taken on the SAME interpreter on the SAME boot # rather than from a citation -- and so that the day #93 lands, this suite says # so by itself instead of continuing to blame it. pt=$(perl -MTest::More -e 'plan tests => 1; ok(1, "ab");' 2>&1); ptrc=$? pt=$(_tn_clean "$pt") case "$pt" in *"panic:"*) bst_notok libtest-needs-perl/plan-tests-no-panic \ "plan tests => N panics on this boot, so the criterion cannot pass either -- $(_bst_1line "$pt")" ;; *"ok 1"*) if [ "$ptrc" -eq 0 ]; then bst_ok libtest-needs-perl/plan-tests-no-panic else bst_notok libtest-needs-perl/plan-tests-no-panic "printed ok but exited $ptrc -- $(_bst_1line "$pt")"; fi ;; *) bst_notok libtest-needs-perl/plan-tests-no-panic "$(_bst_1line "$pt")" ;; esac # the form Test::Needs' skip goes through. This case is written in the direction # that is true on a normal perl -- it is a statement about the PLATFORM, and it # self-flips green the day #93 closes, exactly as libmodule-build-perl's two # reds do. sa=$(perl -MTest::More -e 'plan skip_all => "sa";' 2>&1); sarc=$? sa=$(_tn_clean "$sa") if [ "$sarc" -eq 0 ] && ! printf '%s\n' "$sa" | grep -q '^panic:'; then bst_ok libtest-needs-perl/plan-skipall-no-panic elif printf '%s\n' "$sa" | grep -q 'panic: magic_killbackrefs'; then bst_notok libtest-needs-perl/plan-skipall-no-panic \ "rc=$sarc linux-in-the-browser#93 -- Test2's plan-0-SKIP teardown panics on this arm, and it is the path Test::Needs' own skip takes; rc is 0 on x86-64" else bst_notok libtest-needs-perl/plan-skipall-no-panic "rc=$sarc $(_bst_1line "$sa")" fi # ------------------------- 5. THE CRITERION: prove -r t/ --------------------- # Run from a WRITABLE COPY, not in place: the deployed rootfs is read-only at # browser runtime, and t/lib/TestAPI.pm passes `-It/lib` to every child it # spawns, so CWD has to be the directory that HOLDS t/ -- which is also how # run-ptest invokes it. WD=${TMPDIR:-/tmp}/bst-libtest-needs-perl.$$ PW=$WD/ptest mkdir -p "$PW" && cp -r "$PT/t" "$PW/t" 2>/dev/null check libtest-needs-perl/suite-staged test -f "$PW/t/standalone.t" check libtest-needs-perl/suite-staged-lib test -f "$PW/t/lib/TestAPI.pm" # THE HARNESS ITSELF, BEFORE THE CRITERION THAT RIDES ON IT. `prove` resolving # and parsing is not the same as `prove` running: TAP::Harness names its five # collaborators as bare strings and loads them at run time, so poky's # use-scanning RDEPENDS generator never saw them and an image can carry a # /usr/bin/prove that dies on first invocation. Asserted on a trivial # self-contained test so a broken harness reds HERE, by name, instead of turning # the cases below into an unreadable heap. A .t file is a PROGRAM prove executes, # not a TAP transcript it reads -- so the smoke file is perl that PRINTS the # plan, not the plan itself. cat > "$PW/smoke.t" <<'SMOKE' print "1..1\nok 1 - smoke\n"; SMOKE smoke=$(cd "$PW" && prove smoke.t 2>&1); srn=$? smoke=$(_tn_clean "$smoke") case "$smoke" in *"Result: PASS"*) bst_ok libtest-needs-perl/prove-smoke ;; *) bst_notok libtest-needs-perl/prove-smoke \ "rc=$srn prove cannot run a 1-line passing test -- $(_bst_1line "$smoke")" ;; esac rm -f "$PW/smoke.t" cout=$(cd "$PW" && prove -r t/ 2>&1); crc=$? cout=$(_tn_clean "$cout") # `Result: PASS` is prove's own verdict over the whole run and is the ticket's # "reporting zero failures" verbatim; the exit status is asserted beside it # because a harness that died before printing a verdict leaves neither. The red # carries its ATTRIBUTION rather than just the transcript. case "$cout" in *"Result: PASS"*) bst_ok libtest-needs-perl/prove-result ;; *) if printf '%s\n' "$cout" | grep -q 'panic: magic_killbackrefs'; then why="linux-in-the-browser#93 -- perl's magic_killbackrefs use-after-free in Test2 teardown, reached here through the plan-0-SKIP path that IS this module's deliverable; the identical tree is Result: PASS on x86-64" elif printf '%s\n' "$cout" | grep -q 'panic: '; then why="a perl interpreter panic that is NOT #93's magic_killbackrefs -- read perl-no-panic's detail" elif printf '%s\n' "$cout" | grep -qi "can't fork\|Resource temporarily unavailable"; then why="a fork/exec failure, not a test failure -- see open3-forks in section 4a" else why="no interpreter panic and no fork failure -- this is a genuine test failure and is THIS PORT'S to answer" fi bst_notok libtest-needs-perl/prove-result "rc=$crc $(_bst_1line "$cout") -- $why" ;; esac check_eq libtest-needs-perl/prove-rc 0 "$crc" case "$cout" in *"All tests successful."*) bst_ok libtest-needs-perl/prove-all-successful ;; *) bst_notok libtest-needs-perl/prove-all-successful "$(_bst_1line "$cout")" ;; esac # The file count is pinned EXACTLY -- it is a property of the package, already # asserted on the rootfs in section 1, and a harness that silently ran fewer # files is the failure a green verdict cannot show. files=$(printf '%s\n' "$cout" | sed -n 's/^Files=\([0-9]*\),.*/\1/p' | tail -1) check_eq libtest-needs-perl/prove-files 5 "$files" # The assertion count is bounded from BELOW, not pinned. 227 is what the x86-64 # control ran (24 + 36 + 25 + 71 + 71), with nothing skipped anywhere; bounding # from below rather than pinning keeps a future 0.002011 with an extra case from # reading as a regression, while still catching a file that died part-way. tests=$(printf '%s\n' "$cout" | sed -n 's/^Files=[0-9]*, Tests=\([0-9]*\),.*/\1/p' | tail -1) case "$tests" in ''|*[!0-9]*) bst_notok libtest-needs-perl/prove-tests "no Tests= total in prove's summary -- $(_bst_1line "$cout")" ;; *) if [ "$tests" -ge 227 ]; then bst_ok libtest-needs-perl/prove-tests else bst_notok libtest-needs-perl/prove-tests "Tests=$tests, want >= 227 (x86-64 control ran 227)"; fi ;; esac # THE FIVE FILES, PINNED BY NAME. A skip and a pass are both green to prove, so # without this a regression that turned all five into skips would still report # Result: PASS -- the exact false green the criterion is meant to exclude, and a # particularly live risk for THIS distribution, whose three TestAPI files skip # themselves whole when their framework is absent. for t in find_missing numify_version standalone test-builder test2; do line=$(printf '%s\n' "$cout" | grep -m1 "^t/$t\.t ") case "$line" in '') bst_notok "libtest-needs-perl/ran-$t" "t/$t.t has no line in prove's transcript -- it did not run at all" ;; *skipped*) bst_notok "libtest-needs-perl/ran-$t" "t/$t.t skipped -- its framework was not found, which section 2 says is present: $(_bst_1line "$line")" ;; *ok*) bst_ok "libtest-needs-perl/ran-$t" ;; *) # Attribute by FILE, because the three TestAPI files fail for two different # reasons and one of them is not #93. t/standalone.t's children never load # Test2 -- they are the standalone path -- so they cannot reach the # plan-0-SKIP teardown at all; measured on target, its two reds are # linux-in-the-browser#179 (perl's spurious locale preamble on stderr, # which open3 merges into the child's stdout and which then defeats # TestAPI's `qr/^1\.\.0 # SKIP/i` -- no /m, so ^ anchors at string start). # t/test2.t and t/test-builder.t DO load Test2 and fail on #93 first, # which masks the same #179 mismatch underneath. case "$t" in standalone) bst_notok "libtest-needs-perl/ran-$t" "$(_bst_1line "$line") -- linux-in-the-browser#179: perl's locale preamble precedes the child's TAP, so TestAPI's start-anchored SKIP match fails on a child that skipped correctly. NOT #93: standalone children never load Test2" ;; *) bst_notok "libtest-needs-perl/ran-$t" "$(_bst_1line "$line") -- see prove-result and perl-no-panic for the attribution" ;; esac ;; esac done # WHICH FILES FAILED, BY NAME. Without this a non-zero prove is one red carrying # 100 characters of transcript, and the file that actually broke is whichever one # happened to print first. prove's Test Summary Report names each on a line of # the form `t/.t (Wstat: ...)`, so the names are recoverable and belong in # the report rather than in a later reader's guess. bad=$(printf '%s\n' "$cout" | sed -n 's/^\(t\/[A-Za-z0-9_.-]*\.t\) *(Wstat:.*/\1/p' | sort | tr '\n' ' ') if [ -z "$bad" ]; then bst_ok libtest-needs-perl/prove-failing-files else bst_notok libtest-needs-perl/prove-failing-files "failing files: $bad"; fi # HOW MANY ASSERTIONS ACTUALLY FAILED, as distinct from how the run ENDED. This # is the case that separates the two things #93 confuses: a run where every file # reports `Failed: 0` and prove still says FAIL because a child exited non-zero # in teardown is a PLATFORM result, and a run with a non-zero Failed count is # THIS PORT'S. libmodule-build-perl's entry records exactly the first shape. fail=$(printf '%s\n' "$cout" | sed -n 's/.*Failed \([0-9]*\)\/[0-9]* subtests.*/\1/p' | tr '\n' ' ') if [ -z "$fail" ]; then bst_ok libtest-needs-perl/prove-no-failed-subtests else bst_notok libtest-needs-perl/prove-no-failed-subtests \ "prove reported failed subtests ($fail) -- these are assertion failures, not a teardown exit status"; fi # A perl INTERPRETER PANIC is not a test failure and must not be reported as one. # perl prints `panic: ` from its own guts when an internal invariant # breaks; on this arm that is a platform fact about perl-on-wasm32, not a # statement about Test::Needs. Named separately so it is attributable instead of # being buried in whichever red quoted the transcript first. pan=$(printf '%s\n' "$cout" | grep -m1 'panic: ') if [ -n "$pan" ]; then case "$pan" in *magic_killbackrefs*) pwhy="linux-in-the-browser#93 -- the magic_killbackrefs use-after-free this arm already carries a warnings.pm workaround for" ;; *) pwhy="an interpreter panic #93 does not cover -- unattributed, and worth a brintos.io issue" ;; esac bst_notok libtest-needs-perl/perl-no-panic "$(_bst_1line "$pan") -- $pwhy" else bst_ok libtest-needs-perl/perl-no-panic fi rm -rf "$WD"