. "$BST_LIB" require perl libtest-warnings-perl/present # BST-TIMEOUT: 900 # # Set from a measurement, not guessed. Unlike 36-libtest-needs-perl.test -- whose # 2400 is bought by ~250 CHILD PERLS spawned through IPC::Open3 -- this # distribution's 27 files are IN-PROCESS: no fork, no exec, no open3 anywhere in # t/. The x86-64 control runs all 27 in 1 wallclock second. What this suite does # spend is its own section 4b, which drives 14 separate `perl -e` invocations # because the module's END-block contract can only be observed one plan per # interpreter, and a perl process costs seconds rather than milliseconds on this # arm (linux-in-the-browser#117 measured 46-68x native). 900 covers that with # room and stays well under the runner's 3600 s BST_SUITE_TIMEOUT_MAX, which # would fire on load rather than on breakage. # # Test::Warnings 0.033 (POKY meta/recipes-devtools/perl/libtest-warnings-perl_0.033.bb # -- NOT meta-oe, which the ticket names; same correction libmodule-build-perl, # libtest-fatal-perl and libtest-needs-perl each had to make). Nineteenth Perl # module on this image under BRINTOS-999. Pure Perl with no XS half at all, so # BRINTOS-881 -- the -Uusedl -Uuseshrplib arm that stops every XS module here -- # cannot reach it; section 1 asserts that rather than assuming it. # # WHAT THE MODULE IS. Test::Warnings asserts on the warnings a piece of Perl # emits, and -- the part that distinguishes it from Test::NoWarnings, which it # exists to replace -- fails a test file that warns UNEXPECTEDLY, without the # caller having to declare a test count up front. It does that by installing a # global $SIG{__WARN__} at load time, monkeypatching Test::Builder::done_testing, # and adding one test in an END block. Those three are its whole deliverable and # none of them is a return value, which shapes this suite. # # THIS SUITE'S PASS CRITERION IS THE DISTRIBUTION'S OWN t/, NOT A HAND-WRITTEN # BATTERY -- the ticket's own wording. The port therefore installs the suite # (libtest-warnings-perl-ptest, from PTEST_ENABLED = "1" for this recipe alone) # and section 5 runs `prove -r t/` over it. # # THE PORT'S ONE STRUCTURAL RISK, STATED UP FRONT SO A RED IS ATTRIBUTED RATHER # THAN DISCOVERED. t/06-skip-all.t does `plan skip_all` UNCONDITIONALLY: it evals # `use Does::Not::Exist::`, which always fails, and skips. That is the # Test2 plan-0-SKIP path 36-libdev-checklib-perl.test isolated # linux-in-the-browser#93 to (perl's magic_killbackrefs use-after-free in # Test2::Hub's teardown): `plan tests => N` survives on this arm, plan-0-SKIP # panics. So one of the 27 files is expected to print a perfectly correct # `1..0 # SKIP` and then exit non-zero, which prove reports as a failed file. # # It is worth being precise about how much that costs, because it is much less # than it cost libtest-needs-perl. There the panicking form WAS the deliverable # and 84 of 227 assertions rode on it; here it is one file out of 27 carrying # ZERO assertions (a skipped file contributes none), so the control's Tests=83 # should be reached in full and only prove's verdict changes. Section 4c measures # both plan forms on this boot so section 5's attribution comes from evidence # taken in the same interpreter rather than from this comment. # # THE SECOND RISK IS THE OPPOSITE ONE, AND IT IS QUIETER. FOUR files -- 01-basic, # 21-fail-on-warning, 23-report-warnings, 24-only-report-warnings -- open with # `plan skip_all => 'These tests require Test::Tester 0.108' if not $has_test_tester`. # Those four carry 46 of the control's 83 assertions. If Test::Tester were absent # they would not fail: they would SKIP, prove would still be able to report # PASS over the rest, and the criterion would look green while 55% of it had # silently evaporated -- AND four more files would be routed through the #93 # form. That is the false green section 2 and section 5's per-file pinning exist # to exclude, and it is why the recipe names perl-module-test-tester explicitly. # # EVERY REFERENCE BELOW WAS MEASURED, NOT READ OFF THE TARBALL. The control is # the same 0.033 tree on x86-64 (perl 5.38.2), driven as section 5 drives it: # `prove -Ilib -r t/` gives `All tests successful.`, Files=27, Tests=83, # Result: PASS, exit 0, with nothing skipped. The six CPAN::Meta modules this # image does not carry are absent on the target and cost nothing: the only two # files that touch them (t/00-report-prereqs.t, t/zzz-check-breaks.t) are # information-only and guard every use behind `eval { require ... }`. # # Five layers: # 1. filesystem facts -- what the port SHIPS, asserted before anything starts # perl, so they hold on a boot where the interpreter cannot load a module. # 2. the harness the criterion names, and the modules the suite loads. # 3. can perl load the module on this boot, classified on the diagnostic. # 4. the SEMANTICS, at values, against the x86-64 control -- then (4a) the # TAP-layer contract the module actually exists for, driven one plan per # perl, (4b) the pattern allow-list, and (4c) the two plan forms measured. # 5. the criterion: prove -r t/ over the installed suite. # perl prints a multi-line locale-warning preamble on EVERY invocation on this # image, on stderr (linux-in-the-browser#179, filed by BRINTOS-1323: setlocale # SUCCEEDS and perl reports it as failed). Sections 3-5 capture 2>&1 -- they must: # a prove that died has said so on stderr -- so without this strip every red # would report "Setting locale failed" in the 100-char detail field, the wrong # diagnosis in the one place a later reader attributes by. Same strip as # 36-libtest-fatal-perl.test and 36-libtest-needs-perl.test. _tw_clean() { printf '%s\n' "$1" | awk '/^perl: warning/ { skip=1; next } /^[ \t]/ && skip { next } { skip=0; print }' } V=$(ls -d /usr/lib/perl5/vendor_perl/* 2>/dev/null | head -1) CORE=$(ls -d /usr/lib/perl5/5.* 2>/dev/null | head -1) PT=/usr/lib/libtest-warnings-perl/ptest # ------------------------- 1. what the port SHIPS: filesystem facts ---------- # The VERSION dir, not the arch dir: Test::Warnings is pure Perl, so cpan # installs it to vendor_perl// and NOT to the vendor_perl//-linux/ # an XS module lands in. That placement IS the packaging's statement that no .so # is wanted, which is why BRINTOS-881 cannot reach this port. check libtest-warnings-perl/vendor-dir test -n "$V" check libtest-warnings-perl/module-installed test -f "$V/Test/Warnings.pm" # ${PN} is ONE file and nothing else -- read off the built .deb, not assumed. # Asserted because a recipe that started shipping an auto/ tree here would mean # the module had acquired an XS half, which is the one change that would put # this port back under BRINTOS-881. check libtest-warnings-perl/no-xs-object test ! -d "$V/auto/Test/Warnings" # THE INSTALLED SUITE -- the deliverable the ticket's criterion runs on. poky's # recipe already inherits ptest-perl, but the class deletes its own task unless # PTEST_ENABLED is 1, and this distro's DISTRO_FEATURES has no `ptest`. These # cases are what says this layer's bbappend took effect. check libtest-warnings-perl/ptest-dir test -d "$PT/t" check libtest-warnings-perl/ptest-runptest test -f "$PT/run-ptest" # 27 -- the whole of the distribution's t/. This recipe prunes none of them, so # the count is the tarball's and a recipe that started dropping files reds here. # The x86-64 control ran exactly these 27. check_eq libtest-warnings-perl/ptest-count 27 "$(ls "$PT"/t/*.t 2>/dev/null | wc -l | tr -d ' ')" # The four files that carry 46 of the control's 83 assertions, each by name. # Pinned individually rather than by count because these are the four that skip # themselves whole without Test::Tester -- see the header. A count alone cannot # tell "all 27 present" from "all 27 present and four of them inert". for t in 01-basic 21-fail-on-warning 23-report-warnings 24-only-report-warnings; do check "libtest-warnings-perl/ptest-$t" test -f "$PT/t/$t.t" done # and the file that is expected to go red under #93, by name, so section 5's # attribution has something on the rootfs to point at. check libtest-warnings-perl/ptest-06-skip-all test -f "$PT/t/06-skip-all.t" # t/lib/SilenceStderr.pm is the distribution's ONE fixture and ptest-perl does # not install t/lib by itself -- poky's recipe carries a # do_install_ptest_perl:append that copies it explicitly. Asserted by name # because without it the files that use it die with `Can't locate # SilenceStderr.pm`, which reads as a broken port rather than as a packaging # omission. t/00-report-prereqs.dd is the same shape: a data file beside the .t # that reads it, which a FILES glob over *.t would silently drop. check libtest-warnings-perl/ptest-lib-silencestderr test -f "$PT/t/lib/SilenceStderr.pm" check libtest-warnings-perl/ptest-prereqs-dd test -f "$PT/t/00-report-prereqs.dd" # ------------------- 2. the harness the criterion names ---------------------- # `prove` reached this image under libdbi-perl (perl-misc + perl-module-app-prove) # and WORKS only because libdev-checklib-perl added the 43 perl-module-tap-* # splits. Asserted as ordinary reds, not as a `require`: their absence is a # regression to report, not a reason to skip this port's criterion. check libtest-warnings-perl/prove-installed test -x /usr/bin/prove check libtest-warnings-perl/prove-lib test -f "$CORE/App/Prove.pm" # TEST::TESTER IS THE ONE THAT MATTERS MOST HERE and it is the reason this # port's bbappend names it. Four of the 27 files `plan skip_all` without it -- # see the header -- so its absence would not redden the criterion, it would # HALVE it while leaving prove able to report PASS. It is a perl-module-* # split of this image's 1146-way perl, reached through no other name in the # closure, so nothing but the explicit RDEPENDS puts it on the image. check libtest-warnings-perl/testtester-installed test -f "$CORE/Test/Tester.pm" check libtest-warnings-perl/testmore-installed test -f "$CORE/Test/More.pm" # Test::Builder is not merely the harness for this module -- it is the object # Test::Warnings MONKEYPATCHES (done_testing) and calls through (_builder->ok), # and it is the only name poky puts in RDEPENDS:${PN}. check libtest-warnings-perl/tb-installed test -f "$CORE/Test/Builder.pm" check libtest-warnings-perl/test2api-installed test -f "$CORE/Test2/API.pm" # the `if` pragma: 23 of the 27 files `use if`, so its absence takes out most of # the suite at compile time, before any assertion runs. check libtest-warnings-perl/if-installed test -f "$CORE/if.pm" # ExtUtils::MakeMaker and File::Spec are what t/00-report-prereqs.t loads, and # ExtUtils::MakeMaker::Locale is the one poky's static perl-rdepends.txt does # NOT connect to it -- this port's bbappend names it and the four others in that # family by hand. Asserted so a future perl-rdepends.txt that started declaring # them, or a bbappend that stopped, is visible either way. check libtest-warnings-perl/mm-installed test -f "$CORE/ExtUtils/MakeMaker.pm" check libtest-warnings-perl/mm-locale-installed test -f "$CORE/ExtUtils/MakeMaker/Locale.pm" check libtest-warnings-perl/filespec-installed test -f "$CORE/File/Spec.pm" # CPAN::Meta IS ABSENT ON THIS IMAGE AND THAT IS CORRECT, not an omission, and # it is deliberately NOT asserted here. It is not produced as a perl-module-* # split on this arm at all, and the only two files that touch it -- # t/00-report-prereqs.t and t/zzz-check-breaks.t -- guard every use behind # `eval { require ... }` and are information-only. The x86-64 control reaches # Tests=83 with those modules PRESENT, so the claim that costs nothing here is # the COUNT, and section 5's `prove-tests >= 83` is where it is made. A case # asserting "CPAN::Meta is absent" would go red the day the image gained it # while nothing had regressed, and a case asserting "present or absent, both # fine" asserts nothing at all. # ------------------ 3. can perl LOAD this module on this boot? --------------- # Pure Perl whose only unconditional loads are parent, Exporter and # Test::Builder, so a clean import is the expected -- and the only passing -- # outcome. Failure is classified on the diagnostic so a red says WHOSE defect it # is rather than quoting 100 characters of transcript. pout=$(perl -e ' eval { require Test::Warnings }; if ($@) { print "LOAD-FAILED: $@"; exit 1 } print "LOAD-OK $Test::Warnings::VERSION\n";' 2>&1) pout=$(_tw_clean "$pout") case "$pout" in *"LOAD-OK"*) bst_ok libtest-warnings-perl/require-loads ;; *"Can't locate"*) bst_notok libtest-warnings-perl/require-loads \ "a module in the closure is not on the image: $(_bst_1line "$pout")" ;; *"panic:"*) bst_notok libtest-warnings-perl/require-loads \ "perl PANICKED loading a pure-Perl module -- a platform defect, not this port's: $(_bst_1line "$pout")" ;; *) bst_notok libtest-warnings-perl/require-loads "$(_bst_1line "$pout")" ;; esac # `use Test::Warnings;` with NO import list must not disturb the caller: it # exports nothing by default, and a file that merely loads it (which is the # module's commonest use -- the END-block check needs no imported symbol) must # still run normally. check_out libtest-warnings-perl/import-bare "TW-OK" perl -MTest::Warnings -e 'print "TW-OK\n"' # and `:all`, the form the ticket's own callers write, must import the seven # names into the caller's namespace. check_out libtest-warnings-perl/import-all "TW-ALL-OK" \ perl -e 'use Test::Warnings ":all"; print "TW-ALL-OK\n" if defined &warning && defined &warnings && defined &allow_warnings && defined &had_no_warnings && defined &allow_patterns && defined &disallow_patterns && defined &allowing_warnings;' # ------------------ 4. THE SEMANTICS, at values, vs the x86-64 control ------- # The payload measures; this file names and compares, so a single wrong answer # is one red case rather than one red suite. Everything here is a pure function # or an inspectable property -- the module's TAP-layer contract is section 4a's # business, because each half of it needs its own interpreter and its own plan. out=$(perl "$BST_DATA/libtest-warnings-perl-selftest.pl" 2>&1); rc=$? out=$(_tw_clean "$out") case "$out" in *ALL-TEST-WARNINGS-OK*) all=1 ;; *) all=0 ;; esac if [ "$rc" -eq 0 ] && [ "$all" -eq 1 ]; then bst_ok libtest-warnings-perl/all-ok else bst_notok libtest-warnings-perl/all-ok "rc=$rc $(_bst_1line "$out")"; fi val() { printf '%s\n' "$out" | awk -v t="$1" '$1==t{ $1=""; sub(/^ /,""); print; exit }'; } # --- identity, and that it came from the image ------------------------------ check_eq libtest-warnings-perl/version 0.033 "$(val R1-version)" case "$(val R2-inc-path)" in /usr/*) bst_ok libtest-warnings-perl/installed-path ;; *) bst_notok libtest-warnings-perl/installed-path "Test/Warnings.pm resolved to $(val R2-inc-path)" ;; esac # NOTHING is exported by default. This is an API property with teeth: a caller # may `use Test::Warnings;` purely for the END check, and a module that started # exporting had_no_warnings by default would collide with every caller defining # its own sub of that name. check_eq libtest-warnings-perl/export-default '' "$(val R3-export-default)" EXPORTS='allow_patterns,allow_warnings,allowing_warnings,disallow_patterns,had_no_warnings,warning,warnings' check_eq libtest-warnings-perl/export-ok "$EXPORTS" "$(val R4-export-ok)" # `:all` is exactly @EXPORT_OK -- the tag the ticket's callers use, and the one # whose drift would be invisible to any caller that imports by name. check_eq libtest-warnings-perl/tag-all "$EXPORTS" "$(val R5-tag-all)" # --- the PROTOTYPES, which ARE the API -------------------------------------- # warning(&) and warnings(;&) are what let a caller write `warning { ... }` with # no comma and no `sub`. A prototype that changed is not a subtle regression -- # it is a compile-time syntax error in every caller's t/ -- and the string is the # only thing that sees it before that happens. check_eq libtest-warnings-perl/proto-warning '&' "$(val R6-proto-warning)" check_eq libtest-warnings-perl/proto-warnings ';&' "$(val R7-proto-warnings)" check_eq libtest-warnings-perl/proto-allow-warnings ';$' "$(val R8-proto-allow-warnings)" check_eq libtest-warnings-perl/proto-allowing-warnings '' "$(val R9-proto-allowing-warnings)" check_eq libtest-warnings-perl/proto-had-no-warnings ';$' "$(val R10-proto-had-no-warnings)" check_eq libtest-warnings-perl/proto-allow-patterns '@' "$(val R11-proto-allow-patterns)" # --- warnings{} and warning{}: the collectors ------------------------------- check_eq libtest-warnings-perl/warnings-count 2 "$(val R12-warnings-count)" check_eq libtest-warnings-perl/warnings-text alpha "$(val R13-warnings-first)" check_eq libtest-warnings-perl/warnings-empty 0 "$(val R14-warnings-empty)" # the one case in the module whose purpose is to NOT do its job: `warnings()` # with no arguments returns the string 'warnings', so a namespace that imported # this sub and then wrote `warnings->import` still reaches the pragma. check_eq libtest-warnings-perl/warnings-noargs warnings "$(val R15-warnings-noargs)" # warning{} is warnings{} with a scalar/aggregate distinction, and the two are # one `==` apart in the source: exactly one warning comes back as the STRING # (which is what `like(warning { ... }, qr/../)` depends on), zero or many as an # ARRAY REF (which is what a counting caller depends on). All three arities. check_eq libtest-warnings-perl/warning-one-type string "$(val R16-warning-one-type)" check_eq libtest-warnings-perl/warning-one-text single "$(val R17-warning-one-text)" check_eq libtest-warnings-perl/warning-many-type ARRAY "$(val R18-warning-many-type)" check_eq libtest-warnings-perl/warning-many-count 2 "$(val R19-warning-many-count)" check_eq libtest-warnings-perl/warning-none-type ARRAY "$(val R20-warning-none-type)" check_eq libtest-warnings-perl/warning-none-count 0 "$(val R21-warning-none-count)" # and the block is really RUN -- a stub returning an empty list without calling # the code satisfies warnings-empty, warning-none-type and warning-none-count # entirely, which is what makes this case load-bearing rather than obvious. check_eq libtest-warnings-perl/block-executed yes "$(val R22-block-executed)" # --- allow_warnings / allowing_warnings: the switch ------------------------- # The default is UNDEF, not 0 -- code testing `defined` instead of truth behaves # differently on the first call -- so all three states are pinned. check_eq libtest-warnings-perl/allowing-default '' "$(val R23-allowing-default)" check_eq libtest-warnings-perl/allowing-on 1 "$(val R24-allowing-on)" check_eq libtest-warnings-perl/allowing-off 0 "$(val R25-allowing-off)" # --- allow_patterns' guard object ------------------------------------------- check_eq libtest-warnings-perl/pattern-guard-type ARRAY "$(val R26-pattern-guard-type)" check_eq libtest-warnings-perl/pattern-guard-class Test::Warnings::TemporaryWarning \ "$(val R27-pattern-guard-class)" # without DESTROY the non-void form silently becomes the permanent form, and # every case that merely inspects the returned object still passes. check_eq libtest-warnings-perl/pattern-guard-destroy yes "$(val R28-pattern-guard-destroy)" # --- the collaborator this module rewrites ---------------------------------- check_eq libtest-warnings-perl/done-testing-patched patched "$(val R36-done-testing-patched)" # --- $^X, measured for comparability with the other Perl ports -------------- # This distribution is in-process and does not re-exec perl anywhere, so unlike # 36-libtest-needs-perl.test the criterion here does NOT ride on $^X. It is # measured anyway, and written in the direction true on a normal perl, because # it is how brintos/linux#69 shows itself and a reader comparing the Perl ports # needs the same field in each. case "$(val R32-perl-path)" in /*) bst_ok libtest-warnings-perl/perl-path-absolute ;; *) bst_notok libtest-warnings-perl/perl-path-absolute \ "\$^X is '$(val R32-perl-path)' -- brintos/linux#69: /proc/self/exe is unreadable, so perl's d_procselfexe path falls back to argv[0]. Costs this criterion nothing: no file in t/ re-execs perl" ;; esac # ---------- 4a. THE TAP-LAYER CONTRACT, one plan per interpreter ------------ # This is what Test::Warnings is FOR, and none of it is a return value: each # case below is a whole perl with its own plan, because the END block fires once # per interpreter and its effect is a test line plus an EXIT STATUS. Every # expectation is the x86-64 control's measured output, not the POD's claim. # (i) THE HEADLINE: a clean file gets one EXTRA passing test from the END block. # `tests => 2` with one ok() of its own is how a caller writes it, and the # module's whole convenience is that the count is declared knowing this. cl=$(perl -e 'use Test::More tests=>2; use Test::Warnings; ok(1,"x");' 2>&1); clrc=$? cl=$(_tw_clean "$cl") case "$cl" in *"ok 2 - no (unexpected) warnings (via END block)"*) bst_ok libtest-warnings-perl/end-clean-tap ;; *) bst_notok libtest-warnings-perl/end-clean-tap \ "the END block did not add its passing test: $(_bst_1line "$cl")" ;; esac check_eq libtest-warnings-perl/end-clean-rc 0 "$clrc" # (ii) THE POINT OF THE MODULE: an UNEXPECTED warning fails the file. Without # this case every other one here is satisfied by a module that adds a test which # always passes -- which is precisely what Test::NoWarnings-plus-done_testing # degrades to, and the reason this distribution exists. wn=$(perl -e 'use Test::More tests=>2; use Test::Warnings; warn "boom\n"; ok(1,"x");' 2>&1); wnrc=$? wn=$(_tw_clean "$wn") case "$wn" in *"not ok 2 - no (unexpected) warnings (via END block)"*) bst_ok libtest-warnings-perl/end-warn-fails ;; *) bst_notok libtest-warnings-perl/end-warn-fails \ "a file that warned did NOT fail -- this is the module's whole deliverable: $(_bst_1line "$wn")" ;; esac # and the EXIT STATUS, separately: a file that prints `not ok` and exits 0 is # invisible to prove, to CI, and to the criterion in section 5. check_eq libtest-warnings-perl/end-warn-rc 1 "$wnrc" # (iii) :no_end_test suppresses the check entirely -- the escape hatch a caller # uses when it means to assert the warnings itself. A file that warns must then # PASS, and must run one test rather than two. ne=$(perl -e 'use Test::More tests=>1; use Test::Warnings ":no_end_test"; warn "boom\n"; ok(1,"x");' 2>&1); nerc=$? ne=$(_tw_clean "$ne") case "$ne" in *"1..1"*) case "$ne" in *"no (unexpected) warnings"*) bst_notok libtest-warnings-perl/no-end-test \ ":no_end_test did not suppress the END test: $(_bst_1line "$ne")" ;; *) bst_ok libtest-warnings-perl/no-end-test ;; esac ;; *) bst_notok libtest-warnings-perl/no-end-test "$(_bst_1line "$ne")" ;; esac check_eq libtest-warnings-perl/no-end-test-rc 0 "$nerc" # (iv) THE done_testing MONKEYPATCH, which is the other half of the module's # reason to exist: with no declared plan the check has to run BEFORE # done_testing writes the count, or the count is wrong. The distinct message # text ("via done_testing", not "via END block") is what says the patched path # ran rather than the END fallback. dt=$(perl -e 'use Test::More; use Test::Warnings; ok(1,"x"); done_testing;' 2>&1); dtrc=$? dt=$(_tw_clean "$dt") case "$dt" in *"ok 2 - no (unexpected) warnings (via done_testing)"*) case "$dt" in *"1..2"*) bst_ok libtest-warnings-perl/done-testing-tap ;; *) bst_notok libtest-warnings-perl/done-testing-tap \ "the check ran but done_testing wrote the wrong count -- $(_bst_1line "$dt")" ;; esac ;; *) bst_notok libtest-warnings-perl/done-testing-tap \ "done_testing did not run the warnings check -- $(_bst_1line "$dt")" ;; esac check_eq libtest-warnings-perl/done-testing-rc 0 "$dtrc" # (v) :fail_on_warning reports the warning WHERE IT HAPPENS rather than only at # the end -- an immediate `not ok - unexpected warning`, in addition to the END # test. The control exits 2 here (two failures of three). fw=$(perl -e 'use Test::More tests=>3; use Test::Warnings ":fail_on_warning"; ok(1,"x"); warn "boom\n";' 2>&1) fw=$(_tw_clean "$fw") case "$fw" in *"not ok 2 - unexpected warning"*) bst_ok libtest-warnings-perl/fail-on-warning ;; *) bst_notok libtest-warnings-perl/fail-on-warning \ ":fail_on_warning did not emit its immediate failure: $(_bst_1line "$fw")" ;; esac # (vi) :report_warnings prints the collected warning TEXT in the diagnostic -- # the difference between "something warned" and knowing what. Asserted on the # text of the warning, not on the header line, because a header with an empty # list is exactly the failure this flag exists to prevent. rw=$(perl -e 'use Test::More tests=>2; use Test::Warnings ":report_warnings"; ok(1,"x"); warn "boom\n";' 2>&1) rw=$(_tw_clean "$rw") case "$rw" in *"Got the following unexpected warnings:"*) case "$rw" in *"1: boom"*) bst_ok libtest-warnings-perl/report-warnings ;; *) bst_notok libtest-warnings-perl/report-warnings \ "the report header printed with no warning text under it: $(_bst_1line "$rw")" ;; esac ;; *) bst_notok libtest-warnings-perl/report-warnings "$(_bst_1line "$rw")" ;; esac # (vii) PERL_TEST_WARNINGS_ONLY_REPORT_WARNINGS turns the failure into a report # -- the env-var escape a caller uses to survey warnings without going red. The # file must PASS and still say what it found. ov=$(PERL_TEST_WARNINGS_ONLY_REPORT_WARNINGS=1 perl -e 'use Test::More tests=>1; use Test::Warnings; warn "boom\n"; ok(1,"x");' 2>&1); ovrc=$? ov=$(_tw_clean "$ov") case "$ov" in *"but allowing them because PERL_TEST_WARNINGS_ONLY_REPORT_WARNINGS is set"*) bst_ok libtest-warnings-perl/only-report-env ;; *) bst_notok libtest-warnings-perl/only-report-env \ "the env override did not take effect: $(_bst_1line "$ov")" ;; esac check_eq libtest-warnings-perl/only-report-env-rc 0 "$ovrc" # (viii) allow_warnings() at the TAP layer: an allowed warning is NOTED (a `#` # comment line), not counted. The in-process cases above pin the getter; this # pins what the __WARN__ handler actually does with it, which is the half that # ships. aw=$(perl -e 'use Test::More tests=>2; use Test::Warnings ":all"; allow_warnings(1); warn "boom\n"; ok(1,"x");' 2>&1); awrc=$? aw=$(_tw_clean "$aw") case "$aw" in *"ok 2 - no (unexpected) warnings (via END block)"*) case "$aw" in *"not ok 2"*) bst_notok libtest-warnings-perl/allow-warnings-tap \ "allow_warnings(1) did not stop the warning failing the file: $(_bst_1line "$aw")" ;; *) bst_ok libtest-warnings-perl/allow-warnings-tap ;; esac ;; *) bst_notok libtest-warnings-perl/allow-warnings-tap "$(_bst_1line "$aw")" ;; esac check_eq libtest-warnings-perl/allow-warnings-rc 0 "$awrc" # (ix) THE SUBTEST BOUNDARY. The END check must run once, for the FILE, and not # once per subtest -- the module tests `$builder->in_subtest` for exactly this. # Without the guard a file with N subtests would emit N spurious tests and blow # its own plan, which is the regression a plan-count assertion catches and a # message-text assertion does not. sb=$(perl -e 'use Test::More tests=>3; use Test::Warnings; subtest s1 => sub { plan tests=>1; ok(1,"inner"); }; ok(1,"outer");' 2>&1); sbrc=$? sb=$(_tw_clean "$sb") n_end=$(printf '%s\n' "$sb" | grep -c "no (unexpected) warnings") check_eq libtest-warnings-perl/subtest-one-end-check 1 "$n_end" check_eq libtest-warnings-perl/subtest-rc 0 "$sbrc" # (x) THE CARP PATH. A warning WITHOUT a trailing newline is re-emitted through # Carp::carp, which appends " at FILE line N" -- a documented behaviour, and the # one branch of the __WARN__ handler that loads a module lazily (`require Carp`). # A caller relies on it to locate the warning; a lazy require that could not # resolve would die inside a handler instead. cp=$(perl -e 'use Test::More tests=>2; use Test::Warnings; warn "nonewline"; ok(1,"x");' 2>&1) cp=$(_tw_clean "$cp") case "$cp" in *"nonewline at "*" line "*) bst_ok libtest-warnings-perl/carp-appends-location ;; *) bst_notok libtest-warnings-perl/carp-appends-location \ "a newline-less warning did not go through Carp::carp: $(_bst_1line "$cp")" ;; esac # ---------- 4b. allow_patterns AT THE TAP LAYER, both directions ------------ # The guard's SHAPE is pinned in section 4; its EFFECT can only be seen where a # warning is or is not counted, so it is driven here. Both directions, because # an implementation that allowed EVERYTHING satisfies the first alone. ap=$(perl -e 'use Test::More tests=>2; use Test::Warnings ":all"; allow_patterns(qr/^ok-to-warn/); warn "ok-to-warn here\n"; ok(1,"x");' 2>&1); aprc=$? ap=$(_tw_clean "$ap") if [ "$aprc" -eq 0 ]; then case "$ap" in *"not ok"*) bst_notok libtest-warnings-perl/pattern-allows \ "a warning matching an allowed pattern still failed the file: $(_bst_1line "$ap")" ;; *) bst_ok libtest-warnings-perl/pattern-allows ;; esac else bst_notok libtest-warnings-perl/pattern-allows "rc=$aprc $(_bst_1line "$ap")" fi # the NEGATIVE direction: a warning that does NOT match must still fail, in the # same interpreter that has a pattern installed. an=$(perl -e 'use Test::More tests=>2; use Test::Warnings ":all"; allow_patterns(qr/^ok-to-warn/); warn "something-else\n"; ok(1,"x");' 2>&1); anrc=$? an=$(_tw_clean "$an") if [ "$anrc" -ne 0 ]; then bst_ok libtest-warnings-perl/pattern-still-forbids else bst_notok libtest-warnings-perl/pattern-still-forbids \ "allow_patterns allowed a NON-matching warning -- the allow-list is not discriminating: $(_bst_1line "$an")"; fi # ---------- 4c. THE TWO PLAN FORMS, measured on this boot ------------------- # 36-libdev-checklib-perl.test isolated linux-in-the-browser#93 to ONE of # Test::More's two plan forms. Both are measured here, separately, so section 5 # is attributed from evidence taken on the SAME interpreter on the SAME boot # rather than from a citation -- and so that the day #93 lands, this suite says # so by itself instead of continuing to blame it. pt=$(perl -MTest::More -e 'plan tests => 1; ok(1, "ab");' 2>&1); ptrc=$? pt=$(_tw_clean "$pt") case "$pt" in *"panic:"*) bst_notok libtest-warnings-perl/plan-tests-no-panic \ "plan tests => N panics on this boot, so the criterion cannot pass either -- $(_bst_1line "$pt")" ;; *"ok 1"*) if [ "$ptrc" -eq 0 ]; then bst_ok libtest-warnings-perl/plan-tests-no-panic else bst_notok libtest-warnings-perl/plan-tests-no-panic "printed ok but exited $ptrc -- $(_bst_1line "$pt")"; fi ;; *) bst_notok libtest-warnings-perl/plan-tests-no-panic "$(_bst_1line "$pt")" ;; esac # the form t/06-skip-all.t takes UNCONDITIONALLY. Written in the direction that # is true on a normal perl -- it is a statement about the PLATFORM, and it # self-flips green the day #93 closes, exactly as libtest-needs-perl's twelve # reds do. sa=$(perl -MTest::More -e 'plan skip_all => "sa";' 2>&1); sarc=$? sa=$(_tw_clean "$sa") if [ "$sarc" -eq 0 ] && ! printf '%s\n' "$sa" | grep -q '^panic:'; then bst_ok libtest-warnings-perl/plan-skipall-no-panic elif printf '%s\n' "$sa" | grep -q 'panic: magic_killbackrefs'; then bst_notok libtest-warnings-perl/plan-skipall-no-panic \ "rc=$sarc linux-in-the-browser#93 -- Test2's plan-0-SKIP teardown panics on this arm, and t/06-skip-all.t takes that path unconditionally; rc is 0 on x86-64" else bst_notok libtest-warnings-perl/plan-skipall-no-panic "rc=$sarc $(_bst_1line "$sa")" fi # and the same form WITH Test::Warnings loaded, which is what t/06-skip-all.t # really is: the module's END block is installed and then the file skips. Both # arms are needed because a red on the bare form alone leaves open whether this # port's END block is what breaks the skip, and a red here alone would not say # the platform is at fault. sw=$(perl -e 'use Test::More; use Test::Warnings; plan skip_all => "sw";' 2>&1); swrc=$? sw=$(_tw_clean "$sw") case "$sw" in "1..0 # SKIP sw"*) bst_ok libtest-warnings-perl/skipall-with-tw-tap ;; *) bst_notok libtest-warnings-perl/skipall-with-tw-tap \ "want '1..0 # SKIP sw', got '$(_bst_1line "$sw")' -- the TAP itself is wrong, which WOULD be this port's" ;; esac if [ "$swrc" -eq 0 ]; then bst_ok libtest-warnings-perl/skipall-with-tw-rc elif printf '%s\n' "$sw" | grep -q 'panic: magic_killbackrefs'; then bst_notok libtest-warnings-perl/skipall-with-tw-rc \ "rc=$swrc -- the SKIP printed correctly and perl then panicked in teardown: linux-in-the-browser#93; this is exactly t/06-skip-all.t and rc is 0 on x86-64" else bst_notok libtest-warnings-perl/skipall-with-tw-rc "rc=$swrc want 0 -- $(_bst_1line "$sw")" fi # ------------------------- 5. THE CRITERION: prove -r t/ --------------------- # Run from a WRITABLE COPY, not in place: the deployed rootfs is read-only at # browser runtime. CWD is the directory that HOLDS t/, which is also how # run-ptest invokes it -- t/00-report-prereqs.t reads t/00-report-prereqs.dd by # relative path and the four Test::Tester files load nothing from t/lib, so the # working directory is load-bearing for one file and harmless for the rest. WD=${TMPDIR:-/tmp}/bst-libtest-warnings-perl.$$ PW=$WD/ptest mkdir -p "$PW" && cp -r "$PT/t" "$PW/t" 2>/dev/null check libtest-warnings-perl/suite-staged test -f "$PW/t/01-basic.t" check libtest-warnings-perl/suite-staged-lib test -f "$PW/t/lib/SilenceStderr.pm" # THE HARNESS ITSELF, BEFORE THE CRITERION THAT RIDES ON IT. `prove` resolving # and parsing is not the same as `prove` running: TAP::Harness names its five # collaborators as bare strings and loads them at run time, so poky's # use-scanning RDEPENDS generator never saw them and an image can carry a # /usr/bin/prove that dies on first invocation. Asserted on a trivial # self-contained test so a broken harness reds HERE, by name, instead of turning # the cases below into an unreadable heap. A .t file is a PROGRAM prove executes, # not a TAP transcript it reads -- so the smoke file is perl that PRINTS the # plan, not the plan itself. cat > "$PW/smoke.t" <<'SMOKE' print "1..1\nok 1 - smoke\n"; SMOKE smoke=$(cd "$PW" && prove smoke.t 2>&1); srn=$? smoke=$(_tw_clean "$smoke") case "$smoke" in *"Result: PASS"*) bst_ok libtest-warnings-perl/prove-smoke ;; *) bst_notok libtest-warnings-perl/prove-smoke \ "rc=$srn prove cannot run a 1-line passing test -- $(_bst_1line "$smoke")" ;; esac rm -f "$PW/smoke.t" cout=$(cd "$PW" && prove -r t/ 2>&1); crc=$? cout=$(_tw_clean "$cout") # `Result: PASS` is prove's own verdict over the whole run and is the ticket's # "reporting zero failures" verbatim; the exit status is asserted beside it # because a harness that died before printing a verdict leaves neither. The red # carries its ATTRIBUTION rather than just the transcript. case "$cout" in *"Result: PASS"*) bst_ok libtest-warnings-perl/prove-result ;; *) if printf '%s\n' "$cout" | grep -q 'panic: magic_killbackrefs'; then why="linux-in-the-browser#93 -- perl's magic_killbackrefs use-after-free in Test2 teardown, reached through t/06-skip-all.t's unconditional plan-0-SKIP; the identical tree is Result: PASS on x86-64" elif printf '%s\n' "$cout" | grep -q 'panic: '; then why="a perl interpreter panic that is NOT #93's magic_killbackrefs -- read perl-no-panic's detail" else why="no interpreter panic -- this is a genuine test failure and is THIS PORT'S to answer" fi bst_notok libtest-warnings-perl/prove-result "rc=$crc $(_bst_1line "$cout") -- $why" ;; esac check_eq libtest-warnings-perl/prove-rc 0 "$crc" # The file count is pinned EXACTLY -- it is a property of the package, already # asserted on the rootfs in section 1, and a harness that silently ran fewer # files is the failure a green verdict cannot show. files=$(printf '%s\n' "$cout" | sed -n 's/^Files=\([0-9]*\),.*/\1/p' | tail -1) check_eq libtest-warnings-perl/prove-files 27 "$files" # The assertion count is bounded from BELOW, not pinned. 83 is what the x86-64 # control ran with NOTHING skipped; bounding from below keeps a future 0.034 # with an extra case from reading as a regression, while still catching a file # that died part-way -- and, more to the point here, catching the four # Test::Tester files skipping themselves, which would cost 46 of the 83. tests=$(printf '%s\n' "$cout" | sed -n 's/^Files=[0-9]*, Tests=\([0-9]*\),.*/\1/p' | tail -1) case "$tests" in ''|*[!0-9]*) bst_notok libtest-warnings-perl/prove-tests "no Tests= total in prove's summary -- $(_bst_1line "$cout")" ;; *) if [ "$tests" -ge 83 ]; then bst_ok libtest-warnings-perl/prove-tests else bst_notok libtest-warnings-perl/prove-tests "Tests=$tests, want >= 83 (x86-64 control ran 83 with nothing skipped)"; fi ;; esac # THE FOUR TEST::TESTER FILES, PINNED BY NAME AND BY NOT-SKIPPED. A skip and a # pass are both green to prove, so without this the false green described in the # header -- four files silently inert, 46 assertions gone, Result: PASS over the # remaining 23 -- is indistinguishable from success. This is the single most # load-bearing group in the suite. for t in 01-basic 21-fail-on-warning 23-report-warnings 24-only-report-warnings; do line=$(printf '%s\n' "$cout" | grep -m1 "^t/$t\.t ") case "$line" in '') bst_notok "libtest-warnings-perl/ran-$t" "t/$t.t has no line in prove's transcript -- it did not run at all" ;; *skipped*) bst_notok "libtest-warnings-perl/ran-$t" "t/$t.t SKIPPED -- Test::Tester was not found, so 46 of the criterion's 83 assertions did not run and prove can still say PASS: $(_bst_1line "$line")" ;; *ok*) bst_ok "libtest-warnings-perl/ran-$t" ;; *) bst_notok "libtest-warnings-perl/ran-$t" "$(_bst_1line "$line") -- see prove-result and perl-no-panic for the attribution" ;; esac done # t/06-skip-all.t is EXPECTED to skip -- that is what the file does by design -- # so it is asserted differently from the four above: it must appear, and it must # skip rather than fail on an assertion. A red here means the skip did not even # print, which would be this port's. sl=$(printf '%s\n' "$cout" | grep -m1 '^t/06-skip-all\.t ') case "$sl" in '') bst_notok libtest-warnings-perl/ran-06-skip-all "t/06-skip-all.t has no line in prove's transcript -- it did not run at all" ;; *skipped*) bst_ok libtest-warnings-perl/ran-06-skip-all ;; *ok*) bst_ok libtest-warnings-perl/ran-06-skip-all ;; *) bst_notok libtest-warnings-perl/ran-06-skip-all \ "$(_bst_1line "$sl") -- t/06-skip-all.t skips unconditionally, so a non-skip line here is the plan-0-SKIP teardown (linux-in-the-browser#93), not an assertion failure" ;; esac # WHICH FILES FAILED, BY NAME. Without this a non-zero prove is one red carrying # 100 characters of transcript, and the file that actually broke is whichever one # happened to print first. prove's Test Summary Report names each on a line of # the form `t/.t (Wstat: ...)`, so the names are recoverable and belong in # the report rather than in a later reader's guess. bad=$(printf '%s\n' "$cout" | sed -n 's/^\(t\/[A-Za-z0-9_.-]*\.t\) *(Wstat:.*/\1/p' | sort | tr '\n' ' ') if [ -z "$bad" ]; then bst_ok libtest-warnings-perl/prove-failing-files else bst_notok libtest-warnings-perl/prove-failing-files "failing files: $bad"; fi # HOW MANY ASSERTIONS ACTUALLY FAILED, as distinct from how the run ENDED. This # is the case that separates the two things #93 confuses: a run where every file # reports `Failed: 0` and prove still says FAIL because a child exited non-zero # in teardown is a PLATFORM result, and a run with a non-zero Failed count is # THIS PORT'S. fail=$(printf '%s\n' "$cout" | sed -n 's/.*Failed \([0-9]*\)\/[0-9]* subtests.*/\1/p' | tr '\n' ' ') if [ -z "$fail" ]; then bst_ok libtest-warnings-perl/prove-no-failed-subtests else bst_notok libtest-warnings-perl/prove-no-failed-subtests \ "prove reported failed subtests ($fail) -- these are assertion failures, not a teardown exit status"; fi # A perl INTERPRETER PANIC is not a test failure and must not be reported as one. # perl prints `panic: ` from its own guts when an internal invariant # breaks; on this arm that is a platform fact about perl-on-wasm32, not a # statement about Test::Warnings. Named separately so it is attributable instead # of being buried in whichever red quoted the transcript first. pan=$(printf '%s\n' "$cout" | grep -m1 'panic: ') if [ -n "$pan" ]; then case "$pan" in *magic_killbackrefs*) pwhy="linux-in-the-browser#93 -- the magic_killbackrefs use-after-free this arm already carries a warnings.pm workaround for" ;; *) pwhy="an interpreter panic #93 does not cover -- unattributed, and worth a brintos.io issue" ;; esac bst_notok libtest-warnings-perl/perl-no-panic "$(_bst_1line "$pan") -- $pwhy" else bst_ok libtest-warnings-perl/perl-no-panic fi # THE LOCALE PREAMBLE, REPORTED RATHER THAN ONLY STRIPPED. _tw_clean above # removes it from every detail field in this file, which is what every brintOS # Perl selftest does -- and that private, repeated workaround is precisely why # nothing tracked the defect until BRINTOS-1323 filed # linux-in-the-browser#179. This case makes the strip visible: it reports # whether the preamble is still being emitted, so the day #179 lands the suite # says so instead of silently continuing to filter it. lp=$(perl -e 'print "X\n"' 2>&1) case "$lp" in *"Setting locale failed"*) bst_notok libtest-warnings-perl/no-locale-preamble \ "linux-in-the-browser#179 -- perl prints a locale-warning preamble on stderr on EVERY invocation although setlocale() succeeds; every Perl suite here strips it by hand" ;; *) bst_ok libtest-warnings-perl/no-locale-preamble ;; esac rm -rf "$WD"