. "$BST_LIB" # BST-TIMEOUT: 1200 # ONE python3 start, and that is the whole cost. Every # # assertion below either reads the filesystem/dpkg with # # shell builtins or greps the single probe run's output, so # # the bound is one interpreter load plus a few dozen # # microseconds of list building. It matches # # 99-python3-tomli's 1200 rather than 99-sip-module's 600 # # because the start dominates either way. NOT MEASURED ON # # TARGET YET -- this port's first on-target run is the batch # # verify, and the measured elapsed replaces this note in the # # test plan rather than in this comment, which would cost an # # image build to keep the shipped suite byte-identical. # # python3-more-itertools 10.2.0 (poky meta/recipes-devtools/python, # python3-more-itertools_10.2.0.bb, `inherit pypi python_flit_core ptest`) -- # MORE ROUTINES FOR OPERATING ON ITERABLES, beyond the stdlib `itertools`, on the # wasm32-hwjs glibc image. # # ============================================================================ # WHY THIS PORT IS POSSIBLE AT ALL, AND WHAT THE .so CASE IS REALLY ASSERTING. # ============================================================================ # # This image's python3 is ONE STATIC BINARY with an EMPTY lib-dynload and no # dynamic loader (BRINTOS-1476; 99-python3 asserts both directly). A module that # carried a compiled extension therefore could not be imported here -- not # "would be slow", could not be imported. more-itertools is pure Python, which is # the entire reason it ports. # # So `module-no-so` is not boilerplate. It is a REGRESSION TRIPWIRE on a version # bump: more-itertools grows no extension today, and the day a release adds a # compiled speedup this case goes red at INSTALL time rather than at # `import more_itertools` on somebody else's ticket three rounds later. That is # not idle worry for THIS package -- it is a library of hot inner loops whose own # tracker has carried C-accelerator proposals, and it is one of the most depended- # on packages on PyPI, so the day it accelerates, everything downstream of it here # breaks at once. The case is anchored on the package's own dpkg file list as well # as on a find, because the two fail differently: a find catches a file the package # does not declare, the list catches a declared file that is not Python. # # ============================================================================ # WHAT THE BEHAVIOUR CASES ASSERT, AND WHY THE COMPARISON IS NOT DONE IN SHELL. # ============================================================================ # # data/python3-more-itertools-probe.py runs each routine the ticket names over a # FIXED input and renders the result with repr(), one key=value line each. This # suite compares those lines against the literals. The rendering is repr-based on # purpose: `[1, 2, 3]` and `['1', '2', '3']` are different answers, and a # str()-joined rendering would report them as the same. Shell can build neither # the nested structures nor the comparison, which is why the probe exists at all; # what shell does here is name one case per routine, so a red says WHICH routine # broke rather than reporting "more_itertools is wrong". # # EVERY VALUE ASSERTED BELOW WAS MEASURED, on x86-64, against THIS EXACT PROBE # and more-itertools 10.2.0 -- and then RE-MEASURED against the module extracted # from this image's own python3-more-itertools_10.2.0-r0_wasm32.deb, which agreed # line for line but for the module path (recorded 2026-08-31, in the ticket's # design doc). They are not predictions and they are not read off the source. # What makes the control transferable is that all of it is pure-Python and # content-determined: these routines map lists to lists, and none of that depends # on pointer width, byte order or the platform string. Nothing arch-dependent is # asserted anywhere in this file. # # THE THREE ANSWERS WORTH READING TWICE, because reasoning from one predicts the # others wrongly: # # windowed([1,2,3], 3) -> [(1, 2, 3)] exact fit: ONE window # windowed([1,2], 3) -> [(1, 2, None)] short input: PADDED # windowed([], 3) -> [] empty input: NOTHING, not padding # # `partition` returns (FALSE items, TRUE items) -- in that order, which is the # half of its contract most often mis-remembered, so both halves are asserted. # And `flatten` is ONE level only: the nested list in the fixture survives as a # list, so a change to a recursive flatten reds here. # # ============================================================================ # WHAT IS NOT COVERED, ON PURPOSE. # ============================================================================ # # - THE UPSTREAM TEST SUITE. The recipe inherits `ptest` and poky therefore # builds a python3-more-itertools-ptest package carrying upstream's full # tests/ tree. It is NOT on this image and this suite is not a stand-in for # it: running ~2000 upstream cases here would cost minutes of guest time to # re-answer a question upstream CI answers per release. This suite asserts # the routines the ticket names plus the payload shape, which is what says # the port LANDED, not that more-itertools is correct. # - THE OTHER ~290 ROUTINES. more-itertools is large; the ticket names ten and # those are what is driven against literals. `has_*` covers three more by # presence only, which is enough to catch a half-installed package. # - PERFORMANCE. These are hot loops upstream, but this image's python3 start # dominates any timing this suite could take, so a number here would measure # the interpreter rather than the library. # # THIS PORT CLOSES ON THIS SUITE: non-interactive, no manual browser pass, no # follow-on ticket under BRINTOS-780. P=python3-more-itertools # THE CANARY, which asserts nothing about python3-more-itertools. # # A suite's FIRST BST result line is lost between the guest's printf and the # host's read -- per-suite, deterministic, and NOT a defect in the host tool # (BRINTOS-1149 established that and filed it as linux-in-the-browser#177). The # harness computes its pass count as total-minus-failures, so THE DROPPED LINE # IS COUNTED AS A PASS. A nothing-case therefore stands in that position and the # eaten line costs nothing. It sits ahead of the `require` guards deliberately, # because a `require` that skips would emit its skip as the first line and be # eaten too. # # Remove this case when linux-in-the-browser#177 is fixed. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MI=$SITE/more_itertools PKGLIST=/var/lib/dpkg/info/python3-more-itertools.list D=$BST_DATA PROBE=$D/python3-more-itertools-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-more-itertools.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------ the payload --- # Where the module has to land. `python3 -c 'import more_itertools'` further down # is the functional statement; these are the ones that say WHICH copy answered, # so a module that imported out of some other path is distinguishable from one # that imported out of the package this image installed. check $P/site-packages-dir test -d "$SITE" check $P/module-dir test -d "$MI" check $P/module-init test -f "$MI/__init__.py" # The two source modules the package is made of, named individually rather than # counted: a count moves on any upstream refactor, while the more/recipes split # is the structure more-itertools has documented since 4.0 -- `recipes` being the # itertools-docs recipes and `more` the library's own additions. check $P/module-more test -f "$MI/more.py" check $P/module-recipes test -f "$MI/recipes.py" # PEP 561: more-itertools ships a type marker AND separate stub files, and a # package that lost either stops type-checking for every consumer without # failing a single import. check $P/module-typed test -f "$MI/py.typed" check $P/module-stub-more test -f "$MI/more.pyi" # NO COMPILED EXTENSION, ANYWHERE UNDER THE MODULE. See the header: on this image # a .so could not be loaded at all, so this is the case that fails a version bump # which grew one -- loudly, here, instead of at `import` on a later ticket. check_eq $P/module-no-so "" \ "$(find "$MI" -name '*.so' -o -name '*.so.*' 2>/dev/null)" # ...and the same assertion from the PACKAGE's side rather than the filesystem's. # A find sees what is on disk; the dpkg list sees what the package CLAIMS. A # compiled extension that arrived without being declared shows up in the first, # one that is declared but landed elsewhere shows up in the second. # # The list is copied to tmpfs first and never redirected from the rootfs: a # top-level stdin redirect on the virtiofs root makes dup2(2) return 513 and ash # EXITS, ending the suite silently (brintos/linux#73). cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # EVERY REGULAR FILE THE PACKAGE SHIPS UNDER site-packages IS PYTHON OR ITS # METADATA. The two cases above name the failure that is expected; this one # catches the shape nobody predicted -- a binary, a .pyd, a shared archive -- # without having to enumerate what it might be. # # A dpkg .list names DIRECTORIES too, and without a trailing slash, so a # name-only filter would report every directory in the package as an unexpected # shape. Each entry is therefore tested for being a regular file first. # # `*.pyi` IS LISTED SEPARATELY FROM `*.py` AND THAT IS NOT REDUNDANT: a `*.py` # glob does not match `more.pyi`. Measured against the built .deb -- 10.2.0 ships # three stub files beside its three sources, plus py.typed and the dist-info # metadata, and nothing else. _bad="" while IFS= read -r _f; do case "$_f" in "$SITE"/*) ;; *) continue ;; esac [ -f "$_f" ] || continue case "$_f" in *.py|*.pyi|*.pyc|*.typed) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" # THE UPSTREAM TEST TREE IS NOT ON THIS IMAGE. The recipe inherits ptest, so poky # builds a -ptest package carrying upstream's tests/. This image installs only # the runtime package, and a tests/ directory appearing under the module would # mean the wrong package landed. check_eq $P/pkg-no-tests "" \ "$(grep -E "^$SITE/more_itertools/tests" "$W/pkgf" 2>/dev/null)" # --------------------------------------------------------- the package ------ # dpkg KNOWS the package. Every case above would look identical if some other # package had installed the same files, so this is the one that says # python3-more-itertools is a real package this image installed rather than an # IMAGE_INSTALL line that resolved to nothing. The anchor is exact, so the # -dev/-dbg/-ptest stanzas -- whose names all have this one as a prefix -- cannot # be read in its place. sed -n '/^Package: python3-more-itertools$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" # ---------------------------------------------------------- the interpreter -- # THE IMPORT, ON ITS OWN, BEFORE THE PROBE. If more_itertools cannot be imported # then every probe case below reds at once and each names a different symptom; # this case is what says the cause is the import. It is also the cheapest # possible statement of the deliverable. check $P/import python3 -c 'import more_itertools' # ------------------------------------------------------------- the probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present # ONE interpreter start for everything that follows. bst_probe bounds it and # keeps BOTH streams: a probe that dies early otherwise leaves every case below # reporting got:'' and naming a symptom each, with the actual traceback nowhere. # The bound is generous because a python3 start on this arm is tens of seconds # before any work begins. bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" "$W" OUT="$W/probe" # Read a single key. The probe emits each key at most once, so head -n 1 is # defensive rather than load-bearing; the anchors keep `first_empty_raises` and # `first_basic` distinguishable. pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } # THE EVIDENCE PLANE. If the probe emitted nothing, every case below is reading # an absent file: each would red with got:'' and the set would read exactly like # a broken library, when what actually happened is that no measurement was taken. # The word `unscorable` is PARSED -- brintos_local_verify re-measures a suite # that prints it on a fresh guest rather than trusting a shared boot's numbers # (BRINTOS-1644) -- so it goes in this structural check, not in a comment. # # The cases below still print their reds: withdraw, do not hide. This case is # what says what those reds measure. if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # THE VERSION THE RECIPE PINS, read from the module rather than from dpkg. A # mismatch here means the recipe and what actually landed have gone out of step, # and every expectation in this file is then describing a different # more-itertools. check_eq $P/version "10.2.0" "$(pv version)" # The module answered out of the package this image installed, not from a stray # copy on sys.path. This is what makes the payload cases above and the behaviour # cases below statements about the SAME more_itertools. check_eq $P/module-file "$MI/__init__.py" "$(pv module_file)" # Both source modules imported, not merely present on disk. `recipes` is imported # BY `more`, so a package that shipped half of itself fails here rather than at # the first routine that happens to live in the missing half. check_eq $P/submodule-more "yes" "$(pv submodule_more)" check_eq $P/submodule-recipes "yes" "$(pv submodule_recipes)" # The routines the ticket names, by presence. A missing name is a different # failure from a name that computes the wrong answer, and without these the # behaviour cases below would report both identically. check_eq $P/api-chunked "yes" "$(pv has_chunked)" check_eq $P/api-flatten "yes" "$(pv has_flatten)" check_eq $P/api-unique-everseen "yes" "$(pv has_unique_everseen)" check_eq $P/api-partition "yes" "$(pv has_partition)" check_eq $P/api-first "yes" "$(pv has_first)" check_eq $P/api-last "yes" "$(pv has_last)" check_eq $P/api-windowed "yes" "$(pv has_windowed)" check_eq $P/api-peekable "yes" "$(pv has_peekable)" check_eq $P/api-one "yes" "$(pv has_one)" # Three more re-exported from `recipes`, by presence only -- enough to catch a # package whose top-level __init__ stopped re-exporting that half. check_eq $P/api-spy "yes" "$(pv has_spy)" check_eq $P/api-always-iterable "yes" "$(pv has_always_iterable)" check_eq $P/api-ilen "yes" "$(pv has_ilen)" # ------------------------------------------------------------------ chunked -- # 7 items into chunks of 3, non-divisible ON PURPOSE: the short final chunk is # what a divisible input would hide, and dropping the remainder is the classic # defect. The divisible and empty inputs sit beside it so a red says which shape # broke. check_eq $P/chunked "[[1, 2, 3], [4, 5, 6], [7]]" "$(pv chunked_7_3)" check_eq $P/chunked-exact "[[1, 2, 3], [4, 5, 6]]" "$(pv chunked_6_3)" check_eq $P/chunked-empty "[]" "$(pv chunked_empty)" # LAZY, not eager. A version that returned a list would satisfy every value case # above while breaking every consumer that feeds it an unbounded iterable. check_eq $P/chunked-lazy "yes" "$(pv chunked_is_iterator)" # strict=True turns the short final chunk from a feature into an error -- the # guard a caller uses when a ragged tail means the input was wrong. check_eq $P/chunked-strict-raises "yes" "$(pv chunked_strict_raises)" check_eq $P/chunked-strict-exc "ValueError" "$(pv chunked_strict_exc)" check_eq $P/chunked-strict-ok "[[1, 2], [3, 4]]" "$(pv chunked_strict_exact)" # ------------------------------------------------------------------ flatten -- # ONE LEVEL, which is flatten's whole contract: the nested list in the second # fixture must survive AS A LIST. A recursive flatten would pass the first case # and red on the second, which is why both are here. check_eq $P/flatten "[1, 2, 3, 4, 5]" "$(pv flatten_basic)" check_eq $P/flatten-one-level "[1, [2, 3], 4]" "$(pv flatten_one_level)" check_eq $P/flatten-strings "['a', 'b', 'c']" "$(pv flatten_strings)" check_eq $P/flatten-empty "[]" "$(pv flatten_empty)" # ---------------------------------------------------------- unique_everseen -- # Order-preserving de-duplication over a string with runs AND A REPEAT AFTER A # GAP (the trailing A and B runs), which is exactly what separates # unique_everseen from unique_justseen -- the latter would keep those repeats. check_eq $P/unique-everseen "['A', 'B', 'C', 'D']" "$(pv unique_everseen_str)" # With a key: 'c' collapses into 'C', and the FIRST-seen element is the one kept. check_eq $P/unique-everseen-key "['A', 'B', 'C', 'D']" "$(pv unique_everseen_key)" # An unhashable element takes the seen-list fallback rather than raising -- the # path a set-based implementation would not have. check_eq $P/unique-everseen-unhashable "[[1], [2]]" "$(pv unique_everseen_unhashable)" # -------------------------------------------------------------- partition ---- # (FALSE items, TRUE items) -- IN THAT ORDER. Both halves are asserted because # swapping them is the one defect here that a single-sided test cannot see. check_eq $P/partition-false "[1, 3, 5, 7, 9]" "$(pv partition_false)" check_eq $P/partition-true "[0, 2, 4, 6, 8]" "$(pv partition_true)" check_eq $P/partition-str-false "['apple', 'cherry']" "$(pv partition_str_false)" check_eq $P/partition-str-true "['banana', 'berry']" "$(pv partition_str_true)" # ------------------------------------------------------------ first / last --- check_eq $P/first "1" "$(pv first_basic)" check_eq $P/last "3" "$(pv last_basic)" # WITH a default on an empty iterable -- the case the ticket names. The repr # keeps the quotes, so a default silently coerced to another type would show. check_eq $P/first-default "'fallback'" "$(pv first_default)" check_eq $P/last-default "'fallback'" "$(pv last_default)" # WITHOUT a default: this must RAISE rather than return None, because a caller # that omitted the default asked to be told. Class asserted separately from the # fact of raising -- "did not raise" and "raised the wrong class" are different # defects that one boolean reports identically. check_eq $P/first-empty-raises "yes" "$(pv first_empty_raises)" check_eq $P/first-empty-exc "ValueError" "$(pv first_empty_exc)" check_eq $P/last-empty-raises "yes" "$(pv last_empty_raises)" check_eq $P/last-empty-exc "ValueError" "$(pv last_empty_exc)" # Over a ONE-SHOT ITERATOR rather than a sequence: last() over an iterator is the # case with a real implementation behind it, since it cannot simply index. check_eq $P/first-iterator "7" "$(pv first_iterator)" check_eq $P/last-iterator "9" "$(pv last_iterator)" # ------------------------------------------------------------------ windowed - # THE BOUNDARY the ticket names, in its three distinct shapes -- see the header. # Reasoning from any one of these predicts the wrong answer for the others. check_eq $P/windowed "[(1, 2, 3), (2, 3, 4)]" "$(pv windowed_4_3)" check_eq $P/windowed-exact "[(1, 2, 3)]" "$(pv windowed_exact)" check_eq $P/windowed-short "[(1, 2, None)]" "$(pv windowed_short)" check_eq $P/windowed-empty "[]" "$(pv windowed_empty)" check_eq $P/windowed-fillvalue "[(1, 2, 0)]" "$(pv windowed_fillvalue)" # With a step, the final window is padded too -- the two features compose. check_eq $P/windowed-step "[(1, 2), (3, 4), (5, None)]" "$(pv windowed_step)" # ------------------------------------------------------------------ peekable - # ONE peekable, driven through peek -> next -> prepend -> drain, so what is # measured is the STATE AFTER EACH OPERATION and not four independent objects. # peek must not consume: `peek-twice` and `next-after-peek` all returning 1 is # what says so. check_eq $P/peekable-peek "1" "$(pv peekable_peek)" check_eq $P/peekable-peek-twice "1" "$(pv peekable_peek_twice)" check_eq $P/peekable-next-after-peek "1" "$(pv peekable_next_after_peek)" # prepend puts an item in FRONT of a stream already partly consumed -- the case # the ticket names -- and what follows is the untouched remainder. check_eq $P/peekable-prepend "0" "$(pv peekable_after_prepend)" check_eq $P/peekable-rest "[2, 3]" "$(pv peekable_rest)" # prepend takes several items at once and keeps their order. check_eq $P/peekable-prepend-many "[1, 2, 3]" "$(pv peekable_prepend_many)" # Truthiness is peekable's documented way to ask "is there more" WITHOUT # consuming, and it is False exactly when the stream is exhausted. check_eq $P/peekable-bool-empty "False" "$(pv peekable_bool_empty)" check_eq $P/peekable-bool-nonempty "True" "$(pv peekable_bool_nonempty)" # peek() on an exhausted stream: the default is returned if given, and # StopIteration raised if not. The message is NOT asserted non-empty here -- # measured, StopIteration carries none, and asserting otherwise would red on # correct behaviour. check_eq $P/peekable-peek-default "'none-left'" "$(pv peekable_peek_default)" check_eq $P/peekable-peek-empty-raises "yes" "$(pv peekable_peek_empty_raises)" check_eq $P/peekable-peek-empty-exc "StopIteration" "$(pv peekable_peek_empty_exc)" # ----------------------------------------------------------------------- one - # THE DOCUMENTED EXCEPTION the ticket names. one() returns the single element, # and raises when there is more than one -- and, for a different reason, when # there is none. Both are exercised, and the message is asserted non-empty # because a caller reports it. check_eq $P/one "1" "$(pv one_single)" check_eq $P/one-too-many-raises "yes" "$(pv one_too_many_raises)" check_eq $P/one-too-many-exc "ValueError" "$(pv one_too_many_exc)" check_eq $P/one-too-many-msg "yes" "$(pv one_too_many_msg_nonempty)" check_eq $P/one-empty-raises "yes" "$(pv one_empty_raises)" check_eq $P/one-empty-exc "ValueError" "$(pv one_empty_exc)" # The caller-supplied exception hook: a consumer that wants its own error class # must be able to get it, and this is the only case that says the hook is wired. check_eq $P/one-custom-exc "KeyError" "$(pv one_custom_exc)" # The probe ran to the end. Every case above greps a key, so a probe that died # halfway would leave the remaining keys absent and each case would red with # got:'' -- this case is the one that says the cause was a truncated run. check_eq $P/probe-complete "yes" "$(pv done)" rm -rf "$W"