. "$BST_LIB" # BST-TIMEOUT: 1200 # ONE python3 start, and that is the whole cost. Every # # assertion below either reads the filesystem/dpkg with # # shell builtins or greps the single probe run's output, so # # the bound is one interpreter load plus a parse of a # # 16-line document. It matches 99-python3-flit-core's 1200 # # rather than 99-sip-module's 600 because the start # # dominates either way, and the parse itself is # # microseconds. NOT MEASURED ON TARGET YET -- this port's # # first on-target run is the batch verify, and the measured # # elapsed replaces this note in the test plan rather than in # # this comment, which would cost an image build to keep the # # shipped suite byte-identical. # # python3-tomli 2.0.1 (poky meta/recipes-devtools/python, python3-tomli_2.0.1.bb, # `inherit pypi python_flit_core`) -- A TOML 1.0.0 PARSER, on the wasm32-hwjs # glibc image. # # ============================================================================ # WHY THIS PORT IS POSSIBLE AT ALL, AND WHAT THE .so CASE IS REALLY ASSERTING. # ============================================================================ # # This image's python3 is ONE STATIC BINARY with an EMPTY lib-dynload and no # dynamic loader (BRINTOS-1476; 99-python3 asserts both directly). A module that # carried a compiled extension therefore could not be imported here -- not # "would be slow", could not be imported. tomli is pure Python, which is the # entire reason it ports. # # So `module-no-so` is not boilerplate. It is a REGRESSION TRIPWIRE on a version # bump: tomli grows no extension today, and the day a release adds a compiled # speedup this case goes red at INSTALL time rather than at `import tomli` on # somebody else's ticket three rounds later. That is not a hypothetical for THIS # package -- tomli's own tracker carries a standing request for a C accelerator, # and its successor in the stdlib (`tomllib`) is the same code precisely because # it stayed pure. The case is anchored on the package's own dpkg file list as # well as on a find, because the two fail differently: a find catches a file the # package does not declare, the list catches a declared file that is not a .py. # # ============================================================================ # WHAT THE PARSE CASES ASSERT, AND WHY THE COMPARISON IS NOT DONE IN SHELL. # ============================================================================ # # data/python3-tomli-probe.py writes a TOML document covering the types TOML # 1.0.0 separates, parses it, and compares the result against THE PYTHON LITERAL # THAT DOCUMENT DENOTES -- the two sitting side by side in the probe, as a # matched pair. Shell can neither build that expectation nor compare it, so the # probe does the comparison and this suite asserts on the verdicts: one per TOML # type, so a red names the type that mis-parsed rather than reporting the whole # document as unequal. # # EVERY VALUE ASSERTED BELOW WAS MEASURED, on x86-64, against THIS EXACT PROBE # and tomli 2.0.1 -- the same version this image's recipe pins (recorded # 2026-08-31, and kept in the ticket's design doc). They are not predictions and # they are not read off the source. What makes the control transferable is that # all of it is pure-Python and content-determined: a TOML parse maps text to # str/int/float/bool/datetime/date/list/dict, and none of that depends on # pointer width, byte order or the platform string. Nothing arch-dependent is # asserted anywhere in this file. # # version=2.0.1 doc_equal=yes load_matches_loads=yes # type_*=yes (all ten) invalid_raises=yes dup_key_raises=yes # pytype_local_date=date pytype_offset_datetime=datetime # offset_dt_iso=1979-05-27T07:32:00+00:00 local_date_iso=1979-05-27 # # THE DATE/DATETIME PAIR IS THE ONE WORTH READING TWICE. TOML 1.0.0 distinguishes # an OFFSET DATE-TIME from a bare LOCAL DATE, and tomli returns a tz-aware # `datetime` for the first and a plain `date` for the second. The per-type cases # compare the Python TYPE as well as the value, because `1 == True` and # `1.0 == 1` in Python -- a bool parsed as an int, or a date widened to a # midnight datetime, would satisfy a value-only comparison and is exactly the # class of bug a TOML parser gets wrong. # # ============================================================================ # WHAT IS NOT COVERED, ON PURPOSE. # ============================================================================ # # - WRITING TOML. tomli is a parser; the writer is `tomli-w`, a separate recipe # (meta-python) that is not on this image. # - THE FULL TOML 1.0.0 COMPLIANCE CORPUS. That is toml-test, thousands of # documents, and it is upstream's job -- running it here would cost minutes # of guest time to re-answer a question upstream CI answers per release. This # suite asserts the type coverage the ticket names plus the two error paths, # which is what says the port LANDED, not that tomli is correct. # - `tomllib`. This interpreter is 3.12 and DOES carry tomllib in its stdlib; # that is not what this ticket ports, and asserting the two agree would be a # statement about the interpreter rather than about this package. # # THIS PORT CLOSES ON THIS SUITE: non-interactive, no manual browser pass, no # follow-on ticket under BRINTOS-780. P=python3-tomli # THE CANARY, which asserts nothing about python3-tomli. # # A suite's FIRST BST result line is lost between the guest's printf and the # host's read -- per-suite, deterministic, and NOT a defect in the host tool # (BRINTOS-1149 established that and filed it as linux-in-the-browser#177). The # harness computes its pass count as total-minus-failures, so THE DROPPED LINE # IS COUNTED AS A PASS. A nothing-case therefore stands in that position and the # eaten line costs nothing. It sits ahead of the `require` guards deliberately, # because a `require` that skips would emit its skip as the first line and be # eaten too. # # Remove this case when linux-in-the-browser#177 is fixed. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages TM=$SITE/tomli PKGLIST=/var/lib/dpkg/info/python3-tomli.list D=$BST_DATA PROBE=$D/python3-tomli-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-tomli.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------ the payload --- # Where the module has to land. `python3 -c 'import tomli'` further down is the # functional statement; these are the ones that say WHICH copy answered, so a # module that imported out of some other path is distinguishable from one that # imported out of the package this image installed. check $P/site-packages-dir test -d "$SITE" check $P/module-dir test -d "$TM" check $P/module-init test -f "$TM/__init__.py" # The two source files tomli's public surface is made of. Named individually # rather than counted: a count moves on any upstream refactor, while the parser # and the re-based scanner are the split tomli has documented since 2.0. check $P/module-parser test -f "$TM/_parser.py" check $P/module-re test -f "$TM/_re.py" # PEP 561: tomli ships a type marker, and a package that lost it stops type- # checking for every consumer without failing a single import. check $P/module-typed test -f "$TM/py.typed" # NO COMPILED EXTENSION, ANYWHERE UNDER THE MODULE. See the header: on this image # a .so could not be loaded at all, so this is the case that fails a version bump # which grew one -- loudly, here, instead of at `import` on a later ticket. check_eq $P/module-no-so "" \ "$(find "$TM" -name '*.so' -o -name '*.so.*' 2>/dev/null)" # ...and the same assertion from the PACKAGE's side rather than the filesystem's. # A find sees what is on disk; the dpkg list sees what the package CLAIMS. A # compiled extension that arrived without being declared shows up in the first, # one that is declared but landed elsewhere shows up in the second. # # The list is copied to tmpfs first and never redirected from the rootfs: a # top-level stdin redirect on the virtiofs root makes dup2(2) return 513 and ash # EXITS, ending the suite silently (brintos/linux#73). cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # EVERY REGULAR FILE THE PACKAGE SHIPS UNDER site-packages IS PYTHON OR ITS # METADATA. The two cases above name the failure that is expected; this one # catches the shape nobody predicted -- a binary, a .pyd, a shared archive -- # without having to enumerate what it might be. # # A dpkg .list names DIRECTORIES too, and without a trailing slash, so a # name-only filter would report every directory in the package as an unexpected # shape. Each entry is therefore tested for being a regular file first. Measured # against the built .deb: tomli 2.0.1 ships only .py files, py.typed and its # dist-info metadata, so unlike flit_core this needs no by-name allowance. _bad="" while IFS= read -r _f; do case "$_f" in "$SITE"/*) ;; *) continue ;; esac [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc|*.typed) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" # --------------------------------------------------------- the package ------ # dpkg KNOWS the package. Every case above would look identical if some other # package had installed the same files, so this is the one that says # python3-tomli is a real package this image installed rather than an # IMAGE_INSTALL line that resolved to nothing. # # The stanza is read from a tmpfs copy for the same reason as the file list. The # anchor is exact: `python3-tomli` is a PREFIX of `python3-tomli-w`, a DIFFERENT # recipe (the TOML writer, meta-python), and an unanchored match would report one # stanza as the other. sed -n '/^Package: python3-tomli$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" # ---------------------------------------------------------- the interpreter -- # THE IMPORT, ON ITS OWN, BEFORE THE PROBE. If tomli cannot be imported then # every probe case below reds at once and each names a different symptom; this # case is what says the cause is the import. It is also the cheapest possible # statement of the deliverable. check $P/import python3 -c 'import tomli' # ------------------------------------------------------------- the probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present # ONE interpreter start for everything that follows. bst_probe bounds it and # keeps BOTH streams: a probe that dies early otherwise leaves every case below # reporting got:'' and naming a symptom each, with the actual traceback nowhere. # The bound is generous because a python3 start on this arm is tens of seconds # before any work begins. bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" "$W" OUT="$W/probe" # Read a single key. The probe emits each key at most once, so head -n 1 is # defensive rather than load-bearing; the anchors keep `type_local_date` and # `local_date_iso` distinguishable. pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } # THE EVIDENCE PLANE. If the probe emitted nothing, every case below is reading # an absent file: each would red with got:'' and the set would read exactly like # a broken parser, when what actually happened is that no measurement was taken. # The word `unscorable` is PARSED -- brintos_local_verify re-measures a suite # that prints it on a fresh guest rather than trusting a shared boot's numbers # (BRINTOS-1644) -- so it goes in this structural check, not in a comment. # # The cases below still print their reds: withdraw, do not hide. This case is # what says what those reds measure. if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # THE VERSION THE RECIPE PINS, read from the module rather than from dpkg. A # mismatch here means the recipe and what actually landed have gone out of step, # and every expectation in this file is then describing a different tomli. check_eq $P/version "2.0.1" "$(pv version)" # The module answered out of the package this image installed, not from a stray # copy on sys.path. This is what makes the payload cases above and the behaviour # cases below statements about the SAME tomli. check_eq $P/module-file "$TM/__init__.py" "$(pv module_file)" # tomli's documented public surface, in full. A parser missing TOMLDecodeError is # not merely incomplete -- the error path below has nothing to assert against. check_eq $P/api-loads "yes" "$(pv has_loads)" check_eq $P/api-load "yes" "$(pv has_load)" check_eq $P/api-decode-error "yes" "$(pv has_TOMLDecodeError)" check_eq $P/api-decode-error-is-exception "yes" "$(pv decode_error_is_exception)" # The probe could write its fixture. Separated from the parse so a read-only or # full tmpfs is not reported as a parser failure. check_eq $P/fixture "yes" "$(pv fixture_ok)" # ------------------------------------------------------------- the parse ----- # THE DELIVERABLE. loads() returned, and what it returned EQUALS the dict the # fixture denotes -- the whole document, compared as one structure. check_eq $P/loads "yes" "$(pv loads_ok)" check_eq $P/doc-equal "yes" "$(pv doc_equal)" # The key SET, asserted separately from the equality above. doc-equal already # covers it, but when doc-equal reds this is the case that says whether the # damage was a missing/extra key or a wrong value under a right key. check_eq $P/top-level-keys \ "bare_local_date,count,enabled,inline,items,letters,offset_dt,owner,ratio,title" \ "$(pv top_level_keys)" # ------------------------------------------------- one case per TOML 1.0.0 type # Each compares one key against its literal AND the Python type against the # expected type -- see the header for why the type half is load-bearing. The # pytype_* case beside each verdict is what names the wrong type when the # verdict reds, so the pair reads as "wrong, and here is what it became". check_eq $P/type-string "yes" "$(pv type_string)" check_eq $P/pytype-string "str" "$(pv pytype_string)" check_eq $P/type-integer "yes" "$(pv type_integer)" check_eq $P/pytype-integer "int" "$(pv pytype_integer)" check_eq $P/type-float "yes" "$(pv type_float)" check_eq $P/pytype-float "float" "$(pv pytype_float)" check_eq $P/type-boolean "yes" "$(pv type_boolean)" check_eq $P/pytype-boolean "bool" "$(pv pytype_boolean)" check_eq $P/type-offset-datetime "yes" "$(pv type_offset_datetime)" check_eq $P/pytype-offset-datetime "datetime" "$(pv pytype_offset_datetime)" check_eq $P/type-local-date "yes" "$(pv type_local_date)" check_eq $P/pytype-local-date "date" "$(pv pytype_local_date)" check_eq $P/type-array "yes" "$(pv type_array)" check_eq $P/pytype-array "list" "$(pv pytype_array)" check_eq $P/type-inline-table "yes" "$(pv type_inline_table)" check_eq $P/pytype-inline-table "dict" "$(pv pytype_inline_table)" check_eq $P/type-table "yes" "$(pv type_table)" check_eq $P/pytype-table "dict" "$(pv pytype_table)" check_eq $P/type-array-of-tables "yes" "$(pv type_array_of_tables)" check_eq $P/pytype-array-of-tables "list" "$(pv pytype_array_of_tables)" # THE DATE/DATETIME DISTINCTION, stated as values rather than as verdicts. The # offset date-time carries UTC and round-trips to the offset form; the local date # round-trips to a bare date with no time and no zone. A parser that collapsed # the two would show it here in the ISO strings even if a type check somehow # passed. check_eq $P/offset-dt-utcoffset "0:00:00" "$(pv offset_dt_utcoffset)" check_eq $P/offset-dt-iso "1979-05-27T07:32:00+00:00" "$(pv offset_dt_iso)" check_eq $P/local-date-iso "1979-05-27" "$(pv local_date_iso)" # ------------------------------------------------------- load() from a file --- # The entry point every PEP 517 frontend actually calls on pyproject.toml, and # the only one that touches the filesystem -- so a red here with loads() green is # an I/O or encoding fault rather than a parser fault. check_eq $P/load "yes" "$(pv load_ok)" check_eq $P/load-matches-loads "yes" "$(pv load_matches_loads)" # tomli REFUSES a text handle by design: it decodes UTF-8 itself, and that is how # it guarantees the decode. Asserted so a future relaxation is noticed here # rather than by a consumer. check_eq $P/load-rejects-text-handle "yes" "$(pv load_text_handle_refused)" # ----------------------------------------------------- the invalid document --- # THE TICKET'S OTHER REQUIREMENT. An invalid document must raise TOMLDecodeError # -- not return a partial dict, and not raise something generic. The two cases # split "did not raise" from "raised the wrong class", which a single boolean # reports identically. check_eq $P/invalid-raises "yes" "$(pv invalid_raises)" check_eq $P/invalid-exc "TOMLDecodeError" "$(pv invalid_exc)" # The diagnostic is not empty. A parser that raised the right class with no # message is useless to the frontend that has to report WHERE the document is # wrong. check_eq $P/invalid-msg "yes" "$(pv invalid_msg_nonempty)" # A SECOND malformed document, wrong a different way: the first is lexical (an # unterminated string), this one structural (a duplicated key, which TOML 1.0.0 # makes an error). A parser that merely overwrote would pass every case above. check_eq $P/dup-key-raises "yes" "$(pv dup_key_raises)" # The probe ran to the end. Every case above greps a key, so a probe that died # halfway would leave the remaining keys absent and each case would red with # got:'' -- this case is the one that says the cause was a truncated run. check_eq $P/probe-complete "yes" "$(pv done)" rm -rf "$W"