. "$BST_LIB" # BST-TIMEOUT: 1200 # ONE python3 start, and that is the whole cost. Every # # assertion below either reads the filesystem/dpkg with # # shell builtins or greps the single probe run's output; the # # probe itself parses a 292-byte document four times and # # serialises it once, which is nothing beside the interpreter # # load. It matches 99-python3-iniparse's 1200 for the same # # reason -- the start dominates and the work after it is # # comparable. # # NOT MEASURED ON TARGET YET: this port's first on-target run # # is the batch verify, and the measured elapsed replaces this # # note in the test plan rather than in this comment, which # # would cost an image build to keep the shipped suite # # byte-identical. # # python3-xmltodict 0.13.0 (poky # meta/recipes-devtools/python/python3-xmltodict_0.13.0.bb, `inherit pypi # setuptools3 ptest', carrying the recipe's two CVE-2025-9375 backports) on the # wasm32-hwjs glibc image -- XML AS A dict. `parse' turns a document into nested # dictionaries with attributes prefixed `@' and text under `#text'; `unparse' # turns that structure back into XML. # # ============================================================================ # WHY THIS PORT IS POSSIBLE AT ALL, AND WHAT THE .so CASES ARE REALLY ASSERTING. # ============================================================================ # # This image's python3 is ONE STATIC BINARY with an EMPTY lib-dynload and no # dynamic loader (BRINTOS-1476; 99-python3 asserts both directly). A module # carrying a compiled extension therefore could not be imported here -- not # "would be slow", could not be imported. xmltodict is ONE .py file and ships no # extension of its own, which is the entire reason it ports. # # It does, however, sit ON a C extension: the whole parser is # `xml.parsers.expat'. That is not a package this port installs -- pyexpat is # BUILTIN in this interpreter (99-python3/builtin-modules names it in the set it # asserts), which is exactly why the port works. `pyexpat-builtin' below restates # it from this suite's own side, so an interpreter rebuilt without expat reds HERE # with the cause named rather than presenting as a broken xmltodict. # # So `module-no-so' is not boilerplate. It is a REGRESSION TRIPWIRE on a version # bump: xmltodict grows no extension today, and the day a release adds a compiled # speedup this case goes red at INSTALL time rather than at `import xmltodict' on # somebody else's ticket three rounds later. # # ============================================================================ # IT IS A MODULE, NOT A PACKAGE -- ONE FILE, NO DIRECTORY. # ============================================================================ # # setup.py declares `py_modules=['xmltodict']', so what lands in site-packages is # the single file `xmltodict.py' and NO `xmltodict/' directory. `module-file' and # `no-module-dir' assert both halves: a future release that became a package # would still import, and every behavioural case below would still pass, while # the packaging this image ships had silently changed shape. # # ============================================================================ # WHAT IS ASSERTED IS THE MAPPING, NOT A repr OF IT. # ============================================================================ # # xmltodict 0.13.0 returns plain `dict' on Python 3.7+ (`_dict = dict', with the # OrderedDict fallback gated below 3.7), so a case asserting a repr would be # pinning DICT INSERTION ORDER -- a property of the interpreter, not of this # library. `parse-flat' asserts a sorted, flattened `path=value' list instead: # same content, no order, and a red names one path rather than printing the whole # document back. `empty-is-none' and the `' spelling inside that list carry # the one distinction a str() would collapse -- an EMPTY ELEMENT is None, not the # empty string. # # ============================================================================ # THE ROUND TRIP IS PARSE-TO-PARSE, NOT BYTE-TO-BYTE, AND THAT IS DELIBERATE. # ============================================================================ # # `unparse' emits its own XML declaration and its own attribute order, and MIXED # CONTENT cannot be restored at all: `beforeemphafter' # parses to `#text' = "beforeafter" with the child beside it, so the interleaving # is gone by the time unparse sees it. That is xmltodict's documented behaviour # and not a defect. What the port promises is that the STRUCTURE survives, so the # suite re-parses unparse's output and compares the two PARSES -- and # `roundtrip-diff' names where they differ when they do, instead of leaving # `roundtrip-equal' red with nothing behind it. # # ============================================================================ # THE TWO OPTION CASES CHANGE THE MAPPING, WHICH IS WHY THEY ARE CASES. # ============================================================================ # # `process_namespaces=True' rewrites `dc:creator' to the expanded # `http://purl.org/dc/elements/1.1/:creator' AND consumes the `xmlns:dc' # declaration; `attr_prefix'/`cdata_key' decide which key an attribute and a text # node land under. Both change what a caller READS out of the same bytes. # Each is asserted from both sides -- the new spelling present and the default # spelling ABSENT -- so a build that added one beside the other is a red rather # than a pass. # # ============================================================================ # THE CVE BACKPORTS ARE BEHAVIOUR, SO THE SUITE STATES THEM. # ============================================================================ # # The recipe carries CVE-2025-9375-1.patch and -2.patch: an element or attribute # name (or an `@xmlns' prefix) containing `<' or `>' is rejected with ValueError # instead of being written out to break the tag context. An image built from an # unpatched sdist builds perfectly and reds on the three `cve-rejects-*' cases. # The fourth, `cve-escapes-attr-value', is the other direction: an attribute # VALUE carrying angle brackets is ESCAPED, not rejected, and a patch that # over-reached would break ordinary documents. # # ============================================================================ # EVERY VALUE BELOW WAS MEASURED, NOT PREDICTED. # ============================================================================ # # On x86-64 (CPython 3.10.12), against THIS EXACT PROBE and against THE BYTES # THIS RECIPE BUILDS -- the 0.13.0 sdist the recipe's SRC_URI sha256 pins, # unpacked, with the recipe's two CVE-2025-9375 patches applied and the tree put # on PYTHONPATH. Not a pip copy and not upstream git. Recorded 2026-08-31; the # ticket's design doc carries the transcript. # # What makes that control transferable is that all of it is pure-Python and # content-determined: the mapping xmltodict builds, the document it emits, its # length and digest depend on xmltodict's code and on the fixture, not on pointer # width, byte order, the platform string, the clock or hash randomisation. # Nothing in the probe sorts by hash, formats a float, or reads the environment. # # version=0.13.0 fixture_len=292 fixture_sha=4f1ad446b0fd9bb7 # unparse_len=300 unparse_sha=888e24aff29d58bb # roundtrip_equal=yes repeated_count=2 mixed_text=beforeafter # # THE ONE VALUE THAT IS NOT ASSERTED IS `expat_version'. It is the version of the # expat THIS IMAGE links, not a fact about xmltodict -- the control measured # 2.4.7 against the host's -- so it is emitted for a reader of a red and pinned # by 99-python3/expat-version, which owns it. # # ============================================================================ # WHAT IS NOT COVERED, ON PURPOSE. # ============================================================================ # # - xmltodict's OWN test suite (tests/test_xmltodict.py and # tests/test_dicttoxml.py). The recipe installs them through `ptest', which # this image does not carry, and running them would need python3-pytest. # The port closes on the API this file exercises; the CVE cases above are the # two upstream tests that are about the SHIPPED BYTES rather than about the # test harness. # - STREAMING (`item_depth' + `item_callback') and the generator input path. # Neither is on the ticket's path, and both would make this file a statement # about callback ordering rather than about the mapping. # - `defusedexpat'. xmltodict prefers it when importable and falls back to the # stdlib binding; it is not on this image, so the fallback is what runs, and # `expat-version' records which binding answered. # # THIS PORT CLOSES ON THIS SUITE: non-interactive, no manual browser pass, no # follow-on ticket under BRINTOS-780. P=python3-xmltodict # THE CANARY, which asserts nothing about python3-xmltodict. # # A suite's FIRST BST result line is lost between the guest's printf and the # host's read -- per-suite, deterministic, and NOT a defect in the host tool # (BRINTOS-1149 established that and filed it as linux-in-the-browser#177). The # harness computes its pass count as total-minus-failures, so THE DROPPED LINE # IS COUNTED AS A PASS. A nothing-case therefore stands in that position and the # eaten line costs nothing. It sits ahead of the `require' guards deliberately, # because a `require' that skips would emit its skip as the first line and be # eaten too. # # Remove this case when linux-in-the-browser#177 is fixed. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MOD=$SITE/xmltodict.py DI=$SITE/xmltodict-0.13.0.dist-info PKGLIST=/var/lib/dpkg/info/python3-xmltodict.list PROBE=$BST_DATA/python3-xmltodict-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-xmltodict.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------- the payload --- # Where the module has to land. `import xmltodict' further down is the functional # statement; these are the ones that say WHICH copy answered, so a module that # imported out of some other path is distinguishable from one that imported out # of the package this image installed. check $P/site-packages-dir test -d "$SITE" check $P/module-file test -f "$MOD" # ...AND NO PACKAGE DIRECTORY. setup.py declares `py_modules=['xmltodict']', so a # `xmltodict/' directory here would mean the upstream shape changed under a # recipe that still says otherwise -- invisible to every behavioural case below. check_eq $P/no-module-dir "absent" \ "$(if [ -d "$SITE/xmltodict" ]; then echo present; else echo absent; fi)" # NO COMPILED EXTENSION ANYWHERE THE PACKAGE PUT ANYTHING. See the header: on # this image a .so could not be loaded at all, so this is the case that fails a # version bump which grew one -- loudly, here, instead of at `import' on a later # ticket. # # The list is copied to tmpfs first and never redirected from the rootfs: a # top-level stdin redirect on the virtiofs root makes dup2(2) return 513 and ash # EXITS, ending the suite silently (brintos/linux#73). cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # EVERY REGULAR FILE THE PACKAGE SHIPS IS PYTHON OR ITS METADATA. The case above # names the failure that is expected; this one catches the shape nobody predicted # -- a binary, a .pyd, a stray README -- without having to enumerate what it # might be. # # A dpkg .list names DIRECTORIES too, and without a trailing slash, so a # name-only filter would report every directory in the package as an unexpected # shape; each entry is therefore tested for being a regular file first. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" # ----------------------------------------------------------- the dist-info --- # The installed metadata, which is the second, independent statement of the # version -- `$P/dist-version' below reads it through importlib.metadata, and the # directory name carries it a third time in $DI itself. A package whose module # landed and whose metadata did not would import fine and serve every behavioural # case while reporting no installed version at all. check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 0.13.0" cat "$DI/METADATA" # --------------------------------------------------------------- the package -- # dpkg KNOWS the package. Every case above would look identical if some other # package had installed the same files, so this is the one that says # python3-xmltodict is a real package this image installed rather than an # IMAGE_INSTALL line that resolved to nothing. # # The stanza is read from a tmpfs copy for the same reason as the file list, and # the anchor is exact -- an unanchored match would also take a -ptest or -doc # stanza of the same family. sed -n '/^Package: python3-xmltodict$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" # THE RUNTIME DEPENDENCIES THE RECIPE DECLARES, as dpkg recorded them. All three # are sub-packages of the ported python3 (BRINTOS-1476) and all three are real: # python3-xml carries `xml.parsers.expat' and `xml.sax.saxutils', python3-io # carries the StringIO the module falls back to. # # ONE CASE PER DEPENDENCY, against the extracted line, rather than one case # asserting the whole list as a string. The recipe's RDEPENDS order is not dpkg's # to preserve and the separator spacing is dpkg's own, so an exact-string case # here would be asserting a FORMAT this phase cannot measure. Split, each red also # names which dependency went missing instead of printing the whole line back. grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_out $P/pkg-depends-core "python3-core" cat "$W/dep" check_out $P/pkg-depends-xml "python3-xml" cat "$W/dep" check_out $P/pkg-depends-io "python3-io" cat "$W/dep" # ---------------------------------------------------------- the interpreter --- # THE IMPORT, ON ITS OWN, BEFORE THE PROBE. If xmltodict cannot be imported then # every probe case below reds at once and each names a different symptom; this # case is what says the cause is the import. It is also the cheapest possible # statement of the deliverable. check $P/import python3 -c 'import xmltodict' # ...and the expat binding, separately and first in the failure order that # matters: xmltodict imports it at module scope, so an interpreter without # pyexpat presents AS a broken xmltodict. This case is what tells the two apart. check $P/expat-import python3 -c 'from xml.parsers import expat' # ---------------------------------------------------------------- the probe --- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present # ONE interpreter start for everything that follows. bst_probe bounds it and # keeps BOTH streams: a probe that dies early otherwise leaves every case below # reporting got:'' and naming a symptom each, with the actual traceback nowhere. # The bound is generous because a python3 start on this arm is tens of seconds # before any work begins. bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" # Read a single key. The probe emits each key at most once, so head -n 1 is # defensive rather than load-bearing; the anchors keep `version' distinguishable # from `dist_version', and `ns_parse_ok' from `parse_ok'. pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } # THE EVIDENCE PLANE. If the probe emitted nothing, every case below is reading # an absent file: each would red with got:'' and the set would read exactly like # a broken library, when what actually happened is that no measurement was taken. # The word `unscorable' is PARSED -- brintos_local_verify re-measures a suite that # prints it on a fresh guest rather than trusting a shared boot's numbers # (BRINTOS-1644) -- so it goes in this structural check, not in a comment. # # The cases below still print their reds: withdraw, do not hide. This case is # what says what those reds measure. if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # THE EXPAT BINDING, FROM THE INTERPRETER'S OWN ANSWER. `sys.builtin_module_names' # cannot be satisfied by a file that merely arrived on the path, so this is the # statement that the parser under xmltodict is the one linked into the static # interpreter. The version it reports is emitted but NOT asserted -- see header. check_eq $P/pyexpat-builtin "yes" "$(pv pyexpat_builtin)" # THE VERSION THE RECIPE PINS, twice and independently: the module's own # `__version__' (which setup.py reads to stamp the sdist) and importlib.metadata # over the installed dist-info. The two disagreeing means a mixed install. check_eq $P/version "0.13.0" "$(pv version)" check_eq $P/dist-version "0.13.0" "$(pv dist_version)" # The module answered out of the file this image installed, not from a stray copy # on sys.path. This is what makes the payload cases above and the behaviour cases # below statements about the SAME xmltodict. check_eq $P/module-answered "$MOD" "$(pv module_file)" # THE PUBLIC SURFACE, WHOLE. xmltodict declares no `__all__', so this is `dir()' # minus the underscored names -- which includes the modules it imports at scope # (expat, XMLGenerator, StringIO, ...) deliberately: those are what `from # xmltodict import *' binds, and a release that stopped importing one of them has # changed its parser rather than its formatting. check_eq $P/api-surface \ "AttributesImpl,ParsingInterrupted,StringIO,XMLGenerator,expat,isgenerator,parse,platform,unparse" \ "$(pv api_surface)" # ------------------------------------------------------------- the fixture --- # The document the probe drove, pinned by length and digest, so every case below # is known to be about the document this file describes rather than about an # edited probe. check_eq $P/fixture-len "292" "$(pv fixture_len)" check_eq $P/fixture-sha "4f1ad446b0fd9bb7" "$(pv fixture_sha)" check_eq $P/parse-ok "yes" "$(pv parse_ok)" check_eq $P/root-key "catalog" "$(pv root_key)" check_eq $P/root-type "dict" "$(pv root_type)" # ------------------------------------------------------- the mapping rules --- # One case per rule, so a red names the rule that broke rather than the document. # `parse-flat' below states the whole mapping and catches anything these do not. # # ATTRIBUTES arrive prefixed `@', on the root and on a repeated child alike. check_eq $P/attr-root-version "2" "$(pv attr_root_version)" check_eq $P/attr-book-id "b1" "$(pv attr_book_id)" # A REPEATED ELEMENT BECOMES A LIST. This is the rule a naive dict mapping gets # wrong -- last one wins, and the first is silently gone. Both the type # and the count are asserted: a list of one would be a different failure than a # scalar. check_eq $P/repeated-is-list "yes" "$(pv repeated_is_list)" check_eq $P/repeated-count "2" "$(pv repeated_count)" check_eq $P/child-text "First" "$(pv child_text)" # A DECLARED NAMESPACE, with process_namespaces OFF (the default): the prefix is # part of the key verbatim and the declaration is an ordinary attribute. The `on' # half is asserted further down, and the two together are what say the option # changes the MAPPING. check_eq $P/ns-key-raw "Ada" "$(pv ns_key_raw)" check_eq $P/ns-decl-attr "http://purl.org/dc/elements/1.1/" "$(pv ns_decl_attr)" # MIXED TEXT AND CHILDREN: the text lands under `#text' and the child beside it. # The two runs either side of are CONCATENATED -- xmltodict's documented # behaviour, and the reason the round trip below is parse-to-parse. check_eq $P/mixed-text "beforeafter" "$(pv mixed_text)" check_eq $P/mixed-child "emph" "$(pv mixed_child)" check_eq $P/mixed-attr "high" "$(pv mixed_attr)" # AN EMPTY ELEMENT IS None, NOT THE EMPTY STRING. A str() of either is "", so # this is asserted as an identity in the probe rather than compared as a value. check_eq $P/empty-is-none "yes" "$(pv empty_is_none)" # THE WHOLE MAPPING, flattened and sorted -- the key-by-key assertion against a # literal that the ticket asks for. Everything above names one rule; this states # the document, including the `' that marks the empty element. check_eq $P/parse-flat \ "/catalog/@version=2 /catalog/@xmlns:dc=http://purl.org/dc/elements/1.1/ /catalog/book[0]/@id=b1 /catalog/book[0]/@lang=en /catalog/book[0]/dc:creator=Ada /catalog/book[0]/title=First /catalog/book[1]/@id=b2 /catalog/book[1]/title=Second /catalog/empty= /catalog/note/#text=beforeafter /catalog/note/@level=high /catalog/note/em=emph" \ "$(pv parse_flat)" # -------------------------------------------------------- the round trip ----- # unparse the parsed object, re-parse the result, compare the two PARSES. See the # header for why this is not byte-to-byte. check_eq $P/unparse-ok "yes" "$(pv unparse_ok)" check_eq $P/unparse-declaration "yes" "$(pv unparse_declaration)" # The empty element came back as an element rather than as an empty string or a # dropped key -- the one place a None that had become "" would show. check_eq $P/unparse-empty-element "yes" "$(pv unparse_empty_element)" # The emitted document, pinned. Content-determined like everything else here: # unparse walks the dict this fixture produced, so its length and digest are a # function of the library's code and the fixture. check_eq $P/unparse-len "300" "$(pv unparse_len)" check_eq $P/unparse-sha "888e24aff29d58bb" "$(pv unparse_sha)" check_eq $P/reparse-ok "yes" "$(pv reparse_ok)" # THE DELIVERABLE, IN ONE CASE: the structure survived the trip. check_eq $P/roundtrip-equal "yes" "$(pv roundtrip_equal)" # ...and WHERE it did not, when it did not. Redundant when everything passes and # the whole point when something does not. check_eq $P/roundtrip-diff "" "$(pv roundtrip_diff)" # --------------------------------------------- option: process_namespaces ---- # The option REWRITES THE KEY and CONSUMES THE DECLARATION. Both halves are # asserted, and so is the absence of the prefixed spelling: a build that expanded # the key while leaving `dc:creator' beside it would be neither behaviour. check_eq $P/ns-parse-ok "yes" "$(pv ns_parse_ok)" check_eq $P/ns-expanded-key "yes" "$(pv ns_expanded_key)" check_eq $P/ns-expanded-value "Ada" "$(pv ns_expanded_value)" check_eq $P/ns-prefixed-key-gone "yes" "$(pv ns_prefixed_key_gone)" check_eq $P/ns-decl-consumed "yes" "$(pv ns_decl_consumed)" # ------------------------------------------- option: attr_prefix/cdata_key --- # These decide which key an attribute and a text node land under, so a caller # that sets them reads a different document out of the same bytes. Asserted on # the new spelling AND on the absence of the default, for the same reason as # above. check_eq $P/opt-parse-ok "yes" "$(pv opt_parse_ok)" check_eq $P/opt-attr-key "yes" "$(pv opt_attr_key)" check_eq $P/opt-attr-value "2" "$(pv opt_attr_value)" check_eq $P/opt-default-attr-gone "yes" "$(pv opt_default_attr_gone)" check_eq $P/opt-cdata-key "yes" "$(pv opt_cdata_key)" check_eq $P/opt-cdata-value "beforeafter" "$(pv opt_cdata_value)" check_eq $P/opt-default-cdata-gone "yes" "$(pv opt_default_cdata_gone)" # ----------------------------------------------------------- the CVE guards -- # The two backports the recipe carries, asserted as the behaviour they are. An # image built from an unpatched sdist builds perfectly and reds on these three. check_eq $P/cve-rejects-tag-name "yes" "$(pv cve_rejects_tag_name)" check_eq $P/cve-rejects-attr-name "yes" "$(pv cve_rejects_attr_name)" check_eq $P/cve-rejects-xmlns-prefix "yes" "$(pv cve_rejects_xmlns_prefix)" # ...and the other direction: an attribute VALUE carrying angle brackets is # ESCAPED, not rejected. A patch that over-reached would break ordinary # documents, and this is the case that would say so. check_eq $P/cve-escapes-attr-value "yes" "$(pv cve_escapes_attr_value)" # The probe ran to the end. Every case above greps a key, so a probe that died # halfway would leave the remaining keys absent and each case would red with # got:'' -- this case is the one that says the cause was a truncated run. check_eq $P/probe-complete "yes" "$(pv done)" rm -rf "$W"