. "$BST_LIB" # BST-TIMEOUT: 1500 # THREE python3 starts, and that is the whole cost. Every # # assertion below either reads the filesystem/dpkg with # # shell builtins or greps the single probe run's output; the # # probe parses a 303-byte document three times and serialises # # it once, which is nothing beside the interpreter load. # # 1500 rather than 99-python3-xmltodict's 1200 because that # # suite starts the interpreter twice and this one starts it # # three times -- the third is `soupsieve-imports-bs4', which # # has to be its own process to mean anything. # # NOT MEASURED ON TARGET YET: this port's first on-target run # # is the batch verify, and the measured elapsed replaces this # # note in the test plan rather than in this comment, which # # would cost an image build to keep the shipped suite # # byte-identical. # # python3-beautifulsoup4 4.12.3 (meta-openembedded/meta-python # recipes-devtools/python/python3-beautifulsoup4_4.12.3.bb, `inherit pypi # python_hatchling', no patches) on the wasm32-hwjs glibc image -- HTML AS A # NAVIGABLE TREE. `BeautifulSoup(markup, parser)' builds a tree; `find`/`find_all` # search it by tag, attribute and class; `select` searches it with CSS selectors # through soupsieve. # # ============================================================================ # THIS PORT IS ABOUT THE BACKEND, NOT ABOUT THE PARSE. # ============================================================================ # # bs4 is a FACADE over a tree builder and ships three: `html.parser' (the # stdlib), `lxml' and `html5lib'. Only the first can exist on this image -- # `lxml' is a C extension over libxml2 and this image's python3 is ONE STATIC # BINARY with an EMPTY lib-dynload and no dynamic loader (BRINTOS-1476; # 99-python3 asserts both directly), and `html5lib' is unported. The recipe as # meta-python ships it declares BOTH as hard RDEPENDS, which is why # python3-beautifulsoup4_%.bbappend removes them; the bbappend carries the # measurement behind that. # # WHAT MAKES THE REMOVAL CORRECT AND NOT A HOPE is that bs4 imports both inside # `try: ... except ImportError: pass' (bs4/builder/__init__.py:625-636) and # registers whichever imported. So on this image the registry holds exactly one # builder, and the `backend-*', `builders-registered' and `lookup-*' cases below # are what state that -- a parse alone would have looked identical either way, # which is precisely why the parse is not the evidence. # # `lxml-request' is the other half and the port's honest boundary: a caller who # writes `BeautifulSoup(markup, "lxml")' on this image gets bs4's own # `FeatureNotFound', a named error, rather than a wrong tree. `default-builder' # is the IMAGE-COMPOSITION fact beside it -- on an ordinary desktop install bs4 # picks lxml when you name nothing, and here it picks the stdlib builder and # warns that it chose for you. # # ============================================================================ # SOUPSIEVE IS THE PAIR THIS PORT EXISTS TO CLOSE. # ============================================================================ # # soupsieve is bs4's ONE hard dependency upstream -- pyproject.toml reads # `dependencies = ["soupsieve >1.2"]', with html5lib, lxml, chardet, cchardet and # charset-normalizer all under `[project.optional-dependencies]'. It is kept in # RDEPENDS for that reason and because bs4's `.css' proxy is unusable without it. # # The dependency also runs the OTHER WAY, undeclared: soupsieve's module carries # an unconditional top-level `import bs4' (soupsieve/__init__.py:34, # soupsieve/css_match.py:8) while its recipe declares no target RDEPENDS at all, # so `import soupsieve' fails on any image without bs4. That is the defect # BRINTOS-2089 measured and this port clears. `soupsieve-imports-bs4' states it # from the image's own side: it imports soupsieve IN A SEPARATE INTERPRETER and # asserts bs4 came in with it, which is the one form of the claim that cannot be # satisfied by this suite having imported bs4 first. # # ============================================================================ # NO COMPILED EXTENSION, WHICH IS WHY IT PORTS AT ALL. # ============================================================================ # # The 4.12.3 sdist holds no `.so', no `.c' and no `.pyx' -- bs4 is pure Python, # wheel tag py3-none-any. On this image a compiled extension could not be # imported at all, so `pkg-no-so' is not boilerplate: it is a REGRESSION TRIPWIRE # on a version bump, failing at INSTALL time rather than at `import bs4' on # somebody else's ticket three rounds later. # # ============================================================================ # EVERY VALUE BELOW WAS MEASURED, NOT PREDICTED -- WITH TWO NAMED EXCEPTIONS. # ============================================================================ # # On x86-64 (CPython 3.12.3, hz1) against THIS EXACT PROBE and against THE BYTES # THIS RECIPE BUILDS -- the 4.12.3 sdist the recipe's SRC_URI sha256 pins # (74e3d1928edc070d21748185c46e3fb33490f22f52a3addee9aee0f4f7781051, verified), # unpacked, with no patches (the recipe carries none) and the tree put on # PYTHONPATH beside the soupsieve 2.5 sdist the soupsieve recipe pins # (5663d5a7b3bfaeee0bc4372e7fc48f9cff4940b3eec54a6451cc5299f1097690). Not a pip # copy and not upstream git. Recorded 2026-09-03; the ticket's design doc carries # the transcript. # # What makes that control transferable is that all of it is pure-Python and # content-determined: the tree bs4 builds, the document it emits, its length and # digest depend on bs4's code, on html.parser's normalisation and on the fixture, # not on pointer width, byte order, the platform string, the clock or hash # randomisation. Nothing in the probe sorts by hash, formats a float, or reads # the environment. # # version=4.12.3 fixture_len=303 fixture_sha=94c37b695f1ef876 # decode_len=304 decode_sha=dff3079b8c80dcd6 # builders_registered=HTMLParserTreeBuilder soupsieve_version=2.5 # roundtrip_equal=yes # # THE PACKAGING CASES ARE MEASURED SEPARATELY, AND FROM THE BUILT PACKAGE RATHER # THAN FROM THE PROBE: the recipe-scope build of 2026-09-03 produced # python3-beautifulsoup4_4.12.3-r0_wasm32.deb, whose control stanza reads # # Depends: python3-core, python3-html, python3-logging, python3-soupsieve, # python3-stringold # # -- the two removals and the one addition, as built. The five `pkg-depends-*' # cases and the two `pkg-no-depends-*' cases below are therefore results too, not # statements of intent about a recipe. # # THE TWO THAT ARE NOT MEASURED, and are marked where they appear: # # - `dist-version'. The control ran off PYTHONPATH with no installed metadata, # so it reported `dist_version_error=PackageNotFoundError'. On target the # dist-info IS installed and 4.12.3 is what importlib.metadata must answer. # EXPECTED, NOT YET MEASURED. # - `module-answered' and `soupsieve-answered'. The control's paths are the # unpack dir; on target they are the site-packages paths this file computes. # Same shape of claim, same status. # # THE ONE VALUE THAT IS EMITTED AND ASSERTED BY NOTHING is `chardet_module'. # bs4.dammit tries cchardet, then chardet, then charset_normalizer, each guarded, # and binds the first that imports -- which one answers is a fact about what else # this IMAGE ships, not about bs4, the same class of fact as # 99-python3-xmltodict's `expat_version'. It is emitted for a reader of a red. # # ============================================================================ # WHAT IS NOT COVERED, ON PURPOSE. # ============================================================================ # # - bs4's OWN test suite (bs4/tests/). It needs python3-pytest, and its # test_lxml.py and test_html5lib.py are about the two backends this image # cannot have. The port closes on the API this file exercises. # - THE XML MODE (`BeautifulSoup(markup, "xml")'). It is lxml-only -- there is # no stdlib XML tree builder in bs4 -- so it cannot work here. `lookup-xml' # records it resolving to nothing, which is the whole of what can be said. # - UnicodeDammit's ENCODING DETECTION beyond the binding it chose. Which # detector answers is image composition (above), and the detection itself is # chardet's behaviour rather than bs4's. # - `bs4.diagnose'. It is a debugging helper, is imported by nothing in the # package, and pulls lxml and html5lib at its own module scope -- so it is # the one file here that would not import, and no shipped code path reaches # it. # # THIS PORT CLOSES ON THIS SUITE: non-interactive, no manual browser pass, no # follow-on ticket under BRINTOS-780. P=python3-beautifulsoup4 # THE CANARY, which asserts nothing about python3-beautifulsoup4. # # A suite's FIRST BST result line is lost between the guest's printf and the # host's read -- per-suite, deterministic, and NOT a defect in the host tool # (BRINTOS-1149 established that and filed it as linux-in-the-browser#177). The # harness computes its pass count as total-minus-failures, so THE DROPPED LINE # IS COUNTED AS A PASS. A nothing-case therefore stands in that position and the # eaten line costs nothing. It sits ahead of the `require' guards deliberately, # because a `require' that skips would emit its skip as the first line and be # eaten too. # # Remove this case when linux-in-the-browser#177 is fixed. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MOD=$SITE/bs4/__init__.py DI=$SITE/beautifulsoup4-4.12.3.dist-info SSMOD=$SITE/soupsieve/__init__.py PKGLIST=/var/lib/dpkg/info/python3-beautifulsoup4.list PROBE=$BST_DATA/python3-beautifulsoup4-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-bs4.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------- the payload --- # Where the package has to land. `import bs4' further down is the functional # statement; these are the ones that say WHICH copy answered, so a module that # imported out of some other path is distinguishable from one that imported out # of the package this image installed. # # bs4 is a PACKAGE, not a single module -- `bs4/' with a builder subpackage -- # which is the opposite shape from 99-python3-xmltodict's, and the builder # directory is where the backend story lives, so both are named. check $P/site-packages-dir test -d "$SITE" check $P/package-dir test -d "$SITE/bs4" check $P/module-file test -f "$MOD" check $P/builder-dir test -d "$SITE/bs4/builder" check $P/builder-htmlparser test -f "$SITE/bs4/builder/_htmlparser.py" # NO COMPILED EXTENSION ANYWHERE THE PACKAGE PUT ANYTHING. See the header: on # this image a .so could not be loaded at all, so this is the case that fails a # version bump which grew one -- loudly, here, instead of at `import' on a later # ticket. # # The list is copied to tmpfs first and never redirected from the rootfs: a # top-level stdin redirect on the virtiofs root makes dup2(2) return 513 and ash # EXITS, ending the suite silently (brintos/linux#73). cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # EVERY REGULAR FILE THE PACKAGE SHIPS IS PYTHON, ITS METADATA, OR A FUZZ # FIXTURE. The case above names the failure that is expected; this one catches the # shape nobody predicted -- a binary, a .pyd, a stray README -- without having to # enumerate what it might be. # # THE `.testcase' ARM IS MEASURED, NOT DEFENSIVE, and it is the one thing the # off-target control could not have told us. bs4's wheel ships its own test tree, # and bs4/tests/fuzz/ holds 18 clusterfuzz regression inputs that are raw bytes # with a `.testcase' suffix. Read off the .deb this recipe built (hz1, # 2026-09-03): 77 regular files = 10 module .py + 17 test .py + 27 .pyc + 18 # .testcase + 5 dist-info. Without this arm the case reds on a correct package, # which is exactly the failure a suite written from the sdist alone would have # shipped -- the sdist has the fixtures too, but nothing in it says the wheel # keeps them. # # The arm is narrow on purpose: `bs4/tests/fuzz/' and that suffix only, so a # stray `.testcase' anywhere else, or a genuinely unexpected file inside the fuzz # directory, still reds. # # A dpkg .list names DIRECTORIES too, and without a trailing slash, so a # name-only filter would report every directory in the package as an unexpected # shape; each entry is therefore tested for being a regular file first. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; */bs4/tests/fuzz/*.testcase) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" # ...AND HOW MANY OF THEM THERE ARE. The arm above admits a whole class, so this # is what keeps it honest: the count is pinned, and a release that added or # dropped a fuzz regression input reds here rather than passing silently through # the exemption written for it. check_eq $P/pkg-testcase-count "18" \ "$(grep -c '/bs4/tests/fuzz/.*\.testcase$' "$W/pkgf" 2>/dev/null)" # THE TEST TREE IS SHIPPED, AND IS REACHED BY NOTHING; `bs4/diagnose.py' is # shipped on the same terms. Both are statements about what the wheel puts on the # image rather than about what bs4 does: the tests carry their own pytest # requirement and two of them are about the backends this image cannot have, and # diagnose is the one MODULE in the package that would not import here, because it # pulls lxml and html5lib at its own module scope. Neither is reached by `import # bs4', which is what the header's exclusions rest on -- so this is the case that # would catch that resting place moving. check_eq $P/diagnose-not-imported "absent" \ "$(grep -q 'diagnose' "$MOD" && echo present || echo absent)" # ----------------------------------------------------------- the dist-info --- # The installed metadata, which is the second, independent statement of the # version -- `$P/dist-version' below reads it through importlib.metadata, and the # directory name carries it a third time in $DI itself. A package whose module # landed and whose metadata did not would import fine and serve every behavioural # case while reporting no installed version at all. check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 4.12.3" cat "$DI/METADATA" # UPSTREAM'S OWN DECLARATION, read off the metadata this image installed: # soupsieve is REQUIRED and the other four are EXTRAS. This is the evidence the # bbappend's `RDEPENDS:${PN}:remove' rests on, restated from the target rather # than from the sdist -- so a release that promoted lxml back to a hard # requirement reds HERE, beside the recipe that stopped being right. check_out $P/meta-requires-soupsieve "Requires-Dist: soupsieve" cat "$DI/METADATA" check_eq $P/meta-lxml-is-extra "extra" \ "$(grep '^Requires-Dist: lxml' "$DI/METADATA" 2>/dev/null | grep -q 'extra ==' && echo extra || echo notextra)" check_eq $P/meta-html5lib-is-extra "extra" \ "$(grep '^Requires-Dist: html5lib' "$DI/METADATA" 2>/dev/null | grep -q 'extra ==' && echo extra || echo notextra)" # --------------------------------------------------------------- the package -- # dpkg KNOWS the package. Every case above would look identical if some other # package had installed the same files, so this is the one that says # python3-beautifulsoup4 is a real package this image installed rather than an # IMAGE_INSTALL line that resolved to nothing. # # The stanza is read from a tmpfs copy for the same reason as the file list, and # the anchor is exact -- an unanchored match would also take a -ptest or -doc # stanza of the same family. sed -n '/^Package: python3-beautifulsoup4$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" # THE RUNTIME DEPENDENCIES, as dpkg recorded them -- ONE CASE PER DEPENDENCY # against the extracted line, rather than one case asserting the whole list as a # string. The recipe's RDEPENDS order is not dpkg's to preserve and the separator # spacing is dpkg's own, so an exact-string case here would be asserting a FORMAT # this phase cannot measure. Split, each red also names which dependency went # missing instead of printing the whole line back. # # Each of these is a DIRECT module-scope import reached by `import bs4': # python3-html html.parser (builder/_htmlparser.py:11), html.entities # (dammit.py:12,65) # python3-logging logging (dammit.py:16) # python3-stringold string (dammit.py:17) -- added by the bbappend; see there # python3-soupsieve bs4/css.py:5, and the pair above grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_out $P/pkg-depends-core "python3-core" cat "$W/dep" check_out $P/pkg-depends-html "python3-html" cat "$W/dep" check_out $P/pkg-depends-logging "python3-logging" cat "$W/dep" check_out $P/pkg-depends-stringold "python3-stringold" cat "$W/dep" check_out $P/pkg-depends-soupsieve "python3-soupsieve" cat "$W/dep" # ...AND THE TWO THE BBAPPEND REMOVED, asserted as ABSENT. This is the port's # substantive change and the only case that measures it from the image's side: a # bbappend that stopped being applied would rebuild an image whose bs4 declares # dependencies on two packages that do not exist on this arm, and every case # above would still pass. check_eq $P/pkg-no-depends-lxml "absent" \ "$(grep -q 'python3-lxml' "$W/dep" && echo present || echo absent)" check_eq $P/pkg-no-depends-html5lib "absent" \ "$(grep -q 'python3-html5lib' "$W/dep" && echo present || echo absent)" # ...and neither package is on the image at all, which is the fact that makes the # removal necessary rather than merely tidy. check_eq $P/lxml-not-installed "absent" \ "$(test -e "$SITE/lxml" && echo present || echo absent)" check_eq $P/html5lib-not-installed "absent" \ "$(test -e "$SITE/html5lib" && echo present || echo absent)" # ---------------------------------------------------------- the interpreter --- # THE IMPORT, ON ITS OWN, BEFORE THE PROBE. If bs4 cannot be imported then every # probe case below reds at once and each names a different symptom; this case is # what says the cause is the import. It is also the cheapest possible statement # of the deliverable. check $P/import python3 -c 'import bs4' # THE PAIR, IN THE ONE FORM THAT MEANS ANYTHING -- A SEPARATE INTERPRETER. # soupsieve carries an unconditional top-level `import bs4' and declares no # RDEPENDS saying so, which is the defect BRINTOS-2089 measured; this port is # what clears it. Asserted here as: importing soupsieve ALONE succeeds AND drags # bs4 in. Inside the probe it would prove nothing, because the probe has already # imported bs4 itself. check $P/soupsieve-imports-bs4 python3 -c \ 'import soupsieve, sys; raise SystemExit(0 if "bs4" in sys.modules else 1)' # ---------------------------------------------------------------- the probe --- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present # ONE interpreter start for everything that follows. bst_probe bounds it and # keeps BOTH streams: a probe that dies early otherwise leaves every case below # reporting got:'' and naming a symptom each, with the actual traceback nowhere. # The bound is generous because a python3 start on this arm is tens of seconds # before any work begins. bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" # Read a single key. The probe emits each key at most once, so head -n 1 is # defensive rather than load-bearing; the anchors keep `version' distinguishable # from `dist_version', and `css_select_none' from `css_select_item'. pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } # THE EVIDENCE PLANE. If the probe emitted nothing, every case below is reading # an absent file: each would red with got:'' and the set would read exactly like # a broken library, when what actually happened is that no measurement was taken. # The word `unscorable' is PARSED -- brintos_local_verify re-measures a suite that # prints it on a fresh guest rather than trusting a shared boot's numbers # (BRINTOS-1644) -- so it goes in this structural check, not in a comment. # # The cases below still print their reds: withdraw, do not hide. This case is # what says what those reds measure. if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # THE VERSION THE RECIPE PINS, twice and independently: the module's own # `__version__' (which pyproject.toml's hatch version hook reads to stamp the # sdist) and importlib.metadata over the installed dist-info. The two disagreeing # means a mixed install. check_eq $P/version "4.12.3" "$(pv version)" # EXPECTED, NOT YET MEASURED -- see the header. The control had no installed # metadata and reported PackageNotFoundError; on target the dist-info is there. check_eq $P/dist-version "4.12.3" "$(pv dist_version)" # The package answered out of the files this image installed, not from a stray # copy on sys.path. This is what makes the payload cases above and the behaviour # cases below statements about the SAME bs4. EXPECTED, NOT YET MEASURED: the # control answered from its unpack directory. check_eq $P/module-answered "$MOD" "$(pv module_file)" # THE ADVERTISED SURFACE, whole. bs4 declares `__all__ = ['BeautifulSoup']' -- # one name -- so this is a much narrower statement than a `dir()' would be, and a # release that widened it has changed its contract rather than its formatting. check_eq $P/all-surface "BeautifulSoup" "$(pv all_surface)" # ============================================================ THE BACKENDS ==== # The port's substance. See the header: the two absences are what the bbappend's # RDEPENDS removal is FOR, and the registry holding exactly one builder is what # says bs4's guarded imports took the degraded path rather than that a parse # happened to work. check_eq $P/backend-lxml-absent "absent" "$(pv backend_lxml)" check_eq $P/backend-html5lib-absent "absent" "$(pv backend_html5lib)" check_eq $P/builders-registered "HTMLParserTreeBuilder" "$(pv builders_registered)" # LOOKUP BY FEATURE, which is what a caller actually does. The stdlib builder # answers to both of its feature names; the other three resolve to nothing. # `lookup-xml' is the one that records the header's exclusion: bs4 has no stdlib # XML tree builder, so XML mode is lxml-only and cannot exist here. check_eq $P/lookup-html-parser "HTMLParserTreeBuilder" "$(pv lookup_html_parser)" check_eq $P/lookup-html "HTMLParserTreeBuilder" "$(pv lookup_html)" check_eq $P/lookup-lxml "" "$(pv lookup_lxml)" check_eq $P/lookup-xml "" "$(pv lookup_xml)" check_eq $P/lookup-html5lib "" "$(pv lookup_html5lib)" # WHAT A CALLER GETS WHEN THEY NAME NO PARSER. An image-composition fact, not a # property of bs4: on an ordinary desktop install this is lxml. bs4 warns that it # chose for you, and the warning is part of the answer -- a caller reading it is # being told their result is not portable across installs. check_eq $P/default-builder "HTMLParserTreeBuilder" "$(pv default_builder)" check_eq $P/guessed-warning "yes" "$(pv guessed_warning)" # ...AND WHAT THEY GET WHEN THEY NAME ONE THAT IS NOT HERE: bs4's own # `FeatureNotFound'. A named error rather than a wrong tree is this port's honest # boundary, and this case is where it is written down. check_eq $P/lxml-request "FeatureNotFound" "$(pv lxml_request)" # ============================================================== SOUPSIEVE ===== # The `.css' proxy is bound, which is the in-process half of the pair # (`soupsieve-imports-bs4' above is the other, and the stronger, half). A bs4 # whose soupsieve import had failed would still import, still parse, still # `find_all' -- and warn once and raise on every selector. check_eq $P/soupsieve-bound "yes" "$(pv soupsieve_bound)" check_eq $P/soupsieve-version "2.5" "$(pv soupsieve_version)" # EXPECTED, NOT YET MEASURED, for the same reason as `module-answered'. check_eq $P/soupsieve-answered "$SSMOD" "$(pv soupsieve_file)" # ------------------------------------------------------------- the fixture --- # The document the probe drove, pinned by length and digest, so every case below # is known to be about the document this file describes rather than about an # edited probe. check_eq $P/fixture-len "303" "$(pv fixture_len)" check_eq $P/fixture-sha "94c37b695f1ef876" "$(pv fixture_sha)" check_eq $P/parse-ok "yes" "$(pv parse_ok)" # The parse below named `html.parser' explicitly, so this says the named builder # is the one that ran -- distinct from `default-builder', which is about the # choice bs4 makes when nobody names one. check_eq $P/builder-used "HTMLParserTreeBuilder" "$(pv builder_used)" # ------------------------------------------------------- the mapping rules --- # One case per rule, so a red names the rule that broke rather than the document. # `parse-flat' below states the whole tree and catches anything these do not. # # A MULTI-VALUED ATTRIBUTE BECOMES A LIST. This is the rule a caller coming from # a DOM API gets wrong, and it is the tree builder that applies it -- so it is # also a second, independent statement about which builder ran. The type and the # content are asserted separately: a builder that returned the string # "wrap outer" would join to the same text. check_eq $P/class-is-list "yes" "$(pv class_is_list)" check_eq $P/class-value "wrap|outer" "$(pv class_value)" # ...and an ordinary attribute beside it is NOT split. check_eq $P/data-attr "2" "$(pv data_attr)" check_eq $P/id-attr "main" "$(pv id_attr)" # ENTITIES ARE DECODED, on input, by html.entities -- which is the direct import # that makes python3-html a real RDEPENDS rather than a nominal one. Asserted # from both sides: the character is there and the entity spelling is not. check_eq $P/title-text "Catalog & Index" "$(pv title_text)" check_eq $P/entity-amp "yes" "$(pv entity_amp)" check_eq $P/p-first-text "First " "$(pv p_first_text)" # SEARCH, by tag name and by class. Together with the CSS route below these are # three different code paths to the same two paragraphs, so counting each # separately is what tells a broken search from a broken parse. check_eq $P/find-all-p "2" "$(pv find_all_p)" check_eq $P/find-all-class-item "2" "$(pv find_all_class_item)" check_eq $P/find-all-li "2" "$(pv find_all_li)" check_eq $P/find-href "/x" "$(pv find_href)" # A COMMENT IS NOT TEXT. bs4 models it as a NavigableString SUBCLASS, so a # builder that lost the distinction would put "a comment" into get_text() -- # which is what these three assert, from both sides. check_eq $P/comment-count "1" "$(pv comment_count)" check_eq $P/comment-text "a comment" "$(pv comment_text)" check_eq $P/comment-in-text "no" "$(pv comment_in_text)" # A VOID ELEMENT has no children. `decode-br' below is the output half. check_eq $P/void-is-empty "yes" "$(pv void_is_empty)" # get_text OVER THE WHOLE DOCUMENT, which is the single commonest thing anyone # does with bs4. Separator-joined and stripped, so the answer is content and not # whitespace policy. check_eq $P/get-text "Catalog & Index|First |Second|a|b|go" "$(pv get_text)" # THE WHOLE TREE, flattened and sorted -- the element-by-element assertion. # Everything above names one rule; this states the document. An element carrying # neither attributes nor direct text contributes no path, which is why
is # absent here and is covered by `void-is-empty' and `decode-br' instead. check_eq $P/parse-flat \ "/html/@lang=en /html/body[0]/div[0]/@class=wrap|outer /html/body[0]/div[0]/@data-n=2 /html/body[0]/div[0]/@id=main /html/body[0]/div[0]/a[0]/#text=go /html/body[0]/div[0]/a[0]/@href=/x /html/body[0]/div[0]/a[0]/@title=link /html/body[0]/div[0]/p[0]/#text=First /html/body[0]/div[0]/p[0]/@class=item|first /html/body[0]/div[0]/p[1]/#text=Second /html/body[0]/div[0]/p[1]/@class=item /html/body[0]/div[0]/ul[0]/li[0]/#text=a /html/body[0]/div[0]/ul[0]/li[1]/#text=b /html/head[0]/title[0]/#text=Catalog & Index" \ "$(pv parse_flat)" # ---------------------------------------------------------- the CSS route ---- # soupsieve, reached BOTH ways: through the legacy `.select'/`.select_one' # methods and through the modern `.css' proxy. Both are documented API over the # same engine, and a port that wired only one of them would pass a suite that # drove only the other. check_eq $P/css-select-item "2" "$(pv css_select_item)" check_eq $P/css-select-child "First " "$(pv css_select_child)" check_eq $P/css-select-attr "1" "$(pv css_select_attr)" check_eq $P/css-proxy-select "2" "$(pv css_proxy_select)" check_eq $P/css-proxy-one "a" "$(pv css_proxy_one)" # A SELECTOR THAT MATCHES NOTHING returns empty rather than raising... check_eq $P/css-select-none "0" "$(pv css_select_none)" # ...and a MALFORMED one raises soupsieve's own exception, which is how a caller # tells a typo from an empty page. Named by type: the message is soupsieve's to # reword between releases. check_eq $P/css-bad-selector "SelectorSyntaxError" "$(pv css_bad_selector)" # ---------------------------------------------------------- the output side -- # `str(soup)' is the document echoed back, and it is NOT byte-identical to the # fixture -- html.parser normalises the void element and re-escapes text. Length # and digest pin the formatter; `decode-br' names the one normalisation a reader # of a red would want to see first, and `decode-amp' is the output half of # `entity-amp' -- bs4 decoded on input and re-encodes on output rather than # shipping a raw `&'. check_eq $P/decode-len "304" "$(pv decode_len)" check_eq $P/decode-sha "dff3079b8c80dcd6" "$(pv decode_sha)" check_eq $P/decode-br "
" "$(pv decode_br)" check_eq $P/decode-amp "yes" "$(pv decode_amp)" # A RE-PARSE OF THE OUTPUT AGREES WITH THE FIRST PARSE: the structure survives # the trip even though the bytes do not. `roundtrip-diff' names WHERE it did not, # when it did not, instead of leaving `roundtrip-equal' red with nothing behind # it. check_eq $P/roundtrip-equal "yes" "$(pv roundtrip_equal)" check_eq $P/roundtrip-diff "" "$(pv roundtrip_diff)" # The probe ran to the end. Every case above greps a key, so a probe that died # halfway would leave the remaining keys absent and each case would red with # got:'' -- this case is the one that says the cause was a truncated run. check_eq $P/probe-complete "yes" "$(pv done)" rm -rf "$W"