. "$BST_LIB" # BST-TIMEOUT: 1200 # THREE python3 starts, and they are the whole cost. Every # # assertion below either reads the filesystem/dpkg with # # shell builtins or greps the output of one of those three. # # # # They are a bare `import', a second one whose STDERR is # # the assertion (`import-quiet'), and the probe -- which # # takes ONE start for a hundred and ninety measurements. # # The work inside the probe is regex compilation and # # matching over strings under 60 bytes, microseconds beside # # an interpreter start on this arm, which is tens of # # seconds before the first line of the program runs. # # # # 1200 is 400s per start, the same budget # # 99-python3-wcwidth sets for its three, and roughly six # # times the ~60s a python3 start has measured at here. # # # # A suite-timeout red is a batch-wide cost, so if the three # # starts prove slower than that on target the RIGHT fix is # # this number, not fewer cases. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify, and the measured elapsed replaces # # this note in the test plan rather than in this comment, # # which would cost an image build to keep the shipped suite # # byte-identical. # # python3-parse 1.20.1 (meta-openembedded/meta-python # recipes-devtools/python/python3-parse_1.20.1.bb, `inherit pypi # python_setuptools_build_meta ptest', PyPI sdist) on the wasm32-hwjs glibc # image. # # ============================================================================ # WHAT THE PACKAGE DOES. # ============================================================================ # # `parse()' is the inverse of `str.format()'. Given the same format # specification that rendered a string, it pulls the values back out of it -- # with the type conversion the specification asks for, so `{:d}' returns an int # and `{:ti}' a datetime rather than the matched text. # # That inverse relation is the library's whole contract and it is what the # `rt-*' group asserts directly: render a value with `str.format' under a # specification, hand the SAME specification to `parse()', and the value comes # back equal to what went in. Sixteen of those, plus the three that DO NOT # round trip and are recorded as measurements below. # # ============================================================================ # THE TWO FORMAT LANGUAGES OVERLAP WITHOUT BEING THE SAME, WHICH IS THE ONE # THING A CALLER GETS WRONG. # ============================================================================ # # "The same specification" is true of the specifications both languages accept, # and the alphabets differ at both ends: # # `{:,d}' is legal `str.format' and is NOT legal here -- `err-bad-spec'. # A caller who renders with a thousands separator cannot parse it back with # the same string; the `n' type is this library's answer (`conv-n'), and it # is a different specification. # # `{:l}' is legal here and is NOT legal `str.format' -- `rt-l-not-a-format- # code'. `l' means "letters"; `str.format' has no such code and raises. # # `allowed-types' pins the whole alphabet for that reason: a release that # dropped one turns a working caller's format into a ValueError while every # behavioural case in this file still passes. # # ============================================================================ # `{:s}' DOES NOT MEAN "STRING", AND THAT IS THE MOST LIKELY MISREADING. # ============================================================================ # # The type table is closer to a regex character class than to `str.format': # `s' is WHITESPACE and `S' is non-whitespace, `w' is `\w+', `W' its # complement, `l' letters, `d' digits and `D' non-digits. So `{:s}', which is # exactly what a caller coming from `str.format' reaches for to mean "some # text", matches a run of SPACES -- and `type-s-on-word' asserts that it does # not match a word at all. `conv-s-*' report its value by shape rather than # verbatim, because a value that is nothing but whitespace does not survive # being read through a shell. # # ============================================================================ # THREE ROUND TRIPS THAT DO NOT CLOSE, ALL FOR REASONS THAT ARE UPSTREAM'S. # ============================================================================ # # Width and alignment survive the round trip on a TYPED field: the type gives # the value its own regex, the fill is outside the capture group, and # `rt-int-right', `rt-int-left', `rt-int-centre' and `rt-int-fill' all come # back equal. # # On an UNTYPED field they do not. `{:>10}' compiles to `.{10,}?' with the fill # outside it, and `.' matches a space -- so the padding is captured WITH the # value and `parse' returns `' abc'' where `format' was given `'abc''. # That is upstream's documented "the width is not enforced; it just indicates # there may be whitespace or 0s to strip" read from the other end, and # `rt-untyped-right' and `rt-untyped-fill' record it as a measurement rather # than as a defect of this port. The documented behaviour it does not # contradict is asserted separately: `align-right', `align-centre' and # `align-left' use upstream's own examples, where alignment WITHOUT a width # strips the fill exactly as described. # # `rt-str-type-width' is the third, and it fails for the `s' reason above # rather than for the padding one -- `{:>10s}' asks for padded WHITESPACE. # It is kept apart from the other two so a red in this group says which cause. # # ============================================================================ # `parse()' ANCHORS AT BOTH ENDS; `search()' AND `findall()' DO NOT. # ============================================================================ # # All three are asserted against ONE string -- "the answer is 42 today" -- so # the three answers cannot come from three different inputs. `parse' with a # bare `{:d}' returns None on it, `search' returns 42, and `parse' with the # surrounding text spelled out returns 42 as well. That trio is the whole # distinction, and it is the one a caller reaches for `search' after getting # wrong. # # `findall' returns a ResultIterator rather than a list, so `len()' on it # raises -- `findall-len' asserts the TypeError, because a caller who wrote # `len(findall(...))' gets an exception rather than a count. # # ============================================================================ # THE DOCUMENTED FAILURE PATH IS None, NOT AN EXCEPTION AND NOT AN EMPTY # RESULT. # ============================================================================ # # `nomatch-*' assert the None. `empty-result-bool' is what makes `is None' the # correct test rather than a stylistic preference: a format with no fields # matches and yields an empty `Result', and `if' on that Result is TRUE. A # caller writing `if parse(...)' is right for the wrong reason until the day # their format has no fields in it. # # The two ways the FORMAT can be wrong raise instead, and they raise at parser # construction: an unrecognised specification (`err-bad-spec') and a name used # twice under two different types (`err-repeated-name'). Both are ValueErrors. # # ============================================================================ # ONE UPSTREAM GUARD IS UNREACHABLE, AND IT IS MEASURED RATHER THAN READ. # ============================================================================ # # `TooManyFields' is raised only when `re.compile' rejects the built expression # with "this version only supports 100 named groups" -- a limit CPython's `re' # no longer has. `many-named-fields' parses a format with 150 named fields and # asserts all 150 come back, so the guard is recorded as dead code rather than # as a limit this image has. The class stays part of the API (`api- # toomanyfields') and a caller still catches it through ValueError; what does # not happen is it being raised. # # ============================================================================ # THE UPSTREAM ptest SUITE IS NOT SHIPPED, FOR THE REASON ALREADY RECORDED FOR # python3-wcwidth. # ============================================================================ # # This recipe DOES `inherit ptest' and DOES ship a run-ptest, so "run # upstream's tests" is the obvious reading of this port and it is worth saying # why it is not what happens. # # This distro sets DISTRO_FEATURES without "ptest" (conf/distro/brintos.conf), # so ptest.bbclass never adds ${PN}-ptest to PACKAGES at all and no -ptest deb # is produced. Forcing one would put its RDEPENDS on the image -- # python3-pytest and python3-unittest-automake-output, and pytest's own # RDEPENDS names sixteen packages of which seven are not here and are not this # ticket's. Landing seven unrelated packages as a side effect of a pure-Python # port is what the same decision was refused for on python3-wcwidth. # # What the decision costs is coverage rather than assurance, and it is bounded: # upstream's tests/ is seven files of pytest cases over the same public API # this suite exercises without a framework. `no-ptest-package' asserts the # -ptest package is NOT installed, so a future image that grew one is told to # run it rather than ship an untested duplicate. # # ============================================================================ # WHERE THE EXPECTED VALUES CAME FROM. # ============================================================================ # # Every literal below was measured on x86-64 against parse.py from the sdist # the recipe fetches -- sha256 09002ca350ad42e76629995f71f7b518670bcf93548bdde # 3684fd55d2be51975, which is the recipe's own SRC_URI[sha256sum] -- and the # recipe applies no patch, so the bytes measured are the bytes packaged. The # control interpreter is CPython 3.12.3 and the image ships 3.12, so the two # agree on minor version; that matters here because `many-named-fields' is a # statement about the interpreter's `re' rather than about parse. # # The suite was then run END TO END on x86-64 against the tree unpacked from # the built python3-parse_1.20.1-r0_wasm32.deb, with the site-packages, dpkg # status and dpkg file-list paths redirected at it: 224 of 224 green, no red. # So every literal here has been through the assertion that reads it, not only # through the probe that produced it. # # The on-target result is not yet measured -- the batch verify is this port's # first run on wasm32-hwjs. P=python3-parse # THE CANARY, which asserts nothing about python3-parse. # # A suite's FIRST BST result line is lost between the guest's printf and the # host's read -- per-suite, deterministic, and NOT a defect in the host tool # (BRINTOS-1149 established that and filed it as linux-in-the-browser#177). The # harness computes its pass count as total-minus-failures, so THE DROPPED LINE # IS COUNTED AS A PASS. A nothing-case therefore stands in that position and the # eaten line costs nothing. It sits ahead of the `require' guards deliberately, # because a `require' that skips would emit its skip as the first line and be # eaten too. # # Remove this case when linux-in-the-browser#177 is fixed. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MOD=$SITE/parse.py DI=$SITE/parse-1.20.1.dist-info PKGLIST=/var/lib/dpkg/info/python3-parse.list PROBE=$BST_DATA/python3-parse-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-parse.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------- the payload --- # Where the package has to land. `import parse' further down is the functional # statement; these are the ones that say WHICH copy answered, so a package that # imported out of some other path on sys.path is distinguishable from one that # imported out of what this image installed. For THIS package that is not a # theoretical worry -- `parse' is a plausible name for a local script, and the # probe's own filename was chosen so as not to shadow it. check $P/site-packages-dir test -d "$SITE" check $P/module-file test -s "$MOD" # ONE FILE, NOT A DIRECTORY. pyproject.toml declares `py-modules = ["parse"]', # so one module lands in site-packages. This is the case that fails a release # which moved to a package directory -- loudly, here, rather than nowhere. check $P/no-package-dir test ! -d "$SITE/parse" # The precompiled bytecode, which is what `import-quiet' below turns on. check $P/pycache test -s "$SITE/__pycache__/parse.cpython-312.pyc" # NO SHIPPED SCRIPT. This package is a library and installs no executable -- # unlike python3-smartypants, whose /usr/bin script needed a shebang patch. # Asserted so a release that grew a console_scripts entry point announces # itself here. check $P/no-cli test ! -e /usr/bin/parse # NO COMPILED EXTENSION ANYWHERE THE PACKAGE PUT ANYTHING. On this image a .so # could not be loaded at all -- python3 is ONE STATIC BINARY with an empty # lib-dynload and no dynamic loader (BRINTOS-1476) -- so this is the case that # fails a version bump which grew one, at install rather than at `import'. # # The list is copied to tmpfs first and never redirected from the rootfs: a # top-level stdin redirect on the virtiofs root makes dup2(2) return 513 and ash # EXITS, ending the suite silently (brintos/linux#73). cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # EVERY REGULAR FILE THE PACKAGE SHIPS IS PYTHON OR ITS METADATA. The case # above names the failure that is expected; this one catches the shape nobody # predicted -- a binary, a .pyd, a stray data file -- without having to # enumerate what it might be. # # A dpkg .list names DIRECTORIES too, and without a trailing slash, so a # name-only filter would report every directory in the package as an unexpected # shape; each entry is therefore tested for being a regular file first. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" # THE UPSTREAM TEST SUITE IS NOT IN THE RUNTIME PACKAGE. The sdist ships a # tests/ directory of seven pytest files and the recipe's do_install_ptest # copies them into the -ptest package, which this image does not build. A # tests/ directory appearing under site-packages would mean the split went # wrong and the runtime package absorbed them. check $P/no-tests-dir test ! -d "$SITE/tests" # ----------------------------------------------------------- the dist-info --- # The installed metadata. This module carries its own `__version__' AND # pyproject.toml derives the distribution version from it # (`version = {attr = "parse.__version__"}'), so the two cannot disagree at # build time -- which makes `version-agrees' below a statement about the # INSTALL rather than about upstream: a module landing beside a dist-info from # some other release is what it catches. check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 1.20.1" cat "$DI/METADATA" check_out $P/dist-info-name "Name: parse" cat "$DI/METADATA" # The one-line description, which is upstream's own and is the shortest true # statement of what the package is. check_out $P/dist-info-summary "parse() is the opposite of format()" cat "$DI/METADATA" # top_level.txt names the ONE importable this package installs. It is what an # `import parse' has to match, and a wheel that renamed the module would change # it here first. check_out $P/dist-info-top-level "parse" cat "$DI/top_level.txt" # PURE PYTHON, AS THE WHEEL ITSELF DECLARES IT. `Root-Is-Purelib: true' is the # packaging-side statement of the header's claim -- no platform tag, no # extension module, nothing that could need the absent dynamic loader. check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" # The upstream licence text ships beside the metadata, which is where this # recipe's LIC_FILES_CHKSUM points. check $P/dist-info-licence test -s "$DI/LICENSE" # ------------------------------------------------------------- the package --- # dpkg KNOWS the package. Every case above would look identical if some other # package had installed the same files, so this is the one that says # python3-parse is a real package this image installed rather than an # IMAGE_INSTALL line that resolved to nothing. # # The stanza is read from a tmpfs copy for the same reason as the file list, and # the anchor is exact -- an unanchored match would also take the -dev or -dbg # stanza of the same family, and `python3-parse-type' is a DIFFERENT package in # the same layer whose name has this one as a prefix. sed -n '/^Package: python3-parse$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" # THE RUNTIME DEPENDENCIES THE RECIPE DECLARES, as dpkg recorded them. The # recipe names python3-datetime, python3-logging and python3-numbers, and all # three are SUB-PACKAGES OF THE PORTED python3 (BRINTOS-1476) already on this # image -- which is why this port waits on no other package. Each is asserted # by name, because a Depends line that lost one would still install today (the # sub-package is present for another reason) and break the day it stopped # being. grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_out $P/pkg-depends-core "python3-core" cat "$W/dep" check_out $P/pkg-depends-datetime "python3-datetime" cat "$W/dep" check_out $P/pkg-depends-logging "python3-logging" cat "$W/dep" check_out $P/pkg-depends-numbers "python3-numbers" cat "$W/dep" # NO -ptest PACKAGE IS INSTALLED. See the header: the recipe inherits ptest, # this distro sets DISTRO_FEATURES without "ptest" so none is built, and # forcing one would drag seven unrelated packages onto the image. A future # image that grew one should RUN it rather than ship this suite as an untested # duplicate, and this is the case that says so. check_eq $P/no-ptest-package "" \ "$(sed -n '/^Package: python3-parse-ptest$/p' /var/lib/dpkg/status 2>/dev/null)" # --------------------------------------------------------- the interpreter --- # THE IMPORT, ON ITS OWN, BEFORE THE PROBE. If parse cannot be imported then # every probe case below reds at once and each names a different symptom; this # case is what says the cause is the import. It is also the cheapest possible # statement of the deliverable. check $P/import python3 -c 'import parse' # ...AND THE IMPORT SAYS NOTHING WHILE DOING IT. This is a statement about the # shipped bytecode: the package ships a precompiled .pyc and OE writes it with # CHECKED-HASH invalidation, so the source is hashed and never recompiled. A # red here means the shipped .pyc was REJECTED -- a stale hash, a python3 minor # bump that changed the magic, or a missing __pycache__ -- and the module is # being compiled from source on every import. # # parse.py imports `logging' at module scope and calls getLogger, which # configures nothing and prints nothing; the module has no import-time output # of its own. check_eq $P/import-quiet "" \ "$(python3 -c 'import parse' 2>&1)" # --------------------------------------------------------------- the probe --- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present # ONE interpreter start for everything that follows. bst_probe bounds it and # keeps BOTH streams: a probe that dies early otherwise leaves every case below # reporting got:'' and naming a symptom each, with the actual traceback nowhere. # The bound is generous because a python3 start on this arm is tens of seconds # before any work begins. bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" # Read a single key. The probe emits each key at most once, so head -n 1 is # defensive rather than load-bearing; the anchors keep `conv_s' distinguishable # from `conv_S' -- which differ by case alone and are the pair this suite most # needs to keep apart. pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } # THE EVIDENCE PLANE. If the probe emitted nothing, every case below is reading # an absent file: each would red with got:'' and the set would read exactly like # a broken library, when what actually happened is that no measurement was # taken. The word `unscorable' is PARSED -- brintos_local_verify re-measures a # suite that prints it on a fresh guest rather than trusting a shared boot's # numbers (BRINTOS-1644) -- so it goes in this structural check, not a comment. # # The cases below still print their reds: withdraw, do not hide. This case is # what says what those reds measure. if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # ...and it ran to the END. Every key is emitted in file order, so a probe that # died two thirds of the way through would leave the earlier cases green and the # later ones reading got:'' -- which looks like a partial library rather than a # partial run. This is the case that tells those apart. check_eq $P/probe-complete "yes" "$(pv probe_done)" check_eq $P/probe-count "192" "$(pv emitted_count)" # The module answered out of the file this image installed, not from a stray # copy on sys.path. This is what makes the payload cases above and the # behaviour cases below statements about the SAME parse. check_eq $P/module-answered "$MOD" "$(pv module_file)" # ...and it is a MODULE. See the header: a release that moved to a package # directory would still import and still pass every behavioural case below. check_eq $P/is-module "yes" "$(pv is_module)" # THE VERSION, FROM BOTH PLACES, AND THEN AGAINST EACH OTHER. check_eq $P/version-attr "1.20.1" "$(pv version_attr)" check_eq $P/dist-version "1.20.1" "$(pv dist_version)" check_eq $P/version-agrees "$(pv version_attr)" "$(pv dist_version)" # `__all__' IS SHORTER THAN THE API, and that is worth pinning rather than # deriving: `compile' is public, documented, and NOT in it -- so a caller doing # `from parse import *' does not get `compile', which is the one name they # would reach for next after `parse'. check_eq $P/all-names "parse,search,findall,with_pattern" "$(pv all_names)" # The public surface, by name. Each is reached by at least one behavioural case # below; this group is what says a red further down is about behaviour rather # than about a name that is not there. for _n in parse search findall compile with_pattern Parser Result Match \ ResultIterator TooManyFields RepeatedNameError extract_format; do check_eq $P/api-$(echo "$_n" | tr 'A-Z_' 'a-z-') "yes" "$(pv has_$_n)" done # THE TYPE ALPHABET, PINNED. See the header: this is the set of specifications # the library accepts, so a release that dropped one turns a working caller's # format into a ValueError while every behavioural case here still passes. check_eq $P/allowed-types \ "%,D,F,S,W,b,d,e,f,g,l,n,o,s,ta,tc,te,tg,th,ti,ts,tt,w,x" \ "$(pv allowed_types)" check_eq $P/allowed-types-count "24" "$(pv allowed_types_count)" # ------------------------------------------------------------- the contract --- # THE ROUND TRIP. Each case asserts what `str.format' rendered and whether # `parse' gave the value back equal, so a red says which half moved. The # rendered value is bracketed because three of them carry trailing spaces. check_eq $P/rt-plain-rendered "[a and b]" "$(pv rt_plain_rendered)" check_eq $P/rt-plain "yes" "$(pv rt_plain_eq)" check_eq $P/rt-int-rendered "[42]" "$(pv rt_int_rendered)" check_eq $P/rt-int "yes" "$(pv rt_int_eq)" check_eq $P/rt-int-neg-rendered "[-7]" "$(pv rt_int_neg_rendered)" check_eq $P/rt-int-neg "yes" "$(pv rt_int_neg_eq)" check_eq $P/rt-float-rendered "[3.500000]" "$(pv rt_float_rendered)" check_eq $P/rt-float "yes" "$(pv rt_float_eq)" check_eq $P/rt-exp-rendered "[1.234500e+03]" "$(pv rt_exp_rendered)" check_eq $P/rt-exp "yes" "$(pv rt_exp_eq)" check_eq $P/rt-hex-rendered "[ff]" "$(pv rt_hex_rendered)" check_eq $P/rt-hex "yes" "$(pv rt_hex_eq)" check_eq $P/rt-oct-rendered "[100]" "$(pv rt_oct_rendered)" check_eq $P/rt-oct "yes" "$(pv rt_oct_eq)" check_eq $P/rt-bin-rendered "[101]" "$(pv rt_bin_rendered)" check_eq $P/rt-bin "yes" "$(pv rt_bin_eq)" check_eq $P/rt-pct-rendered "[50.000000%]" "$(pv rt_pct_rendered)" check_eq $P/rt-pct "yes" "$(pv rt_pct_eq)" # NAMED FIELDS, compared as a dict rather than as a tuple -- `named' and # `fixed' are separate collections and this is the case that goes through the # first one. check_eq $P/rt-named-rendered "[bob is 34]" "$(pv rt_named_rendered)" check_eq $P/rt-named "yes" "$(pv rt_named_eq)" # WIDTH AND ALIGNMENT ON A TYPED FIELD, where the padding IS stripped and the # round trip closes. Six of them, covering the width, the zero-pad, all three # alignments and a custom fill. check_eq $P/rt-int-width-rendered "[ 42]" "$(pv rt_int_width_rendered)" check_eq $P/rt-int-width "yes" "$(pv rt_int_width_eq)" check_eq $P/rt-int-zero-rendered "[00042]" "$(pv rt_int_zero_rendered)" check_eq $P/rt-int-zero "yes" "$(pv rt_int_zero_eq)" check_eq $P/rt-int-right-rendered "[ 42]" "$(pv rt_int_right_rendered)" check_eq $P/rt-int-right "yes" "$(pv rt_int_right_eq)" check_eq $P/rt-int-left-rendered "[42 ]" "$(pv rt_int_left_rendered)" check_eq $P/rt-int-left "yes" "$(pv rt_int_left_eq)" check_eq $P/rt-int-centre-rendered "[ 42 ]" "$(pv rt_int_centre_rendered)" check_eq $P/rt-int-centre "yes" "$(pv rt_int_centre_eq)" check_eq $P/rt-int-fill-rendered "[****42*****]" "$(pv rt_int_fill_rendered)" check_eq $P/rt-int-fill "yes" "$(pv rt_int_fill_eq)" # ...AND THE THREE THAT DO NOT CLOSE. See the header. These are assertions that # the round trip FAILS, and each is a documented upstream behaviour rather than # a defect of this port -- so a release that fixed one announces itself here # instead of somewhere a caller would find it. check_eq $P/rt-untyped-right-rendered "[ abc]" "$(pv rt_untyped_right_rendered)" check_eq $P/rt-untyped-right "no" "$(pv rt_untyped_right_eq)" check_eq $P/rt-untyped-fill-rendered "[****abc****]" "$(pv rt_untyped_fill_rendered)" check_eq $P/rt-untyped-fill "no" "$(pv rt_untyped_fill_eq)" check_eq $P/rt-str-type-width-rendered "[ abc]" "$(pv rt_str_type_width_rendered)" check_eq $P/rt-str-type-width "no" "$(pv rt_str_type_width_eq)" # THE OTHER DIRECTION OF THE SAME ASYMMETRY: `{:l}' is this library's letters # type and is not a `str.format' code at all, so the rendering half raises. check_eq $P/rt-l-not-a-format-code "[]" \ "$(pv rt_l_not_a_format_code_rendered)" check_eq $P/rt-l-not-a-format-code-eq "no" "$(pv rt_l_not_a_format_code_eq)" # ----------------------------------------------------------- the converters --- # EACH TYPE'S VALUE AND ITS PYTHON TYPE. The value alone would pass on a # library that returned the matched text -- `{:d}' giving the STRING "42" # satisfies every equality a shell can write -- so the type is the half that # says the conversion happened. check_eq $P/conv-d "42" "$(pv conv_d)" check_eq $P/conv-d-type "int" "$(pv conv_d_type)" # The "#" prefix is handled automatically for `d': all three bases, one type. check_eq $P/conv-d-hex "42" "$(pv conv_d_hex)" check_eq $P/conv-d-oct "42" "$(pv conv_d_oct)" check_eq $P/conv-d-bin "42" "$(pv conv_d_bin)" # ...and all three reach an INT, not the prefixed text. check_eq $P/conv-d-hex-type "int" "$(pv conv_d_hex_type)" check_eq $P/conv-d-oct-type "int" "$(pv conv_d_oct_type)" check_eq $P/conv-d-bin-type "int" "$(pv conv_d_bin_type)" check_eq $P/conv-D "abc" "$(pv conv_D)" check_eq $P/conv-D-type "str" "$(pv conv_D_type)" # `n' is the answer to the thousands separator `err-bad-spec' refuses. check_eq $P/conv-n "1234567" "$(pv conv_n)" check_eq $P/conv-n-type "int" "$(pv conv_n_type)" # A PERCENTAGE IS DIVIDED BY 100, which is the conversion a caller most often # does not expect to be done for them. check_eq $P/conv-pct "0.5" "$(pv conv_pct)" check_eq $P/conv-pct-type "float" "$(pv conv_pct_type)" check_eq $P/conv-f "3.5" "$(pv conv_f)" check_eq $P/conv-f-type "float" "$(pv conv_f_type)" # `F' IS Decimal, NOT float, and that is the difference between the two that a # caller doing money arithmetic needs. check_eq $P/conv-F "3.5" "$(pv conv_F)" check_eq $P/conv-F-type "Decimal" "$(pv conv_F_type)" check_eq $P/conv-e "1.1e-10" "$(pv conv_e)" check_eq $P/conv-e-type "float" "$(pv conv_e_type)" check_eq $P/conv-g "1.5" "$(pv conv_g)" check_eq $P/conv-g-type "float" "$(pv conv_g_type)" check_eq $P/conv-b "11" "$(pv conv_b)" check_eq $P/conv-b-type "int" "$(pv conv_b_type)" check_eq $P/conv-o "15" "$(pv conv_o)" check_eq $P/conv-o-type "int" "$(pv conv_o_type)" check_eq $P/conv-x "255" "$(pv conv_x)" check_eq $P/conv-x-type "int" "$(pv conv_x_type)" # Upper and lower case hex reach the same int -- the type table says "lower and # upper case" and this is the case that holds it to that. check_eq $P/conv-x-upper "255" "$(pv conv_x_upper)" check_eq $P/conv-x-upper-type "int" "$(pv conv_x_upper_type)" check_eq $P/conv-w "he_llo1" "$(pv conv_w)" check_eq $P/conv-w-type "str" "$(pv conv_w_type)" check_eq $P/conv-W "!!!" "$(pv conv_W)" check_eq $P/conv-W-type "str" "$(pv conv_W_type)" check_eq $P/conv-l "hello" "$(pv conv_l)" check_eq $P/conv-l-type "str" "$(pv conv_l_type)" # `s' AND `S'. See the header: `s' is WHITESPACE, and its value is reported by # shape because a value made only of spaces does not survive a shell. check_eq $P/conv-s-len "3" "$(pv conv_s_len)" check_eq $P/conv-s-is-spaces "yes" "$(pv conv_s_is_spaces)" check_eq $P/conv-s-type "str" "$(pv conv_s_type)" check_eq $P/conv-S "a-b_c!" "$(pv conv_S)" check_eq $P/conv-S-type "str" "$(pv conv_S_type)" # ...and the case that says it out loud: `{:s}' does not match a word. check_eq $P/type-s-on-word "" "$(pv type_s_on_word)" # ------------------------------------------------------ the date/time types --- # Reported through `isoformat()', so each value is ASCII and carries its offset. # These are the types with the most upstream machinery behind them -- eight # regexes and a hand-written tzinfo -- and `ti' is the one the ticket names. check_eq $P/dt-ti "1972-01-20T10:21:36+00:00" "$(pv dt_ti)" check_eq $P/dt-ti-type "datetime" "$(pv dt_ti_type)" # The "T" and the "Z" are both optional, and without the "Z" the result is # NAIVE -- no tzinfo at all, rather than UTC. A caller who assumed UTC is wrong # by their own local offset, so the two forms are asserted apart. check_eq $P/dt-ti-naive "1972-01-20T10:21:36" "$(pv dt_ti_naive)" check_eq $P/dt-ti-naive-type "datetime" "$(pv dt_ti_naive_type)" check_eq $P/dt-te "1972-01-20T10:21:36+10:00" "$(pv dt_te)" check_eq $P/dt-te-type "datetime" "$(pv dt_te_type)" check_eq $P/dt-tg "1972-01-20T10:21:36+01:00" "$(pv dt_tg)" check_eq $P/dt-tg-type "datetime" "$(pv dt_tg_type)" # `tg' is day/month and `ta' is month/day, on inputs that differ only in which # way round the first two numbers go. This is the pair a caller gets silently # wrong for eleven days of every month. check_eq $P/dt-ta "1972-01-20T22:21:36+10:30" "$(pv dt_ta)" check_eq $P/dt-ta-type "datetime" "$(pv dt_ta_type)" check_eq $P/dt-tc "1973-09-16T01:03:52" "$(pv dt_tc)" check_eq $P/dt-tc-type "datetime" "$(pv dt_tc_type)" check_eq $P/dt-th "2011-11-21T00:07:11+00:00" "$(pv dt_th)" check_eq $P/dt-th-type "datetime" "$(pv dt_th_type)" # `tc' carries no offset at all, so its datetime is naive -- that is ctime(3)'s # format rather than a loss here. # `tt' is a TIME, not a datetime -- a different class out of the same family. check_eq $P/dt-tt "22:21:36-05:30" "$(pv dt_tt)" check_eq $P/dt-tt-type "time" "$(pv dt_tt_type)" # A STRFTIME FORMAT AS THE TYPE is a separate code path from the eight above: # the regex is built from the 1989 C directives rather than looked up. What it # RETURNS is decided by which directives the format carries, and all three # classes are reachable. check_eq $P/dt-strf-datetime "2023-11-23T12:56:47" "$(pv dt_strf_datetime)" check_eq $P/dt-strf-datetime-type "datetime" "$(pv dt_strf_datetime_type)" check_eq $P/dt-strf-time "10:26:00" "$(pv dt_strf_time)" check_eq $P/dt-strf-time-type "time" "$(pv dt_strf_time_type)" check_eq $P/dt-strf-date "2023-11-25" "$(pv dt_strf_date)" check_eq $P/dt-strf-date-type "date" "$(pv dt_strf_date_type)" # -------------------------------------------------------------- the anchoring --- # See the header. All three answers come from ONE string, so they cannot be # about three different inputs. check_eq $P/anchor-parse "" "$(pv anchor_parse)" check_eq $P/anchor-parse-whole "42" "$(pv anchor_parse_whole)" check_eq $P/anchor-search "42" "$(pv anchor_search)" # `search' returns the FIRST match; `findall' iterates all of them. check_eq $P/search-first "1" "$(pv search_first)" check_eq $P/findall-all "1,2,3" "$(pv findall_all)" check_eq $P/findall-count "3" "$(pv findall_count)" # `findall' returns a ResultIterator, not a list -- so `len()' on it RAISES # rather than counting. A caller who wrote `len(findall(...))' gets a # TypeError, and this is the case that says so. check_eq $P/findall-type "ResultIterator" "$(pv findall_type)" check_eq $P/findall-len "" "$(pv findall_len)" # THE DOCUMENTED FAILURE PATH: None, not an exception and not an empty Result. check_eq $P/nomatch-is-none "yes" "$(pv nomatch_is_none)" check_eq $P/nomatch-search "yes" "$(pv nomatch_search)" check_eq $P/nomatch-findall-count "0" "$(pv nomatch_findall_count)" # ...and the case that makes `is None' the correct test rather than a # preference: a format with no fields matches and yields an EMPTY Result, which # is TRUTHY. `if parse(...)' is right until the day the format has no fields. check_eq $P/empty-result-bool "yes" "$(pv empty_result_bool)" check_eq $P/empty-result-repr "" "$(pv empty_result_repr)" # ----------------------------------------------------------------- the Result --- # THE THREE ACCESSORS. `fixed' is the positional tuple, `named' the dict, and # `spans' the offsets into the input -- the last is what a caller doing # highlighting or replacement needs, and nothing else in this suite reaches it. check_eq $P/result-repr "" "$(pv result_repr)" check_eq $P/result-fixed "a,b" "$(pv result_fixed)" check_eq $P/result-named "name=42" "$(pv result_named)" check_eq $P/result-spans "0:0-1,1:5-6,name:2-4" "$(pv result_spans)" # Indexing takes an int for a positional field and a str for a named one, off # the SAME object; a name that is not there raises KeyError rather than # returning None. check_eq $P/result-index-0 "a" "$(pv result_index_0)" check_eq $P/result-index-name "42" "$(pv result_index_name)" check_eq $P/result-index-missing "" "$(pv result_index_missing)" check_eq $P/result-contains "yes" "$(pv result_contains)" # `evaluate_result=False' RETURNS A Match, NOT A Result, and defers the type # conversion until `evaluate_result()' is called. That is the documented way to # test a match without paying for the converters, so the deferral is the # contract rather than an internal. `Match' defines no `__repr__', so its # default one carries an ADDRESS and is deliberately not asserted. check_eq $P/match-type "Match" "$(pv match_type)" check_eq $P/match-evaluated "42" "$(pv match_evaluated)" check_eq $P/match-evaluated-type "int" "$(pv match_evaluated_type)" # ------------------------------------------------------------------ compile() --- # A COMPILED PARSER IS REUSABLE AND GIVES THE SAME ANSWER AS THE ONE-SHOT CALL. # `compile' is the documented way to hoist the regex build out of a loop, so # what has to hold is that hoisting it changes NOTHING -- `compile-agrees' # compares the two on the same input rather than each against a literal. check_eq $P/compile-type "Parser" "$(pv compile_type)" check_eq $P/compile-repr "" "$(pv compile_repr)" check_eq $P/compile-parse "42,abc" "$(pv compile_parse)" check_eq $P/compile-agrees "yes" "$(pv compile_agrees)" # The same object, three times, on different input -- which is the whole point. check_eq $P/compile-reuse "1,2,3" "$(pv compile_reuse)" # A compiled parser carries the same search/findall surface as the module. check_eq $P/compile-search "42" "$(pv compile_search)" check_eq $P/compile-findall "1,2" "$(pv compile_findall)" # ...and it exposes the regex it built and the fields it found, which is what # makes `compile' worth hoisting rather than an alias for `parse'. check_eq $P/compile-has-expression "yes" "$(pv compile_has_expression)" check_eq $P/compile-named-fields "a,b" "$(pv compile_named_fields)" check_eq $P/compile-fixed-fields "2" "$(pv compile_fixed_fields)" # --------------------------------------------------------------- extra_types --- # A CUSTOM CONVERTER. `with_pattern' attaches the regex a converter matches to # the converter itself; without it `extra_types' has a function and no way to # know what text to hand it. This is the extension point, and the one part of # the public API a caller writes code AGAINST rather than calls. check_eq $P/extra-pattern "\\d+" "$(pv extra_pattern)" check_eq $P/extra-value "42" "$(pv extra_value)" check_eq $P/extra-type "int" "$(pv extra_type)" # ...and through a compiled parser, which is where a caller would actually put # it: `extra_types' is a constructor argument, not a per-call one. check_eq $P/extra-compiled "42,ok" "$(pv extra_compiled)" # A converter with a MULTI-GROUP pattern must declare its group count, or the # numbering downstream of it shifts and the WRONG TEXT reaches the next field. # `extra-group-value' is that next field, and it is what would red. check_eq $P/extra-group-count "2" "$(pv extra_group_count)" check_eq $P/extra-group-value "7,ok" "$(pv extra_group_value)" # AN EXTRA TYPE WHOSE CONVERTER RAISES IS NOT CAUGHT. The exception reaches the # caller, so a caller who assumed `parse()' returns None on every failure is # wrong here -- which is exactly the caller `nomatch-is-none' above taught. check_eq $P/extra-raises "" "$(pv extra_raises)" # ------------------------------------------------------- alignment and width --- # THE DOCUMENTED ALIGNMENT BEHAVIOUR, on upstream's own examples: alignment # WITHOUT a width strips the fill, and the "centre" form does not check that # the value is centred -- it strips both ends. check_eq $P/align-right "a" "$(pv align_right)" check_eq $P/align-centre "lovely" "$(pv align_centre)" check_eq $P/align-left "a" "$(pv align_left)" # WIDTH IS A MINIMUM AND PRECISION IS A MAXIMUM, which is how one format splits # a string with no separator in it at all. Upstream's own three examples, plus # two contiguous numbers -- the case with no delimiter for a regex to find. check_eq $P/width-split "look,at that" "$(pv width_split)" check_eq $P/precision-split "lo,ok" "$(pv precision_split)" check_eq $P/width-precision "look at ,that" "$(pv width_precision)" check_eq $P/width-digits "4,40" "$(pv width_digits)" # A CUSTOM FILL is stripped like whitespace, and one that is a regex # metacharacter is ESCAPED by the library rather than compiled as a class -- # `.' would otherwise match the digits themselves. check_eq $P/fill-star "42" "$(pv fill_star)" check_eq $P/fill-dot "42" "$(pv fill_dot)" # ------------------------------------------------------------------ the errors --- # A WRONG FORMAT RAISES; WRONG INPUT RETURNS None. `{:,d}' is the specific one # worth pinning: the thousands separator is valid `str.format' and is not valid # here, so it renders and will not parse back. check_eq $P/err-bad-spec "" "$(pv err_bad_spec)" check_eq $P/err-repeated-name "" "$(pv err_repeated_name)" # A repeated name with the SAME type is legal, and it constrains the two # occurrences to the same text -- a match on 'xx' and a non-match on 'xy'. That # is the useful half of the feature, and it is why the error above is about the # TYPE rather than about the repeat. check_eq $P/repeat-same-text "" "$(pv repeat_same_text)" check_eq $P/repeat-diff-text "" "$(pv repeat_diff_text)" # TooManyFields IS UNREACHABLE ON THIS INTERPRETER -- see the header. 150 named # fields, which the exception exists to refuse, parse and all 150 come back. check_eq $P/many-named-fields "150" "$(pv many_named_fields)" # Both error classes are ValueErrors, so one `except ValueError' covers the bad # spec, the repeated name and -- were it ever raised -- the field count. check_eq $P/err-too-many-is-valueerror "yes" "$(pv err_too_many_is_valueerror)" check_eq $P/err-repeated-is-valueerror "yes" "$(pv err_repeated_is_valueerror)" # ------------------------------------------------------------- case handling --- # MATCHING IS CASE-INSENSITIVE BY DEFAULT, which is the opposite of what a # caller assumes and is the reason `case_sensitive' exists. Both directions, # on the same pair of strings. check_eq $P/case-default "matched" "$(pv case_default)" check_eq $P/case-sensitive-off "matched" "$(pv case_sensitive_off)" check_eq $P/case-sensitive-on "" "$(pv case_sensitive_on)" # ---------------------------------------------------------------- the corners --- # LITERAL BRACES. `{{' and `}}' are the escape, exactly as in `str.format' -- # what a format string carrying JSON or a C brace needs. check_eq $P/braces "x" "$(pv braces)" # A DOTTED AND AN INDEXED NAME. `{a.b}' and `{c[0]}' are accepted, and the key # in `named' is the WHOLE dotted string rather than a nested structure -- a # caller expecting result.named["a"]["b"] gets nothing. The indexed form loses # its subscript entirely and lands under `c'. check_eq $P/dotted-keys "a.b,c" "$(pv dotted_keys)" check_eq $P/dotted-value "x" "$(pv dotted_value)" # AN EMPTY FIELD MATCHES AT LEAST ONE CHARACTER (`.+?'), so a format asking for # a value where the input has none does not match at all. check_eq $P/empty-input "" "$(pv empty_input)" check_eq $P/one-char "x" "$(pv one_char)" # POSITIONAL AND NAMED FIELDS MIX in one format and stay in separate # collections -- `fixed' does not renumber around the named one. check_eq $P/mixed-fixed-count "2" "$(pv mixed_fixed_count)" check_eq $P/mixed-named-count "1" "$(pv mixed_named_count)" rm -rf "$W"