. "$BST_LIB" # BST-TIMEOUT: 1200 # THREE python3 starts, and they are the whole cost. Every # # other assertion below reads the filesystem or dpkg with # # shell builtins, or greps a line out of the transcript the # # third start left behind. # # # # The three are: a bare `import', a second one whose STDERR # # is the assertion (`import-quiet'), and the probe, which # # takes ONE start for 256 measurements. # # # # The work inside each start is regex construction and # # `str.split' over string literals under 64 characters -- # # `parse.parse' compiles a pattern of the same order. There # # is NO filesystem access, no network, no device and no # # clock anywhere in the package, so nothing here is even # # I/O-bound. The interpreter start is the only term the # # number has to cover. # # # # 1200 is 400s per start, the same per-start budget # # 99-python3-appdirs sets for its three, and roughly six # # times the ~60s a python3 start has measured at here. # # # # A suite-timeout red is a batch-wide cost, so if the three # # starts prove slower than that on target the RIGHT fix is # # this number, not fewer cases. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify, and the measured elapsed replaces # # this note in the test plan rather than in this comment, # # which would cost an image build to keep the shipped suite # # byte-identical. # # python3-parse-type 0.6.2 (meta-openembedded meta-python/recipes-devtools/ # python, python3-parse-type_0.6.2.bb, `inherit pypi ptest setuptools3', PyPI # sdist) on the wasm32-hwjs glibc image. # # ============================================================================ # WHAT THE PACKAGE ACTUALLY IS, AND THEREFORE WHAT HAS TO BE TESTED. # ============================================================================ # # parse_type BUILDS FUNCTIONS. It is a factory for `parse' type converters: you # hand it a mapping, a list of strings, a list of converters or a converter for # one item, and it hands back a callable carrying a `.pattern' attribute that # `parse.Parser' splices into the regex it generates for a `{name:Type}' field. # # So EVERY converter it builds has TWO halves, and they can disagree: # # the PATTERN decides whether `parse.parse' matches the text at all, and # therefore whether the converter is reached; # the CALLABLE decides what the matched text becomes, and what happens when # it cannot be converted. # # A suite that only called the converters would measure the second half and # assert nothing about the first -- and the first is where the package's two # layers disagree with each other. So every case below that builds a converter # asserts its GENERATED PATTERN as a literal string, and a second block hands # the same converters to `parse.parse' and asserts what the parser does. # # TWO LAYERS, AND KEEPING THEM APART IS THE POINT: # # parse_type.cardinality Cardinality (an enum of one, 0..1, 0..*, 1..*) and # a TypeBuilder whose `with_*' methods derive a # converter for T-with-a-cardinality from the # converter for one T. The derived pattern is built # FROM THE ITEM PATTERN, so the parser rejects the # wrong shape before the converter runs. # # parse_type.builder TypeBuilder (a SUBCLASS of the above) adding the # composite constructors -- make_enum, make_choice, # make_choice2, make_variant, make_list -- plus # build_type_dict and parse_anything. # # ============================================================================ # FIVE BEHAVIOURS A CALLER WILL GET WRONG. THESE ARE THE POINT OF THE SUITE. # ============================================================================ # # make_list THROWS THE ITEM PATTERN AWAY. # `with_many(parse_number)' generates `(\d+)(\s*,\s*(\d+))*' # and `make_list(parse_number)' generates # `(.+?)(\s*,\s*(.+?))*' -- from the SAME converter. make_list # passes `pattern=anything_pattern' deliberately ("the parser # accepts anything and the converter needs to fail on errors", # its own docstring), so the two spellings of one intent have # DIFFERENT FAILURE MODES: `{n:Number+}' on `L: a, b' returns # None, and `{n:NumberList}' on the same text lets `int()' # raise ValueError OUT OF `parse.parse'. `p_many_letters' and # `p_mklist_letters' sit next to each other so that is legible # as one defect with two shapes. # # make_list("") IS `['']', NOT `[]'. # make_list is built on Cardinality.one_or_more, whose # converter splits unconditionally with no empty guard, so the # empty string becomes a ONE-item list holding the empty # string. `with_many0' guards and returns `[]'. A caller who # reads "make_list" as "a list, possibly empty" gets a list of # length 1. Asserted as a pair with `many0_empty_len'. # # make_enum RAISES KeyError; make_choice RAISES ValueError. # The same shape of mistake -- text that is not one of the # permitted values -- comes back as two different exception # types from two constructors in one class. A caller who # wrote one `except ValueError' for both is catching half of # them, and the half they are missing escapes `parse.parse'. # # make_enum's CASE FALLBACK IS ONE-WAY. # The parser matches under re.IGNORECASE, so the converter # retries `text.lower()' on a miss. That makes an ALL- # LOWERCASE mapping case-insensitive and leaves a mapping with # an uppercase key -- `{"Yes": True}' -- unable to convert its # own `yes': there is no second retry. Such a converter has a # pattern the parser matches and a callable that then raises. # # make_variant's NO-MATCH ARM IS A BARE `assert'. # `AssertionError("OOPS-VARIANT-MISMATCH: ...")' under a # normal interpreter, and under `python3 -O' the assert is # COMPILED OUT and the function falls through to `return # None'. So the same call is a raise or a silent None # depending on a flag nothing in the library reads. # `variant-optimize-flag' records which interpreter measured # the cases below, so a future image built with -O turns that # one case red rather than turning the verdict silently wrong. # # ============================================================================ # THE RECIPE UNDER-DECLARES ITS RUNTIME DEPENDENCY ON python3-six. # ============================================================================ # # The built .deb reads `Depends: python3-core, python3-parse'. Three of the six # modules in the package import `six' AT MODULE SCOPE -- cardinality_field.py, # parse_util.py and cfparse.py -- so on an image without python3-six they are # unimportable, while `import parse_type' itself still works because # `__init__.py' reaches only cardinality and builder. # # MEASURED rather than read off the source, on an x86-64 control against the # tree unpacked from the built .deb, with a meta_path hook refusing `six' # exactly as a rootfs without that split would: # # parse_type imported OK without six # parse_type.builder imported OK without six # parse_type.cardinality imported OK without six # parse_type.cardinality_field ModuleNotFoundError: No module named 'six' # parse_type.cfparse ModuleNotFoundError: No module named 'six' # parse_type.parse_util ModuleNotFoundError: No module named 'six' # # python3-parse-type_%.bbappend in this layer declares it. THIS IMAGE INSTALLS # python3-six FOR OTHER REASONS (BRINTOS-1744), so declaring it changes no # rootfs and the omission would have stayed invisible here -- it makes the # PACKAGE correct rather than the image different, the same reasoning already # recorded for python3-aniso8601 and python3-distlib in that directory. The # `pkg-depends-exact' case asserts the repaired line by name, so a bbappend that # stopped being applied says so in one case rather than presenting as a broken # library. # # The rest of the closure is covered by the two inherited entries. Measured as # the sys.modules delta across each module's import from a pristine interpreter # and scored against the poky python3 manifest: `import parse_type' reaches # ast, contextlib, copyreg, dis, enum, inspect, linecache, opcode, re, token and # tokenize, ALL of which poky puts in the `core' split; the cfparse arm adds # datetime, decimal, logging, numbers and string, which arrive through # python3-parse's own RDEPENDS. Nothing outside the standard library is reached # but `parse' and `six'. # # ============================================================================ # THERE ARE TWO `parse' MODULES ON THIS ROOTFS, AND THE SUITE SAYS WHICH ANSWERED. # ============================================================================ # # parse_type VENDORS ITS OWN COPY of the parse module at # `parse_type/parse.py' -- a fork of parse 1.19.1, kept for a group-index fix # its own docstring points at -- and python3-parse installs the real one, 1.20.1, # at `site-packages/parse.py'. A top-level `import parse' must reach the LATTER # or every expectation here is about a different library than the one the recipe # RDEPENDS on. `parse-file', `parse-is-vendored' and `parse-version' assert that # directly, and `vendored-version' asserts the fork is still the older one, so a # release that started shadowing the real module turns three cases red rather # than changing answers nobody is watching. # # ============================================================================ # WHY NO FIXTURE IS SHIPPED AND NONE IS BUILT. # ============================================================================ # # There is nothing to build. A converter factory needs no input but string # literals, and every input below is one -- so this suite reaches no device, no # peer, no clock and no writable directory, and its answers cannot depend on # what the runner's environment happened to be. That is the whole reason it is # deterministic. The probe makes no `getenv' call at all. # # ============================================================================ # THE UPSTREAM ptest IS INHERITED AND PRODUCES NOTHING HERE. # ============================================================================ # # The recipe says `inherit pypi ptest setuptools3' and defines # `do_install_ptest', but no -ptest .deb is produced on this arm -- the same # shape python3-parse next door has, and the reason there are no pytest cases # below. `no-ptest-dir' asserts the negative on the rootfs. # # ============================================================================ P=python3-parse-type # The canary is first so that a suite which dies before its own first assertion # is distinguishable from one that never started. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages PKG=$SITE/parse_type DI=$SITE/parse_type-0.6.2.dist-info PKGLIST=/var/lib/dpkg/info/python3-parse-type.list PROBE=$BST_DATA/python3-parse-type-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-parse-type.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ---------------------------------------------------------------- layout ----- # A PACKAGE, NOT A FLAT MODULE -- the opposite shape from python3-appdirs next # door, and the reason `is-package' below is a case rather than an assumption. check $P/site-packages-dir test -d "$SITE" check $P/package-dir test -d "$PKG" check $P/not-a-module test ! -e "$SITE/parse_type.py" # THE SIX MODULES, EACH ASSERTED BY NAME. A release that split or merged one # would keep every behavioural case below green. check $P/mod-init test -s "$PKG/__init__.py" check $P/mod-builder test -s "$PKG/builder.py" check $P/mod-cardinality test -s "$PKG/cardinality.py" check $P/mod-cardinality-field test -s "$PKG/cardinality_field.py" check $P/mod-cfparse test -s "$PKG/cfparse.py" check $P/mod-parse-util test -s "$PKG/parse_util.py" # THE VENDORED parse FORK IS SHIPPED. It is dead weight on this image -- nothing # in the package imports it, and `parse-is-vendored' asserts the real one wins -- # but it is 34 kB of source that a caller could import deliberately, so its # presence is asserted rather than left to be discovered. check $P/mod-vendored-parse test -s "$PKG/parse.py" # Upstream's tests/ are NOT packaged: setup.py's `packages' names parse_type # alone, so the ~40 test modules in the sdist reach no image. check $P/no-tests-dir test ! -d "$PKG/tests" check $P/no-test-dir test ! -d "$SITE/tests" check $P/no-py-typed test ! -e "$PKG/py.typed" # ...and the inherited `ptest' produces no -ptest split on this arm. check $P/no-ptest-dir test ! -d /usr/lib/python3-parse-type/ptest check $P/pycache-init test -s "$PKG/__pycache__/__init__.cpython-312.pyc" check $P/pycache-builder test -s "$PKG/__pycache__/builder.cpython-312.pyc" check $P/pycache-cardinality \ test -s "$PKG/__pycache__/cardinality.cpython-312.pyc" # ------------------------------------------------------------- the .deb ------ cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" # PURE PYTHON, AND NOT MERELY "NO .so OF ITS OWN". This image's python3 is ONE # STATIC BINARY with an empty lib-dynload and no dynamic loader (BRINTOS-1476), # so a module carrying a compiled extension could not be imported here at all. # parse_type ships no C -- no .c, no .pyx, no ext_modules in setup.py -- so this # is a REGRESSION TRIPWIRE on a version bump rather than a statement about today. check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # ...and nothing but Python in the payload, which is the same claim from the # other side. setup.py declares no `entry_points' and no `scripts', so the # package installs NO executable and there is nothing to reach through a shebang # -- which is why this suite has no CLI cases at all. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 0.6.2" cat "$DI/METADATA" check_out $P/dist-info-name "Name: parse_type" cat "$DI/METADATA" check $P/dist-info-record test -s "$DI/RECORD" check $P/dist-info-license test -s "$DI/LICENSE" # `setuptools3' BUILT THIS, WHICH IS VISIBLE IN THE WHEEL METADATA and is the # half of the recipe's `inherit' line that leaves a trace on the rootfs. The # `py2-none-any' tag is upstream's `universal = 1'. `zip-safe' is a one-byte # file setuptools emits from `zip_safe=True' in setup.py -- a setuptools-era # marker that pip no longer reads, asserted because it is the one non-obvious # entry in the dist-info and its disappearance would be a packaging change. check_out $P/dist-info-generator "Generator: bdist_wheel" cat "$DI/WHEEL" check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" check_out $P/dist-info-tag-py3 "Tag: py3-none-any" cat "$DI/WHEEL" check_out $P/dist-info-tag-py2 "Tag: py2-none-any" cat "$DI/WHEEL" check_out $P/dist-info-top-level "parse_type" cat "$DI/top_level.txt" check $P/dist-info-zip-safe test -f "$DI/zip-safe" # The RECORD lists the six modules AND the vendored fork -- seven .py files, no # more. A packaging change that started dropping or adding one says so here. check_eq $P/record-py-count "7" \ "$(grep -c '^parse_type/.*\.py,' "$DI/RECORD" 2>/dev/null)" sed -n '/^Package: python3-parse-type$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" check_out $P/pkg-version "Version: 0.6.2-r0" cat "$W/st" # THE RDEPENDS IS ASSERTED EXACTLY, and the line asserted is the REPAIRED one -- # see the header. Stock meta-python gives `python3-core, python3-parse'; the # bbappend in this layer adds python3-six, which three of the six modules import # at module scope. A bbappend that stopped applying turns THIS ONE CASE red, # ahead of the three `cf-*'/`cfparse-*' blocks that would otherwise be the first # sign, and on an image that dropped python3-six it is the case that explains # them. grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_eq $P/pkg-depends-exact \ "Depends: python3-core, python3-parse, python3-six" \ "$(cat "$W/dep")" # The two runtime dependencies are on the image, asserted directly rather than # inferred from the import working: an import that fails says WHICH module is # missing only if the package it came from was named somewhere. check $P/dep-parse-installed test -s "$SITE/parse.py" check $P/dep-six-installed test -s "$SITE/six.py" # ---------------------------------------------------------------- import ----- check $P/import python3 -c 'import parse_type' # A DeprecationWarning or a stray print on import would be invisible to the case # above, which scores only the exit status. parse_type uses `from __future__ # import absolute_import' throughout, which is a no-op on python3 and warns on # none of it -- so an empty stderr is the expectation, not a hope. check_eq $P/import-quiet "" \ "$(python3 -c 'import parse_type' 2>&1)" # ----------------------------------------------------------------- probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # The probe's own two bookends. `probe-complete' says it reached its last line # and `probe-count' says it emitted every key it meant to: a probe that died # two-thirds of the way through would otherwise leave the missing cases scoring # against an empty string, which `check_eq' would read as a wrong answer rather # than as an absent one. check_eq $P/probe-complete "yes" "$(pv probe_done)" check_eq $P/probe-count "256" "$(pv emitted_count)" # -------------------------------------------------------------- identity ----- # WHICH COPY ANSWERED. Every behavioural case below is a statement about the # package named here. check_eq $P/module-answered "$PKG/__init__.py" "$(pv module_file)" check_eq $P/is-package "yes" "$(pv is_package)" check_eq $P/version "0.6.2" "$(pv version_attr)" check_eq $P/dunder-all "Cardinality,TypeBuilder,build_type_dict" "$(pv dunder_all)" # WHICH `parse' ANSWERED -- see the header. There are two on this rootfs. check_eq $P/parse-version "1.20.1" "$(pv parse_version)" check_eq $P/parse-file "$SITE/parse.py" "$(pv parse_file)" check_eq $P/parse-is-vendored "no" "$(pv parse_is_vendored)" check_eq $P/vendored-version "1.19.1" "$(pv vendored_version)" check_eq $P/vendored-is-older "yes" "$(pv vendored_is_older)" # The exported TypeBuilder is builder's, which SUBCLASSES cardinality's. A # release that flattened the two would keep every behavioural case green and # change what `from parse_type import TypeBuilder' means. check_eq $P/builder-subclasses "yes" "$(pv builder_subclasses_cardinality)" check_eq $P/builder-not-base "yes" "$(pv builder_is_not_base)" check_eq $P/default-strict "yes" "$(pv default_strict)" check_eq $P/anything-pattern '.+?' "$(pv anything_pattern)" check_eq $P/default-pattern '.+?' "$(pv default_pattern)" check_eq $P/parse-anything-pattern '.+?' "$(pv parse_anything_pattern)" check_eq $P/parse-anything-identity 'a b,c' "$(pv parse_anything_identity)" # IGNORECASE is why make_enum has a `lower()' fallback at all, and DOTALL is why # `.+?' spans a newline. Both are asserted because the enum case-fallback # failure below (`enum-upper-lower') is unreadable without them. check_eq $P/re-opts-ignorecase "yes" "$(pv re_opts_ignorecase)" check_eq $P/re-opts-dotall "yes" "$(pv re_opts_dotall)" # ----------------------------------------------------------- Cardinality ----- # FOUR MEMBERS AND SEVEN NAMES. `optional', `many' and `many0' are ALIASES, so # `Cardinality.many.name' is `one_or_more' -- a caller who logs the `.name' of # the cardinality they asked for gets a different word back. check_eq $P/card-members "one,zero_or_one,zero_or_more,one_or_more" "$(pv card_members)" check_eq $P/card-member-count "4" "$(pv card_member_count)" check_eq $P/card-name-count "7" "$(pv card_name_count)" check_eq $P/card-names \ "many,many0,one,one_or_more,optional,zero_or_more,zero_or_one" "$(pv card_names)" check_eq $P/card-alias-optional "yes" "$(pv card_alias_optional)" check_eq $P/card-alias-many "yes" "$(pv card_alias_many)" check_eq $P/card-alias-many0 "yes" "$(pv card_alias_many0)" check_eq $P/card-many-name "one_or_more" "$(pv card_many_name)" check_eq $P/card-optional-name "zero_or_one" "$(pv card_optional_name)" check_eq $P/card-many0-name "zero_or_more" "$(pv card_many0_name)" # The schema strings are the templates every generated pattern below is built # from, asserted here so a pattern change is attributable to the schema rather # than to the builder that used it. check_eq $P/card-one-schema "None" "$(pv card_one_schema)" check_eq $P/card-zoo-schema '(%s)?' "$(pv card_zoo_schema)" check_eq $P/card-zom-schema '(%s)?(\s*%s\s*(%s))*' "$(pv card_zom_schema)" check_eq $P/card-oom-schema '(%s)(\s*%s\s*(%s))*' "$(pv card_oom_schema)" check_eq $P/card-one-gc "0" "$(pv card_one_gc)" check_eq $P/card-zoo-gc "1" "$(pv card_zoo_gc)" check_eq $P/card-zom-gc "3" "$(pv card_zom_gc)" check_eq $P/card-oom-gc "3" "$(pv card_oom_gc)" check_eq $P/card-one-is-many "no" "$(pv card_one_is_many)" check_eq $P/card-zoo-is-many "no" "$(pv card_zoo_is_many)" check_eq $P/card-zom-is-many "yes" "$(pv card_zom_is_many)" check_eq $P/card-oom-is-many "yes" "$(pv card_oom_is_many)" # make_pattern over the item pattern `\d+'. Cardinality one is the IDENTITY -- # it does not wrap, which is the same fact `wc-one-is-same' states about the # converter. check_eq $P/mkpat-one '\d+' "$(pv mkpat_one)" check_eq $P/mkpat-zoo '(\d+)?' "$(pv mkpat_zoo)" check_eq $P/mkpat-zom '(\d+)?(\s*,\s*(\d+))*' "$(pv mkpat_zom)" check_eq $P/mkpat-oom '(\d+)(\s*,\s*(\d+))*' "$(pv mkpat_oom)" check_eq $P/mkpat-oom-semi '(\d+)(\s*;\s*(\d+))*' "$(pv mkpat_oom_semi)" check_eq $P/mkpat-zom-semi '(\d+)?(\s*;\s*(\d+))*' "$(pv mkpat_zom_semi)" # pattern_group_count IS TEXTUAL. It counts `(' and discounts `\(' with a string # replace -- so a NON-CAPTURING `(?:' counts as a group that will never exist, # and the over-count propagates into `regex_group_count', which parse reads to # offset its own groups. Asserted as its own case because no other case in this # suite uses a `(?:' pattern and the defect is therefore invisible from them. check_eq $P/pgc-none "0" "$(pv pgc_none)" check_eq $P/pgc-two "2" "$(pv pgc_two)" check_eq $P/pgc-escaped "0" "$(pv pgc_escaped)" check_eq $P/pgc-mixed "2" "$(pv pgc_mixed)" check_eq $P/pgc-noncapturing "1" "$(pv pgc_noncapturing)" check_eq $P/cgc-one "0" "$(pv cgc_one)" check_eq $P/cgc-zoo "1" "$(pv cgc_zoo)" check_eq $P/cgc-zom "3" "$(pv cgc_zom)" check_eq $P/cgc-oom "3" "$(pv cgc_oom)" # A grouped item pattern is counted ONCE for 0..1 and TWICE for the many arms, # because the schema splices it in twice. check_eq $P/cgc-zoo-grouped "2" "$(pv cgc_zoo_grouped)" check_eq $P/cgc-oom-grouped "5" "$(pv cgc_oom_grouped)" # ------------------------------------------------------ the item converter --- check_eq $P/item-pattern '\d+' "$(pv item_pattern)" check_eq $P/item-call "42" "$(pv item_call)" check_eq $P/item-call-bad "!ValueError" "$(pv item_call_bad)" # --------------------------------------------------- with_cardinality ------- # CARDINALITY ONE RETURNS THE CONVERTER ITSELF -- the same object, not a # wrapper. So a caller who sets an attribute on the "derived" converter has set # it on the original, which is exactly what `build_type_dict' asks them to do # with `.name'. check_eq $P/wc-one-is-same "yes" "$(pv wc_one_is_same)" check_eq $P/wc-opt-is-new "yes" "$(pv wc_opt_is_new)" check_eq $P/wc-opt-alias "yes" "$(pv wc_opt_matches_alias)" check_eq $P/wc-many-alias "yes" "$(pv wc_many_matches_alias)" check_eq $P/wc-many0-alias "yes" "$(pv wc_many0_matches_alias)" # ------------------------------------------------------- optional (0..1) ----- check_eq $P/opt-pattern '(\d+)?' "$(pv opt_pattern)" check_eq $P/opt-group-count "1" "$(pv opt_group_count)" check_eq $P/opt-value "42" "$(pv opt_value)" check_eq $P/opt-value-type "int" "$(pv opt_value_type)" # ABSENT IS None -- AND SO IS WHITESPACE. The converter strips first and treats # the result as absent, so a field that matched only spaces is indistinguishable # from one that matched nothing. `None' as an argument is accepted too, which # is what lets the parser hand an unmatched optional group straight through. check_eq $P/opt-empty "None" "$(pv opt_empty)" check_eq $P/opt-spaces "None" "$(pv opt_spaces)" check_eq $P/opt-none-arg "None" "$(pv opt_none_arg)" check_eq $P/opt-stripped "42" "$(pv opt_stripped)" check_eq $P/opt-bad "!ValueError" "$(pv opt_bad)" # ------------------------------------------------------- many0 (0..*) ----- check_eq $P/many0-pattern '(\d+)?(\s*,\s*(\d+))*' "$(pv many0_pattern)" check_eq $P/many0-group-count "3" "$(pv many0_group_count)" check_eq $P/many0-empty "[]" "$(pv many0_empty)" check_eq $P/many0-spaces "[]" "$(pv many0_spaces)" check_eq $P/many0-one "[1]" "$(pv many0_one)" check_eq $P/many0-three "[1, 2, 3]" "$(pv many0_three)" check_eq $P/many0-type "list" "$(pv many0_type)" check_eq $P/many0-bad "!ValueError" "$(pv many0_bad)" # ------------------------------------------------------- many (1..*) ----- check_eq $P/many-pattern '(\d+)(\s*,\s*(\d+))*' "$(pv many_pattern)" check_eq $P/many-group-count "3" "$(pv many_group_count)" check_eq $P/many-one "[1]" "$(pv many_one)" check_eq $P/many-three "[1, 2, 3]" "$(pv many_three)" # The separator is split on FIRST and each part stripped after, so the spacing # in the text is irrelevant to the result -- which is why the generated pattern # has to carry `\s*' on both sides of the separator for the parser to agree. check_eq $P/many-tight "[1, 2, 3]" "$(pv many_tight)" check_eq $P/many-padded "[1, 2]" "$(pv many_padded)" # ...AND EMPTY IS NOT `[]' HERE. `with_one_or_more' has NO empty guard, so `' # becomes a one-item list holding `', and `int()' then raises. `many0' guards. # This asymmetry is the header's second behaviour and it reaches the public # constructor at `mklist-empty' below. check_eq $P/many-empty "!ValueError" "$(pv many_empty)" check_eq $P/many-semi-pattern '(\d+)(\s*;\s*(\d+))*' "$(pv many_semi_pattern)" check_eq $P/many-semi-value "[1, 2, 3]" "$(pv many_semi_value)" check_eq $P/many0-semi-pattern '(\d+)?(\s*;\s*(\d+))*' "$(pv many0_semi_pattern)" check_eq $P/many0-semi-value "[1, 2]" "$(pv many0_semi_value)" # ------------------------------------------ a converter with NO .pattern ----- # The fallback is `default_pattern' (`.+?'), NOT a failure -- so a plain # undecorated function silently gets a converter whose parser matches anything, # and the only thing rejecting the wrong text is the callable. An explicit # `pattern=' argument restores the protection. check_eq $P/nopat-has-pattern "no" "$(pv nopat_has_pattern)" check_eq $P/nopat-opt-pattern '(.+?)?' "$(pv nopat_opt_pattern)" check_eq $P/nopat-many-pattern '(.+?)(\s*,\s*(.+?))*' "$(pv nopat_many_pattern)" check_eq $P/nopat-explicit-pattern '(\d+)?' "$(pv nopat_explicit_pattern)" # ------------------------------------------------------------- make_list ----- # make_list IS NOT with_many -- see the header. Same item converter, different # generated pattern, because make_list passes `pattern=anything_pattern'. check_eq $P/mklist-pattern '(.+?)(\s*,\s*(.+?))*' "$(pv mklist_pattern)" check_eq $P/mklist-group-count "3" "$(pv mklist_group_count)" check_eq $P/mklist-pattern-differs "yes" "$(pv mklist_pattern_differs_from_many)" check_eq $P/mklist-pattern-anything-based "yes" "$(pv mklist_pattern_is_anything_based)" check_eq $P/mklist-one "[1]" "$(pv mklist_one)" check_eq $P/mklist-three "[1, 2, 3]" "$(pv mklist_three)" check_eq $P/mklist-bad "!ValueError" "$(pv mklist_bad)" # EMPTY IS `[]' FROM NOBODY. make_list is one_or_more, so the empty string is a # ONE-item list -- and with the default item converter (`parse_anything') that # item is the EMPTY STRING rather than a raise. The pair with `many0-empty-len' # is the whole of the claim. check_eq $P/mklist-default-pattern '(.+?)(\s*,\s*(.+?))*' "$(pv mklist_default_pattern)" check_eq $P/mklist-empty "['']" "$(pv mklist_empty)" check_eq $P/mklist-empty-len "1" "$(pv mklist_empty_len)" check_eq $P/many0-empty-len "0" "$(pv many0_empty_len)" check_eq $P/mklist-any-one "['a']" "$(pv mklist_any_one)" check_eq $P/mklist-any-three "['a', 'b', 'c']" "$(pv mklist_any_three)" check_eq $P/mklist-semi-pattern '(.+?)(\s*;\s*(.+?))*' "$(pv mklist_semi_pattern)" check_eq $P/mklist-semi-value "[1, 2, 3]" "$(pv mklist_semi_value)" # ...and once a custom separator is set the DEFAULT is not honoured as well: a # comma in the text is then part of an item, and `int()' rejects it. check_eq $P/mklist-semi-comma "!ValueError" "$(pv mklist_semi_comma)" # ------------------------------------------------------------- make_enum ----- # OVER A DICT: the pattern is the keys joined with `|' IN DICT ORDER, and the # converter returns the mapped VALUE. check_eq $P/enum-dict-pattern "yes|no" "$(pv enum_dict_pattern)" check_eq $P/enum-dict-mappings "no,yes" "$(pv enum_dict_mappings)" check_eq $P/enum-dict-yes "True" "$(pv enum_dict_yes)" check_eq $P/enum-dict-no "False" "$(pv enum_dict_no)" check_eq $P/enum-dict-yes-type "bool" "$(pv enum_dict_yes_type)" # A NON-MEMBER RAISES KeyError, NOT ValueError -- the header's third behaviour. # make_choice raises ValueError for the same shape of mistake, and the message # here is the bare repr of the text rather than a sentence, because KeyError's # `str()' is its argument's repr. check_eq $P/enum-dict-missing "!KeyError" "$(pv enum_dict_missing)" check_eq $P/enum-dict-missing-msg "'maybe'" "$(pv enum_dict_missing_msg)" # THE CASE FALLBACK, AND WHERE IT STOPS -- the header's fourth behaviour. An # all-lowercase mapping is case-insensitive... check_eq $P/enum-dict-upper "True" "$(pv enum_dict_upper)" check_eq $P/enum-dict-mixed "True" "$(pv enum_dict_mixed)" # ...and a mapping with an UPPERCASE key is not, in either direction: lowering # `Yes' finds nothing and there is no second retry. Such a converter has a # pattern the parser matches and a callable that cannot convert what it matched. check_eq $P/enum-upper-pattern "Yes|No" "$(pv enum_upper_pattern)" check_eq $P/enum-upper-exact "True" "$(pv enum_upper_exact)" check_eq $P/enum-upper-lower "!KeyError" "$(pv enum_upper_lower)" check_eq $P/enum-upper-upper "!KeyError" "$(pv enum_upper_upper)" # OVER AN enum.Enum CLASS: `__members__' is taken, so the pattern is the member # NAMES and the converted value is the MEMBER ITSELF -- not its `.value', and # not a copy: `enum-cls-identity' asserts `is Color.red'. check_eq $P/enum-cls-pattern "red|green|blue" "$(pv enum_cls_pattern)" check_eq $P/enum-cls-red "red" "$(pv enum_cls_red)" check_eq $P/enum-cls-red-value "1" "$(pv enum_cls_red_value)" check_eq $P/enum-cls-green "green" "$(pv enum_cls_green)" check_eq $P/enum-cls-blue "blue" "$(pv enum_cls_blue)" check_eq $P/enum-cls-is-member "yes" "$(pv enum_cls_is_member)" check_eq $P/enum-cls-identity "yes" "$(pv enum_cls_identity)" check_eq $P/enum-cls-missing "!KeyError" "$(pv enum_cls_missing)" check_eq $P/enum-cls-upper "red" "$(pv enum_cls_upper)" # The mapping taken from an Enum class is that class's own `__members__' proxy, # so it is READ-ONLY: a caller cannot extend the converter after building it, # which they CAN do for the dict form. check_eq $P/enum-cls-mappings-type "mappingproxy" "$(pv enum_cls_mappings_type)" check_eq $P/enum-cls-mappings-readonly "!TypeError" "$(pv enum_cls_mappings_readonly)" # make_enum takes NO cardinality -- it is a staticmethod with one argument, so # `Foo+' is built by layering `with_many' on afterwards, which is what cfparse # automates. The layered pattern is the enum's alternation spliced into the # many schema, which is only correct because the alternation was passed # explicitly: `with_many' would otherwise have read `.pattern' and got the same # string by luck. check_eq $P/enum-many-pattern '(yes|no)(\s*,\s*(yes|no))*' "$(pv enum_many_pattern)" check_eq $P/enum-many-value "[True, False, True]" "$(pv enum_many_value)" # ----------------------------------------------------------- make_choice ----- # THE CONVERTER RETURNS THE TEXT; make_choice2 returns (index, text). check_eq $P/choice-pattern "red|green|blue" "$(pv choice_pattern)" check_eq $P/choice-choices "red,green,blue" "$(pv choice_choices)" check_eq $P/choice-choices-type "list" "$(pv choice_choices_type)" check_eq $P/choice-red "red" "$(pv choice_red)" check_eq $P/choice-green "green" "$(pv choice_green)" check_eq $P/choice-blue "blue" "$(pv choice_blue)" check_eq $P/choice-return-type "str" "$(pv choice_return_type)" # A NON-CHOICE RAISES ValueError, and the message names every choice -- the # other half of the header's third behaviour. Compare `enum-dict-missing'. check_eq $P/choice-missing "!ValueError" "$(pv choice_missing)" check_eq $P/choice-missing-msg "pink not in: red, green, blue" "$(pv choice_missing_msg)" # STRICTNESS IS ON BY DEFAULT AND CAN BE TURNED OFF, in which case the converter # is the identity and the ONLY thing rejecting a non-choice is the parser's # regex. Called directly, as here, nothing rejects it at all. check_eq $P/choice-loose-pink "pink" "$(pv choice_loose_pink)" check_eq $P/choice-loose-pattern "red|green" "$(pv choice_loose_pattern)" # THE PARSER MATCHES CASE-INSENSITIVELY AND THE CONVERTER DOES NOT. Without a # `transform', `RED' reaches a converter whose list holds only `red' and is # rejected -- the converter is stricter than the pattern that fed it, which is # what `transform' exists to fix. It is applied to the CHOICES at build time as # well as to the text, so an uppercase choice list plus `str.lower' yields a # LOWERCASE pattern -- the surprise being that the pattern changed at all. check_eq $P/choice-upper "!ValueError" "$(pv choice_upper)" check_eq $P/choice-transform-upper "red" "$(pv choice_transform_upper)" check_eq $P/choice-transform-mixed "green" "$(pv choice_transform_mixed)" check_eq $P/choice-transform-pattern "red|green" "$(pv choice_transform_pattern)" check_eq $P/choice-transform-choices "red,green" "$(pv choice_transform_choices)" check_eq $P/choice-transform-choices-pattern "red|green" \ "$(pv choice_transform_choices_pattern)" # AN EMPTY CHOICE LIST BUILDS AN EMPTY PATTERN AND DOES NOT RAISE. `r""' matches # at every position, so such a converter lets the parser match the empty string # and then rejects it -- a converter that can never succeed and says so nowhere. check_eq $P/choice-empty-pattern-len "0" "$(pv choice_empty_pattern_len)" check_eq $P/choice-empty-call "!ValueError" "$(pv choice_empty_call)" check_eq $P/choice2-pattern "red|green|blue" "$(pv choice2_pattern)" check_eq $P/choice2-red "(0, 'red')" "$(pv choice2_red)" check_eq $P/choice2-green "(1, 'green')" "$(pv choice2_green)" check_eq $P/choice2-blue-index "2" "$(pv choice2_blue_index)" check_eq $P/choice2-blue-text "blue" "$(pv choice2_blue_text)" check_eq $P/choice2-type "tuple" "$(pv choice2_type)" check_eq $P/choice2-missing "!ValueError" "$(pv choice2_missing)" # ---------------------------------------------------------- make_variant ----- # THE PATTERN IS THE ALTERNATIVES, EACH PARENTHESISED, and `regex_group_count' # is one per alternative PLUS the groups inside each alternative's own pattern # -- here 2 + 0, because neither `\d+' nor `yes|no' contains a group. check_eq $P/variant-pattern '(\d+)|(yes|no)' "$(pv variant_pattern)" check_eq $P/variant-group-count "2" "$(pv variant_group_count)" check_eq $P/variant-converters-len "2" "$(pv variant_converters_len)" check_eq $P/variant-converters-type "tuple" "$(pv variant_converters_type)" check_eq $P/variant-number "42" "$(pv variant_number)" check_eq $P/variant-number-type "int" "$(pv variant_number_type)" check_eq $P/variant-yes "True" "$(pv variant_yes)" check_eq $P/variant-yes-type "bool" "$(pv variant_yes_type)" # FIRST MATCH WINS, IN LIST ORDER -- the converters are re-matched one at a time # against the whole text, so the alternation in the pattern is not what # dispatches. Reversing the list reverses the pattern and changes nothing else. check_eq $P/variant-rev-pattern '(yes|no)|(\d+)' "$(pv variant_rev_pattern)" check_eq $P/variant-rev-number "42" "$(pv variant_rev_number)" # NO ALTERNATIVE MATCHES -> AssertionError, WHICH `python3 -O' DELETES -- the # header's fifth behaviour. The strict arm is a bare `assert', so under -O the # function falls through to `return None' and a mismatch becomes a silent None. # `variant-optimize-flag' records which interpreter measured these, so an image # built with -O turns ONE case red instead of turning the verdict wrong. check_eq $P/variant-nomatch "!AssertionError" "$(pv variant_nomatch)" check_eq $P/variant-nomatch-msg "OOPS-VARIANT-MISMATCH: zzz" "$(pv variant_nomatch_msg)" check_eq $P/variant-loose-nomatch "None" "$(pv variant_loose_nomatch)" check_eq $P/variant-optimize-flag "0" "$(pv variant_optimize_flag)" # A SINGLE-CONVERTER VARIANT IS THE CONVERTER ITSELF. No wrapper, no # `.converters', no `regex_group_count' -- so code that reads those attributes # off make_variant's result works for two alternatives and raises for one. check_eq $P/variant-single-is-same "yes" "$(pv variant_single_is_same)" check_eq $P/variant-single-no-converters "no" "$(pv variant_single_has_converters)" check_eq $P/variant-empty "!AssertionError" "$(pv variant_empty)" # THE COMPILED ARM MUTATES THE CALLER'S CONVERTERS. `compiled=True' caches a # `matcher' attribute ON EACH CONVERTER FUNCTION -- shared, module-level objects # in ordinary use -- so a second variant built over the same converter with # different `re_opts' silently reuses the FIRST one's compiled matcher. The # before/after pair on `parse_anything', which the library itself exports, is # that mutation measured on the library's own object. check_eq $P/variant-matcher-before "no" "$(pv variant_matcher_before)" check_eq $P/variant-matcher-after "yes" "$(pv variant_matcher_after)" check_eq $P/variant-compiled-pattern '(.+?)|(yes|no)' "$(pv variant_compiled_pattern)" check_eq $P/variant-compiled-value "anything at all" "$(pv variant_compiled_value)" # ------------------------------------------------------- build_type_dict ----- # KEYED BY THE CONVERTER'S `.name' ATTRIBUTE, which nothing sets for you: a # converter without one raises AttributeError here rather than being skipped, # and a non-callable entry trips the `assert' -- which `python3 -O' would # delete, exactly as it deletes make_variant's. check_eq $P/typedict-keys "Number,Number*,Number+,Number?" "$(pv typedict_keys)" check_eq $P/typedict-len "4" "$(pv typedict_len)" check_eq $P/typedict-number-is-item "yes" "$(pv typedict_number_is_item)" check_eq $P/typedict-nameless "!AttributeError" "$(pv typedict_nameless)" check_eq $P/typedict-not-callable "!AssertionError" "$(pv typedict_not_callable)" # --------------------------------------- the generated pattern, in parse ----- # THIS IS THE HALF A DIRECT CALL CANNOT SHOW. Everything above called the # converter itself; these hand it to `parse.parse', where the generated pattern # decides whether the converter is reached at all. check_eq $P/p-number-ok "{'n': 42}" "$(pv p_number_ok)" check_eq $P/p-number-letters "None" "$(pv p_number_letters)" check_eq $P/p-many-ok "{'n': [1, 2, 3]}" "$(pv p_many_ok)" check_eq $P/p-many-one "{'n': [7]}" "$(pv p_many_one)" # THE PATTERN IS WHAT REJECTS THE WRONG TYPE: `Number+' generates a `\d+'-based # alternation, so letters never reach `int()' and `parse' simply returns None -- # and a MIXED list is rejected whole, not truncated. This is the ticket's # "a converter built for one type does not accept another type's input". check_eq $P/p-many-letters "None" "$(pv p_many_letters)" check_eq $P/p-many-mixed "None" "$(pv p_many_mixed)" check_eq $P/p-opt-present "{'n': 42}" "$(pv p_opt_present)" check_eq $P/p-opt-absent "{'n': None}" "$(pv p_opt_absent)" check_eq $P/p-many0-absent "{'n': []}" "$(pv p_many0_absent)" check_eq $P/p-many0-two "{'n': [1, 2]}" "$(pv p_many0_two)" # ...AND make_list THROWS THAT PROTECTION AWAY. Same input, same item converter: # the `.+?' pattern matches, `int()' is reached, and its ValueError PROPAGATES # OUT OF parse.parse -- a raise where `Number+' returned None. The pair is the # header's first behaviour, measured. check_eq $P/p-mklist-ok "{'n': [1, 2]}" "$(pv p_mklist_ok)" check_eq $P/p-mklist-letters "!ValueError" "$(pv p_mklist_letters)" # The enum and the choice through the parser. `p-enum-upper' is the IGNORECASE # match the enum's `lower()' fallback absorbs, and `p-choice-upper' is the same # match with no fallback -- the parser matches and the converter then raises out # of parse.parse. check_eq $P/p-enum-yes "{'a': True}" "$(pv p_enum_yes)" check_eq $P/p-enum-upper "{'a': True}" "$(pv p_enum_upper)" check_eq $P/p-enum-nonmember "None" "$(pv p_enum_nonmember)" check_eq $P/p-choice-red "{'c': 'red'}" "$(pv p_choice_red)" check_eq $P/p-choice-pink "None" "$(pv p_choice_pink)" check_eq $P/p-choice-upper "!ValueError" "$(pv p_choice_upper)" check_eq $P/p-choice-transform-upper "{'c': 'red'}" "$(pv p_choice_transform_upper)" check_eq $P/p-variant-number "{'v': 42}" "$(pv p_variant_number)" check_eq $P/p-variant-yes "{'v': True}" "$(pv p_variant_yes)" # A variant that matches NOTHING is rejected by the PATTERN before the # converter's `assert' can fire, so through parse it is None rather than the # AssertionError a direct call gives. Both are true and they look different. check_eq $P/p-variant-nomatch "None" "$(pv p_variant_nomatch)" # The derived converters are ordinary parse types -- nothing about them is # parse()-only, so search() and findall() take the same type dict. check_eq $P/p-search "{'n': [1, 2]}" "$(pv p_search)" check_eq $P/p-findall "[1, 2]" "$(pv p_findall)" # ------------------------------------------------------ cardinality_field ---- # THIS LAYER IMPORTS `six' -- see the header. It is imported inside the probe # rather than at its top, so an image without python3-six turns these cases red # and leaves every case above green: the missing RDEPENDS is then legible as a # block of reds with a cause, rather than as a blank suite. check_eq $P/cf-import "ok" "$(pv cf_import)" check_eq $P/cf-pattern-chars "?*+" "$(pv cf_pattern_chars)" check_eq $P/cf-matches-plus "yes" "$(pv cf_matches_plus)" check_eq $P/cf-matches-star "yes" "$(pv cf_matches_star)" check_eq $P/cf-matches-question "yes" "$(pv cf_matches_question)" check_eq $P/cf-matches-bare "no" "$(pv cf_matches_bare)" check_eq $P/cf-matches-empty "no" "$(pv cf_matches_empty)" check_eq $P/cf-split-plus-name "Number" "$(pv cf_split_plus_name)" check_eq $P/cf-split-plus-card "one_or_more" "$(pv cf_split_plus_card)" check_eq $P/cf-split-star-name "Number" "$(pv cf_split_star_name)" check_eq $P/cf-split-star-card "zero_or_more" "$(pv cf_split_star_card)" check_eq $P/cf-split-question-card "zero_or_one" "$(pv cf_split_question_card)" # A BARE NAME IS NOT AN ERROR: it splits to itself with Cardinality.one, so a # caller cannot use `split_type' to tell "no suffix" from "suffix meaning one". check_eq $P/cf-split-bare-name "Number" "$(pv cf_split_bare_name)" check_eq $P/cf-split-bare-card "one" "$(pv cf_split_bare_card)" # make_type IS THE INVERSE, and for Cardinality.one it appends NOTHING -- the # round trip is exact for the other three. check_eq $P/cf-make-one "Number" "$(pv cf_make_one)" check_eq $P/cf-make-opt "Number?" "$(pv cf_make_opt)" check_eq $P/cf-make-many "Number+" "$(pv cf_make_many)" check_eq $P/cf-make-many0 "Number*" "$(pv cf_make_many0)" check_eq $P/cf-roundtrip "yes" "$(pv cf_roundtrip)" check_eq $P/cfb-variant-name "Number*" "$(pv cfb_variant_name)" check_eq $P/cfb-variant-pattern '(\d+)?(\s*,\s*(\d+))*' "$(pv cfb_variant_pattern)" check_eq $P/cfb-variant-value "[1, 2]" "$(pv cfb_variant_value)" # The second argument may be a TYPE DICT or the converter itself; both are # accepted and the `.name' is set on the result either way. check_eq $P/cfb-direct-name "Number+" "$(pv cfb_direct_converter_name)" check_eq $P/cfb-direct-value "[3, 4]" "$(pv cfb_direct_converter_value)" # A bare name -> ValueError with a sentence; a name whose base is not in the # dict -> MissingTypeError, which SUBCLASSES KeyError, so a caller catching # KeyError catches it and one catching ValueError does not. check_eq $P/cfb-no-field "!ValueError" "$(pv cfb_no_field)" check_eq $P/cfb-no-field-msg "type_name='Number' has no CardinalityField" \ "$(pv cfb_no_field_msg)" check_eq $P/cfb-missing "!MissingTypeError" "$(pv cfb_missing)" check_eq $P/cfb-missing-is-keyerror "yes" "$(pv cfb_missing_is_keyerror)" check_eq $P/cfb-variants-keys "Number+,Number?" "$(pv cfb_variants_keys)" # create_missing_type_variants SKIPS names already in the dict, so `Number' -- # which is there -- is not rebuilt and only `Number+' comes back. check_eq $P/cfb-missing-variants-keys "Number+" "$(pv cfb_missing_variants_keys)" check_eq $P/cfb-listsep "," "$(pv cfb_listsep)" # ---------------------------------------------------------------- cfparse ---- # The Parser that ties it together: it reads the cardinality suffix out of the # FORMAT and creates the variant itself, so a caller writes `{n:Number+}' and # supplies only `Number'. Everything it produces is what the `cfb-*' cases above # build by hand, which is why the results agree. check_eq $P/cfparse-import "ok" "$(pv cfparse_import)" check_eq $P/cfparse-many "[1, 2, 3]" "$(pv cfparse_many)" check_eq $P/cfparse-no-match "None" "$(pv cfparse_no_match)" check_eq $P/cfparse-opt-present "42" "$(pv cfparse_opt_present)" check_eq $P/cfparse-opt-absent "None" "$(pv cfparse_opt_absent)" check_eq $P/cfparse-many0-absent "[]" "$(pv cfparse_many0_absent)" # THE VARIANT IS CREATED EAGERLY, so an unknown base type is a MissingTypeError # at Parser CONSTRUCTION -- not at parse time, and not a None result. check_eq $P/cfparse-missing-type "!MissingTypeError" "$(pv cfparse_missing_type)" check_eq $P/cfparse-one "42" "$(pv cfparse_one)" rm -rf "$W"