. "$BST_LIB" # BST-TIMEOUT: 1200 # TWO python3 starts, and they are the whole cost. # # Everything else below reads the filesystem or dpkg with # # shell builtins, or greps a line out of the transcript the # # probe left behind. # # # # The two are: a bare `import absl' (`import'), and the # # probe -- ONE start for all 193 measurements, covering # # every flag type, every app.run arm, every logging case # # and the whole absltest run. # # # # 1200 is 600s per start, twice the budget the three-start # # python3 suites beside this one set per start, and roughly # # ten times the ~60s a python3 start has measured at on this # # arm. Nothing here is quadratic in anything, so the start # # is the only thing the number has to cover, and the probe # # is additionally bounded by `bst_probe' so it cannot spend # # the suite's whole budget on its own. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify, and the measured elapsed replaces # # this note in the test plan rather than in this comment, # # which would cost an image build to keep the shipped suite # # byte-identical. # # python3-absl 2.1.0 (meta-openembedded/meta-python # recipes-devtools/python/python3-absl_2.1.0.bb, `inherit pypi setuptools3', # PyPI sdist) on the wasm32-hwjs glibc image. # # ============================================================================ # WHAT THE PACKAGE IS, AND WHY THAT SHAPES THIS SUITE. # ============================================================================ # # absl-py is Google's Abseil Python common libraries: four things that ship # together and are used apart. # # absl.flags a DECLARATIVE flag registry. A flag is declared where it is # used -- with a type, a default, a help string and optional # validators -- and every declaration in the process lands in # one registry, so a library can add a flag without the program # that links it knowing. # absl.app the `main' entry point: it parses that registry out of an # argument vector and calls the program with what is left. # absl.logging a glog-style layer over the standard logging module -- the # `I0908 05:24:27.123456 12345 file.py:12] message' prefix, # `--verbosity', and the every-n counters. # absl.testing `absltest' (a unittest.TestCase with a large set of extra # assertions) and `parameterized' (one method, N tests). # # NONE OF IT IS A COMMAND WITH AN EXIT STATUS. Everything this port has to say # is said INSIDE an interpreter, so a shell case per assertion would be a # python3 START per assertion -- tens of seconds each on this arm # (BRINTOS-1476, brintos/linux#21), and a suite that could not finish. The probe # takes all 193 measurements in one interpreter and this file scores its # transcript. # # A FLAG IS A DECLARATION, SO IT IS ASSERTED AS ONE. Each of the seven types is # declared on a `FlagValues' of its own -- never on the process-wide # `flags.FLAGS', which `app' and `absltest' have both already written to -- then # parsed from an argument vector the test supplies, then scored against a # literal. The parsed TYPE is asserted beside the value, because a parser that # returned "42" rather than 42 would satisfy every string comparison here. # # NOTHING IN THIS SUITE FORKS, SLEEPS, SIGNALS OR OPENS A SOCKET, and that is a # constraint held deliberately rather than one absl happens to satisfy: # # * fork(2)/exec(2) is minutes on this arm and popen(3) is ENOSYS # (brintos/linux#14), so `absltest' is driven through `unittest' IN-PROCESS # and never through `absltest.main', which re-enters `app.run', ends in # sys.exit, and whose usual callers spawn; # * signal delivery does not work at all (brintos/linux#21) and alarm(2) was # unwired before the brintos/linux#77 fix, so nothing here arms a bound of # its own -- `bst_probe's is the only one; # * `absl.logging' reaches `socket.gethostname()' and `getpass.getuser()' ONLY # on the log-to-FILE path (`find_log_dir_and_names'). This suite never opens # a log file, which is why no case needs a hostname or a passwd entry the # guest may not have. Both functions are asserted to EXIST instead. # # ONE absl API IS NEVER CALLED, ON PURPOSE. A FATAL record from an ABSLLogger # ends in `os.abort()' inside `PythonHandler.emit' -- no exception to catch, no # exit code to read, and the interpreter dies mid-run taking every later case # with it. FATAL's `F' is asserted through `converter.get_initial_for_level', # which is the same function that renders the prefix. # # ============================================================================ # THE PORT IS ONE IMAGE_INSTALL LINE, THIS SUITE, ITS PROBE -- AND ONE bbappend. # ============================================================================ # # No patch and no site answer: 22 .py files across four sub-packages, no # configure step, no compile step, no C. MEASURED rather than read off the # recipe -- every .py the package installs is BYTE-IDENTICAL to the sdist the # SRC_URI sha256 pins. Recipe-scope build: 1622 of 1622 tasks, all succeeded, # zero ERROR: lines. # # THE bbappend ADDS THIRTEEN RDEPENDS. The stock recipe declares `python3-core' # and nothing else, and not one of the four public entry points can be imported # with only that. absl reaches thirteen stdlib splits at MODULE scope -- # `logging', `string', `xml.dom.minidom', `pdb', `getpass', `socket', # `tempfile', `timeit', `unittest', `json', `shlex', `random', `difflib' -- # and none of those imports is lazy or under TYPE_CHECKING, so there is no # partial degradation: the import either completes or raises. Measured on an # x86-64 control with a `sys.meta_path' hook refusing each split's modules in # turn (after purging them from `sys.modules', or a cached entry bypasses the # hook and every refusal reads as a pass); the full table is in the bbappend. # `deb-depends-exact' below pins the corrected line, so the claim is # re-measured on every image. # # THE OMISSION IS INVISIBLE ON THIS IMAGE, which is why it is worth stating. # `python3' is on IMAGE_INSTALL and pulls the whole stdlib, so every one of the # thirteen is present whatever the recipe says. The bbappend makes the PACKAGE # correct rather than the image different. # # ============================================================================ # THE THING THAT WILL SURPRISE A READER: THE LEVELS RUN BACKWARDS. # ============================================================================ # # `absl.logging.DEBUG' is 1, INFO is 0, WARNING is -1, ERROR is -2 and FATAL is # -3, so MORE verbose is a LARGER number and the standard module's constants are # not interchangeable with them. A caller who reaches for `logging.INFO' from # the stdlib and passes it to `set_verbosity' gets 20 -- every debug level at # once, on a program that asked for less output, and nothing raises. # # The suite pins both the absl constants and the two converters # (`absl_to_standard', `standard_to_absl') that map between the scales, so the # inversion is on the record rather than rediscovered by whoever hits it. # # ============================================================================ # HOW THE PROBE'S TRANSCRIPT IS READ. # ============================================================================ # # `key=value' lines, one per line, flushed as written -- a probe killed mid-run # still delivers everything before the kill. `pv KEY' below pulls one out. # # EVERY STRING RESULT IS WRAPPED IN ANGLE BRACKETS and the empty string is `<>'. # Load-bearing here: several answers are legitimately empty -- a residual argv # with no positionals, a METADATA with no Requires-Dist -- and `grep | cut' # cannot otherwise tell an empty ANSWER from a key the probe never reached. A # refusal is `!'. # # A LIST IS JOINED WITH `|' after the transport, so the literals below read as # lists. That also makes `|' the separator inside `report_rendered_help', whose # value is a multi-line help block joined the same way. # # THE PROBE'S OWN TEMPORARY DIRECTORY IS REPLACED BY THE LITERAL `WORKDIR', # because `mkdtemp' picks a fresh suffix every boot and the path appears in the # generated test module's name. # # TWO FIELDS ARE DELIBERATELY NOT ASSERTED BY VALUE. The log prefix carries a # timestamp and a thread id; the probe matches the whole prefix against its # documented shape and emits only the stable fields. Asserting the rendered line # would be asserting the clock. # # ALL 193 EXPECTED VALUES BELOW WERE MEASURED on an x86-64 control (CPython # 3.12.3, the same minor this image ships) running THIS probe against THIS # BUILD's own .deb payload, which `diff -r' shows byte-identical to the sdist # the recipe's sha256 pins. The transcript was verified deterministic across # repeated runs and across a changed $TMPDIR. # # ============================================================================ P=python3-absl # The canary is first so that a suite which dies before its own first assertion # is distinguishable from one that never started. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages PKGDIR=$SITE/absl DI=$SITE/absl_py-2.1.0.dist-info PKGLIST=/var/lib/dpkg/info/python3-absl.list PROBE=$BST_DATA/python3-absl-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-absl.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ---------------------------------------------------------------- layout ----- # FOUR SUB-PACKAGES, not a flat module. `absl/' itself carries three modules, # `absl/flags/' ten, `absl/logging/' two and `absl/testing/' seven. check $P/site-packages-dir test -d "$SITE" check $P/package-dir test -d "$PKGDIR" check $P/no-flat-module test ! -e "$SITE/absl.py" check $P/init test -s "$PKGDIR/__init__.py" check $P/app test -s "$PKGDIR/app.py" check $P/command-name test -s "$PKGDIR/command_name.py" check $P/flags-dir test -d "$PKGDIR/flags" check $P/flags-init test -s "$PKGDIR/flags/__init__.py" check $P/flags-defines test -s "$PKGDIR/flags/_defines.py" check $P/flags-flag test -s "$PKGDIR/flags/_flag.py" check $P/flags-flagvalues test -s "$PKGDIR/flags/_flagvalues.py" check $P/flags-argument-parser test -s "$PKGDIR/flags/_argument_parser.py" check $P/flags-validators test -s "$PKGDIR/flags/_validators.py" check $P/flags-validators-classes test -s "$PKGDIR/flags/_validators_classes.py" check $P/flags-exceptions test -s "$PKGDIR/flags/_exceptions.py" check $P/flags-helpers test -s "$PKGDIR/flags/_helpers.py" check $P/flags-argparse test -s "$PKGDIR/flags/argparse_flags.py" check $P/logging-dir test -d "$PKGDIR/logging" check $P/logging-init test -s "$PKGDIR/logging/__init__.py" check $P/logging-converter test -s "$PKGDIR/logging/converter.py" check $P/testing-dir test -d "$PKGDIR/testing" check $P/testing-init test -s "$PKGDIR/testing/__init__.py" check $P/testing-absltest test -s "$PKGDIR/testing/absltest.py" check $P/testing-parameterized test -s "$PKGDIR/testing/parameterized.py" check $P/testing-flagsaver test -s "$PKGDIR/testing/flagsaver.py" check $P/testing-xml-reporter test -s "$PKGDIR/testing/xml_reporter.py" check $P/testing-pretty-print test -s "$PKGDIR/testing/_pretty_print_reporter.py" check $P/testing-bazelize test -s "$PKGDIR/testing/_bazelize_command.py" # THE TWO `.pyi' STUBS SHIP, and only two: `app.pyi' and `logging/__init__.pyi'. # They are the type-checker's view of the two modules whose signatures absl # could not express in the source, and they are `package_data' -- exactly the # kind of thing a packaging step drops without anything noticing at runtime. check $P/stub-app test -s "$PKGDIR/app.pyi" check $P/stub-logging test -s "$PKGDIR/logging/__init__.pyi" # `absl/__init__.py' AND `absl/testing/__init__.py' CARRY NO CODE -- both are # the 13-line Apache header and nothing else, byte-identical to each other. That # is why `import absl' is nearly free and why `absl' itself exports nothing: a # caller must import `absl.flags', `absl.app', `absl.logging' or # `absl.testing.absltest' by name -- which is not asserted here, because doing # so needs a THIRD interpreter start and the probe has already imported every # one of them, so by the time it could be asked the attribute exists. Asserted # instead as "no non-comment, non-blank line" # rather than as a size, so a release that added a re-export says so here # instead of at the first `absl.flags' that resolves by accident. check_eq $P/init-no-code "" \ "$(grep -v '^#' "$PKGDIR/__init__.py" 2>/dev/null | grep -v '^[[:space:]]*$')" check_eq $P/testing-init-no-code "" \ "$(grep -v '^#' "$PKGDIR/testing/__init__.py" 2>/dev/null | grep -v '^[[:space:]]*$')" # The .pyc files ship in the .deb, which is what lets the package import on a # READ-ONLY root: nothing has to be compiled at first import. check $P/pycache-app test -s "$PKGDIR/__pycache__/app.cpython-312.pyc" check $P/pycache-flags test -s "$PKGDIR/flags/__pycache__/__init__.cpython-312.pyc" check $P/pycache-logging test -s "$PKGDIR/logging/__pycache__/__init__.cpython-312.pyc" check $P/pycache-absltest test -s "$PKGDIR/testing/__pycache__/absltest.cpython-312.pyc" # NO CONSOLE SCRIPT. absl is a library and `setup.py' declares no # `entry_points', so nothing of it lands in ${bindir}; a release that grew one # would show up here rather than as a surprise in the image roster. check $P/no-bin-absl test ! -e /usr/bin/absl check $P/no-bin-absltest test ! -e /usr/bin/absltest # UPSTREAM'S OWN tests/ TREE IS NOT ON THE ROOTFS. `find_packages' excludes it, # so the sdist ships it and the wheel does not. check $P/no-tests-dir test ! -d "$PKGDIR/tests" check $P/no-flags-tests test ! -d "$PKGDIR/flags/tests" # ------------------------------------------------------------- the .deb ------ cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" # PURE PYTHON, AND NOT MERELY "NO .so OF ITS OWN". This image's python3 is ONE # STATIC BINARY with an empty lib-dynload and no dynamic loader (BRINTOS-1476), # so a module carrying a compiled extension could not be imported here at all. # absl ships no C -- no .c, no .pyx, no ext_modules -- so this is a REGRESSION # TRIPWIRE on a version bump rather than a statement about today. check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # ...and nothing but Python, the two stubs and the dist-info in the payload. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc|*.pyi) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-expected "" "$_bad" # TWENTY-TWO MODULES. A release that split or merged one would show up here # first, before any behavioural case noticed. check_eq $P/pkg-py-count "22" \ "$(grep -c '/site-packages/absl/.*\.py$' "$W/pkgf" 2>/dev/null)" check_eq $P/pkg-pyi-count "2" \ "$(grep -c '/site-packages/absl/.*\.pyi$' "$W/pkgf" 2>/dev/null)" # ------------------------------------------------------- the dist-info ------- # NAMED FOR THE VERSION THE RECIPE BUILT, which is not automatic: two ports # beside this one on this image install a `0.0.0.dist-info' because their # upstream re-derives the version from a git tree the sdist does not carry. absl # hard-codes it in setup.py, so `importlib.metadata.version' answers 2.1.0 -- # asserted by directory name AND by METADATA, because those are two separate # consequences and a recipe that fixed one without the other would still # mislead a caller's feature guard. check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 2.1.0" cat "$DI/METADATA" check_out $P/dist-info-name "Name: absl-py" cat "$DI/METADATA" check $P/dist-info-record test -s "$DI/RECORD" check $P/no-zero-dist-info test ! -d "$SITE/absl_py-0.0.0.dist-info" # `setuptools3' BUILT THIS, and it leaves three traces the `python_hatchling' # ports beside it on this image do not: the generator names `bdist_wheel', there # IS a `top_level.txt', and the licence lands FLAT in the dist-info rather than # under `licenses/'. All three are asserted, so a recipe moved onto another # backend says so here rather than silently. check_out $P/dist-info-generator "Generator: bdist_wheel (0.42.0)" cat "$DI/WHEEL" check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" check_out $P/dist-info-tag-py3 "Tag: py3-none-any" cat "$DI/WHEEL" check_eq $P/dist-info-no-py2-tag "" "$(grep '^Tag: py2' "$DI/WHEEL" 2>/dev/null)" check_eq $P/dist-info-top-level-name "absl" "$(cat "$DI/top_level.txt" 2>/dev/null)" check $P/dist-info-flat-license test -s "$DI/LICENSE" check $P/dist-info-authors test -s "$DI/AUTHORS" check $P/dist-info-no-licenses-dir test ! -d "$DI/licenses" # THE IMPORT NAME AND THE DISTRIBUTION NAME DIFFER -- `absl' against `absl-py' # -- which is exactly the case where a caller's `importlib.metadata.version' # lookup goes wrong. `dist-absl-unknown' in the probe asserts the other # direction: there is no `absl' distribution to find. check_out $P/metadata-summary "Abseil Python Common Libraries" cat "$DI/METADATA" check_out $P/metadata-requires-python "Requires-Python: >=3.7" cat "$DI/METADATA" # NO Requires-Dist AT ALL. That is what makes the RDEPENDS question purely one # about the STDLIB SPLIT rather than about third-party packages, and it is why # the bbappend adds thirteen stdlib splits and no package. check_eq $P/metadata-requires-count "0" \ "$(grep -c '^Requires-Dist:' "$DI/METADATA" 2>/dev/null)" sed -n '/^Package: python3-absl$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/deb-known test -s "$W/st" check_out $P/deb-installed "Status: install ok installed" cat "$W/st" check_out $P/deb-version "Version: 2.1.0-r0" cat "$W/st" # THE RDEPENDS LINE, EXACTLY -- and this is the case that proves the bbappend # did what it says. With the stock recipe this reads "Depends: python3-core", # and every one of `import absl.flags', `import absl.app', `import # absl.logging' and `import absl.testing.absltest' fails on any image without # the thirteen, with nothing else in this suite noticing on THIS image, where # `python3' brings them all in regardless. grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/deb-depends-line test -s "$W/dep" check_eq $P/deb-depends-exact "Depends: python3-core, python3-datetime, python3-debugger, python3-difflib, python3-io, python3-json, python3-logging, python3-math, python3-misc, python3-shell, python3-stringold, python3-unittest, python3-unixadmin, python3-xml" \ "$(cat "$W/dep")" # ...and the thirteen files those thirteen splits are about are on the rootfs. # Named individually rather than counted, because "twelve of thirteen" is the # failure this catches and a count would pass it. check $P/stdlib-logging test -s /usr/lib/python3.12/logging/__init__.py check $P/stdlib-string test -s /usr/lib/python3.12/string.py check $P/stdlib-minidom test -s /usr/lib/python3.12/xml/dom/minidom.py check $P/stdlib-pdb test -s /usr/lib/python3.12/pdb.py check $P/stdlib-getpass test -s /usr/lib/python3.12/getpass.py check $P/stdlib-socket test -s /usr/lib/python3.12/socket.py check $P/stdlib-tempfile test -s /usr/lib/python3.12/tempfile.py check $P/stdlib-timeit test -s /usr/lib/python3.12/timeit.py check $P/stdlib-unittest test -s /usr/lib/python3.12/unittest/__init__.py check $P/stdlib-json test -s /usr/lib/python3.12/json/__init__.py check $P/stdlib-shlex test -s /usr/lib/python3.12/shlex.py check $P/stdlib-random test -s /usr/lib/python3.12/random.py check $P/stdlib-difflib test -s /usr/lib/python3.12/difflib.py # ---------------------------------------------------------------- import ----- check $P/import python3 -c 'import absl' # ----------------------------------------------------------------- probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # THE PROBE IS ALSO THIS SUITE'S `import-quiet'. It imports every absl entry # point as its first act and nothing below it writes to stderr on a clean run -- # `app.run's two usage arms have their stderr captured inside the probe for # exactly this reason -- so an empty stderr is a statement about the imports as # well as about the probe. check_eq $P/probe-quiet "" "$(cat "$OUT".err 2>/dev/null)" # The probe's own two bookends. `probe-complete' says it reached its last line # and `probe-count' says it emitted every key it meant to: a probe that died # two-thirds of the way through would otherwise leave the missing cases scoring # against an empty string, which `check_eq' reads as a wrong answer rather than # as an absent one. check_eq $P/probe-complete "" "$(pv probe_done)" check_eq $P/probe-count "193" "$(pv emitted_count)" check_eq $P/fixture-built "" "$(pv fixture_built)" check_eq $P/fixture-torn-down "" "$(pv torn_down)" # -------------------------------------------------------------- identity ----- # WHICH COPY ANSWERED. Every behavioural case below is a statement about the # file named here. check_eq $P/module-answered "<$PKGDIR/__init__.py>" "$(pv module_file)" check_eq $P/is-package "" "$(pv is_package)" # THERE IS NO `absl.__version__'. `importlib.metadata' is the only way to ask, # and it answers under the DISTRIBUTION name -- so the pair below is what a # caller's feature guard actually depends on, and `dist-absl-unknown' says the # obvious spelling of that guard finds nothing. check_eq $P/version-attr "!" "$(pv version_attr)" check_eq $P/dist-version "<2.1.0>" "$(pv dist_version)" check_eq $P/dist-name "" "$(pv dist_name)" check_eq $P/dist-absl-unknown "!" "$(pv dist_absl_unknown)" check_eq $P/dist-requires "<>" "$(pv dist_requires)" check_eq $P/dist-top-level "" "$(pv dist_top_level)" check_eq $P/submodule-app "" "$(pv submodule_app)" check_eq $P/submodule-command-name "" "$(pv submodule_command_name)" check_eq $P/submodule-flags "" "$(pv submodule_flags)" check_eq $P/submodule-argparse-flags "" "$(pv submodule_flags_argparse_flags)" check_eq $P/submodule-logging "" "$(pv submodule_logging)" check_eq $P/submodule-converter "" "$(pv submodule_logging_converter)" check_eq $P/submodule-testing "" "$(pv submodule_testing)" check_eq $P/submodule-absltest "" "$(pv submodule_testing_absltest)" check_eq $P/submodule-flagsaver "" "$(pv submodule_testing_flagsaver)" check_eq $P/submodule-parameterized "" "$(pv submodule_testing_parameterized)" check_eq $P/submodule-pretty-print "" "$(pv submodule_testing__pretty_print_reporter)" check_eq $P/submodule-bazelize "" "$(pv submodule_testing__bazelize_command)" check_eq $P/submodule-xml-reporter "" "$(pv submodule_testing_xml_reporter)" # Counted from the INSTALLED TREE by the probe, which is a different reading of # the same fact as `pkg-py-count' above: that one counts what dpkg says it put # there, this one counts what is on the disk the interpreter reads. check_eq $P/tree-py-count "<22>" "$(pv tree_py_count)" check_eq $P/tree-pyi-count "<2>" "$(pv tree_pyi_count)" check_eq $P/tree-so-count "<0>" "$(pv tree_so_count)" # ===================================================================== flags == # SEVEN TYPES DECLARED ON ONE FlagValues, then parsed out of one vector. check_eq $P/flags-declared-count "<7>" "$(pv declared_count)" check_eq $P/flags-declared-names "" \ "$(pv declared_names)" # THE RESIDUAL CARRIES argv[0] AND THE POSITIONALS, and nothing that parsed as a # flag. That contract is what `app.run' hands to `main', so an off-by-one here # is an off-by-one in every absl program. check_eq $P/parse-residual "" "$(pv parse_residual)" check_eq $P/parse-is-parsed "" "$(pv parse_is_parsed)" # EACH TYPE: THE VALUE, AND THE TYPE OF THE VALUE. The second half is the one # that matters -- a parser that handed back the string "42" would satisfy every # value comparison in a shell suite and be wrong. check_eq $P/value-string "" "$(pv value_string)" check_eq $P/value-integer "<42>" "$(pv value_integer)" check_eq $P/value-integer-type "" "$(pv value_integer_type)" check_eq $P/value-float "<2.5>" "$(pv value_float)" check_eq $P/value-float-type "" "$(pv value_float_type)" check_eq $P/value-boolean "" "$(pv value_boolean)" check_eq $P/value-boolean-type "" "$(pv value_boolean_type)" check_eq $P/value-enum "" "$(pv value_enum)" check_eq $P/value-list "" "$(pv value_list)" check_eq $P/value-list-type "" "$(pv value_list_type)" check_eq $P/value-multi-string "" "$(pv value_multi_string)" check_eq $P/value-multi-string-type "" "$(pv value_multi_string_type)" # `present' IS HOW A CALLER TELLS "GIVEN" FROM "DEFAULTED", and for a # multi-string it also counts the repeats. Nothing else in the API can answer # that question, which is why both are asserted. check_eq $P/present-string "<1>" "$(pv present_string)" check_eq $P/present-multi-string "<2>" "$(pv present_multi_string)" # ------------------------------------- the same flags, none of them given ---- # THE DEFAULT IS ASSERTED AS A PARSE RESULT, not by reading `.default' back: a # default that never reaches `.value' is the failure this catches, and reading # the attribute the declaration wrote cannot see it. check_eq $P/default-residual "" "$(pv default_residual)" check_eq $P/default-string "" "$(pv default_string)" check_eq $P/default-integer "<7>" "$(pv default_integer)" check_eq $P/default-float "<1.5>" "$(pv default_float)" check_eq $P/default-boolean "" "$(pv default_boolean)" check_eq $P/default-enum "" "$(pv default_enum)" check_eq $P/default-list "" "$(pv default_list)" check_eq $P/default-multi-string "" "$(pv default_multi_string)" check_eq $P/default-present-zero "<0>" "$(pv default_present_zero)" # --------------------------------------------------- the boolean spellings --- # `--nobflag' IS THE ONE THE TICKET NAMES. The other three are the same parser # arm, and a regression in it would show in only one of them, so all four are # driven -- plus the refusal of a fifth, which is what says the parser is # validating rather than truthy-testing. check_eq $P/bool-no-form "" "$(pv bool_no_form)" check_eq $P/bool-bare-form "" "$(pv bool_bare_form)" check_eq $P/bool-eq-false "" "$(pv bool_eq_false)" check_eq $P/bool-eq-true "" "$(pv bool_eq_true)" check_eq $P/bool-eq-zero "" "$(pv bool_eq_zero)" check_eq $P/bool-eq-garbage "!" "$(pv bool_eq_garbage)" # ------------------------------------------------------------- validators ---- # BOTH DIRECTIONS, ALWAYS. A validator that never fires and one that always # fires are equally broken, and only the pair tells them apart. check_eq $P/validator-accepts "<8080>" "$(pv validator_accepts)" check_eq $P/validator-rejects "!" "$(pv validator_rejects)" # The message the caller sees, whole: absl prefixes the flag and its value onto # the `message=' the declaration gave, and that composition is the deliverable. check_eq $P/validator-message "" \ "$(pv validator_message)" # THE MULTI-FLAG VALIDATOR is the other API shape: it sees the whole dict and # fires on a RELATION between flags rather than on one value. check_eq $P/multi-validator-accepts "<2..5>" "$(pv multi_validator_accepts)" check_eq $P/multi-validator-rejects "!" \ "$(pv multi_validator_rejects)" # `mark_flag_as_required' is the third shape, and it fires at PARSE time on the # flag's absence rather than on its value. check_eq $P/required-given "" "$(pv required_given)" check_eq $P/required-missing "!" "$(pv required_missing)" # --------------------------------------------- what flag_values reports ------ # THE TICKET'S "flag_values reports the declared help text and default". Read # off the REGISTRY rather than off the variable the declaration returned, # because the registry is what `--help' renders from. check_eq $P/report-help-string "" "$(pv report_help_string)" check_eq $P/report-help-integer "" "$(pv report_help_integer)" check_eq $P/report-default-string "" "$(pv report_default_string)" check_eq $P/report-default-integer "<7>" "$(pv report_default_integer)" # `default_as_str' is the QUOTED form `--help' prints, which is a different # string from the default itself -- `'7'' rather than 7 -- and both are what # some caller reads. check_eq $P/report-default-as-str-string "<'sdefault'>" "$(pv report_default_as_str_string)" check_eq $P/report-default-as-str-integer "<'7'>" "$(pv report_default_as_str_integer)" check_eq $P/report-default-as-str-bool "<'true'>" "$(pv report_default_as_str_bool)" check_eq $P/report-default-list "" "$(pv report_default_list)" check_eq $P/report-enum-values "" "$(pv report_enum_values)" check_eq $P/report-name "" "$(pv report_name)" # THE TYPE NAME absl PRINTS IN `--help', per type. `int' rather than `integer' # and `comma separated list of strings' rather than `list' are absl's own # wording, and they are what a user reads when they get the flag wrong. check_eq $P/report-flag-type-string "" "$(pv report_flag_type_string)" check_eq $P/report-flag-type-integer "" "$(pv report_flag_type_integer)" check_eq $P/report-flag-type-enum "" "$(pv report_flag_type_enum)" check_eq $P/report-flag-type-list "" \ "$(pv report_flag_type_list)" check_eq $P/report-flag-type-multi "" "$(pv report_flag_type_multi)" # `serialize' IS THE ROUND TRIP: the string that would re-produce this value on # a command line. For a false boolean that is the `--noflag' form, which is the # same spelling `bool-no-form' parses -- the two cases are the two ends of it. check_eq $P/report-serialize-enum "<--eflag=green>" "$(pv report_serialize_enum)" check_eq $P/report-serialize-bool-false "<--bflag>" "$(pv report_serialize_bool_false)" check_eq $P/report-parser-string "" "$(pv report_parser_string)" check_eq $P/report-parser-integer "" "$(pv report_parser_integer)" check_eq $P/report-parser-enum "" "$(pv report_parser_enum)" # THE RENDERED HELP BLOCK, WHOLE. `--help' is what a caller sees when they get # the flag wrong, so the layout is asserted and not only the pieces it is built # from. Newlines are `|'; the module heading absl puts above the group is # dropped by the probe, because it is the invocation path. check_eq $P/report-rendered-help \ "< --eflag: : an enum flag| (default: 'red')| --sflag: a string flag| (default: 'sdefault')>" \ "$(pv report_rendered_help)" # ----------------------------------------------------- the duplicate name ---- check_eq $P/duplicate-raises "!" "$(pv duplicate_raises)" # ITS POSITION IN THE HIERARCHY IS THE HALF THAT BITES. `DuplicateFlagError' # derives from `flags.Error' and NOT from ValueError, so `except flags.Error' # catches it and `except ValueError' -- the idiom a caller reaches for first -- # does not. The negative is asserted for exactly that reason. check_eq $P/duplicate-is-flags-error "" "$(pv duplicate_is_flags_error)" check_eq $P/duplicate-is-value-error "" "$(pv duplicate_is_value_error)" check_eq $P/illegal-is-flags-error "" "$(pv illegal_is_flags_error)" # ------------------------------------------------ the other parse refusals --- check_eq $P/reject-bad-integer "!" "$(pv reject_bad_integer)" check_eq $P/reject-bad-enum "!" "$(pv reject_bad_enum)" check_eq $P/reject-unknown-flag "!" "$(pv reject_unknown_flag)" # `--' ENDS THE FLAGS: everything after it is positional, even a `--word'. check_eq $P/double-dash-residual "" \ "$(pv double_dash_residual)" # ======================================================================= app == # `app.run' DRIVEN WITH AN ARGUMENT VECTOR, never with the process's own. check_eq $P/app-run-exit "" "$(pv app_run_exit)" check_eq $P/app-main-argv "" "$(pv app_main_argv)" check_eq $P/app-main-argv0 "" "$(pv app_main_argv0)" check_eq $P/app-main-positionals "" "$(pv app_main_positionals)" check_eq $P/app-main-flag "" "$(pv app_main_flag)" check_eq $P/app-flags-parsed "" "$(pv app_flags_parsed)" # `main's RETURN VALUE IS THE EXIT CODE, and `None' is a distinct answer from # `0' -- `sys.exit(None)' and `sys.exit(0)' both succeed but carry different # `SystemExit.code', which is what a caller wrapping app.run reads. check_eq $P/app-run-exit-zero "" "$(pv app_run_exit_zero)" check_eq $P/app-run-exit-none "" "$(pv app_run_exit_none)" # THE TWO REFUSAL ARMS both write usage to stderr and exit 1, rather than # letting the exception out. check_eq $P/app-run-exit-usage "" "$(pv app_run_exit_usage)" check_eq $P/app-usage-mentions-flags "" "$(pv app_usage_mentions_flags)" check_eq $P/app-usage-mentions-error "" "$(pv app_usage_mentions_error)" check_eq $P/app-run-exit-unknown-flag "" "$(pv app_run_exit_unknown_flag)" # `call_after_init' AFTER INITIALISATION runs the callback immediately, in the # caller's thread -- asserted as a synchronous call, which is what it is once # `app.run' has been through `_run_init'. check_eq $P/app-call-after-init "" "$(pv app_call_after_init)" # `make_process_name_useful' RETURNS ON THIS ARM, which is the case worth # asserting rather than what it achieved. It writes /proc/self/comm and, on # EnvironmentError, falls back to a ctypes prctl that cannot load libc.so.6 on a # static interpreter -- and BOTH arms are caught upstream, so whichever this # image takes, `app.run' proceeds. The process name it sets is not observable # from here and is not asserted. check_eq $P/command-name-survives "" "$(pv command_name_survives)" # =================================================================== logging == check_eq $P/log-handler-type "" "$(pv log_handler_type)" check_eq $P/log-python-handler-type "" "$(pv log_python_handler_type)" check_eq $P/log-logger-type "" "$(pv log_logger_type)" check_eq $P/log-logger-name "" "$(pv log_logger_name)" check_eq $P/log-default-verbosity "<0>" "$(pv log_default_verbosity)" # THE LEVELS RUN BACKWARDS FROM THE STDLIB'S -- DEBUG 1, INFO 0, FATAL -3, so # more verbose is a LARGER number. A caller who passes `logging.INFO' from the # standard module gets 20, which is every debug level at once and raises # nothing. See the header; these five and the two converters below are the # record of it. check_eq $P/log-level-debug "<1>" "$(pv log_level_debug)" check_eq $P/log-level-info "<0>" "$(pv log_level_info)" check_eq $P/log-level-warning "<-1>" "$(pv log_level_warning)" check_eq $P/log-level-error "<-2>" "$(pv log_level_error)" check_eq $P/log-level-fatal "<-3>" "$(pv log_level_fatal)" check_eq $P/log-absl-to-standard "<10|20|30|40|50>" "$(pv log_absl_to_standard)" check_eq $P/log-standard-to-absl "<1|0|-1|-2|-3>" "$(pv log_standard_to_absl)" # EACH SEVERITY RENDERS WITH ITS DOCUMENTED PREFIX. The probe matched the whole # prefix against `%c%02d%02d %02d:%02d:%02d.%06d %5d %s:%d]' and emits only the # stable fields; `log-shape-*' IS that match. The source file is the probe's own # because absl reports the CALLER's frame, not its own -- which is the property # that makes the prefix useful and the one a broken frame-skip would lose. check_eq $P/log-shape-info "" "$(pv log_shape_info)" check_eq $P/log-initial-info "" "$(pv log_initial_info)" check_eq $P/log-message-info "" "$(pv log_message_info)" check_eq $P/log-source-info "" "$(pv log_source_info)" check_eq $P/log-shape-warning "" "$(pv log_shape_warning)" check_eq $P/log-initial-warning "" "$(pv log_initial_warning)" check_eq $P/log-message-warning "" "$(pv log_message_warning)" check_eq $P/log-source-warning "" "$(pv log_source_warning)" check_eq $P/log-shape-error "" "$(pv log_shape_error)" check_eq $P/log-initial-error "" "$(pv log_initial_error)" check_eq $P/log-message-error "" "$(pv log_message_error)" check_eq $P/log-source-error "" "$(pv log_source_error)" # THREE LINES, NOT FOUR: the `debug' call at verbosity INFO is the absent one, # and its absence is what makes the count a filter assertion rather than a # coincidence. check_eq $P/log-severity-line-count "<3>" "$(pv log_severity_line_count)" check_eq $P/log-debug-suppressed-at-info "" "$(pv log_debug_suppressed_at_info)" # FATAL'S `F' IS ASSERTED HERE AND NOWHERE ELSE. Emitting a FATAL record calls # os.abort() inside the handler; this is the same function that renders the # prefix, read directly. check_eq $P/log-initials-by-level "" "$(pv log_initials_by_level)" # `--verbosity' FILTERS, IN BOTH DIRECTIONS. At WARNING the info() must not # arrive and the warning() must; a filter that dropped everything would pass the # first case on its own. check_eq $P/log-filter-line-count "<1>" "$(pv log_filter_line_count)" check_eq $P/log-filter-info-absent "" "$(pv log_filter_info_absent)" check_eq $P/log-filter-warning-present "" "$(pv log_filter_warning_present)" check_eq $P/log-debug-present-at-debug "" "$(pv log_debug_present_at_debug)" # A DEBUG record renders with `I', not `D'. absl has no debug initial -- the # converter maps both DEBUG and INFO to `I' -- so a reader grepping for `D' # finds nothing however verbose the program is. check_eq $P/log-initial-debug "" "$(pv log_initial_debug)" # `log_every_n' EMITS THE DOCUMENTED FRACTION: the 1st call and every nth after # it, so ten calls at n=3 are four lines carrying 0, 3, 6 and 9. The MESSAGES # are asserted and not only the count -- a counter that kept the LAST four calls # would give the same count and be wrong. check_eq $P/log-every-n-count "<4>" "$(pv log_every_n_count)" check_eq $P/log-every-n-messages "" \ "$(pv log_every_n_messages)" check_eq $P/log-first-n-count "<2>" "$(pv log_first_n_count)" check_eq $P/log-first-n-messages "" "$(pv log_first_n_messages)" # `%'-STYLE INTERPOLATION IS DEFERRED to the formatter, as in the stdlib module. check_eq $P/log-interpolation "" "$(pv log_interpolation)" # `exception()' RENDERS AT ERROR and appends the traceback under the line. check_eq $P/log-exception-initial "" "$(pv log_exception_initial)" check_eq $P/log-exception-message "" "$(pv log_exception_message)" check_eq $P/log-exception-has-traceback "" "$(pv log_exception_has_traceback)" # `set_verbosity' TAKES A NAME AS WELL AS A NUMBER, and both round-trip through # `get_verbosity' on the absl scale. check_eq $P/log-level-roundtrip "<-1>" "$(pv log_level_roundtrip)" check_eq $P/log-level-named-roundtrip "<-2>" "$(pv log_level_named_roundtrip)" # THE LOG-TO-FILE ARM EXISTS AND IS NOT DRIVEN. It is the one that reaches # `getpass.getuser()' and `socket.gethostname()'; asserting the API is present # says the omission below is deliberate rather than a gap. check_eq $P/log-file-api-present "" "$(pv log_file_api_present)" check_eq $P/log-find-log-dir-present "" "$(pv log_find_log_dir_present)" # ================================================= absltest and parameterized = # A REAL MODULE ON DISK, written by the probe and imported by path -- which is # what `loadTestsFromModule' takes, and what the ticket names. Built to produce # a KNOWN summary: 8 methods becoming 10 tests, with 1 failure, 1 error, 1 skip # and 1 expected failure. check_eq $P/testmod-imported "" "$(pv testmod_imported)" check_eq $P/testmod-case-is-absltest "" "$(pv testmod_case_is_absltest)" check_eq $P/testmod-case-is-unittest "" "$(pv testmod_case_is_unittest)" check_eq $P/testmod-parameterized-is-absltest "" \ "$(pv testmod_parameterized_is_absltest)" # `parameterized' EXPANDS ONE METHOD INTO N TESTS AND NAMES THEM. The generated # names are part of its contract -- `@parameters' numbers them and appends the # repr of the tuple, `@named_parameters' appends the name given -- so they are # asserted whole rather than counted. check_eq $P/testmod-count "<10>" "$(pv testmod_count)" check_eq $P/testmod-names \ "" \ "$(pv testmod_names)" check_eq $P/parameterized-from-parameters "<3>" "$(pv parameterized_from_parameters)" check_eq $P/parameterized-from-named "<2>" "$(pv parameterized_from_named)" # RUN THROUGH absl's OWN RUNNER -- `_pretty_print_reporter.TextTestRunner', the # one `absltest.main' installs -- so the reporter, the TestCase base and both # decorators are the real ones and only the process boundary is absent. check_eq $P/testmod-runner-type "" "$(pv testmod_runner_type)" check_eq $P/testmod-result-type "" "$(pv testmod_result_type)" # THE COUNTS THE MODULE WAS BUILT TO PRODUCE. Each is asserted separately: a # runner that turned the error into a failure would keep the total and be wrong. check_eq $P/testmod-tests-run "<10>" "$(pv testmod_tests_run)" check_eq $P/testmod-failures "<1>" "$(pv testmod_failures)" check_eq $P/testmod-errors "<1>" "$(pv testmod_errors)" check_eq $P/testmod-skipped "<1>" "$(pv testmod_skipped)" check_eq $P/testmod-expected-failures "<1>" "$(pv testmod_expected_failures)" check_eq $P/testmod-unexpected-successes "<0>" "$(pv testmod_unexpected_successes)" check_eq $P/testmod-was-successful "" "$(pv testmod_was_successful)" # ...AND THE SUMMARY LINE THE CONSOLE GETS, which is produced by different code # from the counts above. Only the pair says the reporter agrees with the result. check_eq $P/testmod-summary \ "" \ "$(pv testmod_summary)" check_eq $P/testmod-ran-line "" "$(pv testmod_ran_line)" # WHICH tests failed, not just how many -- a runner that failed the wrong test # would satisfy every count above. check_eq $P/testmod-failed-test-name "" \ "$(pv testmod_failed_test_name)" check_eq $P/testmod-errored-test-name "" \ "$(pv testmod_errored_test_name)" check_eq $P/testmod-skipped-reason "" \ "$(pv testmod_skipped_reason)" check_eq $P/testmod-reports-failure "" "$(pv testmod_reports_failure)" check_eq $P/testmod-reports-error "" "$(pv testmod_reports_error)" check_eq $P/testmod-failure-detail "" "$(pv testmod_failure_detail)" check_eq $P/testmod-error-detail "" "$(pv testmod_error_detail)" # ONE OF THE TEN TESTS EXERCISES absltest's OWN ASSERTIONS -- assertLen, # assertEmpty, assertStartsWith, assertContainsSubset, assertSameElements, # assertBetween, assertRaisesWithLiteralMatch -- which are the reason to # subclass `absltest.TestCase' rather than `unittest.TestCase' at all. Its # outcome is emitted by name rather than inferred from the counts, so this reads # as the claim it is instead of as an arithmetic argument about two other cases. check_eq $P/testmod-assertions-case "" "$(pv testmod_assertions_case)" # ------------------------------------------------ absltest's own helpers ----- # `create_tempdir'/`create_tempfile' are absltest's filesystem APIs, driven # outside the module above so a filesystem refusal is one red case rather than a # wrong test count. The root they land under is `--test_tmpdir', which the probe # removes again if it created it. check_eq $P/helper-tempdir-created "" "$(pv helper_tempdir_created)" check_eq $P/helper-tempfile-roundtrip "" "$(pv helper_tempfile_roundtrip)" check_eq $P/helper-tempdir-under-test-tmpdir "" \ "$(pv helper_tempdir_under_test_tmpdir)" check_eq $P/helper-tempdir-root-left-behind "" \ "$(pv helper_tempdir_root_left_behind)" # `flagsaver' RESTORES A FLAG AFTER THE BLOCK, in both spellings. Asserted in # three places each time -- inside, after, and against the value it started from # -- because a saver that restored the DEFAULT rather than the prior value would # pass a two-case check. check_eq $P/flagsaver-inside "" "$(pv flagsaver_inside)" check_eq $P/flagsaver-restored "" "$(pv flagsaver_restored)" check_eq $P/flagsaver-decorator "" "$(pv flagsaver_decorator)" check_eq $P/flagsaver-restored-after-decorator "" \ "$(pv flagsaver_restored_after_decorator)" # ------------------------------------------------------------ argparse_flags - # absl's argparse bridge: a real `argparse.ArgumentParser' subclass that also # understands absl flags. Both halves are driven in one vector -- an argparse # option, an absl flag, and a positional -- since the whole point of the module # is that they coexist. check_eq $P/argparse-flags-mixed "" \ "$(pv argparse_flags_mixed)" check_eq $P/argparse-flags-is-argparse "" "$(pv argparse_flags_is_argparse)" # -------------------------------------------------------------- xml_reporter - # What `--xml_output_file' installs, exercised as a renderer over the SAME # module into a StringIO: no file written, and the counts must agree with the # text runner's above. check_eq $P/xml-report-counts "" \ "$(pv xml_report_counts)" rm -rf "$W"