. "$BST_LIB" # BST-TIMEOUT: 1200 # THREE python3 starts, and they are the whole cost. # # Everything else here reads the filesystem or dpkg with # # shell builtins, or greps the output of one of the three. # # # # They are a bare `import', a second one whose STDERR is # # the assertion (`import-quiet'), and the probe -- which # # takes ONE start for a hundred and thirty-eight # # measurements. # # # # THE PARSING IS NOT THE COST. Measured on the build host # # against this build's own wasm32 payload under CPython # # 3.12.3: 0.077 s wall for the whole probe, of which the # # interpreter start and `import flexparser' are 0.063 s. # # Every fixture below is under a dozen lines, so this suite # # is three interpreter starts and a rounding error, exactly # # like the other 99-python3-* files. # # # # 1200 is 400 s per start, the same budget # # 99-python3-mpmath and 99-python3-gast set for their # # three, and roughly six times the ~60 s a python3 start # # has measured at on this arm. The margin is for the start, # # not for the parsing. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify, and the measured elapsed # # replaces this note in the test plan rather than in this # # comment, which would cost an image build to keep the # # shipped suite byte-identical. # # python3-flexparser 0.3 (meta-openembedded/meta-python # recipes-devtools/python/python3-flexparser_0.3.bb, `inherit pypi # python_setuptools_build_meta ptest', PyPI sdist) on the wasm32-hwjs glibc # image. # # ============================================================================ # WHAT THE PACKAGE DOES -- AND THE THING IT DOES NOT DO, WHICH DECIDES THE # SHAPE OF THIS FILE. # ============================================================================ # # flexparser is a FRAMEWORK FOR WRITING A PARSER, not a parser. You subclass # `ParsedStatement' once per kind of line, subclass `Block' for anything with an # opening and a closing, and hand the result to `parse(entry_point, spec)'; what # comes back is a `ParsedProject' -- a dict of source location to `ParsedSource' # -- carrying the statements in file order with a line/column position on each. # # It therefore ships NO grammar of its own, and a suite that only imported it # would be asserting nothing about parsing. The grammar the probe defines -- # `Open', `Close', `Comment', `EqualFloat', an `Include', and the two # `ParsingError' subclasses -- is deliberately the one upstream's own # `flexparser/testsuite/common.py' uses, so what is measured here is the shape # upstream documents rather than one invented for this file. # # ============================================================================ # PURE PYTHON, SO NO PATCH AND NO BBAPPEND. # ============================================================================ # # meta-python's recipe builds on this arm as it stands, and the wasm32 build is # a byte-compile and an install: two modules, `flexparser/__init__.py' and # `flexparser/flexparser.py'. That purity is what makes it portable here at all # -- this image's python3 is ONE STATIC BINARY with an empty lib-dynload and no # dynamic loader (BRINTOS-1476), so a module carrying a compiled extension could # not be imported whatever it built to. `pkg-no-so', `pkg-only-python' and # `impure-modules' assert that from three sides -- the packaged file list, the # same list by extension, and the import closure from inside the interpreter -- # so a version bump that grew a compiled speedup fails at install or at import # rather than in some later caller. # # ============================================================================ # NOTHING ABOUT IT WAITS ON ANOTHER PORT, AND ITS ONE PyPI DEPENDENCY IS NOT # INSTALLED -- DELIBERATELY. # ============================================================================ # # The recipe's one DEPENDS, `python3-setuptools-scm-native', is native and # resolves on the host. Its three target RDEPENDS -- `python3-compression', # `python3-email' and `python3-logging' -- are stdlib splits of the interpreter # recipe itself and were already on this image; the built .deb's control stanza # reads `Depends: python3-compression, python3-core, python3-email, # python3-logging' and `pkg-depends-*' pins all four. # # `requires_dist' shows PyPI's own metadata asking for `typing-extensions', and # `typing_extensions_used' is `no': flexparser never reaches for it. That is not # a missing dependency: both uses of it in the source are behind a version # guard -- # # if sys.version_info >= (3, 10): from typing import TypeAlias # if sys.version_info >= (3, 11): from typing import Self # # -- and this image's interpreter is 3.12, so neither branch is taken. # `py_version' is pinned beside them because it is what makes the absence # harmless; an interpreter downgrade below 3.11 would turn `import flexparser' # into an ImportError, and these three cases together are what would say so. # # This deliberately does NOT assert that typing-extensions is uninstalled. The # case did, and the first image it ran on carried python3-typing-extensions -- # pulled in by python3-flexcache and python3-pint, both of which `Depends' on # it -- so the case went red over a sibling's dependency while flexparser's own # `Depends' (pinned by `pkg-depends-*') never mentioned it. # # ============================================================================ # THE TICKET ASKED FOR SIX ERROR PATHS AS `DOCUMENTED DIAGNOSTICS'. ONE OF THEM # IS. THIS IS THIS PORT'S CENTRAL FINDING AND THE SUITE SCORES BOTH KINDS. # ============================================================================ # # flexparser's structured reporting covers the CONTENT of a file it could read, # and nothing else. Errors split cleanly into two kinds, and which kind a given # failure is decides whether a caller sees a positioned object or a traceback: # # ERRORS THAT ARE DATA. A statement the grammar understood but could not # accept comes back INSIDE the tree as a `ParsingError' -- it is never raised, # `parse()' returns normally, and a caller that only looked at the return # value would see a parse that "worked". `bad-has-errors' and the four # `bad-err*' groups pin two of them with their positions and their captured # values. An unterminated block is the same kind: the block's `closing' is an # `UnexpectedEOS' sitting in the tree. # # ERRORS THAT ARE RAISED. A file that does not exist is a bare # `FileNotFoundError' out of `Parser.parse_file''s `path.open("rb")'. A file # that is not utf-8 is a bare `UnicodeDecodeError' out of # `parse_bytes''s `b.decode(self._encoding)'. An include naming an absolute # path, or one escaping the including file's directory, is a `ValueError' out # of `default_locator'. NONE of the four carries a line, a column or a # `ParsingError' anywhere -- `err-missing-file', `err-bad-encoding', # `err-absolute-include' and `err-escaping-include' pin the exact message each # produces, because "produces the documented diagnostic rather than a # traceback out of the framework" is what the ticket asked to be true and it # is true of exactly one of the six. # # Asserting the ticket's expectation instead would have red a package behaving # exactly as its source says, and hidden the real boundary -- that you must # open the file yourself if you want a diagnostic about opening it -- behind a # failure reading "flexparser is broken here". # # ============================================================================ # THE ERROR OBJECT DOES NOT CARRY THE FILENAME. THE TICKET SAYS IT DOES. # ============================================================================ # # The ticket asked for "the documented error object carrying the right # filename, line number and message". It carries the line and the column. It # does NOT carry the filename: `bad-err-fields' is the object's whole field set # -- # # end_col, end_line, is_position_set, raw, start_col, start_line, value # # -- and `bad-err-has-path' is `no'. The filename lives on the source's `BOF', # one level up, so a caller wanting `file:line' has to join the two itself; # `bad-joined-location' is that join, pinned as the thing a consumer must # actually write. # # ============================================================================ # ...AND str() ON THE GRAMMAR'S OWN ERROR IS NOT A DIAGNOSTIC EITHER. # ============================================================================ # # `ParsingError.__str__' is `Statement.__str__', which is the dataclass repr, so # a grammar's own error subclass prints as # # NotAValidIdentifier(start_line=3, start_col=0, end_line=3, end_col=23, # raw='not an identifier = 1.0', value='not an identifier') # # -- which happens to contain the line number, and is not a message anybody # would want to show a user. Only `UnknownStatement', the framework's own class, # overrides `__str__' into the readable `Could not parse '...' (4,0-4,8)' form. # `unknown-str' and `grammar-err-str' pin both, as separate cases, because the # difference is a property of WHICH class raised and a reader who saw only one # of them would draw the wrong conclusion about the other. # # ============================================================================ # AN UNTERMINATED BLOCK REPORTS ITSELF AS CLOSED. THIS IS AN UPSTREAM DEFECT # AND THE SUITE PINS THE WRONG ANSWER ON PURPOSE. # ============================================================================ # # `Block.is_closed()' is `not isinstance(self.closing, EOS)'. On end of source # `consume_body_closing' puts an `UnexpectedEOS' in the closing slot -- and # `UnexpectedEOS' derives from `ParsingError', which derives from `Statement'. # It is NOT an `EOS'. So the isinstance test is false and `is_closed()' returns # TRUE for a block that was never closed: # # unterm-is-closed yes <-- wrong, and asserted as `yes' # unterm-closing-kind UnexpectedEOS # unterm-closing-is-eos no <-- why it is wrong # unterm-closing-is-error yes <-- the error IS in the tree # # The error is reported correctly through `has_errors' and `errors()', so a # caller using those sees it; a caller branching on `is_closed()' does not. # `closed-closing-is-eos' is the properly-closed block for contrast -- also # `no', because ITS closing is the grammar's `Close', which is likewise not an # `EOS'. That pair is what says the predicate is unable to distinguish the two # cases at all rather than merely mislabelling one of them. # # This is a defect in hgrecco/flexparser, a third-party public repository, and # per docs/conventions/no-public-upstream-reports.md we file nothing against # one. It is recorded here and in the design doc, and the four cases above are # what will turn red -- visibly, and with the reason in this header -- on a # version bump that fixes it. # # ============================================================================ # `inherit ptest' BUILDS NO ptest PACKAGE ON THIS DISTRO, AND THAT MATTERS FOR # WHY THIS PORT EXISTS AT ALL. # ============================================================================ # # The recipe inherits `ptest' and carries a `do_install_ptest', but # `DISTRO_FEATURES' is `ipv4 x11 wayland' -- no `ptest' -- so no # `python3-flexparser-ptest' is produced and none is on the image. # `pkg-no-ptest' asserts that. # # It is worth stating because of the reason this package is in the round: # python3-pint declares flexparser under `RDEPENDS:${PN}-ptest'. That is a # package this distro does not build, so nothing on this image will pull # flexparser in on pint's behalf -- it is installed because the image recipe # names it, and `pkg-installed' is what says so. # # Upstream's own testsuite is a different thing and IS installed: # `flexparser/testsuite/' carries an `__init__.py', so it is an ordinary # subpackage and its files ship into site-packages. `tests-installed' is that # packaging fact and `pkg-no-ptest' is the separate statement that nothing runs # them; a reader who conflates the two concludes this image runs upstream's # test suite, which it does not. # # ============================================================================ # WHERE THE EXPECTED VALUES CAME FROM. # ============================================================================ # # Measured, not recalled: the same probe over the same payload this build # packaged for the target, extracted from the wasm32 .deb and run under CPython # 3.12.3 on the build host. Byte-identical over three PYTHONHASHSEED values (0, # 1, 12345), 140 lines, empty stderr, and no `:' token anywhere. # # Eleven of the values are diagnostics or paths that QUOTE the fixture # directory. They are written below with `$BST_DATA' interpolated, and # `data-dir' asserts that the probe read its fixtures from exactly that # directory -- so a probe that had somehow parsed a different copy is visible # as one failed case naming the directory, rather than as ten diagnostics # quoting an unexpected filename. P=python3-flexparser bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages PKGDIR=$SITE/flexparser DI=$SITE/flexparser-0.3.dist-info PKGLIST=/var/lib/dpkg/info/python3-flexparser.list PROBE=$BST_DATA/python3-flexparser-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-flexparser.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------ what shipped --- check $P/site-packages-dir test -d "$SITE" check $P/package-dir test -d "$PKGDIR" check $P/no-module-py test ! -f "$SITE/flexparser.py" check $P/init-file test -s "$PKGDIR/__init__.py" check $P/flexparser-py test -s "$PKGDIR/flexparser.py" check $P/pycache test -s "$PKGDIR/__pycache__/__init__.cpython-312.pyc" check $P/pycache-submodule test -s "$PKGDIR/__pycache__/flexparser.cpython-312.pyc" check $P/no-cli test ! -e /usr/bin/flexparser # NO PEP 561 MARKER. flexparser is a package whose whole selling point is its # use of `typing', and it still ships no `py.typed' -- so a consumer's mypy # reads no annotations from it. Asserted because its absence is what lets # `pkg-only-python' below allow nothing but .py, .pyc and dist-info. check $P/no-py-typed test ! -e "$PKGDIR/py.typed" # UPSTREAM'S TESTSUITE IS INSTALLED -- see the header. This is the packaging # fact; `pkg-no-ptest' below is the separate statement that nothing runs it. check $P/tests-installed test -s "$PKGDIR/testsuite/test_parser.py" # `test -f', not `test -s': upstream ships this __init__.py as a ZERO-BYTE # package marker, so a non-empty test asserts a property it does not have and # never had. It is still a real package marker: the build byte-compiled it to a # 163-byte testsuite/__pycache__/__init__.cpython-312.pyc like any other module. check $P/tests-is-package test -f "$PKGDIR/testsuite/__init__.py" check $P/tests-fixtures test -s "$PKGDIR/testsuite/bla1.txt" cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # EVERY SHIPPED FILE IS PYTHON, its bytecode, its dist-info or one of upstream's # own .txt fixtures. A compiled extension could not be imported on this arm at # all (BRINTOS-1476), so this is the packaging-side statement of what # `impure-modules' says from inside the interpreter. _bad="" while IFS= read -r _f; do [ -d "$_f" ] && continue case "$_f" in *.py|*.pyc|*.txt|*/dist-info/*|*/AUTHORS|*/LICENSE|*/METADATA|*/RECORD|*/WHEEL) ;; "") ;; *) _bad="$_bad $_f" ;; esac done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-name "Name: flexparser" cat "$DI/METADATA" check_out $P/dist-info-version "Version: 0.3" cat "$DI/METADATA" check_out $P/dist-info-summary \ "Summary: Parsing made fun ... using typing." cat "$DI/METADATA" check_out $P/dist-info-license "License: BSD-3-Clause" cat "$DI/METADATA" check_out $P/dist-info-top-level "flexparser" cat "$DI/top_level.txt" # ROOT-IS-PURELIB IS THE WHEEL'S OWN STATEMENT of what `pkg-no-so' asserts from # the file list -- an arch-independent wheel, built on a wasm32 target, which is # what a pure-Python port is supposed to produce. check_out $P/dist-info-generator "Generator: bdist_wheel" cat "$DI/WHEEL" check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" check_out $P/dist-info-tag "Tag: py3-none-any" cat "$DI/WHEEL" # ------------------------------------------------------------------- dpkg --- sed -n '/^Package: python3-flexparser$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" check_out $P/pkg-version "Version: 0.3" cat "$W/st" check_out $P/pkg-arch "Architecture: wasm32" cat "$W/st" grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" # THE FOUR THE PACKAGING DERIVED -- three from the recipe's RDEPENDS and # python3-core from the pypi class. All four are stdlib splits of the ported # interpreter, so none of them waited on another port. check_out $P/pkg-depends-core "python3-core" cat "$W/dep" check_out $P/pkg-depends-compression "python3-compression" cat "$W/dep" check_out $P/pkg-depends-email "python3-email" cat "$W/dep" check_out $P/pkg-depends-logging "python3-logging" cat "$W/dep" # NO ptest PACKAGE -- see the header. The recipe inherits ptest; DISTRO_FEATURES # does not carry it, so nothing was built and nothing is installed. check_eq $P/pkg-no-ptest "" \ "$(grep -c '^Package: python3-flexparser-ptest$' /var/lib/dpkg/status 2>/dev/null | grep -v '^0$')" # ----------------------------------------------------------------- import --- check $P/import python3 -c 'import flexparser' # AND IT IS SILENT. An import that works while writing to stderr is a port that # half-works, and nothing else in this file would notice. check_eq $P/import-quiet "" \ "$(python3 -c 'import flexparser' 2>&1 >/dev/null)" # ------------------------------------------------------------------ probe --- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present OUT=$W/probe bst_probe $P/probe-run "$OUT" 900 python3 "$PROBE" pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi check_eq $P/probe-complete "yes" "$(pv probe_done)" check_eq $P/probe-count "138" "$(pv emitted_count)" # --------------------------------------------------------------- identity --- # WHERE THE FIXTURES WERE READ FROM. Eleven values below quote this directory, # so it is asserted once here rather than eleven times by accident. check_eq $P/data-dir "$BST_DATA" "$(pv data_dir)" check_eq $P/module-file "$PKGDIR/__init__.py" "$(pv module_file)" check_eq $P/submodule-file "$PKGDIR/flexparser.py" "$(pv submodule_file)" check_eq $P/is-package "yes" "$(pv is_package)" # THE VERSION, FROM BOTH SIDES. `__version__' is not a literal in the source -- # `__init__.py' calls `importlib.metadata.version("flexparser")' and falls back # to the string `unknown' if that raises. So the attribute agreeing with the # recipe proves the dist-info shipped and was found, and a packaging change that # dropped it would read `unknown' here rather than failing an import. check_eq $P/version-attr "0.3" "$(pv version_attr)" check_eq $P/version-metadata "0.3" "$(pv version_metadata)" # TWO MODULES, AND BOTH ARE .py. See the header on BRINTOS-1476. check_eq $P/module-count "2" "$(pv module_count)" check_eq $P/impure-modules "" "$(pv impure_modules)" # ------------------------------------- the dependency that is not needed --- # See the header: `typing-extensions' is asked for by PyPI's metadata and is not # needed, because both uses of it are behind a version guard this interpreter # passes. The three cases are one statement. check_eq $P/py-version "3.12" "$(pv py_version)" # NOT `is it installed' -- that is a fact about the rest of the image, and a # sibling package that depends on typing-extensions would flip it while saying # nothing about flexparser. This asks whether flexparser REACHED for it. check_eq $P/typing-extensions-unused "no" "$(pv typing_extensions_used)" check_eq $P/requires-dist \ "pytest ; extra == 'test',pytest-cov ; extra == 'test',pytest-mpl ; extra == 'test',pytest-subtests ; extra == 'test',typing-extensions" \ "$(pv requires_dist)" # THE HASHER flexparser PICKED for its source fingerprints. It walks a fixed # list and takes the first that both exists and runs, so `hasher-name' names # which of the five this image's hashlib actually provides -- a build of python3 # without blake2 would fall through to sha3_512 here and this case is what would # say so. check_eq $P/hasher-name "blake2b" "$(pv hasher_name)" check_eq $P/hasher-list "blake2b,blake2s,sha3_512,sha512,sha1" "$(pv hasher_list)" check_eq $P/hash-algorithm "blake2b" "$(pv hash_algorithm)" # ------------------------------------------------------------- happy path --- check_eq $P/parse-type "ParsedProject" "$(pv parse_type)" check_eq $P/parse-is-dict "yes" "$(pv parse_is_dict)" check_eq $P/root-type "ParsedSource" "$(pv root_type)" check_eq $P/root-has-errors "no" "$(pv root_has_errors)" check_eq $P/root-error-count "0" "$(pv root_error_count)" # THE OPENING OF THE ROOT SOURCE is a `BOF' and names the file it came from, # which is the only place the filename appears -- see the header. check_eq $P/bof-type "BOF" "$(pv bof_type)" check_eq $P/bof-path "$BST_DATA/python3-flexparser-main.txt" "$(pv bof_path)" check_eq $P/bof-is-main "yes" "$(pv bof_is_main)" check_eq $P/source-location "$BST_DATA/python3-flexparser-main.txt" "$(pv source_location)" check_eq $P/root-body-count "6" "$(pv root_body_count)" check_eq $P/root-body-kinds "Comment,Include,EqualFloat,EqualFloat,MyBlock,EqualFloat" \ "$(pv root_body_kinds)" # EVERY ELEMENT OF THE PARSED STRUCTURE, ONE CASE EACH, AGAINST LITERALS -- the # element-by-element assertion the ticket asked for. Positions are # `start_line,start_col-end_line,end_col', lines 1-based and columns 0-based, # which is `format_position' verbatim. check_eq $P/body0-kind "Comment" "$(pv body0_kind)" check_eq $P/body0-pos "1,0-1,52" "$(pv body0_pos)" check_eq $P/body0-raw "# the root definition file for 99-python3-flexparser" "$(pv body0_raw)" check_eq $P/body1-kind "Include" "$(pv body1_kind)" check_eq $P/body1-pos "2,0-2,34" "$(pv body1_pos)" check_eq $P/body1-raw "include python3-flexparser-inc.txt" "$(pv body1_raw)" check_eq $P/body2-kind "EqualFloat" "$(pv body2_kind)" check_eq $P/body2-pos "3,0-3,14" "$(pv body2_pos)" check_eq $P/body2-raw "gravity = 9.81" "$(pv body2_raw)" check_eq $P/body3-kind "EqualFloat" "$(pv body3_kind)" check_eq $P/body3-pos "4,1-4,19" "$(pv body3_pos)" check_eq $P/body3-raw "$(printf 'planck\t=\t6.626e-34')" "$(pv body3_raw)" # A Block IS NOT A Statement, so it carries no `raw' -- only its opening and its # closing do. `' is that fact, reported by the probe, not a # measurement that failed. check_eq $P/body4-kind "MyBlock" "$(pv body4_kind)" check_eq $P/body4-pos "5,0-10,4" "$(pv body4_pos)" check_eq $P/body4-raw "" "$(pv body4_raw)" check_eq $P/body5-kind "EqualFloat" "$(pv body5_kind)" check_eq $P/body5-pos "11,0-11,13" "$(pv body5_pos)" check_eq $P/body5-raw "euler = 2.718" "$(pv body5_raw)" check_eq $P/comment-count "1" "$(pv comment_count)" check_eq $P/include-count "1" "$(pv include_count)" check_eq $P/equalfloat-count "3" "$(pv equalfloat_count)" check_eq $P/block-count "1" "$(pv block_count)" check_eq $P/root-names "gravity,planck,euler" "$(pv root_names)" check_eq $P/root-values "9.81,6.626e-34,2.718" "$(pv root_values)" # THE PARSED VALUE IS A REAL float, not the text that produced it -- which is # what the grammar's `from_string' was for, and the one thing a framework this # thin could still get wrong invisibly. check_eq $P/gravity-value "9.81" "$(pv gravity_value)" check_eq $P/gravity-type "float" "$(pv gravity_type)" # TRAILING WHITESPACE AND A LEADING TAB, which is what the ticket asked about. # `planck' is written in the fixture as a TAB-indented line, with tabs around # the `=' and three trailing spaces. `_strip_spaces' is on by default, so `raw' # is the STRIPPED text -- the outer whitespace is gone, the inner tabs survive # -- and the COLUMNS moved with it: the statement starts at column 1, not 0, # and ends at 19 rather than at the physical end of the line. check_eq $P/planck-name "planck" "$(pv planck_name)" check_eq $P/planck-value "6.626e-34" "$(pv planck_value)" check_eq $P/planck-raw "$(printf 'planck\t=\t6.626e-34')" "$(pv planck_raw)" check_eq $P/planck-pos "4,1-4,19" "$(pv planck_pos)" check_eq $P/planck-raw-has-tab "yes" "$(pv planck_raw_has_tab)" # THE BLOCK, opening to closing. check_eq $P/block-opening-kind "Open" "$(pv block_opening_kind)" check_eq $P/block-closing-kind "Close" "$(pv block_closing_kind)" check_eq $P/block-is-closed "yes" "$(pv block_is_closed)" check_eq $P/block-body-count "3" "$(pv block_body_count)" check_eq $P/block-body-kinds "Comment,EqualFloat,EqualFloat" "$(pv block_body_kinds)" check_eq $P/block-names "avogadro,boltzmann" "$(pv block_names)" check_eq $P/block-values "6.02214076e+23,1.380649e-23" "$(pv block_values)" # BLANK LINES ARE SKIPPED BY THE ITERATOR rather than parsed into anything -- # `StatementIterator._get_next_strip' loops `while not part'. So the two # statements around the blank line are two lines apart in the file and adjacent # in the tree, and `blank-line-gap' is what says the line numbers survived the # skip. A framework that renumbered would pass every other case in this file. check_eq $P/avogadro-pos "7,0-7,24" "$(pv avogadro_pos)" check_eq $P/boltzmann-pos "9,0-9,24" "$(pv boltzmann_pos)" check_eq $P/blank-line-gap "2" "$(pv blank_line_gap)" # COMMENTS ARE STATEMENTS HERE, not something the framework drops. flexparser # has no comment concept of its own -- the `#' line is a comment only because # the grammar says so, and it lands in the tree like anything else, inside the # block as well as outside it. The ticket asked for comments to be "handled as # documented"; what is documented is that the framework does not handle them. check_eq $P/comment-text "# the root definition file for 99-python3-flexparser" \ "$(pv comment_text)" check_eq $P/block-comment-text "# a comment inside the block" "$(pv block_comment_text)" # ---------------------------------------------------------------- include --- check_eq $P/project-len "2" "$(pv project_len)" check_eq $P/project-keys "None,(python3-flexparser-main.txt,python3-flexparser-inc.txt)" \ "$(pv project_keys)" check_eq $P/include-target "python3-flexparser-inc.txt" "$(pv include_target)" check_eq $P/include-stmt-pos "2,0-2,34" "$(pv include_stmt_pos)" # THE RELATIVE TARGET RESOLVED AGAINST THE INCLUDING FILE, not against the # process cwd -- which is the whole point of `default_locator', the property the # ticket asked about, and the one thing an include feature can get wrong # invisibly: a cwd-relative resolution works from the directory the fixtures # happen to be in and fails everywhere else. check_eq $P/locator-result "$BST_DATA/python3-flexparser-inc.txt" "$(pv locator_result)" check_eq $P/locator-is-inc "yes" "$(pv locator_is_inc)" check_eq $P/included-bof-path "$BST_DATA/python3-flexparser-inc.txt" "$(pv included_bof_path)" check_eq $P/included-is-inc "yes" "$(pv included_is_inc)" check_eq $P/included-body-count "3" "$(pv included_body_count)" check_eq $P/included-names "light_speed,electron_mass" "$(pv included_names)" check_eq $P/included-values "299792458.0,9.109e-31" "$(pv included_values)" # ITERATION ORDER ACROSS THE TWO FILES. `iter_statements' walks INTO the include # at the point the include statement appears, so the included names sit between # the root's comment and `gravity' rather than after everything in the root -- # which is the difference between an include and an append, and is not visible # from either file's own parse. check_eq $P/iter-names "light_speed,electron_mass,gravity,planck,avogadro,boltzmann,euler" \ "$(pv iter_names)" check_eq $P/iter-count "16" "$(pv iter_count)" check_eq $P/iter-blocks-count "12" "$(pv iter_blocks_count)" # THE CONTENT HASH. flexparser fingerprints every source it reads; these three # say the digest is stable across two parses of the same bytes and differs # between two different files, which is what a fingerprint has to do to be worth # computing. check_eq $P/hash-stable "yes" "$(pv hash_stable)" check_eq $P/content-hash-algorithm "blake2b" "$(pv content_hash_algorithm)" check_eq $P/content-hash-differs "yes" "$(pv content_hash_differs)" # ------------------------------------------------- errors that are DATA ----- # See the header. `parse()' RETURNED NORMALLY for a file with two bad # statements, and the errors are in the tree. check_eq $P/bad-has-errors "yes" "$(pv bad_has_errors)" check_eq $P/bad-error-count "2" "$(pv bad_error_count)" check_eq $P/bad-error-kinds "NotAValidIdentifier,CannotParseToFloat" "$(pv bad_error_kinds)" # IT IS AN Exception AND IT WAS NEVER RAISED. `ParsingError' derives from # `Exception', so a caller can raise it if it wants to -- and flexparser does # not, which is the whole design and the thing a reader most often assumes # wrongly. check_eq $P/bad-error-is-exception "yes" "$(pv bad_error_is_exception)" check_eq $P/bad-err0-kind "NotAValidIdentifier" "$(pv bad_err0_kind)" check_eq $P/bad-err0-pos "3,0-3,23" "$(pv bad_err0_pos)" check_eq $P/bad-err0-raw "not an identifier = 1.0" "$(pv bad_err0_raw)" check_eq $P/bad-err0-line "3" "$(pv bad_err0_line)" check_eq $P/bad-err0-value "not an identifier" "$(pv bad_err0_value)" check_eq $P/bad-err1-kind "CannotParseToFloat" "$(pv bad_err1_kind)" check_eq $P/bad-err1-pos "4,0-4,22" "$(pv bad_err1_pos)" check_eq $P/bad-err1-raw "planck = six point six" "$(pv bad_err1_raw)" check_eq $P/bad-err1-line "4" "$(pv bad_err1_line)" check_eq $P/bad-err1-value "six point six" "$(pv bad_err1_value)" # WHICH FILE THE ERROR IS IN IS NOT ON THE ERROR OBJECT -- see the header. This # is the ticket's "carrying the right filename" and the answer is that it does # not carry one. check_eq $P/bad-err-has-path "no" "$(pv bad_err_has_path)" check_eq $P/bad-err-fields \ "end_col,end_line,is_position_set,raw,start_col,start_line,value" \ "$(pv bad_err_fields)" check_eq $P/bad-source-path "$BST_DATA/python3-flexparser-bad.txt" "$(pv bad_source_path)" check_eq $P/bad-joined-location "python3-flexparser-bad.txt:3" "$(pv bad_joined_location)" # str() ON THE TWO ERROR CLASSES -- see the header. The framework's own class # reads as a diagnostic; a grammar's own subclass reads as a dataclass repr. check_eq $P/unknown-str "Could not parse 'nonsense' (4,0-4,8)" "$(pv unknown_str)" check_eq $P/unknown-names-line "yes" "$(pv unknown_names_line)" check_eq $P/unknown-is-parsing-error "yes" "$(pv unknown_is_parsing_error)" check_eq $P/grammar-err-str \ "NotAValidIdentifier(start_line=3, start_col=0, end_line=3, end_col=23, raw='not an identifier = 1.0', value='not an identifier')" \ "$(pv grammar_err_str)" check_eq $P/grammar-err-names-line "yes" "$(pv grammar_err_names_line)" # AN UNTERMINATED BLOCK. The error is data, like the two above -- and # `is_closed()' is wrong about it. See the header; this group is the finding. check_eq $P/unterm-has-errors "yes" "$(pv unterm_has_errors)" check_eq $P/unterm-error-count "1" "$(pv unterm_error_count)" check_eq $P/unterm-error-kinds "UnexpectedEOS" "$(pv unterm_error_kinds)" check_eq $P/unterm-block-count "1" "$(pv unterm_block_count)" check_eq $P/unterm-closing-kind "UnexpectedEOS" "$(pv unterm_closing_kind)" # THE POSITION IS THE LINE AFTER THE LAST STATEMENT -- `consume_body_closing' # sets it to `last_line + 1' on StopIteration, so it points at the end of the # source rather than at the unclosed opening. That is a defensible choice and it # is not the one a reader expects, so it is pinned rather than described. check_eq $P/unterm-err-pos "4,0-4,0" "$(pv unterm_err_pos)" check_eq $P/unterm-closing-is-error "yes" "$(pv unterm_closing_is_error)" # THE THREE CASES BELOW ARE THE UPSTREAM DEFECT, asserted at the value the # package actually produces. `unterm-is-closed' is `yes' for a block that was # never closed; `unterm-closing-is-eos' is `no', which is why; and # `closed-closing-is-eos' is `no' for the properly-closed block too, which is # what says the predicate cannot tell the two apart at all. They turn red on a # version bump that fixes it -- which is the point of pinning them. check_eq $P/unterm-is-closed "yes" "$(pv unterm_is_closed)" check_eq $P/unterm-closing-is-eos "no" "$(pv unterm_closing_is_eos)" check_eq $P/closed-closing-is-eos "no" "$(pv closed_closing_is_eos)" # --------------------------------------- errors that are RAISED, not data --- # THE FOUR PATHS THE TICKET ASKED FOR AS DIAGNOSTICS, and the answer is the same # for all four: flexparser does not wrap them. What comes out is the plain # stdlib exception from the open(), the decode() or the locator, with no line, # no column and no ParsingError anywhere. check_eq $P/err-missing-file \ "FileNotFoundError: [Errno 2] No such file or directory: '$BST_DATA/python3-flexparser-does-not-exist.txt'" \ "$(pv err_missing_file)" check_eq $P/err-missing-is-oserror "yes" "$(pv err_missing_is_oserror)" check_eq $P/err-missing-errno "2" "$(pv err_missing_errno)" # THE ENCODING IS HARD-WIRED to utf-8 on the Parser class and this fixture is # latin-1, so the failure is a decode of the whole file rather than of one # statement -- position 41 is a BYTE offset into the source, not a line. check_eq $P/err-bad-encoding \ "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 41: invalid continuation byte" \ "$(pv err_bad_encoding)" check_eq $P/err-bad-encoding-type "UnicodeDecodeError" "$(pv err_bad_encoding_type)" check_eq $P/err-encoding-position "41" "$(pv err_encoding_position)" check_eq $P/parser-encoding "utf-8" "$(pv parser_encoding)" # THE LOCATOR'S TWO REFUSALS. An include that escapes the tree IS caught -- but # by the locator, as a ValueError, and again with no position. Both messages # quote the INCLUDING file, which is the one place a filename reaches a # diagnostic at all. check_eq $P/err-escaping-include \ "ValueError: Cannot refer to locations above the current location ($BST_DATA/python3-flexparser-escaping.txt, ../python3-flexparser-inc.txt)" \ "$(pv err_escaping_include)" check_eq $P/err-absolute-include \ "ValueError: Cannot refer to absolute paths in import statements ($BST_DATA/python3-flexparser-absolute.txt, /etc/hostname)." \ "$(pv err_absolute_include)" check_eq $P/err-locator-absolute \ "ValueError: Cannot refer to absolute paths in import statements ($BST_DATA/python3-flexparser-main.txt, /etc/hostname)." \ "$(pv err_locator_absolute)" check_eq $P/err-locator-escaping \ "ValueError: Cannot refer to locations above the current location ($BST_DATA/python3-flexparser-main.txt, ../python3-flexparser-inc.txt)" \ "$(pv err_locator_escaping)" check_eq $P/err-locator-type \ "TypeError: Cannot handle type , use str or pathlib.Path for files or (package: str, resource_name: str) tuple for a resource." \ "$(pv err_locator_type)" # parse_bytes REFUSES AN INCLUDE outright rather than resolving it against # nothing, which is the documented limitation -- and the one place the framework # itself raises about content rather than about I/O. check_eq $P/err-parse-bytes-include \ "ValueError: parse_bytes does not support using an IncludeStatement" \ "$(pv err_parse_bytes_include)" check_eq $P/parse-bytes-ok "1" "$(pv parse_bytes_ok)" check_eq $P/err-parse-type \ "TypeError: Cannot handle type , use str or pathlib.Path for files or (package: str, resource_name: str) tuple for a resource." \ "$(pv err_parse_type)" check_eq $P/err-bad-spec \ 'TypeError: `spec` must be of type Parser, Block, RootBlock or tuple of type Block or ParsedStatement, not ' \ "$(pv err_bad_spec)" # --------------------------------------------------------- module surface --- # THE EXPORTED NAMES, pinned as a set. This is the API a consumer imports, so a # version bump that adds or drops one is a compatibility fact worth one red. check_eq $P/all-count "23" "$(pv all_count)" check_eq $P/all-names \ "BOF,BOR,BOS,Block,DelimiterAction,DelimiterInclude,EOS,IncludeStatement,NullableParsedResult,ParsedProject,ParsedSource,ParsedStatement,Parser,ParsingError,RootBlock,SPLIT_EOL,StatementIterator,UnexpectedEOF,UnexpectedEOS,UnknownStatement,__version__,parse,parse_bytes" \ "$(pv all_names)" # `UnexpectedEOF' IS THE SAME OBJECT as `UnexpectedEOS', not a subclass -- the # source marks it "Deprecate in 0.3" and 0.3 still carries it, so a consumer # catching the old name still works. check_eq $P/eof-alias "yes" "$(pv eof_alias)" check_eq $P/split-eol-keys "'\r\n','\n','\r'" "$(pv split_eol_keys)" rm -rf "$W"