. "$BST_LIB"
# BST-TIMEOUT: 1200  # THREE python3 starts, and they are the whole cost. Every
#                    # other assertion below reads the filesystem or dpkg with
#                    # shell builtins, or greps a line out of the transcript the
#                    # third start left behind.
#                    #
#                    # The three are: a bare `import', a second one whose STDERR
#                    # is the assertion (`import-quiet'), and the probe, which
#                    # takes ONE start for a hundred and twenty-eight
#                    # measurements.
#                    #
#                    # The work inside the probe is `re.finditer' over inputs
#                    # under 30 characters and a recursive descent over at most
#                    # a dozen tokens. Nothing here is quadratic in anything, so
#                    # the interpreter start is the only thing the number has to
#                    # cover, and on this arm that is tens of seconds before the
#                    # first line of the program runs.
#                    #
#                    # 1200 is 400s per start, the same per-start budget
#                    # 99-python3-imagesize sets for its three.
#                    #
#                    # NOT MEASURED ON TARGET YET: this port's first on-target
#                    # run is the batch verify, and the measured elapsed replaces
#                    # this note in the test plan rather than in this comment,
#                    # which would cost an image build to keep the shipped suite
#                    # byte-identical.
#
# python3-textparser 0.24.0 (meta-openembedded/meta-python
# recipes-devtools/python/python3-textparser_0.24.0.bb, `inherit pypi
# setuptools3', PyPI sdist) on the wasm32-hwjs glibc image.
#
# ============================================================================
# WHAT THE PACKAGE DOES.
# ============================================================================
#
# One flat 926-line module. A `Parser' subclass declares `token_specs()' -- an
# ordered list of (kind, regex) or (kind, friendly-name, regex) -- and
# `grammar()', a tree of combinators. `parse(text)' runs the token specs as ONE
# alternated regex over the text, then matches the grammar against the resulting
# token stream by recursive descent, and returns a parse tree of the token
# VALUES (or of `Token' namedtuples, with `token_tree=True').
#
# The combinators are `Sequence', `Choice'/`ChoiceDict' (and the `choice()'
# factory that picks between them), `Repeated' and its `ZeroOrMore' /
# `OneOrMore' / `*Dict' specialisations, `DelimitedList', `Optional', `Any',
# `AnyUntil', `And', `Not', `NoMatch', `Tag' and `Forward'. No dependency
# outside the standard library, no compiled extension, no C anywhere.
#
# ============================================================================
# WHAT EACH ASSERTION IS AGAINST, AND WHAT IT IS NOT.
# ============================================================================
#
# THE PARSE TREES ARE ASSERTED AGAINST LITERALS, NOT AGAINST A SECOND PARSE.
# A suite that parsed the same text twice and compared would pass on a build
# whose parser had stopped working, so every tree below is written out. They are
# rendered by the probe's `flat()' as a bare bracket shape -- `[]' for a list,
# `{}' for a dict, `()' for a tuple -- which keeps the container types visible
# while staying inside one line and out of three levels of quoting.
#
# THE EXPECTED VALUES ARE A HOST CONTROL. Every literal here was measured by
# running this exact probe against textparser 0.24.0 on an x86-64 CPython 3.12
# during the recipe phase, which is the only place a pure-Python answer can come
# from before the image exists. They are what the target is asserted to
# reproduce, not a record of what it did.
#
# ============================================================================
# THREE ANSWERS THAT ARE SHARP EDGES RATHER THAN FEATURES.
# ============================================================================
#
# Each is upstream's behaviour in 0.24.0, is asserted with the literal wrong-
# looking value, and is stated here so a future release that changes one turns
# this suite red at the case that names it rather than at a case that merely
# uses it.
#
#   anyuntil-no-terminator-raises
#                   `AnyUntil(p)' loops until `p' matches, and NOTHING stops it
#                   at end of input: `__EOF__' does not match `p', so the loop
#                   consumes it and asks for the next token, and `_Tokens.get_
#                   value' indexes off the end. What escapes `Parser.parse' is
#                   therefore `IndexError' -- NOT `ParseError', which is what
#                   every other malformed input in this suite produces, and not
#                   something `except textparser.Error' catches. This is the
#                   documented boundary of `AnyUntil' asserted as the thing it
#                   actually is.
#
#   forward-unset-raises
#                   A `Forward()' never assigned through `<<=' keeps
#                   `_pattern = None' and `match' calls `None.match', so a
#                   grammar with an unclosed forward declaration fails with
#                   `AttributeError' at parse time rather than at construction.
#
#   mismatch-is-truthy
#                   `MISMATCH' is a bare object, so it is TRUE in a boolean
#                   context. A caller who writes `if pattern.match(tokens):' to
#                   test for a match has written the check that always passes.
#                   Asserted directly, because it is the opposite of what the
#                   name suggests.
#
# ============================================================================
# `choice()' IS TWO CLASSES, AND WHICH ONE YOU GET CHANGES THE SEMANTICS.
# ============================================================================
#
# `choice()' returns `ChoiceDict' when the first token kind of every alternative
# is distinct, and falls back to `Choice' when it is not. `ChoiceDict' dispatches
# on the next token's kind and never backtracks; `Choice' tries the alternatives
# in order. So `choice-picks-dict' and `choice-picks-choice' are not trivia --
# they say which of the two `choice-first-alternative' and `choice-order-matters'
# are about.
#
# `choice-order-matters' is the consequence, stated as a failure: with the
# SHORTER alternative first, `Choice' takes it, the trailing token is then
# unconsumed, and the parse raises rather than backtracking into the longer
# alternative. Reordering the same two alternatives parses the same text.
#
# ============================================================================
# THE PACKAGING SECTION, AND THE ONE DEPENDENCY LINE.
# ============================================================================
#
# The module's only module-scope imports are `re', `collections' and `operator'.
# All three are in `python3-core' per poky's `python3-manifest.json' -- CHECKED
# there rather than assumed -- and the built .deb's dependency line is exactly
# `Depends: python3-core', which `pkg-depends-exact' asserts as a whole string
# so an added or dropped name fails rather than passing a substring test.
#
# `pkg-only-python' walks the packaged file list and refuses anything that is
# not a `.py', a `.pyc' or dist-info metadata, which is the shape a pure-Python
# port has and the shape a mis-split one loses.

P=python3-textparser

# The suite reached its first line. A suite that ends itself silently -- a
# stdin redirect on the virtiofs root, an unhandled exit -- is otherwise
# indistinguishable from one that passed every case it emitted.
bst_ok $P/suite-canary

require python3 $P/python3-present

SITE=/usr/lib/python3.12/site-packages
MOD=$SITE/textparser.py
DI=$SITE/textparser-0.24.0.dist-info
PKGLIST=/var/lib/dpkg/info/python3-textparser.list
PROBE=$BST_DATA/python3-textparser-probe.py
tmp=${TMPDIR:-/tmp}
W=$tmp/bst-textparser.$$
mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; }

# ---------------------------------------------------------------- layout ---
# A FLAT module, not a package: `no-package-dir' is the assertion that the
# recipe did not silently start shipping a directory of the same name.
check $P/site-packages-dir test -d "$SITE"
check $P/module-file       test -s "$MOD"
check $P/no-package-dir    test ! -d "$SITE/textparser"
check $P/no-test-dir       test ! -d "$SITE/tests"
check $P/no-py-typed       test ! -e "$SITE/py.typed"

# The byte-compiled form is packaged, so the first import on target does not
# write to a read-only site-packages and fall back to compiling every start.
check $P/pycache-module test -s "$SITE/__pycache__/textparser.cpython-312.pyc"

# ------------------------------------------------------------ dpkg shape ---
cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf"
check $P/pkg-list-present test -s "$W/pkgf"

# Nothing compiled: a pure-Python port that grew a shared object has either
# picked up a build dependency it should not have or been packaged wrong.
check_eq $P/pkg-no-so "" \
  "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)"

# Every packaged regular file is a .py, a .pyc or dist-info metadata.
_bad=""
while IFS= read -r _f; do
  [ -f "$_f" ] || continue
  case "$_f" in
    *.py|*.pyc)      continue ;;
    */__pycache__/*) continue ;;
    */*.dist-info/*) continue ;;
  esac
  _bad="$_bad $_f"
done < "$W/pkgf"
check_eq $P/pkg-only-python "" "$_bad"

check $P/dist-info-dir      test -d "$DI"
check $P/dist-info-metadata test -f "$DI/METADATA"
check_out $P/dist-info-version "Version: 0.24.0" cat "$DI/METADATA"
check_out $P/dist-info-name    "Name: textparser" cat "$DI/METADATA"
check_out $P/dist-info-license-field "License: MIT" cat "$DI/METADATA"
check $P/dist-info-license  test -s "$DI/LICENSE"
check $P/dist-info-record   test -s "$DI/RECORD"

check_out $P/dist-info-generator "Generator: bdist_wheel" cat "$DI/WHEEL"
check_out $P/dist-info-purelib   "Root-Is-Purelib: true"  cat "$DI/WHEEL"
check_out $P/dist-info-tag-py3   "Tag: py3-none-any"      cat "$DI/WHEEL"
check_out $P/dist-info-top-level "textparser"             cat "$DI/top_level.txt"

sed -n '/^Package: python3-textparser$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null
check $P/pkg-known test -s "$W/st"
check_out $P/pkg-installed "Status: install ok installed" cat "$W/st"
check_out $P/pkg-version "Version: 0.24.0-r0" cat "$W/st"

# The whole line, not a substring: an added or dropped name has to fail here.
grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep"
check $P/pkg-depends-line test -s "$W/dep"
check_eq $P/pkg-depends-exact \
  "Depends: python3-core" \
  "$(cat "$W/dep")"

# ---------------------------------------------------------------- import ---
check $P/import python3 -c 'import textparser'

# A DeprecationWarning or a stray print at import time is a regression that
# every other case here would still pass through.
check_eq $P/import-quiet "" \
  "$(python3 -c 'import textparser' 2>&1)"

# ----------------------------------------------------------------- probe ---
if [ ! -f "$PROBE" ]; then
  bst_notok $P/probe-present "$PROBE did not ship"
  rm -rf "$W"
  exit 0
fi
bst_ok $P/probe-present

bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE"

OUT="$W/probe"
pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; }

if [ -s "$OUT" ]; then
  bst_ok $P/probe-output
else
  bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable"
fi

# The probe reached its last line, and emitted every measurement it declares.
# A short transcript would otherwise read as a run of empty `got:''` reds whose
# common cause nothing states.
check_eq $P/probe-complete "yes" "$(pv probe_done)"
check_eq $P/probe-count    "128" "$(pv emitted_count)"

# The module the probe imported is the one this package installed, and not a
# copy that arrived some other way.
check_eq $P/module-answered "$MOD" "$(pv module_file)"

# ------------------------------------------------------------- identity ---
# `__version__' and the dist-info version are two independent statements of the
# same fact, so they are asserted separately.
check_eq $P/is-package                         no "$(pv is_package)"
check_eq $P/version                            0.24.0 "$(pv version)"
check_eq $P/author                             'Erik Moqvist' "$(pv author)"
check_eq $P/has-all                            no "$(pv has_all)"
check_eq $P/dist-version                       0.24.0 "$(pv dist_version)"

# ------------------------------------------------------------------ API ---
# `api-missing' is empty or it names exactly what went. `api-count' is the
# roster's size, so an assertion that silently checked nothing cannot pass.
#
# `mismatch-is-truthy' is the sharp edge the header names: MISMATCH is a plain
# object and therefore true, so `if p.match(t):' is the caller's bug that never
# reports itself.
check_eq $P/api-missing                        '' "$(pv api_missing)"
check_eq $P/api-count                          33 "$(pv api_count)"
check_eq $P/mismatch-is-truthy                 yes "$(pv mismatch_is_truthy)"
check_eq $P/mismatch-is-singleton              yes "$(pv mismatch_is_singleton)"

# --------------------------------------------------- dependency closure ---
# The delta introduces nothing outside the standard library, and all three
# module-scope imports resolved. The delta is NOT asserted as a list: CPython
# imports `collections' and `operator' during its own startup, so what is in it
# is a fact about the interpreter build rather than about this package.
check_eq $P/import-delta-nonstdlib             textparser "$(pv import_delta_nonstdlib)"
check_eq $P/import-delta-count-nonstdlib       1 "$(pv import_delta_count_nonstdlib)"
check_eq $P/deps-resolved                      collections,operator,re "$(pv deps_resolved)"
check_eq $P/deps-all-stdlib                    yes "$(pv deps_all_stdlib)"

# ================================== A COMPLETE ARITHMETIC GRAMMAR ==========
# Built in-process from `Forward', `choice', `Sequence', `ZeroOrMore',
# `DelimitedList' and `Optional'; every tree below is asserted against a
# literal.
#
# `arith-precedence' is the one that says the grammar is a grammar and not a
# token list: `1 + 2 * 3' nests the multiplication INSIDE the second addend,
# where `1 + 2 * 3' parsed flat would not.
#
# `arith-whitespace-same' asserts the SKIP spec by consequence -- two spellings
# of the same expression produce the same tree -- and `arith-token-tree-offsets'
# then shows the offsets that spelling did carry, which is what a caller uses to
# point at an error.
check_eq $P/arith-number                       '[[[1 []] []] []]' "$(pv arith_number)"
check_eq $P/arith-add                          '[[[1 []] [[+ [2 []]]]] []]' "$(pv arith_add)"
check_eq $P/arith-precedence                   '[[[1 []] [[+ [2 [[* 3]]]]]] []]' "$(pv arith_precedence)"
check_eq $P/arith-parens                       '[[[[( [[1 []] [[+ [2 []]]]] )] [[* 3]]] []] []]' "$(pv arith_parens)"
check_eq $P/arith-call                         '[[[[max ( [[[[1 []] []] [[2 []] []]]] )] []] []] []]' "$(pv arith_call)"
check_eq $P/arith-call-nested                  '[[[[f ( [[[[[g ( [[[[1 []] []]]] )] []] []] [[2 []] []]]] )] []] []] []]' "$(pv arith_call_nested)"
check_eq $P/arith-call-empty                   '[[[[f ( [] )] []] []] []]' "$(pv arith_call_empty)"
check_eq $P/arith-semi                         '[[[1 []] [[+ [2 []]]]] [;]]' "$(pv arith_semi)"
check_eq $P/arith-no-semi-optional-empty       yes "$(pv arith_no_semi_optional_empty)"
check_eq $P/arith-negative                     '[[[-1 []] [[+ [-2 []]]]] []]' "$(pv arith_negative)"
check_eq $P/arith-float                        '[[[1.5e-3 []] []] []]' "$(pv arith_float)"
check_eq $P/arith-whitespace-same              yes "$(pv arith_whitespace_same)"
check_eq $P/arith-token-tree-kinds             NUMBER,+,NUMBER "$(pv arith_token_tree_kinds)"
check_eq $P/arith-token-tree-offsets           0,2,4 "$(pv arith_token_tree_offsets)"

# ==================================== A KEY/VALUE GRAMMAR, AS A DICT =======
# `ZeroOrMoreDict' keys each entry on the first element of its match.
#
# `kv-repeated-key' is the property this grammar exists to state: a repeated key
# ACCUMULATES into a list rather than overwriting, so a caller reading
# `tree['a']' gets both entries and a caller expecting a scalar gets a list.
check_eq $P/kv-empty                           '{}' "$(pv kv_empty)"
check_eq $P/kv-one                             '{a:[[a = 1 ;]]}' "$(pv kv_one)"
check_eq $P/kv-three                           '{a:[[a = 1 ;]] b:[[b = two ;]] c:[[c = 3 ;]]}' "$(pv kv_three)"
check_eq $P/kv-repeated-key                    '{a:[[a = 1 ;] [a = 2 ;]]}' "$(pv kv_repeated_key)"
check_eq $P/kv-is-dict                         dict "$(pv kv_is_dict)"
check_eq $P/kv-keys                            a,b "$(pv kv_keys)"

# ================================= EACH COMBINATOR, ON ITS OWN ============
# From here down every case is driven over a MINIMAL grammar, so a red names the
# combinator rather than the expression grammar that happens to contain it.
#
# ---- Optional: present, absent, and the shape of each.
# Both answers are LISTS -- `[]' on mismatch and a one-element list on a match --
# so a caller cannot use the result directly without unwrapping it.
check_eq $P/optional-present                   '[1 [x]]' "$(pv optional_present)"
check_eq $P/optional-absent                    '[1 []]' "$(pv optional_absent)"
check_eq $P/optional-absent-is-empty-list      yes "$(pv optional_absent_is_empty_list)"
check_eq $P/optional-present-is-list           yes "$(pv optional_present_is_list)"

# ---- ZeroOrMore: at zero, at one and at several.
check_eq $P/zeroormore-zero                    '[[] []]' "$(pv zeroormore_zero)"
check_eq $P/zeroormore-one                     '[[1] []]' "$(pv zeroormore_one)"
check_eq $P/zeroormore-several                 '[[1 2 3 4] []]' "$(pv zeroormore_several)"

# ---- Repeated at least once: `OneOrMore', and the `Repeated(minimum=)' it is.
# `oneormore-zero-raises' is the ticket's "asserted to fail at zero", stated by
# the exception the whole grammar then produces. `repeated-min2-*' shows the
# same threshold at a minimum the specialisations do not expose, and
# `repeateddict-min2-one-key-raises' shows that the dict-valued sibling counts
# DISTINCT KEYS rather than matches -- two matches under one key are one.
check_eq $P/oneormore-one                      '[1]' "$(pv oneormore_one)"
check_eq $P/oneormore-several                  '[1 2 3]' "$(pv oneormore_several)"
check_eq $P/oneormore-zero-raises              ParseError "$(pv oneormore_zero_raises)"
check_eq $P/oneormore-minimum                  1 "$(pv oneormore_minimum)"
check_eq $P/repeated-min2-at-one-raises        ParseError "$(pv repeated_min2_at_one_raises)"
check_eq $P/repeated-min2-at-two               '[1 2]' "$(pv repeated_min2_at_two)"
check_eq $P/oneormoredict-one                  '{a:[[a 1]]}' "$(pv oneormoredict_one)"
check_eq $P/oneormoredict-zero-raises          ParseError "$(pv oneormoredict_zero_raises)"
check_eq $P/repeateddict-min2-one-key-raises   ParseError "$(pv repeateddict_min2_one_key_raises)"

# ---- choice: the first matching alternative wins, and which class you got
# decides whether "first" means "in order" or "by token kind". See the header.
check_eq $P/choice-first-alternative           '[1 x]' "$(pv choice_first_alternative)"
check_eq $P/choice-falls-through               1 "$(pv choice_falls_through)"
check_eq $P/choice-order-matters               ParseError "$(pv choice_order_matters)"
check_eq $P/choice-picks-dict                  ChoiceDict "$(pv choice_picks_dict)"
check_eq $P/choice-picks-choice                Choice "$(pv choice_picks_choice)"
check_eq $P/choicedict-duplicate-raises        Error "$(pv choicedict_duplicate_raises)"
check_eq $P/choicedict-unsupported-raises      Error "$(pv choicedict_unsupported_raises)"
check_eq $P/choicedict-map-keys                NUMBER,WORD "$(pv choicedict_map_keys)"

# ---- Forward: a recursive grammar at three depths, and the unset case.
check_eq $P/forward-depth0                     1 "$(pv forward_depth0)"
check_eq $P/forward-depth1                     '[! 1]' "$(pv forward_depth1)"
check_eq $P/forward-depth3                     '[! [! [! 1]]]' "$(pv forward_depth3)"
check_eq $P/forward-unset-raises               AttributeError "$(pv forward_unset_raises)"

# ---- AnyUntil: the terminator is neither consumed nor included, and the
# documented boundary is `IndexError' rather than a syntax error. See header.
check_eq $P/anyuntil-several                   '[[a 1 b] !]' "$(pv anyuntil_several)"
check_eq $P/anyuntil-empty                     '[[] !]' "$(pv anyuntil_empty)"
check_eq $P/anyuntil-terminator-excluded       yes "$(pv anyuntil_terminator_excluded)"
check_eq $P/anyuntil-no-terminator-raises      IndexError "$(pv anyuntil_no_terminator_raises)"

# ---- The remaining combinators the two grammars above lean on.
# `and-consumes-nothing' and `not-empty-list' are the lookahead pair: both
# become an empty list in the tree and neither advances the token stream.
# `delimitedlist-trailing-delim-stops' shows that a trailing delimiter is left
# for the enclosing pattern rather than swallowed.
check_eq $P/any-token                          '[1 x]' "$(pv any_token)"
check_eq $P/any-at-eof-raises                  ParseError "$(pv any_at_eof_raises)"
check_eq $P/tag-shape                          '(num 1)' "$(pv tag_shape)"
check_eq $P/tag-is-tuple                       yes "$(pv tag_is_tuple)"
check_eq $P/and-empty-list                     '[[] 1]' "$(pv and_empty_list)"
check_eq $P/and-consumes-nothing               yes "$(pv and_consumes_nothing)"
check_eq $P/not-empty-list                     '[[] 1]' "$(pv not_empty_list)"
check_eq $P/not-rejects                        ParseError "$(pv not_rejects)"
check_eq $P/nomatch-raises                     ParseError "$(pv nomatch_raises)"
check_eq $P/delimitedlist-several              '[1 2 3]' "$(pv delimitedlist_several)"
check_eq $P/delimitedlist-one                  '[1]' "$(pv delimitedlist_one)"
check_eq $P/delimitedlist-zero-raises          ParseError "$(pv delimitedlist_zero_raises)"
check_eq $P/delimitedlist-trailing-delim-stops '[[1 2] ,]' "$(pv delimitedlist_trailing_delim_stops)"
check_eq $P/delimitedlist-custom-delim         '[1 2]' "$(pv delimitedlist_custom_delim)"
check_eq $P/sequence-mismatch-raises           ParseError "$(pv sequence_mismatch_raises)"
check_eq $P/zeroormoredict-key-fn              '{1:[[a 1]] 2:[[b 2]]}' "$(pv zeroormoredict_key_fn)"

# ================================= ParseError CARRIES A POSITION ==========
# The fault is deliberately on the SECOND line, so the line (2), the column (5)
# and the offset (11) are each distinguishable from one another and from 1.
#
# `tokenize-fault-*' is the same exception from the other cause: a tokenizer
# failure and a grammar failure both surface as `ParseError' out of
# `Parser.parse', and only the offset says which. The two underlying exceptions
# are asserted directly beside them.
#
# `line' and `column' are the module functions the position is built from, and
# `column-at-zero' is 1 rather than 0 -- columns are one-based.
check_eq $P/parse-error-raised                 yes "$(pv parse_error_raised)"
check_eq $P/parse-error-offset                 11 "$(pv parse_error_offset)"
check_eq $P/parse-error-line                   2 "$(pv parse_error_line)"
check_eq $P/parse-error-column                 5 "$(pv parse_error_column)"
check_eq $P/parse-error-text-is-input          yes "$(pv parse_error_text_is_input)"
check_eq $P/parse-error-message                'Invalid syntax at line 2, column 5: "b = >>!<<;"' "$(pv parse_error_message)"
check_eq $P/parse-error-is-error               yes "$(pv parse_error_is_error)"
check_eq $P/tokenize-fault-type                ParseError "$(pv tokenize_fault_type)"
check_eq $P/tokenize-fault-offset              7 "$(pv tokenize_fault_offset)"
check_eq $P/tokenize-fault-column              8 "$(pv tokenize_fault_column)"
check_eq $P/tokenizeerror-offset               2 "$(pv tokenizeerror_offset)"
check_eq $P/grammarerror-offset                7 "$(pv grammarerror_offset)"
check_eq $P/grammarerror-message               'Invalid syntax at offset 7.' "$(pv grammarerror_message)"
check_eq $P/tokenizeerror-is-error             yes "$(pv tokenizeerror_is_error)"
check_eq $P/line-first                         1 "$(pv line_first)"
check_eq $P/line-second                        2 "$(pv line_second)"
check_eq $P/column-first                       2 "$(pv column_first)"
check_eq $P/column-second                      2 "$(pv column_second)"
check_eq $P/column-at-zero                     1 "$(pv column_at_zero)"

# ==================================== markup_line RENDERS THE CARET =======
# `markup-doc' is the docstring's own example, verbatim. The five beside it are
# the rendering's edges: an offset inside the first line, an offset at the end
# of a text with no trailing newline, an offset at a line START (the marker
# lands before the first character and nothing precedes it), a caller-chosen
# marker, and the last line.
check_eq $P/markup-doc                         '1>>!<<234' "$(pv markup_doc)"
check_eq $P/markup-first-line                  'a>>!<<bc' "$(pv markup_first_line)"
check_eq $P/markup-no-trailing-newline         'abc>>!<<' "$(pv markup_no_trailing_newline)"
check_eq $P/markup-line-start                  '>>!<<cd' "$(pv markup_line_start)"
check_eq $P/markup-custom-marker               'a^bc' "$(pv markup_custom_marker)"
check_eq $P/markup-last-line                   'c>>!<<d' "$(pv markup_last_line)"

# ========================== THE TOKENIZER SKIPS, NAMES, AND REFUSES =======
# What the spec declares skippable is skipped: `tokenize-skips-whitespace'
# compares the kinds of a spaced and an unspaced spelling.
#
# `tokenize-friendly-name' is the three-tuple form of a token spec -- the
# grammar sees `=' rather than `EQ'. `keyword-promoted' is the other rewrite:
# a WORD whose value is in `keywords()' becomes a kind of its own.
#
# An unrecognised character is REPORTED, not dropped: `TokenizeError' at the
# offset of the character itself, with the caret message naming its column.
check_eq $P/tokenize-kinds                     '__SOF__,WORD,=,NUMBER,;' "$(pv tokenize_kinds)"
check_eq $P/tokenize-values                    '__SOF__,a,=,1,;' "$(pv tokenize_values)"
check_eq $P/tokenize-offsets                   0,0,2,4,5 "$(pv tokenize_offsets)"
check_eq $P/tokenize-skips-whitespace          yes "$(pv tokenize_skips_whitespace)"
check_eq $P/tokenize-sof-first                 __SOF__ "$(pv tokenize_sof_first)"
check_eq $P/tokenize-friendly-name             = "$(pv tokenize_friendly_name)"
check_eq $P/tokenize-unrecognised-raises       TokenizeError "$(pv tokenize_unrecognised_raises)"
check_eq $P/tokenize-unrecognised-offset       4 "$(pv tokenize_unrecognised_offset)"
check_eq $P/tokenize-unrecognised-message      'Invalid syntax at line 1, column 5: "a = >>!<<@;"' "$(pv tokenize_unrecognised_message)"
check_eq $P/keyword-promoted                   if,WORD "$(pv keyword_promoted)"
check_eq $P/tokenize-init-sof                  __SOF__ "$(pv tokenize_init_sof)"
check_eq $P/tokenize-init-regex                '(?P<NUMBER>\d+)|(?P<WORD>\w+)' "$(pv tokenize_init_regex)"

# ==================================== replace_blocks =====================
# Blocks are replaced by spaces of the same length, so offsets into the result
# still index the original text. `replace-blocks-multiline' keeps the newline
# for the same reason -- line numbers survive the substitution.
check_eq $P/replace-blocks-simple              'a{  }d' "$(pv replace_blocks_simple)"
check_eq $P/replace-blocks-nested              'a{     }e' "$(pv replace_blocks_nested)"
check_eq $P/replace-blocks-multiline           'a{ \n  }e' "$(pv replace_blocks_multiline)"
check_eq $P/replace-blocks-custom              'a[  ]d' "$(pv replace_blocks_custom)"

# ==================================== Grammar ON A HAND-BUILT STREAM ======
# `Grammar' is drivable without a `Parser', which is what a caller that
# tokenizes for itself uses. Trailing tokens are an error, not a prefix match.
check_eq $P/grammar-on-tokens                  1 "$(pv grammar_on_tokens)"
check_eq $P/grammar-token-tree                 'Token(NUMBER 1 0)' "$(pv grammar_token_tree)"
check_eq $P/grammar-trailing-tokens-raise      GrammarError "$(pv grammar_trailing_tokens_raise)"

rm -rf "$W"