. "$BST_LIB" # BST-TIMEOUT: 1800 # FOUR python3 starts, and they are the whole cost. Two are # # bare `import' checks (one of them measuring STDERR), one is # # the probe, one runs upstream's own 95-test suite. Every # # other assertion reads the filesystem or dpkg with shell # # builtins, or greps a transcript already on disk. 1800 # # rather than the 1200 the one-start python3 ports declare # # because this suite pays the start four times and the # # upstream run is a second body of work after it. # # NOT MEASURED ON TARGET YET: this port's first on-target run # # is the batch verify, and the measured elapsed replaces this # # note in the test plan rather than in this comment, which # # would cost an image build to keep the shipped suite # # byte-identical. # # python3-simpleeval 0.9.13 (meta-openembedded # meta-python/recipes-devtools/python/python3-simpleeval_0.9.13.bb, `inherit # pypi python_setuptools_build_meta ptest', PyPI sdist) on the wasm32-hwjs glibc # image -- A SINGLE-EXPRESSION EVALUATOR THAT IS SAFE BY WHITELIST. It parses # with the stdlib `ast' and then walks the tree ITSELF, dispatching on node type # through a dict it owns; a node type absent from that dict is a refusal, not a # fallback to the interpreter. So the security property is not a blacklist of # dangerous spellings -- it is that the evaluator has no path to anything it did # not name. # # ============================================================================ # THE REFUSALS ARE THE PACKAGE. THEY GET ONE CASE EACH, WITH THE CLASS. # ============================================================================ # # Twelve of the cases below assert that an expression RAISES, and each asserts # WHICH exception. That is deliberate and it is not pedantry: a caller sandboxing # untrusted input distinguishes `FeatureNotAvailable' (you asked for something # this evaluator does not do) from `NumberTooHigh' (you asked for something it # refuses to spend time on) from `NameNotDefined' (you asked for something that # is not there), and a build in which all three collapsed into one class would # still "raise on everything dangerous" while breaking every caller that catches # them apart. The probe therefore emits two keys per refusal -- did it raise, and # with what type -- and both are asserted. # # ============================================================================ # ATTRIBUTE ACCESS IS *NOT* REFUSED WHOLESALE. THE TICKET SAYS IT IS. MEASURED. # ============================================================================ # # BRINTOS-2002's own description says "attribute access raises # FeatureNotAvailable". Only SOME attribute access does, and the distinction is # the whole of _eval_attribute: the refusal fires for a name starting with one of # DISALLOW_PREFIXES ("_", "func_") or equal to one of DISALLOW_METHODS ("format", # "format_map", "mro"). An ORDINARY PUBLIC ATTRIBUTE RESOLVES AND CALLS -- # `s.upper()' with names={"s": "ab"} returns "AB", measured, and chained calls # work too. Both halves are asserted here (`attr-public-method', # `attr-public-chained' beside the six refusals), because a reader who took the # ticket at its word would conclude this image's build was broken the first time # an expression reached a method. # # The blocked names are not arbitrary either -- `__globals__' and `func_globals' # are how a function object hands out the interpreter's namespace, and `format' # is a string method that reads attributes off its arguments. Blocking three # METHOD NAMES rather than "methods" is the entire mitigation, so # `disallow-prefixes' and `disallow-methods' pin those tables exactly: a release # that dropped an entry would pass every behavioural case here and quietly reopen # the hole. # # ============================================================================ # THERE ARE THREE EVALUATORS IN THIS PACKAGE AND THEY ADMIT DIFFERENT GRAMMARS. # ============================================================================ # # `simple_eval(expr)' builds a fresh `SimpleEval' per call and throws it away; # `SimpleEval()' is the same evaluator kept, so names and functions added to the # instance persist across `eval' calls; `EvalWithCompoundTypes' is a SUBCLASS # that adds the list/tuple/dict/set literal nodes and comprehensions. # # THE CONSEQUENCE IS THE MOST SURPRISING FACT ABOUT THE DEFAULT CONFIGURATION, # and it is a second correction to this ticket, which asks for "string and list # literals with indexing and slicing" as though both worked in one evaluator. # `[1, 2, 3]' in the DEFAULT evaluator raises FeatureNotAvailable -- ast.List is # not in SimpleEval.nodes at all -- while `'hello'[1:3]' works, because the # SUBSCRIPT node is in the default evaluator and only the LITERAL is not. A list # reached through names= is therefore indexable and sliceable in the default # evaluator (`names-list-index', `names-list-slice'), and a list WRITTEN OUT is # not. Both are asserted from both ends: `list-literal-default-raises' with its # class, and `list-literal-compound' returning the list. # # Likewise the comprehension. The ticket says "a comprehension raises", and in # the default evaluator it does -- FeatureNotAvailable, for the same reason. The # MAX_COMPREHENSION_LENGTH limit the ticket also names belongs to the SUBCLASS, # where comprehensions exist at all; there it raises IterableTooLong past 10000 # elements. Two different refusals for two different evaluators, one case each. # # ============================================================================ # THE LIMITS ARE MUTABLE MODULE GLOBALS, READ AT CALL TIME. # ============================================================================ # # MAX_POWER, MAX_STRING_LENGTH, MAX_COMPREHENSION_LENGTH, MAX_SHIFT and # MAX_SHIFT_BASE are module attributes the guard functions look up when they run, # not values captured at construction -- which is why upstream's own suite can # lower MAX_POWER to 100, watch `101**2' start raising, and put it back. They are # pinned here as values because a caller's sandbox budget is expressed in them. # # `power-allowed-*' is worth reading before it is copied. It asserts that a big # but PERMITTED exponentiation still computes, and it does so IN BITS rather than # digits, because CPython 3.11 added an int->str conversion limit (4300 digits by # default) -- so the obvious `len(str(9 ** 9 ** 5))' raises ValueError about a # value simpleeval computed correctly. That limit is the INTERPRETER's, not this # package's, and `int-str-digit-limit' states it as its own fact so a red here # cannot be misread as a simpleeval regression. # # ============================================================================ # UPSTREAM'S OWN 95 TESTS RUN HERE, UNMODIFIED, WITHOUT pytest. # ============================================================================ # # This is the part of BRINTOS-2002 that other pure-Python ports in this Epic # could NOT do, and the reason is in the payload rather than in this arm. # # The recipe DOES `inherit ptest', and its run-ptest is `pytest --automake' -- # but this distro sets DISTRO_FEATURES without "ptest" # (conf/distro/brintos.conf), so ptest.bbclass never adds ${PN}-ptest to PACKAGES # and NO -ptest deb is produced. MEASURED off the recipe-scope build: # python3-simpleeval, -dev and -dbg, and no -ptest. `no-ptest-package' asserts # that from the target's own dpkg database. Forcing one would put python3-pytest # and python3-unittest-automake-output on the image, and pytest's RDEPENDS name # seven further packages nobody ported -- refused before, for python3-wcwidth and # python3-ansicolors. # # WHAT IS DIFFERENT HERE is that upstream's test_simpleeval.py needs none of # that: it imports `ast, gc, operator, os, platform, sys, unittest, warnings' and # nothing else, and its cases are unittest.TestCase methods rather than bare # `test_*' functions. pytest was only ever the RUNNER. So the file is shipped # byte-identical as data/python3-simpleeval-upstream-tests.py (sha256 # 68b1637a9385d33a3bd193a766a30f940a3e4eaa1ecf4c90c62fffd85b1b1314, the sdist's # own), and data/python3-simpleeval-upstream-run.py supplies the reporter so each # upstream test lands as ITS OWN case -- `python3-simpleeval/upstream/.'. # Ninety-five cases with names, not one "OK". # # The reporter exists because `-m unittest -v' cannot be grepped safely: a test # WITH a docstring prints the id, the docstring, then `... ok' on the docstring's # line, so a grep for `... ok' loses it. Measured: 95 tests ran, 94 lines matched. # # Two upstream tests need the environment arranged rather than the code changed, # and this suite arranges it: test_load_file WRITES testfile.txt into the current # directory, and the deployed rootfs is READ-ONLY at browser runtime -- so the # runner is handed the suite's own $TMPDIR workdir to chdir into. TestReferenceCleanup # drives gc.set_debug(gc.DEBUG_SAVEALL) and asserts no cycle survives constructing # a SimpleEval; it is a real assertion about this interpreter's gc and is left to # run. # # ============================================================================ # WHY IT PORTS AT ALL: PURE PYTHON, AND ONE DEPENDENCY THAT IS ALREADY HERE. # ============================================================================ # # This image's python3 is ONE STATIC BINARY with an EMPTY lib-dynload and no # dynamic loader (BRINTOS-1476; 99-python3 asserts both directly). A module # carrying a compiled extension could not be imported here -- not "would be # slow", could not be imported. simpleeval is ONE FILE, simpleeval.py, declared # as py-modules with no ext_modules, and it imports `ast', `operator', `sys', # `warnings' and `random' -- stdlib, all of it. The packaged file is # BYTE-IDENTICAL to the sdist's, so there is no patch of ours to go looking for. # # The recipe's RDEPENDS names python3-math and nothing else; the packaging # derived `Depends: python3-core, python3-math'. python3-math is where poky's # manifest puts random.py, which is what `from random import random' reaches -- # and it is a SUB-PACKAGE OF THE PORTED python3 already on this image, so this # port waits on no other package. `pkg-depends-exact' asserts the whole line off # dpkg rather than the presence of one name: a line that lost python3-math would # still install today and break the day the sub-package left. # # It is a MODULE, NOT A PACKAGE -- one simpleeval.py in site-packages, the same # shape as python3-parse and the opposite of python3-wcwidth's directory. A # release that moved to a package would still import and still pass every # behavioural case, so it is asserted directly. P=python3-simpleeval bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MOD=$SITE/simpleeval.py DI=$SITE/simpleeval-0.9.13.dist-info PKGLIST=/var/lib/dpkg/info/python3-simpleeval.list PROBE=$BST_DATA/python3-simpleeval-probe.py UPRUN=$BST_DATA/python3-simpleeval-upstream-run.py UPSRC=$BST_DATA/python3-simpleeval-upstream-tests.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-simpleeval.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ------------------------------------------------------------- packaging ----- check $P/site-packages-dir test -d "$SITE" check $P/module-file test -f "$MOD" check_eq $P/no-package-dir "absent" \ "$(if [ -d "$SITE/simpleeval" ]; then echo present; else echo absent; fi)" check $P/pycache test -f "$SITE/__pycache__/simpleeval.cpython-312.pyc" cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" check_eq $P/pkg-no-so "" "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" check_eq $P/no-ptest-payload "absent" \ "$(if [ -d /usr/lib/python3-simpleeval/ptest ]; then echo present; else echo absent; fi)" check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 0.9.13" cat "$DI/METADATA" check_out $P/dist-info-name "Name: simpleeval" cat "$DI/METADATA" check_out $P/dist-info-top-level "simpleeval" cat "$DI/top_level.txt" check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" check $P/dist-info-licence test -f "$DI/LICENCE" sed -n '/^Package: python3-simpleeval$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" check_out $P/pkg-version "Version: 0.9.13-r0" cat "$W/st" grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_eq $P/pkg-depends-exact "Depends: python3-core, python3-math" \ "$(cat "$W/dep" 2>/dev/null)" check_eq $P/no-ptest-package "absent" \ "$(if grep -q '^Package: python3-simpleeval-ptest$' /var/lib/dpkg/status 2>/dev/null; then echo present; else echo absent; fi)" # ---------------------------------------------------------------- import ----- check $P/import python3 -c 'import simpleeval' check $P/import-api python3 -c 'from simpleeval import SimpleEval, EvalWithCompoundTypes, simple_eval, InvalidExpression' # The import must also be QUIET. simpleeval builds its node table out of # ast.Num / ast.Str / ast.Index, all of which CPython 3.12 deprecates, and a # DeprecationWarning reaching stderr would land in every caller's output. The # default warning filter suppresses it outside __main__; this asserts the # outcome rather than the filter. check_eq $P/import-quiet "" \ "$(python3 -c 'import simpleeval' 2>&1 >/dev/null)" # ----------------------------------------------------------------- probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present bst_probe $P/probe-run "$W/probe" 700 python3 "$PROBE" OUT="$W/probe" pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi check_eq $P/probe-no-raised "" "$(pv probe_failed_keys)" check_eq $P/probe-done "yes" "$(pv probe_done)" # ------------------------------------------------------------- identity ------ check_eq $P/module-answered "$MOD" "$(pv module_file)" check_eq $P/py-version "3.12" "$(pv py_version)" check_eq $P/module-not-package "yes" "$(pv is_module_not_package)" check_eq $P/python3-flag "True" "$(pv python3_flag)" check_eq $P/python35-flag "True" "$(pv python35_flag)" check_eq $P/api-surface \ "ATTR_INDEX_FALLBACK,AssignmentAttempted,AttributeDoesNotExist,DEFAULT_FUNCTIONS,DEFAULT_NAMES,DEFAULT_OPERATORS,DISALLOW_FUNCTIONS,DISALLOW_METHODS,DISALLOW_PREFIXES,EvalWithCompoundTypes,FeatureNotAvailable,FunctionNotDefined,InvalidExpression,IterableTooLong,MAX_COMPREHENSION_LENGTH,MAX_POWER,MAX_SHIFT,MAX_SHIFT_BASE,MAX_STRING_LENGTH,MultipleExpressions,NameNotDefined,NumberTooHigh,OperatorNotDefined,PYTHON3,PYTHON35,SimpleEval,ast,op,random,random_int,safe_add,safe_lshift,safe_mult,safe_power,safe_rshift,simple_eval,sys,warnings" \ "$(pv api_surface)" # ------------------------------------------------------------- the limits ---- check_eq $P/max-string-length "100000" "$(pv max_string_length)" check_eq $P/max-comprehension-length "10000" "$(pv max_comprehension_length)" check_eq $P/max-power "4000000" "$(pv max_power)" check_eq $P/max-shift "10000" "$(pv max_shift)" check_eq $P/max-shift-base-is-float-max "True" "$(pv max_shift_base_is_float_max)" check_eq $P/disallow-prefixes "_,func_" "$(pv disallow_prefixes)" check_eq $P/disallow-methods "format,format_map,mro" "$(pv disallow_methods)" check_eq $P/disallow-functions-core \ "compile,eval,exec,getattr,isinstance,open,repr,setattr,type" \ "$(pv disallow_functions_core)" # `help' joins DISALLOW_FUNCTIONS only when the interpreter has it, which is a # property of whether `site' ran. Reported as its own case rather than folded # into the list above, whose nine entries must not move. check_eq $P/disallow-has-help "yes" "$(pv disallow_has_help)" check_eq $P/default-functions "float,int,rand,randint,str" "$(pv default_functions)" check_eq $P/default-names "False,None,True" "$(pv default_names)" check_eq $P/default-operator-count "26" "$(pv default_operator_count)" check_eq $P/attr-index-fallback "True" "$(pv attr_index_fallback)" check_eq $P/int-str-digit-limit "4300" "$(pv int_str_digit_limit)" # ------------------------------------------------- exception hierarchy ------- check_eq $P/exc-base-is-exception "Exception" "$(pv exc_invalidexpression_base)" for _e in functionnotdefined namenotdefined attributedoesnotexist \ operatornotdefined featurenotavailable numbertoohigh iterabletoolong; do check_eq $P/exc-$_e-is-invalid "True" "$(pv exc_${_e}_is_invalid)" done check_eq $P/exc-assignmentattempted-is-warning "True" "$(pv exc_assignmentattempted_is_warning)" check_eq $P/exc-multipleexpressions-is-warning "True" "$(pv exc_multipleexpressions_is_warning)" # ---------------------------------------------------------- arithmetic ------- check_eq $P/arith-precedence "0" "$(pv arith_precedence)" check_eq $P/arith-mul-before-add "14" "$(pv arith_mul_before_add)" check_eq $P/arith-pow-right-assoc "512" "$(pv arith_pow_right_assoc)" check_eq $P/arith-truediv "3.5" "$(pv arith_truediv)" check_eq $P/arith-floordiv "3" "$(pv arith_floordiv)" check_eq $P/arith-mod "1" "$(pv arith_mod)" check_eq $P/arith-unary-minus "-3" "$(pv arith_unary_minus)" check_eq $P/arith-unary-plus "5" "$(pv arith_unary_plus)" check_eq $P/arith-invert "-3" "$(pv arith_invert)" check_eq $P/arith-float "3.0" "$(pv arith_float)" check_eq $P/arith-bitand "8" "$(pv arith_bitand)" check_eq $P/arith-bitor "14" "$(pv arith_bitor)" check_eq $P/arith-bitxor "6" "$(pv arith_bitxor)" check_eq $P/arith-lshift "1024" "$(pv arith_lshift)" check_eq $P/arith-rshift "128" "$(pv arith_rshift)" # --------------------------------------------------------- comparisons ------- check_eq $P/cmp-lt "True" "$(pv cmp_lt)" check_eq $P/cmp-eq "True" "$(pv cmp_eq)" check_eq $P/cmp-ne "True" "$(pv cmp_ne)" check_eq $P/cmp-ge "True" "$(pv cmp_ge)" check_eq $P/cmp-chained-true "True" "$(pv cmp_chained_true)" check_eq $P/cmp-chained-false "False" "$(pv cmp_chained_false)" check_eq $P/cmp-in-string "True" "$(pv cmp_in_string)" check_eq $P/cmp-notin-string "True" "$(pv cmp_notin_string)" check_eq $P/cmp-is-none "True" "$(pv cmp_is_none)" check_eq $P/cmp-isnot "True" "$(pv cmp_isnot)" # `and'/`or' RETURN AN OPERAND, NOT A BOOL, and `or' returns the last value even # when it is falsy -- upstream fixed that deliberately, so a release that # regressed to False reds bool-or-last while every other boolean case passes. check_eq $P/bool-and-true "True" "$(pv bool_and_true)" check_eq $P/bool-and-shortcircuit "False" "$(pv bool_and_shortcircuit)" check_eq $P/bool-or-true "True" "$(pv bool_or_true)" check_eq $P/bool-or-last "''" "$(pv bool_or_last)" check_eq $P/bool-and-value "2" "$(pv bool_and_value)" check_eq $P/bool-not "True" "$(pv bool_not)" check_eq $P/ifexp-true "y" "$(pv ifexp_true)" check_eq $P/ifexp-false "n" "$(pv ifexp_false)" # ------------------------------------------------ literals and subscripts ---- check_eq $P/str-literal "hello" "$(pv str_literal)" check_eq $P/str-concat "ab" "$(pv str_concat)" check_eq $P/str-index "e" "$(pv str_index)" check_eq $P/str-index-negative "o" "$(pv str_index_negative)" check_eq $P/str-slice "el" "$(pv str_slice)" check_eq $P/str-slice-step "hlo" "$(pv str_slice_step)" check_eq $P/str-slice-open "llo" "$(pv str_slice_open)" check_eq $P/fstring "2 apples" "$(pv fstring)" check_eq $P/fstring-format-spec "007" "$(pv fstring_format_spec)" check_eq $P/list-literal-default-raises "yes" "$(pv list_literal_default_raised)" check_eq $P/list-literal-default-class "FeatureNotAvailable" "$(pv list_literal_default_type)" check_eq $P/list-literal-compound "[1, 2, 3]" "$(pv list_literal_compound)" check_eq $P/list-index-compound "1" "$(pv list_index_compound)" check_eq $P/list-slice-compound "[2, 3]" "$(pv list_slice_compound)" check_eq $P/tuple-compound "(1, 2)" "$(pv tuple_compound)" check_eq $P/dict-compound "1" "$(pv dict_compound)" check_eq $P/set-compound "[1, 2]" "$(pv set_compound)" check_eq $P/compound-extra-functions "dict,list,set,tuple" "$(pv compound_extra_functions)" check_eq $P/names-list-index "20" "$(pv names_list_index)" check_eq $P/names-list-slice "[10, 20]" "$(pv names_list_slice)" # ------------------------------------------------------ default functions ---- check_eq $P/fn-int "43" "$(pv fn_int)" check_eq $P/fn-float "3.0" "$(pv fn_float)" check_eq $P/fn-str "42!" "$(pv fn_str)" check_eq $P/fn-randint-type "int" "$(pv fn_randint_type)" check_eq $P/fn-randint-range "True" "$(pv fn_randint_in_range)" check_eq $P/fn-rand-type "float" "$(pv fn_rand_type)" check_eq $P/fn-rand-range "True" "$(pv fn_rand_in_range)" # ----------------------------------------------------- names and functions --- check_eq $P/names-supplied "42" "$(pv names_supplied)" check_eq $P/names-string "hello world" "$(pv names_string)" check_eq $P/names-dict-attr-fallback "7" "$(pv names_dict_attr_fallback)" check_eq $P/functions-supplied "42" "$(pv functions_supplied)" check_eq $P/functions-kwargs "42" "$(pv functions_kwargs)" check_eq $P/names-callable "42" "$(pv names_callable)" check_eq $P/instance-carries-state "15" "$(pv instance_carries_state)" check_eq $P/names-not-mutated "['a']" "$(pv names_not_mutated_by_eval)" # An unknown NAME falls back to the functions table before giving up, so a bare # `int' is the function object. Not obvious, and a caller can rely on it. check_eq $P/bare-function-name "int" "$(pv bare_function_name)" # ---------------------------------------------------------- the refusals ----- check_eq $P/unknown-name-raises "yes" "$(pv unknown_name_raised)" check_eq $P/unknown-name-class "NameNotDefined" "$(pv unknown_name_type)" check_eq $P/unknown-function-raises "yes" "$(pv unknown_function_raised)" check_eq $P/unknown-function-class "FunctionNotDefined" "$(pv unknown_function_type)" check_eq $P/lambda-raises "yes" "$(pv lambda_literal_raised)" check_eq $P/lambda-class "FeatureNotAvailable" "$(pv lambda_literal_type)" check_eq $P/attr-dunder-raises "yes" "$(pv attr_dunder_raised)" check_eq $P/attr-dunder-class "FeatureNotAvailable" "$(pv attr_dunder_type)" check_eq $P/attr-func-prefix-raises "yes" "$(pv attr_func_prefix_raised)" check_eq $P/attr-func-prefix-class "FeatureNotAvailable" "$(pv attr_func_prefix_type)" check_eq $P/attr-underscore-raises "yes" "$(pv attr_underscore_raised)" check_eq $P/attr-underscore-class "FeatureNotAvailable" "$(pv attr_underscore_type)" check_eq $P/attr-format-raises "yes" "$(pv attr_format_raised)" check_eq $P/attr-format-class "FeatureNotAvailable" "$(pv attr_format_type)" check_eq $P/attr-format-map-raises "yes" "$(pv attr_format_map_raised)" check_eq $P/attr-format-map-class "FeatureNotAvailable" "$(pv attr_format_map_type)" check_eq $P/attr-mro-raises "yes" "$(pv attr_mro_raised)" check_eq $P/attr-mro-class "FeatureNotAvailable" "$(pv attr_mro_type)" # An attribute that is merely ABSENT is a DIFFERENT refusal from one that is # forbidden, and a caller distinguishing "you may not" from "there is none" # depends on the two classes staying apart. check_eq $P/attr-absent-raises "yes" "$(pv attr_absent_raised)" check_eq $P/attr-absent-class "AttributeDoesNotExist" "$(pv attr_absent_type)" check_eq $P/import-raises "yes" "$(pv import_stmt_raised)" check_eq $P/import-class "FeatureNotAvailable" "$(pv import_stmt_type)" check_eq $P/comprehension-default-raises "yes" "$(pv comprehension_default_raised)" check_eq $P/comprehension-default-class "FeatureNotAvailable" "$(pv comprehension_default_type)" check_eq $P/power-raises "yes" "$(pv power_too_high_raised)" check_eq $P/power-class "NumberTooHigh" "$(pv power_too_high_type)" check_eq $P/string-mult-raises "yes" "$(pv string_mult_too_long_raised)" check_eq $P/string-mult-class "IterableTooLong" "$(pv string_mult_too_long_type)" check_eq $P/string-add-raises "yes" "$(pv string_add_too_long_raised)" check_eq $P/string-add-class "IterableTooLong" "$(pv string_add_too_long_type)" check_eq $P/lshift-raises "yes" "$(pv lshift_too_high_raised)" check_eq $P/lshift-class "NumberTooHigh" "$(pv lshift_too_high_type)" check_eq $P/rshift-raises "yes" "$(pv rshift_too_high_raised)" check_eq $P/rshift-class "NumberTooHigh" "$(pv rshift_too_high_type)" check_eq $P/empty-raises "yes" "$(pv empty_expression_raised)" check_eq $P/empty-class "InvalidExpression" "$(pv empty_expression_type)" check_eq $P/no-operators-raises "yes" "$(pv no_operators_raised)" check_eq $P/no-operators-class "OperatorNotDefined" "$(pv no_operators_type)" check_eq $P/no-functions-raises "yes" "$(pv no_functions_raised)" check_eq $P/no-functions-class "FunctionNotDefined" "$(pv no_functions_type)" # Handing the evaluator a forbidden builtin is refused AT CONSTRUCTION, not at # eval -- so a caller cannot build a dangerous evaluator and discover it later. check_eq $P/forbidden-function-raises "yes" "$(pv forbidden_function_at_construction_raised)" check_eq $P/forbidden-function-class "FeatureNotAvailable" "$(pv forbidden_function_at_construction_type)" # ATTRIBUTE ACCESS IS NOT REFUSED WHOLESALE -- see the header. These two are the # counterweight to the eight refusals above. check_eq $P/attr-public-method "AB" "$(pv attr_public_method)" check_eq $P/attr-public-chained "AB" "$(pv attr_public_chained)" check_eq $P/comprehension-compound "[2, 4, 6]" "$(pv comprehension_compound)" check_eq $P/comprehension-too-long-raises "yes" "$(pv comprehension_too_long_raised)" check_eq $P/comprehension-too-long-class "IterableTooLong" "$(pv comprehension_too_long_type)" check_eq $P/power-allowed-exact "True" "$(pv power_allowed_exact)" check_eq $P/power-allowed-bits "187181" "$(pv power_allowed_bits)" # An assignment is NOT refused: it WARNS and evaluates the right-hand side. A # caller who believed `a = 1' was blocked is wrong in a way no exception tells # them about, so the warning class and the returned value are asserted together. check_eq $P/assignment-warns "3:AssignmentAttempted" "$(pv assignment_warns)" check_eq $P/multiple-expressions-warn "2:MultipleExpressions" "$(pv multiple_expressions_warn)" # ------------------------------------------------------- upstream's suite ---- if [ ! -f "$UPRUN" ] || [ ! -f "$UPSRC" ]; then bst_notok $P/upstream-present "$UPRUN or $UPSRC did not ship" rm -rf "$W" exit 0 fi bst_ok $P/upstream-present bst_probe $P/upstream-run "$W/upstream" 700 python3 "$UPRUN" "$UPSRC" "$W" UP="$W/upstream" if [ -s "$UP" ]; then bst_ok $P/upstream-output else bst_notok $P/upstream-output "the upstream runner emitted nothing; every upstream case below is unscorable" fi upv() { grep "^$1 " "$UP" 2>/dev/null | head -n 1 | cut -d' ' -f2-; } # 95 is what upstream 0.9.13 ships and what this port measured off the .deb's # own tree. A release that added or dropped a test reds THIS case and names the # new count, rather than leaving a quietly smaller run looking clean. check_eq $P/upstream-total "95" "$(upv RESULT_TOTAL)" check_eq $P/upstream-loaded "95" "$(upv RESULT_LOADED)" check_eq $P/upstream-failed "0" "$(upv RESULT_FAILED)" check_eq $P/upstream-skipped "0" "$(upv RESULT_SKIPPED)" # Each upstream test as its own case, so a red names the method rather than the # file. The transcript is under $TMPDIR, so this loop's redirect is a tmpfs read. grep '^RESULT ' "$UP" > "$W/upcases" 2>/dev/null || : > "$W/upcases" while IFS= read -r _l; do _verdict=$(printf '%s' "$_l" | cut -d' ' -f2) _name=$(printf '%s' "$_l" | cut -d' ' -f3) _detail=$(printf '%s' "$_l" | cut -s -d' ' -f4-) [ -n "$_name" ] || continue case "$_verdict" in ok) bst_ok "$P/upstream/$_name" ;; skip) bst_skip "$P/upstream/$_name" "${_detail:-skipped upstream}" ;; notok) bst_notok "$P/upstream/$_name" "${_detail:-failed}" ;; esac done < "$W/upcases" rm -rf "$W"