. "$BST_LIB" # BST-TIMEOUT: 1200 # THREE python3 starts and nothing else. # # # # The three are: a bare `import pefile', a second one whose # # STDERR is the assertion (`import-quiet'), and the probe, # # which takes ONE start for 187 measurements. # # # # This suite is CHEAPER than its execve-driven neighbours # # and the number says so: pefile forks nothing, opens no # # device, reaches no peer and touches no terminal. Every # # byte it parses is a byte the probe wrote a moment # # earlier under $TMPDIR. 1200 is 400s per interpreter # # start, which is the per-start budget the neighbouring # # python3 suites use, and no allowance beyond it. # # # # A suite-timeout red is a batch-wide cost, so if the # # interpreter proves slower than that on target the RIGHT # # fix is this number, not fewer cases. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify, and the measured elapsed # # replaces this note in the test plan rather than in this # # comment, which would cost an image build to keep the # # shipped suite byte-identical. # # python3-pefile 2023.2.7 (meta-openembedded meta-python/recipes-devtools/python, # python3-pefile_2023.2.7.bb, `inherit pypi setuptools3 ptest', PyPI sdist) on # the wasm32-hwjs glibc image. # # ============================================================================ # WHAT THE PACKAGE ACTUALLY IS, AND THEREFORE WHAT HAS TO BE TESTED. # ============================================================================ # # `pefile' PARSES AND WRITES Portable Executable images. `PE(data=...)' walks # the DOS stub, the `PE\0\0' signature, the COFF header, the optional header, # the sixteen data directories and the section table; `write()' serialises the # parsed object back out. It is structure work over a byte buffer and nothing # else -- no fork, no execve, no device, no peer, no terminal. # # That is what makes this port close on its suite, and it is also what makes # the suite unusually strong for a port of this size: there is no environment # to disagree about. Every byte parsed here is a byte the probe wrote a moment # earlier, so every expectation below is a LITERAL and a red is a statement # about the parser rather than about the image's furniture. # # ============================================================================ # THE FIXTURE IS BUILT BY THE PROBE, NOT SHIPPED. # ============================================================================ # # `_build()' in the probe assembles a PE32 i386 image out of `struct.pack': # a 64-byte DOS header, `PE\0\0' at 0x80, a 20-byte COFF header, a 224-byte # optional header carrying its sixteen data directories, a two-entry section # table, and the sections' raw bytes. 1536 bytes in total, asserted by sha256 # so that a fixture drift shows up as ONE red named `fixture-sha256' rather # than as forty unexplained ones. # # `.rdata' carries a REAL EXPORT DIRECTORY -- one name, `pefile_fixture_export', # at ordinal base 1, pointing into `.text'. That is what makes `fast_load' # observable at all: with sixteen EMPTY directories the fast and full parses # are indistinguishable, because there is nothing for the full one to find. # # The fixture is also deliberately built so that `get_warnings()' comes back # EMPTY: `.text' is a 512-byte ramp with no NUL in it and `.rdata' is filled # out with printable filler, because pefile warns about a mostly-zero file and # a permanent warning would hide a real one behind it. `warnings-none' is that # claim, and every `*-warned' case below leans on it. # # ============================================================================ # THE ROUND TRIP IS THE STRONGEST CASE IN THE FILE. # ============================================================================ # # Parse, change ONE field, `write()', re-parse, and assert BOTH halves: the # field moved, and the eleven other fields this suite names did not. A # serialiser that quietly normalised a neighbour would pass a "did my edit # take" test and fail this one. `rt-bytes-differing' tightens it further -- # 0x1000 -> 0x1234 differs in exactly two bytes of the whole image, and any # rewriting anywhere else raises that count. # # `rt-identity' is the same claim with no edit at all: re-emitting an untouched # parse reproduces the input BYTE FOR BYTE. That is what says write() is a # serialiser rather than a normaliser. # # ============================================================================ # FOUR THINGS A CALLER WILL GET WRONG. THESE ARE THE POINT OF THE SUITE. # ============================================================================ # # THE CONSTRUCTOR DOES NOT REJECT A MALFORMED FILE. The ticket expected a # PEFormatError from a section table running past the end of # the image, and there is none: with NumberOfSections # claiming 4096 in a file holding two, pefile parses the two, # records the warning `Invalid section 2. Contents are # null-bytes.' and returns a usable object. The same is true # of an optional-header magic that is neither 0x10b nor # 0x20b, and of a data directory whose RVA is past every # section. `sections-past-eof-*', `bad-opt-magic-*' and # `baddir-*' measure all three. # # The line pefile actually draws is READ versus BELIEVE: it # refuses what it cannot READ off the buffer -- no `MZ', a # truncated DOS header, an `e_lfanew' pointing off the end or # at the wrong offset, a body truncated mid-optional-header -- # and tolerates everything it can read but cannot believe. # Six refusal cases sit beside the three tolerances so the # boundary is a measurement rather than a claim. # # A caller who treats the constructor as a validator has no # validation. get_warnings() is the only signal and it is # advisory. # # pe.sections IS SORTED BY VirtualAddress, NOT LEFT IN SECTION-TABLE ORDER. # parse_sections() sorts the list and hangs a # `next_section_virtual_address' off each entry so a caller # can spot overlaps. So `sections[0]' is not the first entry # of the section table -- it is whichever section has the # lowest RVA. `sorted-first-name' shows it directly (a # fixture whose table is written high-then-low comes back # low-then-high), and `rt-sec-index0-name' shows what it # costs: rename `.text' to `.code', move it above `.rdata', # write, re-parse, and the section you were holding index 0 # for is now index 1. The FILE is unchanged -- # `rt-table-first-name' reads the raw table and still finds # `.code' first -- so the reordering is presentational, which # is exactly why it is easy to miss. # # THE IMPORT HASH OF AN IMPORT-FREE BINARY IS THE EMPTY STRING, not the md5 # of nothing. `imphash' asserts it. A caller keying a corpus # on imphash collides every import-free file into one bucket. # # THERE IS NO `__all__', so `from pefile import *' leaks the module's whole # namespace -- `struct' and `os' among it. # `star-leaks-struct' and `star-leaks-os' say so by name, # because the neighbouring python3 suites assert the opposite # and a reader should see that the difference was measured. # # ============================================================================ # THE PORT NEEDS NO PATCH, NO bbappend AND NO SITE ANSWER. # ============================================================================ # # There is no configure step, no compile step and no C anywhere -- `py_modules' # plus one `packages' entry in setup.py -- so there is nothing on this arm for # an override to reach. meta-python's recipe is taken unmodified. # # What the recipe DOES carry is an RDEPENDS on three python3 splits -- # python3-mmap, python3-netclient and python3-stringold -- and all three are # load-bearing rather than decorative: pefile.py imports `mmap' for the # file-backed form, `string' for its printable-character tables, and the # netclient split carries `ordlookup''s consumers. `pkg-depends-exact' asserts # the built .deb's line verbatim, so a lean image that dropped one would say so # here once rather than present as a broken library. # # ============================================================================ # THREE FILES SHIP, NOT ONE. # ============================================================================ # # `py_modules=["pefile", "peutils"]' AND `packages=["ordlookup"]', so # site-packages holds `pefile.py', `peutils.py' and an `ordlookup/' DIRECTORY. # Nothing else would notice if the recipe stopped installing the last two, so # `peutils-*' and `ordlookup-*' are here for that: two ordinal lookups against # the shipped tables, and an empty SignatureDatabase asserted by its three # counts so that `sigdb-match-none' is a measurement rather than a tautology. # # ============================================================================ P=python3-pefile # The canary is first so that a suite which dies before its own first assertion # is distinguishable from one that never started. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MOD=$SITE/pefile.py PEUTILS=$SITE/peutils.py ORD=$SITE/ordlookup DI=$SITE/pefile-2023.2.7.dist-info PKGLIST=/var/lib/dpkg/info/python3-pefile.list PROBE=$BST_DATA/python3-pefile-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-pefile-suite.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ---------------------------------------------------------------- layout ----- # TWO FLAT MODULES AND ONE PACKAGE -- see the header. check $P/site-packages-dir test -d "$SITE" check $P/module-file test -s "$MOD" check $P/peutils-file test -s "$PEUTILS" check $P/ordlookup-dir test -d "$ORD" check $P/ordlookup-init test -s "$ORD/__init__.py" check $P/ordlookup-oleaut32 test -s "$ORD/oleaut32.py" check $P/ordlookup-ws2-32 test -s "$ORD/ws2_32.py" check $P/not-a-package test ! -d "$SITE/pefile" check $P/pycache-mod test -s "$SITE/__pycache__/pefile.cpython-312.pyc" check $P/pycache-peutils test -s "$SITE/__pycache__/peutils.cpython-312.pyc" check $P/pycache-ordlookup \ test -s "$ORD/__pycache__/__init__.cpython-312.pyc" # ------------------------------------------------------------- the .deb ------ cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" # Upstream's tests/ tree, its README and fix_dmp_files.py are not packaged: # setup.py declares py_modules and one package, and nothing else. # # The claim is made against pefile's OWN file list rather than against $SITE. # site-packages is SHARED, so `test ! -d "$SITE/tests"' is a statement about # the whole image: python3-reedsolo 2.0.13 installs its upstream tests/ tree # there as a TOP-LEVEL `tests' package, which reddened this case on an image # pefile is merely a co-tenant of. check_eq $P/no-tests-dir "" \ "$(grep -E '/site-packages/tests(/|$)' "$W/pkgf" 2>/dev/null)" check_eq $P/no-readme "" \ "$(grep -E '/site-packages/README([.][^/]*)?$' "$W/pkgf" 2>/dev/null)" check_eq $P/no-fix-dmp-files "" \ "$(grep -E '/site-packages/fix_dmp_files[.]py$' "$W/pkgf" 2>/dev/null)" # PURE PYTHON, AND NOT MERELY "NO .so OF ITS OWN". This image's python3 is ONE # STATIC BINARY with an empty lib-dynload and no dynamic loader (BRINTOS-1476), # so a module carrying a compiled extension could not be imported here at all. # pefile ships no C -- no .c, no ext_modules in setup.py -- so this is a # REGRESSION TRIPWIRE on a version bump rather than a statement about today. check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # NOTHING LANDS IN /usr/bin. setup.py declares no `entry_points' and no # `scripts', so pefile is importable and nothing else. check_eq $P/pkg-no-bin "" \ "$(grep -E '^/usr/(s?bin)/' "$W/pkgf" 2>/dev/null)" # ...and nothing but Python in the payload, which is the same claim from the # other side: a data file or a script appearing here would be a packaging # change nobody asked for. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" # THE DIST-INFO IS NAMED FOR THE DISTRIBUTION, and here -- unlike its # python3-editor neighbour -- the distribution and the primary module share a # name, so `pefile-2023.2.7.dist-info' sits beside `pefile.py'. top_level.txt # is the file that says three things were installed rather than one. check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-name "Name: pefile" cat "$DI/METADATA" check_out $P/dist-info-version "Version: 2023.2.7" cat "$DI/METADATA" check $P/dist-info-record test -s "$DI/RECORD" check $P/dist-info-license test -s "$DI/LICENSE" # `setuptools3' BUILT THIS, WHICH IS VISIBLE IN THE WHEEL METADATA. There is no # `universal = 1' in setup.cfg -- setup.cfg carries nothing but egg_info -- so # there is ONE tag and it is py3. check_out $P/dist-info-generator "Generator: bdist_wheel" cat "$DI/WHEEL" check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" check_out $P/dist-info-tag-py3 "Tag: py3-none-any" cat "$DI/WHEEL" check_eq $P/dist-info-no-py2-tag "" \ "$(grep '^Tag: py2' "$DI/WHEEL" 2>/dev/null)" check_out $P/dist-info-top-pefile "pefile" cat "$DI/top_level.txt" check_out $P/dist-info-top-peutils "peutils" cat "$DI/top_level.txt" check_out $P/dist-info-top-ordlookup "ordlookup" cat "$DI/top_level.txt" sed -n '/^Package: python3-pefile$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" check_out $P/pkg-version "Version: 2023.2.7-r0" cat "$W/st" # THE RDEPENDS IS ASSERTED EXACTLY -- see the header. A lean image that dropped # one of the three splits meta-python names would say so here once rather than # present as a broken library. grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_eq $P/pkg-depends-exact \ "Depends: python3-core, python3-mmap, python3-netclient, python3-stringold" \ "$(cat "$W/dep")" # ---------------------------------------------------------------- import ----- check $P/import python3 -c 'import pefile' check $P/import-peutils python3 -c 'import peutils' check $P/import-ordlookup python3 -c 'import ordlookup' # A DeprecationWarning, a SyntaxWarning out of peutils' unescaped regex, or a # stray print on import would be invisible to the cases above, which score only # the exit status. The .pyc files ship precompiled, so nothing recompiles at # import and the expected output is nothing at all. check_eq $P/import-quiet "" \ "$(python3 -c 'import pefile, peutils, ordlookup' 2>&1)" # ----------------------------------------------------------------- probe ----- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # The probe's own two bookends. `probe-complete' says it reached its last line # and `probe-count' says it emitted every key it meant to: a probe that died # two-thirds of the way through would otherwise leave the missing cases scoring # against an empty string, which `check_eq' would read as a wrong answer rather # than as an absent one. check_eq $P/probe-complete "yes" "$(pv probe_done)" check_eq $P/probe-count "187" "$(pv emitted_count)" # --------------------------------------------------------------- fixture ----- # THE FIXTURE ITSELF, FIRST AND BY HASH. Every literal below is a statement # about THESE 1536 bytes, so a builder that drifted would otherwise present as # forty unexplained reds instead of this one. check_eq $P/fixture-len "1536" "$(pv fixture_len)" check_eq $P/fixture-sha256 \ "afee79f68dea5940e73c37ad6bec023f2276ea072e721622440b58bb90311a15" \ "$(pv fixture_sha256)" # -------------------------------------------------------------- identity ----- # WHICH COPY ANSWERED. Every behavioural case below is a statement about the # module named here. check_eq $P/module-answered "$MOD" "$(pv module_file)" check_eq $P/peutils-answered "$PEUTILS" "$(pv peutils_file)" check_eq $P/is-package "no" "$(pv is_package)" check_eq $P/version-attr "2023.2.7" "$(pv version_attr)" check_eq $P/author-attr "Ero Carrera" "$(pv author_attr)" # NO `__all__', so a star-import leaks the module's own imports -- see header. check_eq $P/has-dunder-all "no" "$(pv has_dunder_all)" check_eq $P/star-leaks-struct "yes" "$(pv star_leaks_struct)" check_eq $P/star-leaks-os "yes" "$(pv star_leaks_os)" check_eq $P/has-pe "yes" "$(pv has_pe)" check_eq $P/has-peformaterror "yes" "$(pv has_peformaterror)" check_eq $P/has-structure "yes" "$(pv has_structure)" check_eq $P/has-sectionstructure "yes" "$(pv has_sectionstructure)" check_eq $P/has-datacontainer "yes" "$(pv has_datacontainer)" check_eq $P/has-dump "yes" "$(pv has_dump)" check_eq $P/has-max-symbol-export-count "yes" \ "$(pv has_max_symbol_export_count)" check_eq $P/has-unicode-postprocessor "yes" \ "$(pv has_unicodestringwrapperpostprocessor)" # PEFormatError DERIVES FROM Exception DIRECTLY -- not from ValueError and not # from OSError. A caller who catches ValueError around PE() catches nothing. check_eq $P/peformaterror-bases "Exception" "$(pv peformaterror_bases)" check_eq $P/peformaterror-not-valueerror "no" "$(pv peformaterror_is_valueerror)" check_eq $P/peformaterror-not-oserror "no" "$(pv peformaterror_is_oserror)" # THE LOOKUP TABLES a caller reads field values through. `MACHINE_TYPE' is a # TWO-WAY dict -- name to value and value back to name -- which is why the # reverse lookup is asserted beside the forward ones. check_eq $P/machine-i386 "332" "$(pv machine_i386)" check_eq $P/machine-amd64 "34404" "$(pv machine_amd64)" check_eq $P/machine-reverse "IMAGE_FILE_MACHINE_I386" \ "$(pv machine_reverse_14c)" check_eq $P/subsystem-console "3" "$(pv subsystem_console)" check_eq $P/dir-entry-export "0" "$(pv dir_entry_export)" check_eq $P/dir-entry-import "1" "$(pv dir_entry_import)" check_eq $P/dir-entry-resource "2" "$(pv dir_entry_resource)" check_eq $P/dir-entry-basereloc "5" "$(pv dir_entry_basereloc)" # ----------------------------------------------------- the headers, parsed --- # Each of these is a literal the builder wrote, read back through pefile's own # structure walk. check_eq $P/dos-e-magic "0x5a4d" "$(pv dos_e_magic)" check_eq $P/dos-e-lfanew "0x80" "$(pv dos_e_lfanew)" check_eq $P/nt-signature "0x00004550" "$(pv nt_signature)" check_eq $P/file-machine "0x014c" "$(pv file_machine)" check_eq $P/file-nsections "2" "$(pv file_nsections)" check_eq $P/file-sizeofoptionalheader "224" \ "$(pv file_sizeofoptionalheader)" check_eq $P/file-characteristics "0x0102" "$(pv file_characteristics)" check_eq $P/opt-magic "0x010b" "$(pv opt_magic)" check_eq $P/opt-entrypoint "0x1000" "$(pv opt_entrypoint)" check_eq $P/opt-imagebase "0x400000" "$(pv opt_imagebase)" check_eq $P/opt-sectionalignment "0x1000" "$(pv opt_sectionalignment)" check_eq $P/opt-filealignment "0x200" "$(pv opt_filealignment)" check_eq $P/opt-sizeofimage "0x3000" "$(pv opt_sizeofimage)" check_eq $P/opt-sizeofheaders "0x200" "$(pv opt_sizeofheaders)" check_eq $P/opt-subsystem "3" "$(pv opt_subsystem)" check_eq $P/opt-numberofrvaandsizes "16" "$(pv opt_numberofrvaandsizes)" check_eq $P/opt-checksum-field "0" "$(pv opt_checksum_field)" # THE SHAPE PREDICATES. Characteristics 0x0102 is EXECUTABLE_IMAGE | # 32BIT_MACHINE with no DLL bit and no SYSTEM bit, so exactly one of the three # is true -- asserted on all three sides so a fixture drift shows up as a # disagreement rather than as one silent flip. check_eq $P/is-exe "yes" "$(pv is_exe)" check_eq $P/is-dll "no" "$(pv is_dll)" check_eq $P/is-driver "no" "$(pv is_driver)" # --------------------------------------------------------- the section table - check_eq $P/nsections-parsed "2" "$(pv nsections_parsed)" # `Name' IS THE RAW 8 BYTES, NUL-padded -- not a str and not stripped. Asserted # as hex first, because that is the value a caller comparing against b'.text' # is actually holding. check_eq $P/sec0-name-raw "2e74657874000000" "$(pv sec0_name_raw)" check_eq $P/sec0-name ".text" "$(pv sec0_name)" check_eq $P/sec0-va "0x1000" "$(pv sec0_va)" check_eq $P/sec0-vsize "0x200" "$(pv sec0_vsize)" check_eq $P/sec0-rawsize "0x200" "$(pv sec0_rawsize)" check_eq $P/sec0-rawptr "0x200" "$(pv sec0_rawptr)" check_eq $P/sec0-characteristics "0x60000020" "$(pv sec0_characteristics)" check_eq $P/sec1-name-raw "2e72646174610000" "$(pv sec1_name_raw)" check_eq $P/sec1-name ".rdata" "$(pv sec1_name)" check_eq $P/sec1-va "0x2000" "$(pv sec1_va)" check_eq $P/sec1-vsize "0x200" "$(pv sec1_vsize)" check_eq $P/sec1-rawsize "0x200" "$(pv sec1_rawsize)" check_eq $P/sec1-rawptr "0x400" "$(pv sec1_rawptr)" check_eq $P/sec1-characteristics "0x40000040" "$(pv sec1_characteristics)" # `Misc' AND `Misc_VirtualSize' ARE THE SAME FIELD under two names -- the COFF # union. A caller who sets one and reads the other gets the value they set. check_eq $P/sec0-misc-is-vsize "yes" "$(pv sec0_misc_is_vsize)" # SECTION DATA COMES BACK BYTE FOR BYTE, padded to SizeOfRawData. check_eq $P/sec0-data-sha256 \ "136fad966989628fbe9044f27b8ff176577be90355bd0e875f8272ed544b2435" \ "$(pv sec0_data_sha256)" check_eq $P/sec1-data-sha256 \ "fec100efe4c4133754b168d5991e291c945c9dd305dd9673fea6822027c5f3bc" \ "$(pv sec1_data_sha256)" check_eq $P/sec1-data-len "512" "$(pv sec1_data_len)" # THE LIST IS SORTED BY VirtualAddress -- see the header. `sorted-first-name' # is the direct measurement: a fixture whose section TABLE is written # high-then-low comes back low-then-high. check_eq $P/sections-in-va-order "yes" "$(pv sections_in_va_order)" check_eq $P/sorted-first-name ".lo" "$(pv sorted_first_name)" check_eq $P/sorted-second-name ".hi" "$(pv sorted_second_name)" # ...and each entry carries the NEXT section's RVA, which is what the sort is # for: it is how a caller detects overlapping sections. The last one is None. check_eq $P/sec0-next-va "0x2000" "$(pv sec0_next_va)" check_eq $P/sec1-next-va "None" "$(pv sec1_next_va)" # ------------------------------------------------ address translation -------- check_eq $P/section-for-rva-text ".text" "$(pv section_for_rva_text)" check_eq $P/section-for-rva-rdata ".rdata" "$(pv section_for_rva_rdata)" # AN RVA IN NO SECTION RETURNS None rather than raising -- unlike # get_offset_from_rva below, which is the one lookup in this group that raises. # The pair is asserted side by side because the asymmetry is the point. check_eq $P/section-for-rva-gap "None" "$(pv section_for_rva_gap)" check_eq $P/offset-from-rva-text "0x200" "$(pv offset_from_rva_text)" check_eq $P/offset-from-rva-rdata "0x400" "$(pv offset_from_rva_rdata)" check_eq $P/rva-from-offset-rdata "0x2000" "$(pv rva_from_offset_rdata)" check_eq $P/offset-from-rva-bad "!PEFormatError" "$(pv offset_from_rva_bad)" # STRINGS ARE READ AT AN RVA AND STOP AT THE NUL, so both of these come back # without their terminator and without the filler that follows. check_eq $P/string-at-rva-dllname "FIXTURE.DLL" "$(pv string_at_rva_dllname)" check_eq $P/string-at-rva-export "pefile_fixture_export" \ "$(pv string_at_rva_export)" # ------------------------------------------------------ the data directories - check_eq $P/ndirs-parsed "16" "$(pv ndirs_parsed)" check_eq $P/dir0-name "IMAGE_DIRECTORY_ENTRY_EXPORT" "$(pv dir0_name)" check_eq $P/dir0-rva "0x2000" "$(pv dir0_rva)" check_eq $P/dir0-size "0x80" "$(pv dir0_size)" check_eq $P/dir1-name "IMAGE_DIRECTORY_ENTRY_IMPORT" "$(pv dir1_name)" check_eq $P/dir1-rva "0" "$(pv dir1_rva)" check_eq $P/dir2-name "IMAGE_DIRECTORY_ENTRY_RESOURCE" "$(pv dir2_name)" check_eq $P/dir5-name "IMAGE_DIRECTORY_ENTRY_BASERELOC" "$(pv dir5_name)" # THE EXPORT DIRECTORY, PARSED. Every value is a literal the probe's `_rdata()' # wrote, which is what lets the whole directory be asserted rather than just # its presence. check_eq $P/has-export-attr "yes" "$(pv has_export_attr)" check_eq $P/export-dll-name "FIXTURE.DLL" "$(pv export_dll_name)" check_eq $P/export-base "1" "$(pv export_base)" check_eq $P/export-nfunctions "1" "$(pv export_nfunctions)" check_eq $P/export-nnames "1" "$(pv export_nnames)" check_eq $P/export-symbol-count "1" "$(pv export_symbol_count)" check_eq $P/export-symbol-name "pefile_fixture_export" \ "$(pv export_symbol_name)" check_eq $P/export-symbol-rva "0x1000" "$(pv export_symbol_rva)" check_eq $P/export-symbol-ordinal "1" "$(pv export_symbol_ordinal)" # A ZERO DIRECTORY LEAVES THE ATTRIBUTE ABSENT rather than empty, which is the # distinction a caller has to code for: `hasattr', not `if pe.DIRECTORY_...'. check_eq $P/has-import-attr "no" "$(pv has_import_attr)" check_eq $P/has-resource-attr "no" "$(pv has_resource_attr)" check_eq $P/has-reloc-attr "no" "$(pv has_reloc_attr)" # THE IMPORT HASH OF AN IMPORT-FREE BINARY IS THE EMPTY STRING -- see header. check_eq $P/imphash "" "$(pv imphash)" # THE FIXTURE PRODUCES NO WARNINGS AT ALL, which is what every `*-warned' case # below leans on -- see the header. check_eq $P/warnings-none "0" "$(pv warnings_count)" check_eq $P/warnings-empty "" "$(pv warnings_joined)" # ------------------------------------------------------------- the checksum -- # THE CHECKSUM IS COMPUTED INDEPENDENTLY IN THE PROBE and the two are compared, # so this is a statement about the algorithm rather than about pefile agreeing # with itself. The offset is asserted too: it is the same in PE32 and PE32+, # which is why pefile hardcodes +0x40 off the optional header. check_eq $P/checksum-offset "0xd8" "$(pv checksum_offset)" check_eq $P/checksum-generated "0x6a79" "$(pv checksum_generated)" check_eq $P/checksum-independent "0x6a79" "$(pv checksum_independent)" check_eq $P/checksum-agrees "yes" "$(pv checksum_agrees)" # ...and the whole contract from the caller's side: an image with CheckSum 0 # does NOT verify, and one with the generated value written back does. check_eq $P/checksum-zero-fails-verify "yes" "$(pv verify_checksum_zero)" check_eq $P/checksum-written "0x6a79" "$(pv checksum_written)" check_eq $P/checksum-verifies "yes" "$(pv checksum_verifies)" # ------------------------------------------------------------- round trip ---- # write() RETURNS A bytearray, NOT bytes. A caller who type-checks the result, # or hands it to something that requires an immutable buffer, will be surprised. check_eq $P/write-type "bytearray" "$(pv write_type)" check_eq $P/write-len-same "yes" "$(pv write_len_same)" check_eq $P/write-bytes-changed "yes" "$(pv write_bytes_changed)" # THE EDIT TOOK... check_eq $P/rt-entrypoint "0x1234" "$(pv rt_entrypoint)" # ...AND NOTHING ELSE MOVED. Eleven named fields compared before and after; the # second case lists any that disagreed, so a red there names the field rather # than merely denying the first. check_eq $P/rt-unchanged-count "11" "$(pv rt_unchanged_count)" check_eq $P/rt-changed-others "" "$(pv rt_changed_others)" check_eq $P/rt-sec-data-intact "yes" "$(pv rt_sec_data_intact)" check_eq $P/rt-bytes-differing "2" "$(pv rt_bytes_differing)" # NO EDIT AT ALL IS THE IDENTITY -- write() is a serialiser, not a normaliser. check_eq $P/rt-identity "yes" "$(pv rt_identity)" # A SECTION-HEADER EDIT ROUND-TRIPS, AND THE RE-PARSE RE-SORTS -- see header. # `.text' renamed to `.code' and moved above `.rdata' comes back at index 1. check_eq $P/rt-sec-index0-name ".rdata" "$(pv rt_sec_index0_name)" check_eq $P/rt-sec-index1-name ".code" "$(pv rt_sec_index1_name)" check_eq $P/rt-sec-renamed-va "0x3000" "$(pv rt_sec_renamed_va)" check_eq $P/rt-sec-rdata-untouched "0x2000" "$(pv rt_sec_rdata_untouched)" # ...while the FILE's own table still has `.code' first: the reordering is # presentational and write() did not move anything. check_eq $P/rt-table-first-name ".code" "$(pv rt_table_first_name)" # write(filename=...) IS THE FILE-BACKED FORM and it writes the same bytes the # in-memory form returns. check_eq $P/write-file-matches "yes" "$(pv write_file_matches)" check_eq $P/write-file-len "1536" "$(pv write_file_len)" # ------------------------------------------------------------- from a file --- # PE(name=...) AND PE(data=...) AGREE. The file form mmaps -- which is what the # recipe's RDEPENDS on python3-mmap is for -- and the data form does not. # Nothing above depends on which was used, and this is why. check_eq $P/file-form-machine "0x014c" "$(pv file_form_machine)" check_eq $P/file-form-nsections "2" "$(pv file_form_nsections)" check_eq $P/file-form-sec0 ".text" "$(pv file_form_sec0)" check_eq $P/file-form-write-matches "yes" "$(pv file_form_write_matches)" # A MISSING FILE IS CPython's FileNotFoundError, not a PEFormatError -- the # open happens before any parsing. An EMPTY buffer is the parser's own error. check_eq $P/missing-file "!FileNotFoundError" "$(pv missing_file)" check_eq $P/empty-data "!PEFormatError" "$(pv empty_data)" # -------------------------------------------------------------- fast_load ---- # fast_load=True PARSES THE HEADERS AND STOPS. The fixture carries a REAL # export directory precisely so that this is observable -- see the header. check_eq $P/fast-machine "0x014c" "$(pv fast_machine)" check_eq $P/fast-nsections "2" "$(pv fast_nsections)" check_eq $P/fast-has-export "no" "$(pv fast_has_export)" # full_load() FILLS THEM IN, and returns None: it mutates in place, so a caller # who writes `pe = pe.full_load()' loses their object. check_eq $P/full-load-returns "None" "$(pv full_load_returns)" check_eq $P/full-has-export "yes" "$(pv full_has_export)" check_eq $P/full-export-dll-name "FIXTURE.DLL" "$(pv full_export_dll_name)" check_eq $P/full-export-symbol-name "pefile_fixture_export" \ "$(pv full_export_symbol_name)" # ...and it is IDEMPOTENT: a second call neither raises nor duplicates the # symbol list. check_eq $P/full-load-twice "1" "$(pv full_load_twice_symbols)" # A FAST-LOADED OBJECT STILL SERIALISES TO THE ORIGINAL BYTES. The directories # it skipped are still in the buffer; skipping them is a parse decision, not a # truncation. check_eq $P/fast-write-identity "yes" "$(pv fast_write_identity)" # ---------------------------------------------------- a bad directory RVA ---- # THE TICKET'S THIRD ERROR PATH, AND IT DOES NOT RAISE. A directory entry whose # RVA is past every section is recorded as a warning and the object comes back # usable -- pefile is deliberately tolerant here, because real packed binaries # carry exactly this. fast_load=True skips the parse entirely and produces no # warning at all, which is the cleanest statement of what fast_load buys. check_eq $P/baddir-parses "yes" "$(pv baddir_parses)" check_eq $P/baddir-has-warnings "yes" "$(pv baddir_has_warnings)" check_eq $P/baddir-import-attr "no" "$(pv baddir_import_attr)" check_eq $P/baddir-dir1-rva "0x900000" "$(pv baddir_dir1_rva)" # The bad-directory fixture zeroes the export entry the good one carries, so # the tolerated failure is measured on an object with NO directories parsed at # all -- which is what keeps it a statement about the bad RVA alone. check_eq $P/baddir-export-attr "no" "$(pv baddir_export_attr)" check_eq $P/baddir-fast-warnings "0" "$(pv baddir_fast_warnings)" # ============================================================ malformed ====== # ONE FIELD CORRUPTED PER CASE, each from the same builder, so a PEFormatError # is attributable to the corruption named in the key. # # THE REFUSALS -- what pefile cannot READ off the buffer. check_eq $P/no-mz "!PEFormatError" "$(pv no_mz)" check_eq $P/no-mz-raised "yes" "$(pv no_mz_raised)" # The message is asserted verbatim because it is the one a caller grepping for # a not-a-PE will grep for, and because it names ZM specifically: pefile knows # about the older `ZM' executable format and says so rather than reporting a # generic magic mismatch. check_eq $P/no-mz-message "'Probably a ZM Executable (not a PE file).'" \ "$(pv no_mz_text)" check_eq $P/truncated-dos "!PEFormatError" "$(pv truncated_dos)" check_eq $P/truncated-at-sig "!PEFormatError" "$(pv truncated_at_sig)" check_eq $P/lfanew-past-eof "!PEFormatError" "$(pv lfanew_past_eof)" check_eq $P/lfanew-wrong-offset "!PEFormatError" "$(pv lfanew_wrong_offset)" check_eq $P/truncated-sectable "!PEFormatError" "$(pv truncated_sectable)" # THE TOLERANCES -- what pefile can read but cannot BELIEVE. The ticket expected # the first of these to raise; it does not. See the header. check_eq $P/sections-past-eof "no-raise" "$(pv sections_past_eof)" check_eq $P/sections-past-eof-declared "4096" "$(pv sections_past_eof_declared)" check_eq $P/sections-past-eof-parsed "2" "$(pv sections_past_eof_parsed)" check_eq $P/sections-past-eof-warned "yes" "$(pv sections_past_eof_warned)" check_eq $P/sections-past-eof-warning \ "Invalid section 2. Contents are null-bytes." \ "$(pv sections_past_eof_warning)" check_eq $P/bad-opt-magic "no-raise" "$(pv bad_opt_magic)" check_eq $P/bad-opt-magic-value "0x0bad" "$(pv bad_opt_magic_value)" check_eq $P/bad-opt-magic-warned "yes" "$(pv bad_opt_magic_warned)" # AN UNKNOWN MACHINE IS DATA, NOT STRUCTURE, so it comes back verbatim and the # caller decides -- with no warning at all, unlike the two above. check_eq $P/unknown-machine-parses "yes" "$(pv unknown_machine_parses)" check_eq $P/unknown-machine-value "0xdead" "$(pv unknown_machine_value)" # ZERO SECTIONS IS A VALID PE, and the section list comes back empty rather # than the parse failing. check_eq $P/zero-sections-parses "yes" "$(pv zero_sections_parses)" check_eq $P/zero-sections-count "0" "$(pv zero_sections_count)" # ================================================== peutils and ordlookup ==== # THE SIBLING MODULES -- see the header. Nothing else in this suite would # notice if the recipe stopped installing them. check_eq $P/peutils-has-signaturedatabase "yes" \ "$(pv peutils_has_signaturedatabase)" check_eq $P/peutils-has-is-probably-packed "yes" \ "$(pv peutils_has_is_probably_packed)" check_eq $P/peutils-has-is-suspicious "yes" "$(pv peutils_has_is_suspicious)" check_eq $P/peutils-has-is-valid "yes" "$(pv peutils_has_is_valid)" # THE SIGNATURE DATABASE IS THREE TREES, NOT ONE, and an empty one is what an # image with no userdb.txt has -- which is this image. Asserting the counts is # what makes `sigdb-match-none' a measurement rather than a tautology: it says # the match returned None because there was nothing to match, and names how # nothing. check_eq $P/sigdb-eponly-true "0" "$(pv sigdb_eponly_true)" check_eq $P/sigdb-eponly-false "0" "$(pv sigdb_eponly_false)" check_eq $P/sigdb-section-start "0" "$(pv sigdb_section_start)" check_eq $P/sigdb-match-none "None" "$(pv sigdb_match_none)" check_eq $P/sigdb-match-all-empty "yes" "$(pv sigdb_match_all_empty)" # is_probably_packed() IS AN ENTROPY TEST over the sections, and the fixture is # built to fail it: `.text' is a 512-byte ramp and `.rdata' is repeating # filler, so neither section is dense enough to look packed. check_eq $P/is-probably-packed "no" "$(pv is_probably_packed)" check_eq $P/section-entropy-under-7 "yes" "$(pv section_0_entropy_under_7)" # ORDINAL LOOKUPS against the two tables `ordlookup' ships. A DLL it has no # table for returns None with make_name=False and a synthesised `ordN' with # make_name=True -- the two halves of the same call, asserted together because # a caller who leaves the default on cannot tell a real name from a made-up one. check_eq $P/ordlookup-is-package "yes" "$(pv ordlookup_is_package)" check_eq $P/ordlookup-ws2-32 "accept" "$(pv ordlookup_ws2_32)" check_eq $P/ordlookup-oleaut32 "SysAllocString" "$(pv ordlookup_oleaut32)" check_eq $P/ordlookup-unknown-dll "None" "$(pv ordlookup_unknown_dll)" check_eq $P/ordlookup-make-name "ord7" "$(pv ordlookup_make_name)" # --------------------------------------------------------------- filesystem -- # The probe built every fixture it used and took it away again. check_eq $P/workdir-gone "yes" "$(pv workdir_gone)" check_eq $P/cwd-restored "yes" "$(pv cwd_restored)" rm -rf "$W"