. "$BST_LIB" # BST-TIMEOUT: 1200 # THREE python3 starts, and they are the whole cost. Every # # other assertion below reads the filesystem or dpkg with # # shell builtins, or greps a line out of the transcript the # # third start left behind. # # # # The three are: a bare `import', a second one whose STDERR # # is the assertion (`import-quiet'), and the probe, which # # takes ONE start for two hundred and fifty-five # # measurements. # # # # NOTHING IN THIS SUITE OPENS A SOCKET, which is the reason # # the number can be this small for an HTTP client. The work # # inside the probe is string parsing, dict operations and a # # few hundred bytes through zlib; microseconds beside an # # interpreter start on this arm, which is tens of seconds # # before the first line of the program runs. # # # # 1200 is 400s per start, the same per-start budget # # 99-python3-libconf and 99-python3-imagesize each set for # # their three, and roughly six times the ~60s a python3 # # start has measured at here. # # # # A suite-timeout red is a batch-wide cost, so if the three # # starts prove slower than that on target the RIGHT fix is # # this number, not fewer cases. # # # # NOT MEASURED ON TARGET YET: this port's first on-target # # run is the batch verify, and the measured elapsed replaces # # this note in the test plan rather than in this comment, # # which would cost an image build to keep the shipped suite # # byte-identical. # # python3-urllib3 2.2.2 (poky/meta recipes-devtools/python/python3-urllib3_2.2.2.bb, # `inherit pypi python_hatchling', PyPI sdist plus five CVE patches) on the # wasm32-hwjs glibc image. # # ============================================================================ # WHAT THE PACKAGE DOES. # ============================================================================ # # The HTTP client `requests' is built on: a thread-safe connection pool, a # retry/backoff state machine, a URL parser, a multi-valued case-insensitive # header mapping and a response object that decodes transfer and content # encodings. Thirty-odd pure-Python modules; no C extension, no `.so' anywhere in # the payload, nothing to configure on this arm. # # ============================================================================ # THE SUITE OPENS NO SOCKET, AND THAT IS A DESIGN DECISION. # ============================================================================ # # The obvious way to test an HTTP client is to make a request. This suite must # not: the on-target verify has no network, so a case that waited for one would # HANG the guest rather than fail it -- and a wedged suite is quarantined, which # costs its owner the round. Every measurement is therefore taken either on the # half of urllib3 that runs before a connection exists (URL parsing, retry and # timeout arithmetic, header containers, multipart encoding, pool construction) # or on `HTTPResponse' driven from a `BytesIO' body, which is the seam upstream's # own test suite uses to exercise decoding without a server. # # What that cannot cover is the wire. Nothing here asserts that a GET completes, # that TLS negotiates, or that a redirect is followed. It asserts that the # machinery those depend on computes the right answers, and that the package # imports and is complete on this image. # # ============================================================================ # THE FOUR FINDINGS. # ============================================================================ # # ONE. THIS IMAGE'S urllib3 CANNOT DECODE `br' OR `zstd', AND SAYS SO ONLY BY # WHAT IT ADVERTISES. `response.py' imports brotli and zstandard inside # try/except; neither package is on the image, so `response.brotli' is None and # `HAS_ZSTD' is False. Two consequences, and the second is the one that reaches a # server. `CONTENT_DECODERS' is `gzip, x-gzip, deflate' rather than the five a # desktop urllib3 carries, so `make_headers(accept_encoding=True)' -- the default # for every request this image makes -- sends `Accept-Encoding: gzip,deflate'. # And a `Content-Encoding' urllib3 does not know is PASSED THROUGH UNDECODED with # no exception, so a server that ignored the advertisement and sent brotli anyway # hands the caller bytes it cannot use and nothing to tell it so. # `br-passthrough' and `zstd-passthrough' assert exactly that silence. # # TWO. `urllib3.contrib.pyopenssl' IS THE ONLY IMPORTER OF cryptography AND # pyopenssl IN THE PACKAGE, AND IT DOES NOT IMPORT HERE. That single deprecated # module is why poky's recipe declares both, and `import urllib3' never touches # it. Neither package is on this image -- python3-cryptography is a Rust # extension and there is no wasm32 Rust target in this tree, so it does not even # build (measured: `do_rust_gen_targets: No rust target defined for wasm32') -- # so the bbappend drops both from RDEPENDS. `pkg-depends-exact' is the assertion # that the shipped control stanza agrees, and `import-pyopenssl' is the assertion # that dropping them cost nothing `import urllib3' needed. # # `http2' is unimportable for the same shape of reason (no h2), and so is # `contrib.emscripten', which wants `js' and `pyodide' -- worth naming on THIS # image precisely because its name suggests it might be the one that works here, # and it is not: it targets Pyodide's browser fetch API, which has nothing to do # with this wasm32 Linux. # # `contrib.socks' IS THE ONE THAT IMPORTS, and this file predicted the opposite # before the image existed. PySocks 1.7.1 is installed -- not because urllib3 # asks for it (upstream declares it `extra == "socks"' and this port leaves it # out of RDEPENDS) but because poky's python3-requests RDEPENDS on it and # python3-requests is on this image. So `import-socks' asserts `ok'. What it # measures is the image's package set, not this package's declaration: it is the # case that goes red if requests ever leaves the image, and that is the signal # it is here for. # # THREE. `python3-idna' STAYS IN RDEPENDS AND A CLOSURE MEASUREMENT ALONE WOULD # HAVE DROPPED IT. `util/url.py:_idna_encode()' does `import idna' INSIDE the # function, reached only when a host label is non-ASCII, so idna is absent from # the `import urllib3' module closure while being a genuine runtime dependency of # a real path. `url-idna-host' drives that path with a Cyrillic hostname and # asserts its punycode; without the package it would raise LocationParseError # instead. This is the "in either direction" half of the ticket: the same # measurement that removes three declarations keeps this one. # # FOUR. `NewConnectionError' DERIVES FROM `ConnectTimeoutError'. A connection # refused, or a name that does not resolve, is caught by `except # ConnectTimeoutError' although nothing timed out -- so a caller distinguishing # "the server is slow" from "the server is not there" by exception type gets the # wrong answer, and the retry policy keyed on it treats the two alike. # `exc-NewConnectionError-bases' states it as the inheritance it is. # # ============================================================================ # THREE SMALLER DIVERGENCES, EACH ASSERTED BECAUSE IT SURPRISES. # ============================================================================ # # BACKOFF IS ZERO UNTIL THE SECOND RETRY. `get_backoff_time' returns 0 while # fewer than two entries are in the history, so the first retry is immediate # whatever `backoff_factor' says. `retry-backoff-0' through `retry-backoff-4' # state the whole curve rather than the rule, so a release that changed the # off-by-one turns exactly one case red. # # A `Timeout' WITH A TOTAL AND NO `start_connect' SILENTLY ENFORCES NO TOTAL. # `read_timeout' hands back the bare read timeout instead of consulting the # budget, and raises nothing. The state machine objects only to being STARTED # twice. So a Timeout reused across requests without being re-started degrades # from "5s overall" to "2s per read, unbounded" with no diagnostic at all. # # PERCENT-ESCAPES ARE UPPERCASED AND NOTHING IS DECODED. `%2f' comes back `%2F' # and `%7E' comes back `%7E' rather than the `~' it stands for, so two URLs # differing only in whether a byte was escaped stay different after parsing. # # ============================================================================ # HOW THE EXPECTED VALUES WERE OBTAINED. # ============================================================================ # # EVERY value below was MEASURED, not read off the source: the probe was run # against this recipe's own patched tree under CPython 3.12.3 on the build host, # with `OpenSSL', `cryptography', `certifi', `brotli', `brotlicffi', `zstandard', # `h2' and `socks' hidden behind a `sys.meta_path' blocker so the interpreter saw # exactly the module set this image has. The package is pure Python and # arch-independent and the target runs the same CPython 3.12 series, but the # suite's first ON-TARGET run is this batch's verify and the elapsed time is # recorded in the test plan rather than here. # # The one thing that could not be measured that way is `pkg-depends-exact': the # control stanza is written by bitbake, so its value comes from the .deb this # ticket's own recipe-scope build produced. P=python3-urllib3 # The canary is first so an empty capture is distinguishable from a suite that # never started. bst_ok $P/suite-canary require python3 $P/python3-present SITE=/usr/lib/python3.12/site-packages MOD=$SITE/urllib3 DI=$SITE/urllib3-2.2.2.dist-info PKGLIST=/var/lib/dpkg/info/python3-urllib3.list PROBE=$BST_DATA/python3-urllib3-probe.py tmp=${TMPDIR:-/tmp} W=$tmp/bst-urllib3.$$ mkdir -p "$W" || { bst_notok $P/workdir "cannot create $W"; exit 0; } # ---------------------------------------------------------------- payload ---- # urllib3 IS A PACKAGE, not a flat module -- the opposite of most of this Epic's # ports -- so the directory and its two subpackages are asserted individually. check $P/site-packages-dir test -d "$SITE" check $P/package-dir test -d "$MOD" check $P/init test -s "$MOD/__init__.py" check $P/no-flat-module test ! -e "$SITE/urllib3.py" check $P/util-subpackage test -d "$MOD/util" check $P/contrib-subpackage test -d "$MOD/contrib" check $P/emscripten-subpackage test -d "$MOD/contrib/emscripten" # The modules a caller reaches by name. Each is a separate case so a packaging # change that dropped one names it. check $P/mod-connectionpool test -s "$MOD/connectionpool.py" check $P/mod-poolmanager test -s "$MOD/poolmanager.py" check $P/mod-response test -s "$MOD/response.py" check $P/mod-connection test -s "$MOD/connection.py" check $P/mod-exceptions test -s "$MOD/exceptions.py" check $P/mod-fields test -s "$MOD/fields.py" check $P/mod-filepost test -s "$MOD/filepost.py" check $P/mod-http2 test -s "$MOD/http2.py" check $P/mod-util-retry test -s "$MOD/util/retry.py" check $P/mod-util-timeout test -s "$MOD/util/timeout.py" check $P/mod-util-url test -s "$MOD/util/url.py" check $P/mod-util-ssl test -s "$MOD/util/ssl_.py" # THE DEPRECATED SHIM SHIPS EVEN THOUGH IT CANNOT IMPORT. The package is # installed whole; dropping cryptography and pyopenssl from RDEPENDS removes the # DEPENDENCY, not the file. Both facts are asserted, because a reader who saw # only the failing import would reasonably conclude the file was missing. check $P/pyopenssl-file-ships test -s "$MOD/contrib/pyopenssl.py" check $P/socks-file-ships test -s "$MOD/contrib/socks.py" check $P/py-typed test -s "$MOD/py.typed" check $P/pycache test -d "$MOD/__pycache__" # --------------------------------------------------------------- packaging --- cat "$PKGLIST" > "$W/pkgf" 2>/dev/null || : > "$W/pkgf" check $P/pkg-list-present test -s "$W/pkgf" # NO SHARED OBJECT ANYWHERE. urllib3 is pure Python; a `.so' in this package # would mean a compiled dependency had been vendored into it, which on this arm # is the failure mode that matters. check_eq $P/pkg-no-so "" \ "$(grep -E '\.so($|\.)' "$W/pkgf" 2>/dev/null)" # AND ONE FILE THAT IS NOT PYTHON AT ALL. `contrib/emscripten' ships a 3655-byte # `emscripten_fetch_worker.js' -- a Web Worker for Pyodide's fetch API -- so this # pure-Python package installs a JavaScript file into site-packages. It is inert # here (the submodule that would load it does not import; see `import-emscripten' # below) and it is asserted rather than tolerated, because a sweep looking for # non-Python payload in a Python package will find it and should find it # ACCOUNTED FOR rather than as a surprise. check $P/pkg-ships-fetch-worker-js \ test -s "$MOD/contrib/emscripten/emscripten_fetch_worker.js" check_eq $P/pkg-js-count "1" \ "$(grep -c '\.js$' "$W/pkgf" 2>/dev/null)" # Beyond that one, nothing but Python and its metadata. _bad="" while IFS= read -r _f; do [ -f "$_f" ] || continue case "$_f" in *.py|*.pyc) continue ;; */__pycache__/*) continue ;; */*.dist-info/*) continue ;; */py.typed) continue ;; *emscripten_fetch_worker.js) continue ;; esac _bad="$_bad $_f" done < "$W/pkgf" check_eq $P/pkg-only-python "" "$_bad" check $P/dist-info-dir test -d "$DI" check $P/dist-info-metadata test -f "$DI/METADATA" check_out $P/dist-info-version "Version: 2.2.2" cat "$DI/METADATA" check_out $P/dist-info-name "Name: urllib3" cat "$DI/METADATA" check $P/dist-info-record test -s "$DI/RECORD" # THE LICENCE IS UNDER `licenses/', not at the top of the dist-info. That is # hatchling's layout and it differs from the `setuptools3' ports in this Epic, # so it is asserted at the path it actually has. check $P/dist-info-license test -s "$DI/licenses/LICENSE.txt" check $P/dist-info-no-top-license test ! -e "$DI/LICENSE" check_out $P/dist-info-generator "Generator: hatchling 1.21.1" cat "$DI/WHEEL" check_out $P/dist-info-purelib "Root-Is-Purelib: true" cat "$DI/WHEEL" check_out $P/dist-info-tag-py3 "Tag: py3-none-any" cat "$DI/WHEEL" sed -n '/^Package: python3-urllib3$/,/^$/p' /var/lib/dpkg/status > "$W/st" 2>/dev/null check $P/pkg-known test -s "$W/st" check_out $P/pkg-installed "Status: install ok installed" cat "$W/st" check_out $P/pkg-version "Version: 2.2.2-r0" cat "$W/st" # ============================================================================ # THE CONTROL STANZA IS THIS TICKET'S DELIVERABLE. # ============================================================================ # # This one case is what the bbappend exists for. Poky declares nine runtime # dependencies; three of them -- python3-cryptography, python3-pyopenssl and # python3-certifi -- are imported by no code path this image can reach, and one # of the three does not build on this arm at all. The bbappend removes exactly # those three and touches nothing else, so this line is the shipped evidence # that the declaration and the imports now agree. # # It is asserted WHOLE, not by absence of the three names: a `grep -v' would pass # just as happily on a stanza that had lost python3-idna, which is the one # dependency here that a closure measurement would wrongly have removed. grep '^Depends:' "$W/st" > "$W/dep" 2>/dev/null || : > "$W/dep" check $P/pkg-depends-line test -s "$W/dep" check_eq $P/pkg-depends-exact "Depends: python3-core, python3-email, python3-idna, python3-json, python3-logging, python3-netclient, python3-threading" "$(cat "$W/dep")" # ------------------------------------------------------------------ import --- check $P/import python3 -c 'import urllib3' # A CLEAN IMPORT IS ITSELF AN ASSERTION HERE. urllib3 warns on stderr for several # degraded configurations, and this image is a degraded configuration -- no # brotli, no zstd, no pyopenssl. A DependencyWarning at import would mean one of # those degradations had become loud, which is a behavioural change even though # nothing failed. check_eq $P/import-quiet "" \ "$(python3 -c 'import urllib3' 2>&1)" # ------------------------------------------------------------------- probe --- if [ ! -f "$PROBE" ]; then bst_notok $P/probe-present "$PROBE did not ship" rm -rf "$W" exit 0 fi bst_ok $P/probe-present bst_probe $P/probe-run "$W/probe" 900 python3 "$PROBE" OUT="$W/probe" pv() { grep "^$1=" "$OUT" 2>/dev/null | head -n 1 | cut -d= -f2-; } if [ -s "$OUT" ]; then bst_ok $P/probe-output else bst_notok $P/probe-output "the probe emitted nothing -- python3 did not start, or died before its first line; every CASE below is unscorable" fi # `emitted_count' is the probe's own tally and `probe_done' its last line, so the # two together say the transcript below is whole rather than truncated. check_eq $P/probe-complete "yes" "$(pv probe_done)" check_eq $P/probe-count "255" "$(pv emitted_count)" # --------------------------------------------------------------- identity ---- check_eq $P/module-answered "$MOD/__init__.py" "$(pv module_file)" check_eq $P/is-package "yes" "$(pv is_package)" check_eq $P/version '2.2.2' "$(pv version)" check_eq $P/version-module 'urllib3' "$(pv version_module)" check_eq $P/all-count '16' "$(pv all_count)" check_eq $P/all-sorted 'BaseHTTPResponse,HTTPConnectionPool,HTTPHeaderDict,HTTPResponse,HTTPSConnectionPool,PoolManager,ProxyManager,Retry,Timeout,add_stderr_logger,connection_from_url,disable_warnings,encode_multipart_formdata,make_headers,proxy_from_url,request' "$(pv all_sorted)" # --------------------------------------------- the image's decoder set ------- # Finding ONE. See the header. check_eq $P/brotli-absent 'yes' "$(pv brotli_absent)" check_eq $P/has-zstd 'no' "$(pv has_zstd)" check_eq $P/content-decoders 'gzip,x-gzip,deflate' "$(pv content_decoders)" check_eq $P/decoder-count '3' "$(pv decoder_count)" check_eq $P/redirect-statuses '301,302,303,307,308' "$(pv redirect_statuses)" # THE ONE MEASUREMENT A REMOTE SERVER WOULD SEE. A desktop urllib3 with brotli # and zstandard installed sends four codings here; this image sends two. check_eq $P/accept-encoding-default 'gzip,deflate' "$(pv accept_encoding_default)" check_eq $P/accept-encoding-explicit 'gzip' "$(pv accept_encoding_explicit)" # ------------------------------------------- what does and does not import --- # Finding TWO. `import_contrib' is `ok' and `import_pyopenssl' is not, which is # the pair that says the package is whole and one submodule's dependency is not. check_eq $P/import-contrib 'ok' "$(pv import_contrib)" # The refusal that decides this port's RDEPENDS. `ModuleNotFoundError' rather # than `ImportError' names WHICH failure it is: the dependency is absent, not # present and broken. check_eq $P/import-pyopenssl '!ModuleNotFoundError' "$(pv import_pyopenssl)" check_eq $P/import-http2 '!ModuleNotFoundError' "$(pv import_http2)" # MEASURED, and it contradicts what the recipe lane predicted: PySocks is on the # image, pulled in by python3-requests' RDEPENDS, so this one imports. Header, # finding TWO, says what that does and does not prove. check_eq $P/import-socks 'ok' "$(pv import_socks)" # Named because it invites the wrong guess on THIS image. `contrib.emscripten' # targets Pyodide's browser fetch API and wants `js' and `pyodide'; it has # nothing to do with this wasm32 Linux and does not import here either. check_eq $P/import-emscripten '!ModuleNotFoundError' "$(pv import_emscripten)" check_eq $P/import-util-url 'ok' "$(pv import_util_url)" check_eq $P/import-ssltransport 'ok' "$(pv import_ssltransport)" check_eq $P/import-connection 'ok' "$(pv import_connection)" check_eq $P/import-poolmanager 'ok' "$(pv import_poolmanager)" check_eq $P/import-request-methods 'ok' "$(pv import_request_methods)" # ------------------------------------------------------- public API surface --- # `has-Rreq' is a NEGATIVE CONTROL: there is no such attribute and there never # was. It is here so a `hasattr' that had started answering yes to everything -- # a `__getattr__' added upstream, a probe bug -- turns a case red instead of # turning the eleven above it green for the wrong reason. check_eq $P/has-PoolManager 'yes' "$(pv has_PoolManager)" check_eq $P/has-ProxyManager 'yes' "$(pv has_ProxyManager)" check_eq $P/has-HTTPConnectionPool 'yes' "$(pv has_HTTPConnectionPool)" check_eq $P/has-HTTPSConnectionPool 'yes' "$(pv has_HTTPSConnectionPool)" check_eq $P/has-HTTPResponse 'yes' "$(pv has_HTTPResponse)" check_eq $P/has-BaseHTTPResponse 'yes' "$(pv has_BaseHTTPResponse)" check_eq $P/has-HTTPHeaderDict 'yes' "$(pv has_HTTPHeaderDict)" check_eq $P/has-Retry 'yes' "$(pv has_Retry)" check_eq $P/has-Timeout 'yes' "$(pv has_Timeout)" check_eq $P/has-Rreq 'no' "$(pv has_Rreq)" check_eq $P/callable-request 'yes' "$(pv callable_request)" check_eq $P/callable-connection-from-url 'yes' "$(pv callable_connection_from_url)" check_eq $P/callable-proxy-from-url 'yes' "$(pv callable_proxy_from_url)" check_eq $P/callable-encode-multipart-formdata 'yes' "$(pv callable_encode_multipart_formdata)" check_eq $P/callable-make-headers 'yes' "$(pv callable_make_headers)" check_eq $P/callable-disable-warnings 'yes' "$(pv callable_disable_warnings)" check_eq $P/callable-add-stderr-logger 'yes' "$(pv callable_add_stderr_logger)" # -------------------------------------------------- exception hierarchy ------ # Finding FOUR is `exc-NewConnectionError-bases'. Each edge is a separate case so # a re-parented exception names itself. check_eq $P/exc-HTTPError-base 'Exception' "$(pv exc_HTTPError_base)" check_eq $P/exc-PoolError-base 'HTTPError' "$(pv exc_PoolError_base)" check_eq $P/exc-RequestError-base 'PoolError' "$(pv exc_RequestError_base)" check_eq $P/exc-MaxRetryError-base 'RequestError' "$(pv exc_MaxRetryError_base)" check_eq $P/exc-TimeoutError-base 'HTTPError' "$(pv exc_TimeoutError_base)" check_eq $P/exc-ConnectTimeoutError-bases 'TimeoutError,HTTPError' "$(pv exc_ConnectTimeoutError_bases)" check_eq $P/exc-ReadTimeoutError-is-httperror 'yes' "$(pv exc_ReadTimeoutError_is_httperror)" check_eq $P/exc-LocationParseError-base 'LocationValueError' "$(pv exc_LocationParseError_base)" check_eq $P/exc-NewConnectionError-bases 'ConnectTimeoutError,TimeoutError' "$(pv exc_NewConnectionError_bases)" check_eq $P/exc-ProtocolError-base 'HTTPError' "$(pv exc_ProtocolError_base)" check_eq $P/exc-ProtocolError-is-ConnectionError 'yes' "$(pv exc_ProtocolError_is_ConnectionError)" check_eq $P/exc-DecodeError-base 'HTTPError' "$(pv exc_DecodeError_base)" check_eq $P/exc-SSLError-base 'HTTPError' "$(pv exc_SSLError_base)" check_eq $P/exc-ResponseError-base 'HTTPError' "$(pv exc_ResponseError_base)" check_eq $P/exc-HTTPWarning-base 'Warning' "$(pv exc_HTTPWarning_base)" check_eq $P/exc-InsecureRequestWarning-base 'SecurityWarning' "$(pv exc_InsecureRequestWarning_base)" check_eq $P/exc-IncompleteRead-bases 'HTTPError,IncompleteRead' "$(pv exc_IncompleteRead_bases)" check_eq $P/exc-InvalidChunkLength-bases 'HTTPError,IncompleteRead' "$(pv exc_InvalidChunkLength_bases)" check_eq $P/exc-HTTPError-is-oserror 'no' "$(pv exc_HTTPError_is_oserror)" # -------------------------------------------------------------- URL parsing --- check_eq $P/url-scheme 'http' "$(pv url_scheme)" check_eq $P/url-host 'a.com' "$(pv url_host)" check_eq $P/url-path '/x' "$(pv url_path)" check_eq $P/url-port-none 'None' "$(pv url_port_none)" check_eq $P/url-port '8080' "$(pv url_port)" check_eq $P/url-port-type 'int' "$(pv url_port_type)" check_eq $P/url-query 'y=1' "$(pv url_query)" check_eq $P/url-fragment 'f' "$(pv url_fragment)" check_eq $P/url-auth 'u:p' "$(pv url_auth)" check_eq $P/url-netloc 'a.com:81' "$(pv url_netloc)" check_eq $P/url-request-uri '/x?y=1' "$(pv url_request_uri)" check_eq $P/url-url-roundtrip 'http://a.com/x?y=1#f' "$(pv url_url_roundtrip)" check_eq $P/url-str 'http://a.com/x?y=1' "$(pv url_str)" check_eq $P/url-scheme-lowered 'http' "$(pv url_scheme_lowered)" check_eq $P/url-host-lowered 'a.com' "$(pv url_host_lowered)" check_eq $P/url-path-case-kept '/X' "$(pv url_path_case_kept)" # THE EMPTY PATH IS `None', NOT `"/"'. urllib3 does not normalise it -- # `request_uri' is what supplies the "/" a request line needs. check_eq $P/url-empty-path 'None' "$(pv url_empty_path)" check_eq $P/url-empty-path-request-uri '/' "$(pv url_empty_path_request_uri)" check_eq $P/url-root-path '/' "$(pv url_root_path)" check_eq $P/url-bare-host 'a.com' "$(pv url_bare_host)" check_eq $P/url-bare-scheme 'None' "$(pv url_bare_scheme)" # `a.com:80' IS HOST AND PORT, not scheme and path. A caller passing an # unvalidated string here gets a parse, not an error. check_eq $P/url-bare-hostport-host 'a.com' "$(pv url_bare_hostport_host)" check_eq $P/url-bare-hostport-port '80' "$(pv url_bare_hostport_port)" check_eq $P/url-relative-path '/x/y' "$(pv url_relative_path)" check_eq $P/url-relative-host 'None' "$(pv url_relative_host)" check_eq $P/url-ipv6-host '[::1]' "$(pv url_ipv6_host)" check_eq $P/url-ipv6-port '8080' "$(pv url_ipv6_port)" check_eq $P/url-ipv6-netloc '[::1]:8080' "$(pv url_ipv6_netloc)" check_eq $P/url-ipv4-host '127.0.0.1' "$(pv url_ipv4_host)" # Percent-escapes are UPPERCASED and nothing is decoded: `%2f' -> `%2F' and # `%7E' stays `%7E' rather than becoming the `~' it stands for. check_eq $P/url-pct-upper '/%2F' "$(pv url_pct_upper)" check_eq $P/url-pct-unreserved '/%7Euser' "$(pv url_pct_unreserved)" check_eq $P/url-space-encoded '/a%20b' "$(pv url_space_encoded)" check_eq $P/url-query-encoded 'a=b%20c' "$(pv url_query_encoded)" check_eq $P/url-empty-port 'None' "$(pv url_empty_port)" check_eq $P/url-bad-port '!LocationParseError' "$(pv url_bad_port)" check_eq $P/url-port-out-of-range '!LocationParseError' "$(pv url_port_out_of_range)" check_eq $P/url-bad-ipv6 '!LocationParseError' "$(pv url_bad_ipv6)" # A NUL BYTE IN THE PATH DOES NOT RAISE -- it is percent-encoded like any other # byte. Only the port and the IPv6 literal have refusals. check_eq $P/url-null-byte 'a.com' "$(pv url_null_byte)" check_eq $P/url-bad-port-message 'LocationParseError: Failed to parse: http://a.com:x/' "$(pv url_bad_port_message)" # Finding THREE: the lazy `import idna' inside `_idna_encode'. This case is the # whole reason python3-idna stays in RDEPENDS; without the package it is a # LocationParseError instead of a punycode host. check_eq $P/url-idna-host 'xn--e1afmkfd.xn--p1ai' "$(pv url_idna_host)" check_eq $P/url-idna-scheme 'http' "$(pv url_idna_scheme)" check_eq $P/url-idna-mixed 'www.xn--e1afmkfd.xn--p1ai' "$(pv url_idna_mixed)" check_eq $P/url-idna-module-importable 'ok' "$(pv url_idna_module_importable)" check_eq $P/url-ascii-host-no-idna 'xn--e1afmkfd.xn--p1ai' "$(pv url_ascii_host_no_idna)" check_eq $P/url-is-tuple 'yes' "$(pv url_is_tuple)" check_eq $P/url-fields 'scheme,auth,host,port,path,query,fragment' "$(pv url_fields)" check_eq $P/url-eq 'yes' "$(pv url_eq)" check_eq $P/url-hostname-alias 'yes' "$(pv url_hostname_alias)" # --------------------------------------------------------------- Retry ------- check_eq $P/retry-default-total '3' "$(pv retry_default_total)" check_eq $P/retry-default-connect 'None' "$(pv retry_default_connect)" check_eq $P/retry-backoff-max '120' "$(pv retry_backoff_max)" check_eq $P/retry-allowed-methods 'DELETE,GET,HEAD,OPTIONS,PUT,TRACE' "$(pv retry_allowed_methods)" check_eq $P/retry-status-forcelist '413,429,503' "$(pv retry_status_forcelist)" check_eq $P/retry-default-redirect 'None' "$(pv retry_default_redirect)" check_eq $P/retry-raise-on-redirect 'True' "$(pv retry_raise_on_redirect)" check_eq $P/retry-raise-on-status 'True' "$(pv retry_raise_on_status)" check_eq $P/retry-respect-retry-after 'True' "$(pv retry_respect_retry_after)" # BACKOFF IS ZERO UNTIL THE SECOND RETRY. The curve is stated entry by entry # rather than as a rule, so a release that moved the off-by-one turns exactly # one case red and names which. check_eq $P/retry-backoff-0 '0' "$(pv retry_backoff_0)" check_eq $P/retry-backoff-1 '0' "$(pv retry_backoff_1)" check_eq $P/retry-backoff-2 '2.0' "$(pv retry_backoff_2)" check_eq $P/retry-backoff-3 '4.0' "$(pv retry_backoff_3)" check_eq $P/retry-backoff-4 '8.0' "$(pv retry_backoff_4)" check_eq $P/retry-backoff-capped '120.0' "$(pv retry_backoff_capped)" check_eq $P/retry-backoff-factor-half-2 '1.0' "$(pv retry_backoff_factor_half_2)" check_eq $P/retry-is-exhausted-fresh 'False' "$(pv retry_is_exhausted_fresh)" check_eq $P/retry-total-after-increment '1' "$(pv retry_total_after_increment)" check_eq $P/retry-exhausted-raises 'MaxRetryError' "$(pv retry_exhausted_raises)" check_eq $P/retry-history-len '2' "$(pv retry_history_len)" check_eq $P/retry-history-type 'RequestHistory' "$(pv retry_history_type)" check_eq $P/retry-from-int-total '3' "$(pv retry_from_int_total)" check_eq $P/retry-from-int-none '7' "$(pv retry_from_int_none)" check_eq $P/retry-from-int-bool-true 'Retry' "$(pv retry_from_int_bool_true)" check_eq $P/retry-repr-prefix 'Retry' "$(pv retry_repr_prefix)" # METHOD-SENSITIVE: POST is not in DEFAULT_ALLOWED_METHODS, so an identical 503 # retries for GET and does not for POST. check_eq $P/retry-is-retry-get-503 'True' "$(pv retry_is_retry_get_503)" check_eq $P/retry-is-retry-post-503 'False' "$(pv retry_is_retry_post_503)" check_eq $P/retry-is-retry-get-200 'False' "$(pv retry_is_retry_get_200)" check_eq $P/retry-parse-retry-after-seconds '120' "$(pv retry_parse_retry_after_seconds)" check_eq $P/retry-parse-retry-after-bad 'InvalidHeader' "$(pv retry_parse_retry_after_bad)" # --------------------------------------------------------------- Timeout ----- check_eq $P/timeout-default-sentinel '_TYPE_DEFAULT.token' "$(pv timeout_default_sentinel)" check_eq $P/timeout-from-float-connect '3.0' "$(pv timeout_from_float_connect)" check_eq $P/timeout-from-float-read '3.0' "$(pv timeout_from_float_read)" check_eq $P/timeout-total-none 'None' "$(pv timeout_total_none)" check_eq $P/timeout-clone-type 'Timeout' "$(pv timeout_clone_type)" check_eq $P/timeout-negative-refused 'ValueError' "$(pv timeout_negative_refused)" check_eq $P/timeout-string-refused 'ValueError' "$(pv timeout_string_refused)" check_eq $P/timeout-bool-refused 'ValueError' "$(pv timeout_bool_refused)" check_eq $P/timeout-repr-prefix 'Timeout' "$(pv timeout_repr_prefix)" # A TOTAL BUDGET WITH NO `start_connect' ENFORCES NO TOTAL, silently: the bare # read timeout comes back and nothing is raised. `timeout-start-twice' is the # one thing the state machine does object to. check_eq $P/timeout-read-without-start '2' "$(pv timeout_read_without_start)" check_eq $P/timeout-read-after-start '2' "$(pv timeout_read_after_start)" check_eq $P/timeout-start-twice 'TimeoutStateError' "$(pv timeout_start_twice)" # ------------------------------------------------------- HTTPHeaderDict ------ check_eq $P/hdr-case-insensitive-get 'text/plain' "$(pv hdr_case_insensitive_get)" check_eq $P/hdr-case-preserved 'Content-Type' "$(pv hdr_case_preserved)" check_eq $P/hdr-in-lower 'True' "$(pv hdr_in_lower)" # `add' ACCUMULATES WHERE `[]=' REPLACES, and reading a multi-valued header back # with `[]' joins with ", ". `getlist' is the only way to get the values apart. check_eq $P/hdr-add-joins 'a=1, b=2' "$(pv hdr_add_joins)" check_eq $P/hdr-getlist 'a=1,b=2' "$(pv hdr_getlist)" check_eq $P/hdr-getlist-absent '[]' "$(pv hdr_getlist_absent)" check_eq $P/hdr-len '2' "$(pv hdr_len)" check_eq $P/hdr-setitem-replaces 'c=3' "$(pv hdr_setitem_replaces)" check_eq $P/hdr-eq-case 'True' "$(pv hdr_eq_case)" # Equality against a PLAIN dict is case-insensitive too, which is the direction # that surprises: the asymmetry a caller expects is not there. check_eq $P/hdr-eq-plain-dict 'True' "$(pv hdr_eq_plain_dict)" check_eq $P/hdr-copy-type 'HTTPHeaderDict' "$(pv hdr_copy_type)" check_eq $P/hdr-discard-missing 'None' "$(pv hdr_discard_missing)" check_eq $P/hdr-from-pairs '1, 2' "$(pv hdr_from_pairs)" check_eq $P/hdr-repr-prefix 'HTTPHeaderDict' "$(pv hdr_repr_prefix)" # ------------------------------------------------- fields and multipart ------ check_eq $P/mp-content-type 'multipart/form-data; boundary=BOUND' "$(pv mp_content_type)" check_eq $P/mp-body-is-bytes 'yes' "$(pv mp_body_is_bytes)" check_eq $P/mp-line-count '11' "$(pv mp_line_count)" # CRLF ONLY, never a bare LF. A body assembled by hand that used "\n" is # accepted by some servers and rejected by others; this asserts urllib3 does # not produce one. check_eq $P/mp-crlf-only 'yes' "$(pv mp_crlf_only)" check_eq $P/mp-boundary-count '3' "$(pv mp_boundary_count)" check_eq $P/mp-has-terminator 'yes' "$(pv mp_has_terminator)" check_eq $P/mp-has-filename 'yes' "$(pv mp_has_filename)" check_eq $P/mp-has-name 'yes' "$(pv mp_has_name)" check_eq $P/mp-declared-ctype 'yes' "$(pv mp_declared_ctype)" check_eq $P/mp-payload-present 'yes' "$(pv mp_payload_present)" check_eq $P/mp-random-boundary-len '32' "$(pv mp_random_boundary_len)" # `render_headers' READS `self.headers', WHICH IS EMPTY UNTIL `make_multipart' # HAS RUN -- so a field rendered without that call emits nothing and no error. check_eq $P/rf-headers-before-multipart '0' "$(pv rf_headers_before_multipart)" check_eq $P/rf-render-headers-lines '2' "$(pv rf_render_headers_lines)" check_eq $P/rf-disposition 'Content-Disposition: form-data; name="name"; filename="f.txt"' "$(pv rf_disposition)" check_eq $P/rf-from-tuples-type 'RequestField' "$(pv rf_from_tuples_type)" check_eq $P/rf-guess-content-type-txt 'text/plain' "$(pv rf_guess_content_type_txt)" check_eq $P/rf-guess-content-type-unknown 'application/octet-stream' "$(pv rf_guess_content_type_unknown)" check_eq $P/rf-guess-content-type-none 'application/octet-stream' "$(pv rf_guess_content_type_none)" # urllib3 2.x DEFAULTS TO THE HTML5 FORM of a header parameter: it # percent-encodes inside the quoted string rather than emitting an RFC 2231 # `name*=' parameter. The two `rfc2231' cases below drive the deprecated # function that still does the old thing, so the difference is visible. check_eq $P/rf-param-ascii 'name="v"' "$(pv rf_param_ascii)" check_eq $P/rf-param-quote 'name="a%22b"' "$(pv rf_param_quote)" check_eq $P/rf-param-newline 'name="a%0Ab"' "$(pv rf_param_newline)" check_eq $P/rf-param-rfc2231-ascii 'name="v"' "$(pv rf_param_rfc2231_ascii)" check_eq $P/rf-param-rfc2231-utf8 "name*=utf-8''%C3%A9" "$(pv rf_param_rfc2231_utf8)" # ---------------------------------------------------- response decoding ------ check_eq $P/resp-read-all 'hello' "$(pv resp_read_all)" check_eq $P/resp-status-default '0' "$(pv resp_status_default)" check_eq $P/resp-closed-after-read 'False' "$(pv resp_closed_after_read)" check_eq $P/resp-gzip 'hello gzip' "$(pv resp_gzip)" check_eq $P/resp-gzip-xgzip 'hello gzip' "$(pv resp_gzip_xgzip)" check_eq $P/resp-deflate 'hello deflate' "$(pv resp_deflate)" # THE DEFLATE DECODER ACCEPTS BOTH SHAPES. `deflate' is specified as a zlib # stream, but servers ship raw DEFLATE often enough that urllib3 retries # headerless on the first failure -- so the wrong shape decodes too and a caller # cannot use urllib3 to tell which one it was sent. check_eq $P/resp-deflate-raw 'raw deflate' "$(pv resp_deflate_raw)" check_eq $P/resp-identity 'raw' "$(pv resp_identity)" # Finding ONE, as behaviour rather than as a flag: an unknown Content-Encoding # is PASSED THROUGH UNDECODED and nothing is raised. These two are the brotli # and zstd magic numbers coming back byte-for-byte -- which is what a caller on # this image gets from a server that ignored the Accept-Encoding above. check_eq $P/resp-br-passthrough '0b028061626303' "$(pv resp_br_passthrough)" check_eq $P/resp-zstd-passthrough '28b52ffd' "$(pv resp_zstd_passthrough)" check_eq $P/resp-decode-content-false '30' "$(pv resp_decode_content_false)" check_eq $P/resp-corrupt-gzip 'DecodeError' "$(pv resp_corrupt_gzip)" check_eq $P/resp-read-amt '0123' "$(pv resp_read_amt)" check_eq $P/resp-read-rest '456789' "$(pv resp_read_rest)" check_eq $P/resp-tell '10' "$(pv resp_tell)" check_eq $P/resp-headers-type 'HTTPHeaderDict' "$(pv resp_headers_type)" check_eq $P/resp-getheader '1' "$(pv resp_getheader)" check_eq $P/resp-stream-chunks 'ab,cd,ef' "$(pv resp_stream_chunks)" check_eq $P/resp-readinto '3' "$(pv resp_readinto)" check_eq $P/resp-readable 'True' "$(pv resp_readable)" check_eq $P/resp-isclosed-fresh 'False' "$(pv resp_isclosed_fresh)" check_eq $P/resp-data-property 'xyz' "$(pv resp_data_property)" check_eq $P/resp-json "{'k': 1}" "$(pv resp_json)" check_eq $P/resp-json-bad 'JSONDecodeError' "$(pv resp_json_bad)" check_eq $P/resp-url-none 'None' "$(pv resp_url_none)" check_eq $P/resp-retries-none 'None' "$(pv resp_retries_none)" check_eq $P/resp-version-default '0' "$(pv resp_version_default)" check_eq $P/resp-reason-default 'None' "$(pv resp_reason_default)" # `False', NOT `None', when there is nothing to redirect to -- and `False' again # for a 200 that carries a Location header, because the STATUS decides. check_eq $P/resp-get-redirect-location-none 'False' "$(pv resp_get_redirect_location_none)" check_eq $P/resp-get-redirect-location-302 '/x' "$(pv resp_get_redirect_location_302)" check_eq $P/resp-get-redirect-location-200-with-header 'False' "$(pv resp_get_redirect_location_200_with_header)" # ----------------------------------------------------------- ssl helpers ----- # `import urllib3' reaches `ssl' unconditionally, so a python3 whose _ssl # extension had not loaded would have failed `import' far above. These assert # that urllib3's own view of this image's OpenSSL 3.x agrees with 99-python3's. check_eq $P/ssl-module-importable 'ok' "$(pv ssl_module_importable)" check_eq $P/ssl-is-pyopenssl 'False' "$(pv ssl_is_pyopenssl)" check_eq $P/ssl-has-sni 'True' "$(pv ssl_has_sni)" check_eq $P/ssl-context 'ok' "$(pv ssl_context)" check_eq $P/ssl-context-min-version 'TLSv1_2' "$(pv ssl_context_min_version)" check_eq $P/ssl-context-check-hostname 'True' "$(pv ssl_context_check_hostname)" check_eq $P/ssl-context-verify-mode 'CERT_REQUIRED' "$(pv ssl_context_verify_mode)" check_eq $P/ssl-context-no-compression 'True' "$(pv ssl_context_no_compression)" check_eq $P/ssl-context-cert-none-no-hostname 'False' "$(pv ssl_context_cert_none_no_hostname)" check_eq $P/ssl-resolve-cert-reqs-none 'CERT_NONE' "$(pv ssl_resolve_cert_reqs_none)" check_eq $P/ssl-resolve-cert-reqs-required 'CERT_REQUIRED' "$(pv ssl_resolve_cert_reqs_required)" check_eq $P/ssl-resolve-cert-reqs-default 'CERT_REQUIRED' "$(pv ssl_resolve_cert_reqs_default)" check_eq $P/ssl-resolve-cert-reqs-bare 'CERT_REQUIRED' "$(pv ssl_resolve_cert_reqs_bare)" # md5, sha1 and sha256 of the three bytes `abc'. `assert_fingerprint' picks the # hash by HEX LENGTH, so these three also assert that dispatch; `None' is the # return of a match and the SSLError is the mismatch. check_eq $P/ssl-assert-fingerprint-md5 'None' "$(pv ssl_assert_fingerprint_md5)" check_eq $P/ssl-assert-fingerprint-sha1 'None' "$(pv ssl_assert_fingerprint_sha1)" check_eq $P/ssl-assert-fingerprint-bad 'SSLError' "$(pv ssl_assert_fingerprint_bad)" check_eq $P/ssl-assert-fingerprint-sha256 'None' "$(pv ssl_assert_fingerprint_sha256)" # ------------------------------------------------- pools, without any I/O ---- check_eq $P/pm-pool-type 'HTTPConnectionPool' "$(pv pm_pool_type)" check_eq $P/pm-pool-type-https 'HTTPSConnectionPool' "$(pv pm_pool_type_https)" # TWO URLS ON ONE HOST SHARE ONE POOL and two hosts do not, which is the whole # of what a PoolManager is. Neither call opens a connection. check_eq $P/pm-pool-count-after-two '1' "$(pv pm_pool_count_after_two)" check_eq $P/pm-pool-count-distinct-hosts '2' "$(pv pm_pool_count_distinct_hosts)" check_eq $P/pm-pool-host 'a.com' "$(pv pm_pool_host)" check_eq $P/pm-pool-port '81' "$(pv pm_pool_port)" check_eq $P/pm-default-port-http '80' "$(pv pm_default_port_http)" check_eq $P/pm-default-port-https '443' "$(pv pm_default_port_https)" check_eq $P/pm-scheme 'http' "$(pv pm_scheme)" check_eq $P/pm-bad-scheme 'URLSchemeUnknown' "$(pv pm_bad_scheme)" check_eq $P/pm-proxy-from-url-type 'ProxyManager' "$(pv pm_proxy_from_url_type)" check_eq $P/pm-proxy-host 'p' "$(pv pm_proxy_host)" check_eq $P/pm-key-fields-count '29' "$(pv pm_key_fields_count)" check_eq $P/pm-default-pools-size '10' "$(pv pm_default_pools_size)" check_eq $P/pool-default-maxsize '1' "$(pv pool_default_maxsize)" # NOT `repr()': HTTPConnectionPool has no `__repr__' of its own, so the default # one carries a memory address and could never be asserted. `str()' is defined # and is stable. check_eq $P/pool-class-name 'HTTPConnectionPool' "$(pv pool_class_name)" check_eq $P/pool-str "HTTPConnectionPool(host='a.com', port=81)" "$(pv pool_str)" check_eq $P/pool-default-headers-empty '0' "$(pv pool_default_headers_empty)" check_eq $P/pool-block-default 'False' "$(pv pool_block_default)" check_eq $P/top-level-poolmanager-type 'PoolManager' "$(pv top_level_poolmanager_type)" check_eq $P/disable-warnings-returns-none 'None' "$(pv disable_warnings_returns_none)" rm -rf "$W"