|)h^dZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ejeZdZej$fdZGdd eZd ZGd d eZy) a This module allows opening and reading local and remote files and decompress them on-the-fly if needed. Remote files are read using urllib (except of "ssh://" URLs, which are handled differently). Supported file extentions are: 'bz2', 'gz', 'xz', 'lzo', 'zst' and a "tar" version of them: 'tar.bz2', 'tbz2', 'tbz', 'tb2', 'tar.gz', 'tgz', 'tar.xz', 'txz', 'tar.lzo', 'tzo', 'tar.lz4', 'tlz4', '.tar.zst', 'tzst'. This module uses the following system programs for decompressing: pbzip2, bzip2, gzip, pigz, xz, lzop, lz4, zstd, tar and unzip. N)parse) BmapHelpers) bz2gzxzlzolz4zstztar.gzztar.bz2ztar.xzztar.lzoztar.lz4ztar.zstzipc|tjk(r|}nF|tjk(r||z}n-tdtjtj|fz||krtd||fz||z }|}|dkDr4t |d}|j |}|sn|t |z}|dkDr4|dkrtd||z |fz||z S)z This function implements the 'seek()' method for file object 'file_obj'. Only seeking forward and is allowed, and 'whence' may be either 'os.SEEK_SET' or 'os.SEEK_CUR'. zP'seek()' method requires the 'whence' argument to be %d or %d, but %d was passedzT''seek()' method supports only seeking forward, seeking from %d to %d is not allowedrz seeked too far: %d instead of %d)osSEEK_SETSEEK_CURErrorminreadlen) file_objcur_posoffsetwhencenew_poslengthto_read chunk_sizebufs 7/usr/lib/python3.12/site-packages/bmaptool/TransRead.py_fake_seek_forwardr Is 2;; F" 035;; V2T U   36=w5G H  w FG A++. mmJ' 3s8 A+{6'G:KW9UUVV W ceZdZdZy)rz A class for exceptions generated by this module. We currently support only one type of exceptions, and we basically throw human-readable problem description in case of errors. N)__name__ __module__ __qualname____doc__r!rrros   r!rc|dk(rd}|S|dk(rd}|S|dk(rd}|S|dk(rd}|S|d k(rd }|S|d k(rd }|S|d k(rd}|Sd}|S)z A helper function which converts "sshpass" command-line tool's exit code into a human-readable string. See "man sshpass". rzinvalid command line argumentzconflicting arguments givenzgeneral run-time errorz,unrecognized response from ssh (parse error)zinvalid/incorrect passwordzHhost public key is unknown. sshpass exits without confirming the new keyz ssh errorunknownr')coderesults r_decode_sshpass_exit_coder2ys  qy0( M' .$ M# ) M ? M - M  %  M  M Mr!cleZdZdZdZdZdZdZdZdZ ddZ e jfd Z d Zd Zd Zy ) TransReadz This class implement the transparent reading functionality. Instances of this class are file-like objects which you can read and seek only forward. cP||_d|_d|_d|_d|_g|_d|_d|_g|_d|_ d|_ |jdk(r4|jjtjjn/|jjt|jd|j+y#t $rJ}|j"t"j$k(r|j'|nt)d|d|Yd}~_d}~wwxYw) zx Class constructor. The 'filepath' argument is the full path to the file to read transparently. NnoneFr-rbzcannot open file '': )namesizecompression_type bz2file_foundis_url_child_processes_rthread_done_f_objs _fake_seek_posappendsysstdinbufferopenIOErrorerrnoENOENT _open_urlr_open_compressed_file)selffilepatherrs r__init__zTransRead.__init__s   &" "     KyyC ##CII$4$45 ##DD$9: ""$ KyyELL(x(8SIJJ) KsA2C D%AD  D%cd|_t|dr(|jD]}|jd|_t|dr!|jj d|_t|drJ|j D]3}|j|j|j5g|_yy)z/The class destructor which closes opened files.TrBNr@r?) rAgetattrrBcloser@joinr?pollkillwait)rOrchilds r__del__zTransRead.__del__s 4 # LL )DL 4 $ MM    DM 4+ ,..::<'JJLJJL/%'D ! -r!cd} |js2|j|}|sn|j||js2|jy#|jwxYw)a7 This function is used when reading compressed files. It runs in a spearate thread, reads data from the 'f_from' file-like object, and writes them to the 'f_to' file-like object. 'F_from' may be a urllib object, while 'f_to' is usually stdin of the decompressor process. rN)rArwriterU)rOf_fromf_torrs r _read_threadzTransRead._read_threadsR!  jjkk*- 3 jj JJLDJJLs =AA%cLd}d}d}d}d}d}d}d}d } d } d } d } d } ||js||jr=d|_tjdrd}nd}||jrd}nd} d}n||js||jr=d|_tjdrd}nd}||jrd}nd} d}n| |js||jr%d|_d}||jrd}nPd} d}nJ| |js||jr%d|_d}||jrd}nd} d}n|jj dr d|_d}d}n| |js||jr#d|_d}||jrd}nd} d }n| |js||jr#d!|_d"}||jrd#}nQd} d$}nL|j s?t j|jd%jj|_ y | dk(r|d&z }tj|std'|d(|jd)| r1tj| std'| d*|jd)| r | d+z|z}n|d+z|z}|j rtj}n|jd%j}tj|d,d-|tj.}|tjk(rl|jd%|j f}t#j$|j&|/|_d,|j(_|j(j-d,|_|jj1|j2|j4j1|y )0z Detect file compression type and open it with the corresponding compression module, or just plain 'open() if the file is not compressed. cl|jds"|jdr|jdsyy)z8Returns 'True' if file 'name' is compressed with 'gzip'.z.gzipz.gz.tar.gzTFendswithr:s ris_gzipz0TransRead._open_compressed_file..is_gzips+}}W% e$T]]9-Er!cJ|jdr|jdsyy)z9Returns 'True' if file 'name' is compressed with 'bzip2'.z.bz2.tar.bz2TFrdrfs ris_bzip2z1TransRead._open_compressed_file..is_bzip2}}V$T]]:-Fr!cJ|jdr|jdsyy)z6Returns 'True' if file 'name' is compressed with 'xz'.z.xz.tar.xzTFrdrfs ris_xzz.TransRead._open_compressed_file..is_xz s}}U#DMM),Dr!cJ|jdr|jdsyy)z8Returns 'True' if file 'name' is compressed with 'lzop'.z.lzo.tar.lzoTFrdrfs ris_lzopz0TransRead._open_compressed_file..is_lzoprkr!cJ|jdr|jdsyy)z7Returns 'True' if file 'name' is compressed with 'lz4'.z.lz4.tar.lz4TFrdrfs ris_lz4z/TransRead._open_compressed_file..is_lz4rkr!cJ|jdr|jdsyy)z8Returns 'True' if file 'name' is compressed with 'zstd'.z.zst.tar.zstTFrdrfs ris_zstz/TransRead._open_compressed_file..is_zstrkr!cJ|jds|jdryy)zl Returns 'True' if file 'name' is a tar archive compressed with 'gzip'. rcz.tgzTFrdrfs r is_tar_gzz2TransRead._open_compressed_file..is_tar_gz%s! }}Y'4==+@r!c|jds3|jds"|jds|jdryy)zm Returns 'True' if file 'name' is a tar archive compressed with 'bzip2'. riz.tbzz.tbz2z.tb2TFrdrfs r is_tar_bz2z3TransRead._open_compressed_file..is_tar_bz2/s: j)==(==)==(r!cJ|jds|jdryy)z^ Returns 'True' if file 'name' is a tar archive compressed with 'xz'. rmz.txzTFrdrfs r is_tar_xzz2TransRead._open_compressed_file..is_tar_xz>s! }}Y'4==+@r!cJ|jds|jdryy)zl Returns 'True' if file 'name' is a tar archive compressed with 'lzop'. rpz.tzoTFrdrfs r is_tar_lzoz3TransRead._open_compressed_file..is_tar_lzoGs! }}Z(DMM&,Ar!cJ|jds|jdryy)zk Returns 'True' if file 'name' is a tar archive compressed with 'lz4'. rsz.tlz4TFrdrfs r is_tar_lz4z3TransRead._open_compressed_file..is_tar_lz4Q! }}Z(DMM',Br!cJ|jds|jdryy)zl Returns 'True' if file 'name' is a tar archive compressed with 'zstd'. rvz.tzstTFrdrfs r is_tar_zstz3TransRead._open_compressed_file..is_tar_zst[rr!Ngzippigzz-d -ctarz-x -z -Obzip2pbzip2z-x -j -Orz-x -J -Or lzopz -x --lzo -Oz.zipr funzipr z -x -Ilz4 -Or zstdz-dz -x -Izstd -Oz -P -C /zthe "z=" program is not available but it is required decompressing ""z7" program is not available but it is required reading " Tr)shellbufsizerGstdout)targetargs)r:r<rprogram_is_availablerer>rfstatrBfilenost_sizer;r subprocessPIPEPopenrG threadingThreadr`r@daemonstartrCrErr?)rOrgrjrnrqrtrwryr{r}rrrarchiver decompressorr child_stdin child_processs rrNzTransRead._open_compressed_files                  TYY 7499#5$*D !//7% % tyy! !  "htyy&9$+D !//9' &  " ! tyy !U499%5$(D !LTYY !  "gdii&8$)D !!Ltyy! $ YY   '$)D !#LD  "fTYY&7$)D ! Ldii  $  "fTYY&7$)D !!Ldii  %;;HHT\\"%5%<%<%>?GG  u  J D// =1=tyyJ  K<D(D#%,D ;;$//K,,r*113K"(( ??   *// ) R ##D&,,D4E4EDQDM#'DMM MM   ! M001 $$]3r!c z|j}|j}|j}|j}|r|dz|z}t j ds t d|r.dd|zddddd d |g }t j dst d ddddd d d|g}tj|dgz}|j|j}|dk7rt|} t d|| |fzd|zdz|z} tj|| gzd}|j|jdk7rt d|d|dtj|d|zgztj}|jj|j|j j|d|_d|_y)z This function opens a file on a remote host using SSH. The URL has to have this format: "ssh://username@hostname:path". Currently we only support password-based authentication. @sshz[the "ssh" program is not available but it is required for downloading over the ssh protocolsshpassz-pz-o StrictHostKeyChecking=noz-o ClearAllForwardings=yesz-o ForwardX11=noz-o PubkeyAuthentication=noz-o PasswordAuthentication=yesz_the "sshpass" program is not available but it is required for password-based SSH authenticationz-o PubkeyAuthentication=yesz-o PasswordAuthentication=noz-o BatchMode=yestruerz*cannot connect to "%s": %s (error code %d)ztest -f z && test -r r)rrz" on "zW" cannot be read: make sure it exists, is a regular file, and you have read permissionszcat )rTN)usernamepasswordpathhostnamerrrrrrY returncoder2rrBrErr?r>rC) rO parsed_urlrrrr popen_argsrretcodedecodedcommands r _open_url_sshzTransRead._open_url_sshs&&&&&& #~0H//6A  x-,",/ J33E:H -,"-." J#((vh)>? ** a</8G<Wg./ t#n4t;"((wi)?U   # #q ("&x1 #(( &4- ( M001 $$]3 r!c d}ddl}ddlm}ddlm}ddlm}tj|}|jdk(r|j|y|j}|j} |sL| sJ|jdvr< tj} | j|j} | r | d}| d } |r| rt/|} |j0r|jd |j0| d<n|j| d<tj2| }|j5}|j7d||| |j9|}|j;|}n|j;}dg|_|j?|dD]} |jA||}d|_)|jTjWy#t$rYt j"$rG} t$j'd | j(d | j*d | j,Yd} ~ ld} ~ wwxYw#|jB$r&}| ||ntEd|d|Yd}~d}~w|$rG}|)tG|jH|jBr ||ntEd|d|Yd}~d}~wtJtL|jNf$r}tEd|d|d}~w|jP$rtEd|zwxYw)zV Open an URL 'url' and return the file-like object of the opened URL. c4tjd|zy)z This is a small helper function for printing a warning if we cannot open the URL for some time. z^failed to open the URL with %d sec timeout, is the proxy configured correctly? Keep trying ...N)_logwarningtimeouts r_print_warningz+TransRead._open_url.._print_warning;s LL>@GH r!rN) http_client)request)URLErrorr)httphttpsr)zError parsing line z of z: :r)z User-Agentz Mozilla/5.0) Nrzcannot open URL 'r9zWcannot open URL '%s': server responds with an HTTP status code that we don't understandT),socket six.movesrsix.moves.urllibrsix.moves.urllib.errorrurlparseschemerrrnetrcauthenticatorsrFileNotFoundErrornetcNetrcParseErrorrerrorlinenofilenamemsglistport urlunparseHTTPPasswordMgrWithDefaultRealm add_passwordHTTPBasicAuthHandler build_opener addheadersinstall_openerrIrr isinstancereasonrJ ValueError InvalidURL BadStatusLiner>rBrE)rOurlrrhttpliburllibrrrrnauthenew_urlpassword_manager auth_handleropenerrf_objrQs rrMzTransRead._open_url6s   463&&s+    %   z * &&&&Z->->BS-S VKKM'' (;(;<#AwH#AwH :&G(2(;(;Z__M '00 %%g.C &EEG   ) )$Xx H!667GHL((6F((*F:;f%"G  C 9".  E"w% '' V 0 $qzzl"QUUGTUU VJ>> I&"7+c3 GHH, I&:cjj&..+Q"7+c3 GHH,Z););< E#sCDD(( @BEF sN;;G H* H'H' rTrBAttributeError)rOr:s r __getattr__zTransRead.__getattr__s4  F *4;;4<<+T2 2 r!N)r)r#r$r%r&rRr[r`rNrrMrrrrrrUrr'r!rr4r4sN 0%d'(*a4FYva#F #%++ + !r!r4)r&rrrKrFloggingrrrrrrrr getLoggerr#rSUPPORTED_COMPRESSION_TYPESrr Exceptionrr2objectr4r'r!rrsr"   .w""":<#L I