}tjP dZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z ddl m Z ejeZdZeGdd Zed Gd d ZeGddZeGddZdeddfdZde dedefdZde dedefdZde dededzfdZde dedefdZde dedededdf dZde de fdZde ddfdZdedefd ZdS)!u$Contains utilities to handle the `../.cache/huggingface` folder in local directories. First discussed in https://github.com/huggingface/huggingface_hub/issues/1738 to store download metadata when downloading files from the hub to a local directory (without using the cache). ./.cache/huggingface folder structure: [4.0K] data ├── [4.0K] .cache │ └── [4.0K] huggingface │ └── [4.0K] download │ ├── [ 16] file.parquet.metadata │ ├── [ 16] file.txt.metadata │ └── [4.0K] folder │ └── [ 16] file.parquet.metadata │ ├── [6.5G] file.parquet ├── [1.5K] file.txt └── [4.0K] folder └── [ 16] file.parquet Download metadata file structure: ``` # file.txt.metadata 11c5a3d5811f50298f278a704980280950aedb10 a16a55fda99d2f2e7b69cce5cf93ff4ad3049930 1712656091.123 # file.parquet.metadata 11c5a3d5811f50298f278a704980280950aedb10 7c5d3f4b8b76583b422fcb9189ad6c89d5d97a094541ce8932dce3ecabde1421 1712656091.123 } ``` N) dataclass)Path PurePosixPathPureWindowsPath) WeakFileLockzSignature: 8a477f597d28d172789f06886806bc55 # This file is a cache directory tag created by huggingface_hub. # For information about cache directory tags, see: # https://bford.info/cachedir/ cBeZdZUdZeed<eed<eed<dedefdZdS) LocalDownloadFilePathsa Paths to the files related to a download process in a local dir. Returned by [`get_local_download_paths`]. Attributes: file_path (`Path`): Path where the file will be saved. lock_path (`Path`): Path to the lock file used to ensure atomicity when reading/writing metadata. metadata_path (`Path`): Path to the metadata file. file_path lock_path metadata_pathetagreturnc4|jjt|jjd|dz }t |}t jdkr:t|dkr'|dstd|z}|S)z^Return the path where a file will be temporarily downloaded before being moved to `file_path`..z .incompletent\\?\) r parent _short_hashnamestrresolveoslen startswithr)selfrpath resolved_paths [/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/huggingface_hub/_local_folder.pyincomplete_pathz&LocalDownloadFilePaths.incomplete_path\s!(k$:L:Q.R.R+f+fUY+f+f+ffDLLNN++  7d??s=11C77 @X@XYb@c@c7 M122D N)__name__ __module__ __qualname____doc__r__annotations__rr!r"r r r Hsb  OOOOOOCDr"r T)frozenc<eZdZUdZeed<eed<eed<eed<dS)LocalUploadFilePathsa Paths to the files related to an upload process in a local dir. Returned by [`get_local_upload_paths`]. Attributes: path_in_repo (`str`): Path of the file in the repo. file_path (`Path`): Path where the file will be saved. lock_path (`Path`): Path to the lock file used to ensure atomicity when reading/writing metadata. metadata_path (`Path`): Path to the metadata file. path_in_repor r r N)r#r$r%r&rr'rr(r"r r+r+gsH OOOOOOr"r+c<eZdZUdZeed<eed<eed<eed<dS)LocalDownloadFileMetadataaG Metadata about a file in the local directory related to a download process. Attributes: filename (`str`): Path of the file in the repo. commit_hash (`str`): Commit hash of the file in the repo. etag (`str`): ETag of the file in the repo. Used to check if the file has changed. For LFS files, this is the sha256 of the file. For regular files, it corresponds to the git hash. timestamp (`int`): Unix timestamp of when the metadata was saved i.e. when the metadata was accurate. filename commit_hashr timestampN)r#r$r%r&rr'floatr(r"r r.r.sH  MMM IIIr"r.ceZdZUdZeed<dZedzed<dZe dzed<dZ e dzed<dZ e dzed<dZ e dzed<d Ze ed <d Ze ed <d ed dfdZdS)LocalUploadFileMetadatazT Metadata about a file in the local directory related to an upload process. sizeNr1 should_ignoresha256 upload_mode remote_oidF is_uploaded is_committedpathsrc zt|j5|jd5}t j}|t |dz|t |j|d|j4|t t|j|d|j ||j |d|j ||j |d|j ||j |d|t t|j dz|t t|jdzdddn #1swxYwY||_ddddS#1swxYwYdS)zSave the metadata to disk.w N)rr r opentimewriterr5r6intr7r8r9r:r;r1)rr<f new_timestamps r savezLocalUploadFileMetadata.savesO %/ * * + +$))#.. z._validate_relative_filename..s*,c,ct_T-B-B,c,c,c,c,c,cr"zj': cannot be an absolute, drive-relative or UNC path. Please ask the repository owner to rename this file.N)replacesplit ValueErrorrdriveroot)r/ pure_paths pure_paths r _validate_relative_filenamerWs& xc**005555 C C C C    ))d,c,cx~~^aObOb,c,c,cdJ ? in GXGGG  r" local_dircBt|tjj|d}||z }t |dz |dz }|d}tjdkrt| dsttj |dkrtdtj |z}tdtj |z}tdtj |z}|j dd |j dd t||| S) aCompute paths to the files related to a download process. Folders containing the paths are all guaranteed to exist. Args: local_dir (`Path`): Path to the local directory in which files are downloaded. filename (`str`): Path of the file in the repo. Return: [`LocalDownloadFilePaths`]: the paths to the files (file_path, lock_path, metadata_path, incomplete_path). rJdownload .metadata.lockrrrTparentsexist_ok)r r r )rWrrjoinrQ_huggingface_dir with_suffixrrrrabspathrrmkdirr rXr/sanitized_filenamer r r s r get_local_download_pathsrgsg )))x~~c':':;..I$Y//* bXbbbcccc$   E""""""""@ A"""""""""""""""F 4sH'DBC9- D9C= =DC= DH' F"&F6EF F %F?FF  F H'F""H'&-G? G?>H'? H H'HH''H+.H+c <t||}t|j5|jr |j5}t |}t|}|}|dkrdntt|}|}|dkrdn|} |} | dkrdn| } | dvrtd|j d| |} | dkrdn| } tt|}tt|}t|||| | | ||}dddn #1swxYwYn#t$r}td|jd|d |jn<#t$r/}td |jd|Yd}~nd}~wwxYwt|jj cYd}~cdddSd}~wwxYw|j4|jr-|js&t1j|jz d krd |_ |j5|jj|jkr|cdddStd |dn#t6$rYnwxYwdddn #1swxYwYt|jj S)aRead metadata about a file in the local directory related to an upload process. TODO: factorize logic with `read_download_metadata`. Args: local_dir (`Path`): Path to the local directory in which files are downloaded. filename (`str`): Path of the file in the repo. Return: `[LocalUploadFileMetadata]` or `None`: the metadata if it exists, `None` otherwise. N)Nregularlfsz Invalid upload mode in metadata rm)r1r5r6r7r8r9r:r;rlrnro)r5i@Frprq)rjrr r rrr@r2rsrtrCrGrRr,r4rurvrwrxr ryst_sizer1r:r;rArzr{r|)rXr/r<rDr1r5_should_ignorer6_sha256r7 _upload_moder8 _remote_oidr9r:r;r}r~s r read_upload_metadatares #9h 7 7E eo & &@@   % % ' '? , T(--//1 %ajjll&8&8&:&: ; ;Iqzz||113344D%&ZZ\\%7%7%9%9N,:b,@,@DDd3~K^K^F_F_Mjjll0022G%,]]TTF#$::<<#5#5#7#7L*6"*<*<$$,K"*BBB()oEL^)o)obm)o)oppp"#**,,"4"4"6"6K)4):): J"&s1::<<+=+=+?+?'@'@"A"AK#'AJJLL,>,>,@,@(A(A#B#BL6"+!&3%$/#-$/%1   H-@ T T TlU-@llAlllk'..0000 kkkNN#iuOb#i#ifg#i#ijjjjjjjjk/EO4H4H4J4J4RSSSSSSSS]@@@@@@@@F T".(/ -/IKK("44y@@',$ %1eo6J6J6L6L6UYaYk6k6k#y@@@@@@@@z bXbbbcccc$    }@@@@@@@@@@@@@@@F #(<(<(>(>(F G G GGsO'I(G6I I(I I(#I $I('O'( L12&L,J32L,3 K,=%K'"L,'K,,.L,L1O',L11>O'0/O ,O  O' OO'OO''O+.O+r0rc Bt||}t|j5|jd5}||d|dt jddddn #1swxYwYddddS#1swxYwYdS)zWrite metadata about a file in the local directory related to a download process. Args: local_dir (`Path`): Path to the local directory in which files are downloaded. r>r?N)rgrr r r@rBrA)rXr/r0rr<rDs r write_download_metadatars! %Y 9 9E eo & &??  % %c * * ?a GG{==d==dikk=== > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ???????????????????s5B0A<0 B<B BB BBBcv|dz dz }|}tjdkrYtj|}t |dkr'|dst d|z}|ddt||dz }|d z }| s t|d 5| d d d d n #1swxYwYn#t$rYnt$rYnwxYw |n#t$rYnwxYw|S)zKReturn the path to the `.cache/huggingface` directory in a local directory.z.cache huggingfacerrT)r_r^z .gitignorez.gitignore.lockg?)timeout*N)rrrrcrrrrd_create_cachedir_tagrrr write_text IndexErrorOSErrorrx)rXrtargetabs_path gitignoregitignore_locks r raras x - /DF w$7??4(( x==3  x':':9'E'E )h.//F LL$L---   %I//N       nc::: * *$$S))) * * * * * * * * * * * * * * *    D    D    ! ! # # # #    D  KsN9C8 C, C8,C00C83C04C88 D DDD)) D65D6 cache_dirc|dz }|s. |tdS#t$rYdSwxYwdS)zCreate a CACHEDIR.TAG file in ``cache_dir`` if one does not already exist. The tag follows the `Cache Directory Tagging Standard `_ so that backup tools can recognize and skip cache directories. z CACHEDIR.TAGN)rrrCACHEDIR_TAG_CONTENTr)rtag_paths r rrsl >)H ??       4 5 5 5 5 5    DD s7 AActjtj|SrL)base64urlsafe_b64encodehashlibsha1encodedigestdecode)r/s r rrs?  #GL1B1B$C$C$J$J$L$L M M T T V VVr") r&rrloggingrrA dataclassesrpathlibrrrutilsr getLoggerr#rvrr r+r.r4rrWrgrjrrrrarrr(r"r rs##J  !!!!!!8888888888  8 $ $(  < $.  , -+-+-+-+-+-+-+ -+` # $    F i i i@V i i i iF"d"c">R""""J0d0c0>WZ^>^0000fRHDRHCRH