}tjBdZddlZddlZddlZddlZddlZddlmZddlm Z m Z ddl m Z ddl mZddlmZmZddlmZdd lmZmZdd lmZmZmZmZmZmZd d lmZd d l m!Z!d dl"m#Z#d dl$m%Z%m&Z&m'Z'd dl(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3d dl(m4Z5d dl6m7Z7erd dl8m9Z9m:Z:e0j;e<Z=edZ>dZ?dZ@dZAeGddZBeGddZCeGddZDdeEdeEfdZFeeDeCeBfZGd eHeGddfd!ZIe3dd"ddd#d$eHeDd%eEd&eEd'eJeEeEfd(eEdzd)eKd*eEdzd+eLdzfd,ZMd$eHeDd)eKddfd-ZNe3dd"d.d/eHeJeEefd0eJeEeDfd'eJeEeEfd(eEdzd)eKf d1ZOe3dddd2d$eHeDd%eEd&eEd'eJeEeEfd(eEdzd*eEdzd+eLdzfd3ZPd4eJfd5ZQe3 dHd$e eDd%eEd&eEd'eJeEeEfd*eEd(eEdzd+eLd7eEdzddfd8ZRGd9d:eZSe3 dId;e eCd%eEd&eEd'eJeEeEfd*eEd(eEdzdeJeSedeJeSedeJeSedV # #!.77<,FUVV V   )d.>.FUVV V * ).F.Fr=)r>r?r@rArBrCrHrIrJrKrLrMr7r<rDr=r;rFrFZs:#L#*###"Kt""" $M3:$$$ HcDj!IsTz    ND   WWWWWr=rFceZdZUdZeed<eezezezed<e ddZ e ed<e dddZ e dzed <e dddZedzed <e dddZedzed <e dddZeed <e dddZeed <ddZeddedeefdZdefdZededzfdZdS)CommitOperationAdda Data structure holding necessary info to upload a file to a repository on the Hub. Args: path_in_repo (`str`): Relative filepath in the repo, for example: `"checkpoints/1fec34a/weights.bin"` path_or_fileobj (`str`, `Path`, `bytes`, or `BinaryIO`): Either: - a path to a local file (as `str` or `pathlib.Path`) to upload - a buffer of bytes (`bytes`) holding the content of the file to upload - a "file object" (subclass of `io.BufferedIOBase`), typically obtained with `open(path, "rb")`. It must support `seek()` and `tell()` methods. Raises: [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If `path_or_fileobj` is not one of `str`, `Path`, `bytes` or `io.BufferedIOBase`. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If `path_or_fileobj` is a `str` or `Path` but not a path to an existing file. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If `path_or_fileobj` is a `io.BufferedIOBase` but it doesn't support both `seek()` and `tell()`. r/path_or_fileobjF)initrepr upload_infoN)rRrSdefault _upload_mode_should_ignore _remote_oid _is_uploaded _is_committedreturncvt|j|_t|jtrt |j|_t|jt rt jt j |j|_t j |jstd|jdn5t|jtj tfstdt|jtj ri |j|jdt jn)#t$t&f$r}td|d}~wwxYwt|jt r t)j|j|_dSt|jtr t)j|j|_dSt)j|j|_dS)z6Validates `path_or_fileobj` and compute `upload_info`.zProvided path: 'z(' is not a file on the local file systemzpath_or_fileobj must be either an instance of str, bytes or io.BufferedIOBase. If you passed a file-like object, make sure it is in binary mode.rzNpath_or_fileobj is a file-like object but does not implement seek() and tell()N)r4r/r6rQr rBospathnormpath expanduserisfiler8ioBufferedIOBasebytestellseekSEEK_CUROSErrorAttributeErrorr from_pathrT from_bytes from_fileobj)r:excs r;r<z CommitOperationAdd.__post_init__s243DEE d*D 1 1 =#&t';#<#>$"677 t !rD4H!r!r!rsss tD023De2LMM #  d*B,= > >  $))+++$))!R[9999^,    d  d*C 0 0 M)3D4HIID    ,e 4 4 M)4T5IJJD   )6t7KLLD   s?>E>>F$FF$ with_tqdmc#Kt|jtst|jtri|r3t |j5}|VddddS#1swxYwYdSt |jd5}|VddddS#1swxYwYdSt|jt rtj|jVdSt|jtj rI|j }|jV|j |tj dSdS)u A context manager that yields a file-like object allowing to read the underlying data behind `path_or_fileobj`. Args: with_tqdm (`bool`, *optional*, defaults to `False`): If True, iterating over the file object will display a progress bar. Only works if the file-like object is a path to a file. Pure bytes and buffers are not supported. Example: ```python >>> operation = CommitOperationAdd( ... path_in_repo="remote/dir/weights.h5", ... path_or_fileobj="./local/weights.h5", ... ) CommitOperationAdd(path_in_repo='remote/dir/weights.h5', path_or_fileobj='./local/weights.h5') >>> with operation.as_file() as file: ... content = file.read() >>> with operation.as_file(with_tqdm=True) as file: ... while True: ... data = file.read(1024) ... if not data: ... break config.json: 100%|█████████████████████████| 8.19k/8.19k [00:02<00:00, 3.72kB/s] >>> with operation.as_file(with_tqdm=True) as file: ... httpx.put(..., data=file) config.json: 100%|█████████████████████████| 8.19k/8.19k [00:02<00:00, 3.72kB/s] ``` Nrb) r6rQrBr r#openrdrbBytesIOrcrerfSEEK_SET)r:rnfileprev_poss r;as_filezCommitOperationAdd.as_filesH d*C 0 0 =Jt?SUY4Z4Z = %d&:;;tJJJ$.55JJJ ,e 4 4 =*T122 2 2 2 2 2 ,b.? @ @ =+0022H& & & &  % %h < < < < < = =s$ AA#&A#BBBc|5}tj|cdddS#1swxYwYdS)z[ The base64-encoded content of `path_or_fileobj` Returns: `bytes` N)rvbase64 b64encodereadr:rts r; b64contentzCommitOperationAdd.b64contents \\^^ 1t#DIIKK00 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1s&AA A c|jdS|jdkr|jjS|5}t j|cdddS#1swxYwYdS)aReturn the OID of the local file. This OID is then compared to `self._remote_oid` to check if the file has changed compared to the remote one. If the file did not change, we won't upload it again to prevent empty commits. For LFS files, the OID corresponds to the SHA256 of the file content (used a LFS ref). For regular files, the OID corresponds to the SHA1 of the file content. Note: this is slightly different to git OID computation since the oid of an LFS file is usually the git-SHA1 of the pointer file content (not the actual file content). However, using the SHA256 is enough to detect changes and more convenient client-side. Nr*)rVrTsha256hexrvr"git_hashrzr{s r; _local_oidzCommitOperationAdd._local_oids   $4  % ' '#*..00 0 14|DIIKK00 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1s&A::A>A>r[N)F)r>r?r@rArBrCr rdrr rTrrV UploadModerWr7rXrYrZr<rrrvr|propertyrrDr=r;rPrPs.4Z%'(2222#eU;;;K;;; ',eUD&Q&Q&QL*t#QQQ#(%U"M"M"MND4KMMM$eUDIIIKtIIIEuEEEL$EEE %UFFFM4FFF!M!M!M!MF/=/=/=(82D/=/=/=^/=b1E11111C$J111X111r=rPr/r[c|dr |dd}|dks|dks|drtd|d|dr |d d}tD]Ftfd |dDrtd d |d G|S)Nr3r.z..z../z,Invalid `path_in_repo` in CommitOperation: ''z./c3$K|] }|kV dSNrD).0part forbiddens r; z)_validate_path_in_repo..?s(EETty EEEEEEr=zHInvalid `path_in_repo` in CommitOperation: cannot update files under a 'z/' folder (path: 'z').) startswithr8ranysplit)r/rs @r;r4r46ss##(#ABB' sld22l6M6Me6T6T2W WWWXXXt$$(#ABB' & EEEE\-?-?-D-DEEE E E '[d''!'''   r= operationsc&tt}|D]}|j}t|trh||dkrt jd|d||xxdz cc<t|jD]}|t|xxdz cc< t|tr^|tt|dkr8|j rt jd|dt jd|ddS)a Warn user when a list of operations is expected to overwrite itself in a single commit. Rules: - If a filepath is updated by multiple `CommitOperationAdd` operations, a warning message is triggered. - If a filepath is updated at least once by a `CommitOperationAdd` and then deleted by a `CommitOperationDelete`, a warning is triggered. - If a `CommitOperationDelete` deletes a filepath that is then updated by a `CommitOperationAdd`, no warning is triggered. This is usually useless (no need to delete before upload) but can happen if a user deletes an entire folder and then add new files to it. rzBAbout to update multiple times the same file in the same commit: 'z9'. This can cause undesired inconsistencies in your repo.rz_About to delete a folder containing files that have just been updated within the same commit: 'zLAbout to delete a file that have just been updated within the same commit: 'N) rintr/r6rPwarningswarnr parentsrBr.r1)rnb_additions_per_path operationr/parents r;_warn_on_overwriting_operationsrJs-8,<,<  - i!3 4 4 8$\2Q66 "%""" ", / / /1 4 / / /' 55= 8 8&c&kk222a72222 i!6 7 7 $S|)D)D%E%EFJJ& MI=IIII M9*6999/r=)endpoint num_threadsrevision create_pr additions repo_typerepo_idheadersrrrrc td|D}tr4|st|||||||dStdt ||g} i} t |tD]} d| D} td| D|||||dd d g \} }}|r1d d |D}td|| | | D]#}|| |j j <$t| dkrt!| | |||dSdS)a3 Uploads the files through the Xet protocol if possible, otherwise through the legacy LFS protocol. The Xet path does not require any Python-side sha256 computation: hashing happens inside `hf_xet` while chunking the files (single read pass) and is backfilled on the operations afterwards. c3TK|]#}t|jtjV$dSr)r6rQrbrcrops r;rz _upload_files..s3eeUWz"*OPPeeeeeer=)rrrrrrrNzcUploading files as a binary IO buffer is not supported by Xet Storage. Falling back to HTTP upload.)r) chunk_sizecg|]}|SrDrDrs r; z!_upload_files..s)))Rb)))r=cg|] }|j SrD)rTrs r;rz!_upload_files..s>>>R".>>>r=basic multipart) upload_infosrrrrrtoken transfers c g|]D}d|dd|didES)z$Encountered error for file with OID oidz: `errormessageget)rerrs r;rz!_upload_files..sbt3775>>sscggV]_aNbNbNfNfgpNqNqssr=zLFS batch API returned errors: r)actions oid2addoprrr)rr&_upload_xet_filesloggerwarning_compute_missing_sha256srUPLOAD_BATCH_MAX_NUM_FILESrjoinr8extendrTr~rlen_upload_lfs_files)rrrrrrrrhas_buffered_io_data lfs_actions lfs_oid2addopchunk chunk_list actions_chunk errors_chunk_rrs r; _upload_filesrxs$ee[deeeee #  ##!!#     F q   YK@@@@(*K35M 6PQQQ<<))5))) )<>>:>>> , * * * & |Q  Kii+G IIIJJ J=))) < .s"III0HI"IIIr=rNzComputing sha256 for z files.r) max_workersc|jjSr)rTr~rs r;z*_compute_missing_sha256s..s R^%:r=)rrinforTr~rlistmap)rr not_hashedrexecutors r;rrsIIyIIIJ :! KK@J@@@AAA :! qM % ,  4 4 4I X\\::J G GHHHIIIIIIIIIIIIIIIIIIs%B66B:=B:)rrrrcg}|D]\}|d0td|djdG||]dfd }t |dkr-td ||d dStd t |d |d t ||dt |d|tdS)a Uploads the content of `additions` to the Hub using the large file storage protocol. Relevant external documentation: - LFS Batch API: https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md Args: actions (`list[dict[str, Any]]`): LFS batch actions returned by the server. oid2addop (`dict[str, CommitOperationAdd]`): A dictionary mapping the OID of the file to the corresponding `CommitOperationAdd` object. headers (`dict[str, str]`): Headers to use for the request, including authorization headers and user agent. endpoint (`str`, *optional*): The endpoint to use for the request. Defaults to `constants.ENDPOINT`. num_threads (`int`, *optional*): The number of concurrent threads to use when uploading. Defaults to 5. Raises: [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) If an upload failed for any reason [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) Type of the repo to upload to: `"model"`, `"dataset"` or `"space"`. repo_id (`str`): A namespace (user or an organization) and a repo name separated by a `/`. headers (`dict[str, str]`): Headers to use for the request, including authorization headers and user agent. num_threads (`int`, *optional*): The number of concurrent threads to use when uploading. Defaults to 5. revision (`str`, *optional*): The git revision to upload to. Raises: [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) If an upload failed for any reason [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If the server returns malformed responses [`HfHubHTTPError`] If the LFS batch endpoint returned an HTTP error. rNzContent of file rz/ is already present upstream - skipping upload.r[c |d}t||dS#t$r}td|jd|d}~wwxYw)Nr)rlfs_batch_actionrrzError while uploading 'z ' to the Hub.)r Exception RuntimeErrorr/) batch_actionrrmrrrs r;_wrapped_lfs_uploadz._upload_lfs_files.._wrapped_lfs_upload sz i!,u"56I \SZem n n n n n n i i i^9O^^^__eh h is!& AA  ArzUploading 1 LFS file to the Hubrz Uploading z" LFS files to the Hub using up to z threads concurrentlyzUpload z LFS files)descr tqdm_classr)rrdebugr/appendrrhf_tqdm)rrrrrfiltered_actionsactionrs ``` r;rrsof,, ::i ( LLy9VE]#;#Hyyy      # #F + + + +iiiiiiii !! 6777,Q/00000 t-.. t tR] t t t      <3/00<<<#       r=)rrrc t|dkrdSddlm}m}m} m} m} ddlm} | |j ||||} |r| dz } | |}ddl | }d}dtffd } ts| }|j }nd}d |D}d |D}g}|| ||| 5}|D]I}|||t#|j|| fJ|D]<}||||j|| f= dddn #1swxYwY|D]Q\}}|jjs@t,|jj|j_Rn#t6$r |wxYw ||dSdS#||wwxYw)a Uploads the content of `additions` to the Hub using the xet storage protocol. This chunks the files and deduplicates the chunks before uploading them to xetcas storage. Args: additions (`` of `CommitOperationAdd`): The files to be uploaded. repo_type (`str`): Type of the repo (e.g. `"model"`, `"dataset"`, `"space"`). repo_id (`str`): A namespace (user or an organization) and a repo name separated by a `/`. headers (`dict[str, str]`): Headers to use for the request, including authorization headers and user agent. endpoint: (`str`, *optional*): The endpoint to use for the xetcas service. Defaults to `constants.ENDPOINT`. revision (`str`, *optional*): The git revision to upload to. create_pr (`bool`, *optional*): Whether or not to create a Pull Request with that commit. Raises: [`EnvironmentError`](https://docs.python.org/3/library/exceptions.html#EnvironmentError) If an upload failed for any reason. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If the server returns malformed responses or if the user is unauthorized to upload to xet storage. [`HfHubHTTPError`] If the LFS batch endpoint returned an HTTP error. **How it works:** The file upload system uses Xet storage, which is a content-addressable storage system that breaks files into chunks for efficient storage and transfer. ``session.new_upload_commit()`` manages uploading files by: - Registering upload tasks and starting upload immediately in the background - Breaking files into smaller chunks for efficient storage - Avoiding duplicate storage by recognizing identical chunks across files - Connecting to a storage server (CAS server) that manages these chunks Authentication works transparently: the upload commit accepts a ``token_refresh_url`` that is used to refresh the short-lived xet write token as needed. The upload process works like this: 1. Upload tasks run in parallel: 1.1. Read the file content from a path, bytes array, or a stream. 1.2. Split the file content into smaller chunks based on content patterns: each chunk gets a unique ID based on what's in it. 1.3. For each chunk: - Check if it already exists in storage. - Skip uploading chunks that already exist. 1.4. Group chunks into larger blocks for efficient transfer. 1.5. Upload these blocks to the storage server. 1.6. Assemble the file manifest locally and send it to the server for validation. 2. Return reference files that contain information about the uploaded files, which can be used later to download them. rNr) XetTokenTypeabort_xet_sessionget_xet_sessionxet_connection_info_refresh_urlxet_headers_without_auth)XetUploadProgressReporter) token_typerrrrz ?create_pr=1rcf|jjr|jjnjSr)rTrr~rCOMPUTE_SHA256)rhf_xets r; _sha256_argz&_upload_xet_files.._sha256_args./1n.Far~$((***FLaar=cFg|]}t|jt|SrD)r6rQrdrs r;rz%_upload_xet_files..s+YYY:b>PRW3X3XYYYYr=cTg|]%}t|jttf#|&SrD)r6rQrBr rs r;rz%_upload_xet_files..s1___:b>PSVX\R]3^3^____r=)token_refresh_urltoken_refresh_headerscustom_headersprogress_callback)r~)r utils._xetrrrrrutils._xet_progress_reportingrWRITErrPrupdate_progressnew_upload_commitrstart_upload_filerBrQstart_upload_bytesrTrrdfromhexresultxet_infor~KeyboardInterruptclose)rrrrrrrrrrrrr refresh_url xet_headerssessionprogressrr all_bytes_ops all_paths_opshandlescommitrhandlers @r;rr#slB 9~~IHHHHH11% K&~% **733KMMMoGHb*bbbbbb )++ %0022H ( 8   $ YYiYYY __i___ 8:  & &)")&/ '  l # p pF$<$PYdYdegYhYh$=$i$ijkkkk l l l l l l l l l l l l l l l l" W WJB>+ W(- fmmoo6N6U(V(V% W   W   NN      8  NN     sE0AF>B E F>E""F>%E"&AF>=G2>GG22H preupload_infoc~|d}t|tstd|D]}t|trZt|dt r2t|dt r |ddvstd|S)Nfilesz&preupload_info is improperly formattedr^ uploadModer)z'preupload_info is improperly formatted:)rr6rr8dictrB)rr file_infos r;_validate_preupload_infors   w ' 'E eT " "CABBBHH y$ ' ' H9==00#66 H9==66<< H<(,>>>FGG G? r=Fgitignore_contentc ||n tj}i}i} i} t|dD]} dd| Di} ||| d<td|d|d|d || ||rd d ind } t | t | }|jdid |dD| jdid|dD| jdid|dD|D]8}||j|_ | |j|_ | |j|_ 9|D]}|j j dkrd|_ dS)a Requests the Hub "preupload" endpoint to determine whether each input file should be uploaded as a regular git blob, as a git LFS blob, or as a XET file. Input `additions` are mutated in-place with the upload mode. Args: additions (`Iterable` of :class:`CommitOperationAdd`): Iterable of :class:`CommitOperationAdd` describing the files to upload to the Hub. repo_type (`str`): Type of the repo to upload to: `"model"`, `"dataset"` or `"space"`. repo_id (`str`): A namespace (user or an organization) and a repo name separated by a `/`. headers (`dict[str, str]`): Headers to use for the request, including authorization headers and user agent. revision (`str`): The git revision to upload the files to. Can be any valid git revision. gitignore_content (`str`, *optional*): The content of the `.gitignore` file to know which files should be ignored. The order of priority is to first check if `gitignore_content` is passed, then check if the `.gitignore` file is present in the list of files to commit and finally default to the `.gitignore` file already hosted on the Hub (if any). Raises: [`~utils.HfHubHTTPError`] If the Hub API returned an error. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If the Hub API response is improperly formatted. Nr,rcg|]F}|jtj|jjd|jjdGS)ascii)r^samplesize)r/rxryrTrdecoderrs r;rz'_fetch_upload_modes..s_  O$.r~/DEELLWUUN/r= gitIgnorePOST/api/s/z /preupload/r1)jsonrparamsc,i|]}|d|dS)r^rrDrrts r; z'_fetch_upload_modes..s#dddDtF|T,-?dddr=c,i|]}|d|dS)r^ shouldIgnorerDr%s r;r&z'_fetch_upload_modes..s#$l$l$lDT&\43G$l$l$lr=cFi|]}|d|dS)r^rrr%s r;r&z'_fetch_upload_modes..s(]]]T4<%]]]r=rr+rD)rENDPOINTrr rrr"updater/rVrWrXrTr)rrrrrrrr upload_modesshould_ignore_infooid_inforpayloadrespradditions r;_fetch_upload_modesr2sN$/xxY5GH+-L*,&(H 3//__       (#4GK   I Ii I I7 I Ix I I)2> eeddN[bLcdddeee!!mm$l$lTbcjTk$l$l$lmmm^^]]^T[E\]]]^^^^?? ,X-B C"4X5J"K'(=>..   $ ) )$-H !..r=cJeZdZUedzed<edzed<eed<edzed<dS) _CopySourceNrIrJr^r)r>r?r@rBrCrDr=r;r4r4sKt: IIIDjr=r4copiesr(c ddlm}m}|||}t|}i} i} d|D} t | t D]N} ||t| ||} | D]#}t||s|j| |j |f<$O| dt|dD]\\}}}}t|}|d u}|p|}|p|}|p|rd n|}d |D}t |t D]}||t|||}|D]}t||rtd t|||j |}|jr|| |<nUt|||||j }t!||}t%||j| |<|s|j| |j |f<|D]}t|||j|} | | vr)|rd|d|nd}!t+d|jd||!d| |j|f|_|s&| |j|jf|_| S)az Fetch information about the files to copy. For LFS files, we only need their metadata (file size and sha256) while for regular files we need to download the raw content from the Hub. Args: copies (`Iterable` of :class:`CommitOperationCopy`): Iterable of :class:`CommitOperationCopy` describing the files to copy on the Hub. repo_type (`str`): Type of the repo to upload to: `"model"`, `"dataset"` or `"space"`. repo_id (`str`): A namespace (user or an organization) and a repo name separated by a `/`. headers (`dict[str, str]`): Headers to use for the request, including authorization headers and user agent. revision (`str`): The git revision to upload the files to. Can be any valid git revision. Returns: `dict[_CopySource, Union[RepoFile, bytes]]]` Key is `(src_repo_id, src_repo_type, path, revision)`. For intra-repo copies, `src_repo_id` and `src_repo_type` are `None`. Value is the raw content as bytes (for regular files) or the file information as a RepoFile (for LFS files). Raises: [`~utils.HfHubHTTPError`] If the Hub API returned an error. [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If the Hub API response is improperly formatted. r)HfApi RepoFolder)rrcg|] }|j SrD)r/rs r;rz(_fetch_files_to_copy..Es333b"/333r=)rpathsrrc6|jpd|jpd|jpdfS)NrIrJrHrs r;rz&_fetch_files_to_copy..Rs$ 4"b6F6L"boNcacdr=)keyc*|j|j|jfSrr=rs r;rz&_fetch_files_to_copy..Ts0@"/Rr=Nmaincg|] }|j SrD)rGrs r;rz(_fetch_files_to_copy..\s>>>RR(>>>r=z$Copying a folder is not implemented.)rrrrfilename)rz from r r<z Cannot copy z at revision z: file is missing on repo.)hf_apir7r8rrFETCH_LFS_BATCH_SIZEget_paths_infor6blob_idr^sortr NotImplementedErrorr4r*rrrrcontentrGrr/rLrHrK)"r5rrrrrr7r8rC files_to_copyr. dest_pathsbatchdest_repo_filesrtsrc_repo_id_keysrc_repo_type_keysrc_revision_keygroupr is_cross_repo eff_repo_id eff_repo_type eff_revision src_paths paths_batchsrc_repo_files src_repo_filesourceurlresponserr> source_descs" r;_fetch_files_to_copyr^sP*))))))) UHg 6 6 6F &\\FACM9;H43F333J ,@AA ? ? //u++ 0   $ ? ?DdJ// ?26,$)X./ ?  KKddKeeeIPRRJJJ/h/hE>+-=%[[ 't3 %0 )6Y 'Rm,QFF >>:>>> ))5IJJ ] ]K#22#;''%' 3N "0 ] ] mZ88V-.TUUU$_6GI[]mnn $ =,9M&))$!)"/ +!-!.!3 C +}}00g0FFH'111,4,valuerzSkipping file 'z(' in commit (ignored by gitignore file).rr+rtrx)rIr^encodingr*lfsFiler~)r^algorr deletedFolder deletedFiler^)r^rjrz_Malformed files_to_copy (should be raw file content as bytes or RepoFile objects with LFS info.z(Unknown operation to commit. Operation: z. Upload mode: rVzSkipped z/ file(s) in commit (ignored by gitignore file).)r6rPrWrrr/rVr|rrTr~rrr.r1rFr4rIrJrGrHrdrxryr*r8getattrr) rrJr_r`ra header_valuenb_ignored_filesrrZ file_to_copys r;_prepare_commit_payloadrqsi"0B/M++SU .>PQQL '4 ^$\ 2 2222 FF i!3 4 4 9Q  LLk9+Akkk l l l  !   i!3 4 4> 9OS\9\9\(3355<<>>%2 (     #5 6 64 9;QUZ;Z;Z %2$$07;;==%16      #8 9 9) *3*=P= )"89      #6 7 7#  %'*& F )0L,.. !#)#3L#A#A#H#H#J#J ) 6$,! $ ) 6 (+/6!u ?9??I~t<<??  ! `/```aaaaar=)rrar hot_reloadretry_on_errorrrrsr'c ||n tj}t||||| } dd| D}ddi|}|d|d|d |}i}| rd |d <| rd |d <| rt d ||||}n%t ||||}t|d|}ddl m }||d|||d| r|dnd|S)u Build the ndjson payload for `operations` and POST it to the `/commit` endpoint of the Hub. `revision` must already be URL-quoted. Operations must be ready to commit: upload modes fetched and LFS/Xet content uploaded. If `retry_on_error` is `True`, the POST is retried with exponential backoff on transient errors. Caution: if a commit succeeds but its response is lost, the retry will create a duplicate commit (and a duplicate pull request if `create_pr` is set) — only enable it when this is acceptable. Returns a [`CommitInfo`] built from the server response. Raises: [`~utils.HfHubHTTPError`] If the Hub API returned an error. N)rrJr_r`rar=c3hK|]-}tj|dzV.dS) N)r"dumpsencode)ritems r;rz_send_commit..s=JJ$DJt$$++--5JJJJJJr=z Content-Typezapplication/x-ndjsonrr z/commit/r!rrrr)rrIr#r ) endpoint_namer)r' commitUrl commitOidpullRequestUrl) commit_urlr_r`rpr_url _endpoint) rr*rqrr rpostrr"rCr')rrJr_r`rrrrrrarrrrsr/datar[r#r\ commit_datar's r; _send_commitrss@$/xxY5GH%#%-# G 88JJ'JJJ J JD5AAG  D DI D D D D( D DCF"!{#"|YWdSYZZZ==%%c7DQW%XX9999--//K"""""" :{+%-  $09C{+,,t    r=)NFNr)NN)XrArxrbr"r]r collectionsrcollections.abcrrconcurrent.futuresr contextlibr dataclassesrr itertoolsr pathlibr r typingr rrrrrr<rerrorsr file_downloadrr*rrrutilsrrrrrrr r!r"r#r$r%rutils._runtimer&rCr'r( get_loggerr>rrrDDUPLICATE_LFS_BATCH_SIZErr.rFrPrBr4CommitOperationrrrrr7rrrrrr2r4rdr^rqrrDr=r;rs ######........111111%%%%%%((((((((''''''''KKKKKKKKKKKKKKKK&&&&&&%%%%%%<<<<<<<<<<                          #""""",,,,,,-,,,,,,,,  H % %% &    < 2W2W2W2W2W2W2W 2Wj b1b1b1b1b1b1b1 b1J"*,?AVVW+_0E+$++++\ !F F F &'F F  F #s(^ F Dj F F DjF d{F F F F R I-?(@ Is IW[ I I I I   P P P $sCx. !P C++,P #s(^ P Dj P  P P P P f !BBB&'BB B #s(^ B Dj BDjBd{BBBBJ T     $(R.R.*+R.R.R.#s(^ R.  R. Dj R.R.TzR. R.R.R.R.j* nn ( )nnn#s(^ n  n Dj n +uZ./ /0nnnnj&* $ ebeb)eb U:u+<%==>ebebd eb : eb d38n ebebebebd $ CCC)C U:u+<%==>C C  C  CC#s(^CCDjC:CCCCCCCCCCr=