}tjddlZddlZddlmZddlmZmZddlZddlm Z ddl m Z ddl m Z mZmZmZmZmZmZddlmZmZmZmZmZdd lmZdd lmZdd lmZm Z m!Z!e j"e#Z$ee%j&d z d z Z'ee%j&d z dz Z(ej)dZ*GddZ+Gdde+Z,Gdde+Z-Gdde+Z.de/dedfdZ0de/ezde1dzfdZ2de/ezde1ddfdZ3dd ddddd!d"e/d#e/d$e/d%e/d&e/d'ed(e/d)e/d*e/dzd+e4d,e/dzd-e/dzd.e/dzd/e/dzde1fd0Z5e!dd ddddd dd1d2e/d3e1d4e/dzd5e4d6e/dzd7e/dzd8e/dzd9e/dzd:e4d;e/dzde/fd<Z6dS)=N)Path)AnyLiteral)hf_hub_download) upload_file)CardDataDatasetCardData EvalResult ModelCardData SpaceCardDataeval_results_to_model_indexmodel_index_to_eval_results)HfHubHTTPError get_sessionhf_raise_for_statusis_jinja_available yaml_dump) constants)EntryNotFoundError)SoftTemporaryDirectoryloggingvalidate_hf_hub_args templateszmodelcard_template.mdzdatasetcard_template.mdzE^(\s*---(?:\r\n|\r|\n))([\S\s]*?)((?:\r\n|\r|\n)---[ \t]*(\r\n|\n|$))cneZdZeZeZdZddede fdZ e dZ e j defdZ dZd eezfd Ze dd eezd ed zded zde fdZdd ed zfdZ ddeded zd ed zded zded zded zde d zded zfdZe d deded zded zfdZd S)!RepoCardmodelFcontentignore_metadata_errorsc"||_||_dS)aInitialize a RepoCard from string content. The content should be a Markdown file with a YAML block at the beginning and a Markdown body. Args: content (`str`): The content of the Markdown file. Example: ```python >>> from huggingface_hub.repocard import RepoCard >>> text = ''' ... --- ... language: en ... license: mit ... --- ... ... # My repo ... ''' >>> card = RepoCard(text) >>> card.data.to_dict() {'language': 'en', 'license': 'mit'} >>> card.text '\n# My repo\n' ``` > [!TIP] > Raises the following error: > > - [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) > when the content of the repo card metadata is not a dictionary. N)rr)selfrrs V/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/huggingface_hub/repocard.py__init__zRepoCard.__init__*sD'=# ct|jpd}d||j||j|d||jS)zLThe content of the RepoCard, including the YAML block and the Markdown body. ---) line_breakoriginal_order)_detect_line_ending_contentdatato_yaml_original_ordertext)r!r(s r"rzRepoCard.contentOs)77?4 WZW!2!2jY]Ym!2!n!nWpzWW@JWLPLUWW Wr$c||_t|}|rs|d}||d|_t j|}|i}t|tstdn#t di}||_|j di|d|ji|_t!||_dS)z Set the content of the RepoCard.N)repo card metadata block should be a dictzBRepo card metadata block was not found. Setting CardData to empty.r)r+REGEX_YAML_BLOCKsearchgroupendr/yaml safe_load isinstancedict ValueErrorloggerwarningcard_data_classrr,listkeysr.)r!rmatch yaml_block data_dicts r"rzRepoCard.contentUs   ''00  QJ .DIz22I  i.. N !LMMM N NN_ ` ` `IDI(D(ii9iiTMhiii #INN$4$455r$c|jSN)r)r!s r"__str__zRepoCard.__str__ps |r$filepathct|}|jddt|ddd5}|t |ddddS#1swxYwYdS)a{Save a RepoCard to a file. Args: filepath (`Union[Path, str]`): Filepath to the markdown file to save. Example: ```python >>> from huggingface_hub.repocard import RepoCard >>> card = RepoCard("---\nlanguage: en\n---\n# This is a test repo card") >>> card.save("/tmp/test.md") ``` T)parentsexist_okwutf-8modenewlineencodingN)rparentmkdiropenwritestr)r!rHfs r"savez RepoCard.savess>>dT::: (b7 C C C q GGCII                     s#A//A36A3Nrepo_id_or_path repo_typetokenct|rt|}nZt|tr2tt |t j|p|j|}ntd|d| ddd5}|| |cd d d S#1swxYwYd S) aInitialize a RepoCard from a Hugging Face Hub repo's README.md or a local filepath. Args: repo_id_or_path (`Union[str, Path]`): The repo ID associated with a Hugging Face Hub repo or a local filepath. repo_type (`str`, *optional*): The type of Hugging Face repo to push to. Defaults to None, which will use "model". Other options are "dataset" and "space". Not used when loading from a local filepath. If this is called from a child class, the default value will be the child class's `repo_type`. token (`str`, *optional*): Authentication token, obtained with `huggingface_hub.HfApi.login` method. Will default to the stored token. ignore_metadata_errors (`str`): If True, errors while parsing the metadata section will be ignored. Some information might be lost during the process. Use it at your own risk. Returns: [`huggingface_hub.repocard.RepoCard`]: The RepoCard (or subclass) initialized from the repo's README.md file or filepath. Example: ```python >>> from huggingface_hub.repocard import RepoCard >>> card = RepoCard.load("nateraw/food") >>> assert card.data.tags == ["generated_from_trainer", "image-classification", "pytorch"] ``` )r[r\z.Cannot load RepoCard: path not found on disk (z).rrMrNrO)rN) ris_filer:rWrr REPOCARD_NAMEr[r<rUread)clsrZr[r\r card_pathrXs r"loadz RepoCard.loadsBH  ( ( * * c_--II  - - c#+'83= IIaoaaabb b^^b7^ C C Pq3qvvxx8NOOO P P P P P P P P P P P P P P P P P Ps$CCCc|p|j}|t|d}ddi} td||}t |dS#t $r&}|jdkrt|j|d}~wwxYw)aValidates card against Hugging Face Hub's card validation logic. Using this function requires access to the internet, so it is only called internally by [`huggingface_hub.repocard.RepoCard.push_to_hub`]. Args: repo_type (`str`, *optional*, defaults to "model"): The type of Hugging Face repo to push to. Options are "model", "dataset", and "space". If this function is called from a child class, the default will be the child class's `repo_type`. > [!TIP] > Raises the following errors: > > - [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) > if the card fails validation checks. > - [`HTTPError`](https://requests.readthedocs.io/en/latest/api/#requests.HTTPError) > if the request to the Hub API fails for any other reason. )repoTyperAcceptz text/plainz(https://huggingface.co/api/validate-yaml)jsonheadersiN) r[rWrpostrr status_coder<r/)r!r[bodyriresponseexcs r"validatezRepoCard.validates(/ "4yy  \* "}}))*T[_ip)qqH  ) ) ) ) )   #s** ///  s3A B !BBrepo_idcommit_messagecommit_descriptionrevision create_pr parent_commitc p|p|j}||t5} t| tjz } | t|dtt| tj|||||||| } dddn #1swxYwY| S)aBPush a RepoCard to a Hugging Face Hub repo. Args: repo_id (`str`): The repo ID of the Hugging Face Hub repo to push to. Example: "nateraw/food". token (`str`, *optional*): Authentication token, obtained with `huggingface_hub.HfApi.login` method. Will default to the stored token. repo_type (`str`, *optional*, defaults to "model"): The type of Hugging Face repo to push to. Options are "model", "dataset", and "space". If this function is called by a child class, it will default to the child class's `repo_type`. commit_message (`str`, *optional*): The summary / title / first line of the generated commit. commit_description (`str`, *optional*) The description of the generated commit. revision (`str`, *optional*): The git revision to commit from. Defaults to the head of the `"main"` branch. create_pr (`bool`, *optional*): Whether or not to create a Pull Request with this commit. Defaults to `False`. parent_commit (`str`, *optional*): The OID / SHA of the parent commit, as a hexadecimal string. Shorthands (7 first characters) are also supported. If specified and `create_pr` is `False`, the commit will fail if `revision` does not point to `parent_commit`. If specified and `create_pr` is `True`, the pull request will be created from `parent_commit`. Specifying `parent_commit` ensures the repo has not changed before committing the changes, and can be especially useful if the repo is updated / committed too concurrently. Returns: `str`: URL of the commit which updated the card metadata. )r[rN)rR) path_or_fileobj path_in_reporpr\r[rqrrrtrsruN) r[rorrrr` write_textrWr) r!rpr\r[rqrrrsrtrutmpdirtmp_pathurls r" push_to_hubzRepoCard.push_to_hubsR/  *** # % % F||i&==H   D G  < < < #H &4#-#5#!+   C                sA1B++B/2B/ card_data template_path template_strc trddl}ntd|}|||!t |}|&t |j}| |}|j dd| i|}||S)a"Initialize a RepoCard from a template. By default, it uses the default template. Templates are Jinja2 templates that can be customized by passing keyword arguments. Args: card_data (`huggingface_hub.CardData`): A huggingface_hub.CardData instance containing the metadata you want to include in the YAML header of the repo card on the Hugging Face Hub. template_path (`str`, *optional*): A path to a markdown file with optional Jinja template variables that can be filled in with `template_kwargs`. Defaults to the default template. template_str (`str`, *optional*): A raw Jinja template string with optional variables. Used when neither `template_path` nor the default template is appropriate. Ignored if `template_path` is also provided. Returns: [`huggingface_hub.repocard.RepoCard`]: A RepoCard instance with the specified card data and content from the template. rNzjUsing RepoCard.from_template requires Jinja2 to be installed. Please install it with `pip install Jinja2`.r~r3) rjinja2 ImportErrorto_dictcopyupdater read_textdefault_template_pathTemplaterenderr-) rbr~rrtemplate_kwargsrkwargstemplaters r" from_templatezRepoCard.from_template!s6     MMMM9  ""$$))++ o&&&  $ ..88::L   9::DDFFL??<00!(/JJI,=,=,?,?J6JJs7||r$)F)NNFrF)NNNNNNNNN)__name__ __module__ __qualname__rr?TEMPLATE_MODELCARD_PATHrr[rWboolr#propertyrsetterrGrrY classmethodrdror}rr3r$r"rr%s>O3I###T####JWWXW  ^6s666^64TCZ(!% ', 3P3Pt3P:3PTz 3P !% 3P3P3P[3Pj###*####P! $%))-#!%$(===Tz=: = d =  $J =*=$;=Tz====~%)#' +++Tz+Dj +++[+++r$rc ZeZdZeZeZdZe ddede dzde dzffd Z xZ S) ModelCardrNr~rrc <tj|||fi|S)a Initialize a ModelCard from a template. By default, it uses the default template, which can be found here: https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md Templates are Jinja2 templates that can be customized by passing keyword arguments. Args: card_data (`huggingface_hub.ModelCardData`): A huggingface_hub.ModelCardData instance containing the metadata you want to include in the YAML header of the model card on the Hugging Face Hub. template_path (`str`, *optional*): A path to a markdown file with optional Jinja template variables that can be filled in with `template_kwargs`. Defaults to the default template. template_str (`str`, *optional*): A raw Jinja template string with optional variables. Used when neither `template_path` nor the default template is appropriate. Ignored if `template_path` is also provided. Returns: [`huggingface_hub.ModelCard`]: A ModelCard instance with the specified card data and content from the template. Example: ```python >>> from huggingface_hub import ModelCard, ModelCardData, EvalResult >>> # Using the Default Template >>> card_data = ModelCardData( ... language='en', ... license='mit', ... library_name='timm', ... tags=['image-classification', 'resnet'], ... datasets=['beans'], ... metrics=['accuracy'], ... ) >>> card = ModelCard.from_template( ... card_data, ... model_description='This model does x + y...' ... ) >>> # Including Evaluation Results >>> card_data = ModelCardData( ... language='en', ... tags=['image-classification', 'resnet'], ... eval_results=[ ... EvalResult( ... task_type='image-classification', ... dataset_type='beans', ... dataset_name='Beans', ... metric_type='accuracy', ... metric_value=0.9, ... ), ... ], ... model_name='my-cool-model', ... ) >>> card = ModelCard.from_template(card_data) >>> # Using a Custom Template >>> card_data = ModelCardData( ... language='en', ... tags=['image-classification', 'resnet'] ... ) >>> card = ModelCard.from_template( ... card_data=card_data, ... template_path='./src/huggingface_hub/templates/modelcard_template.md', ... custom_template_var='custom value', # will be replaced in template if it exists ... ) ``` superrrbr~rrr __class__s r"rzModelCard.from_templateUs)X%uww$Y |_____r$r) rrrr r?rrr[rrWr __classcell__rs@r"rrPs#O3I%)#' K`K` K`TzK`Dj K`K`K`K`K`[K`K`K`K`K`r$rc ZeZdZeZeZdZe ddede dzde dzffd Z xZ S) DatasetCarddatasetNr~rrc <tj|||fi|S)a Initialize a DatasetCard from a template. By default, it uses the default template, which can be found here: https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/datasetcard_template.md Templates are Jinja2 templates that can be customized by passing keyword arguments. Args: card_data (`huggingface_hub.DatasetCardData`): A huggingface_hub.DatasetCardData instance containing the metadata you want to include in the YAML header of the dataset card on the Hugging Face Hub. template_path (`str`, *optional*): A path to a markdown file with optional Jinja template variables that can be filled in with `template_kwargs`. Defaults to the default template. template_str (`str`, *optional*): A raw Jinja template string with optional variables. Used when neither `template_path` nor the default template is appropriate. Ignored if `template_path` is also provided. Returns: [`huggingface_hub.DatasetCard`]: A DatasetCard instance with the specified card data and content from the template. Example: ```python >>> from huggingface_hub import DatasetCard, DatasetCardData >>> # Using the Default Template >>> card_data = DatasetCardData( ... language='en', ... license='mit', ... annotations_creators='crowdsourced', ... task_categories=['text-classification'], ... task_ids=['sentiment-classification', 'text-scoring'], ... multilinguality='monolingual', ... pretty_name='My Text Classification Dataset', ... ) >>> card = DatasetCard.from_template( ... card_data, ... pretty_name=card_data.pretty_name, ... ) >>> # Using a Custom Template >>> card_data = DatasetCardData( ... language='en', ... license='mit', ... ) >>> card = DatasetCard.from_template( ... card_data=card_data, ... template_path='./src/huggingface_hub/templates/datasetcard_template.md', ... custom_template_var='custom value', # will be replaced in template if it exists ... ) ``` rrs r"rzDatasetCard.from_templates)x%uww$Y |_____r$r) rrrr r?TEMPLATE_DATASETCARD_PATHrr[rrWrrrs@r"rrs%O5I%)#' ;`;`";`Tz;`Dj ;`;`;`;`;`[;`;`;`;`;`r$rceZdZeZeZdZdS) SpaceCardspaceN)rrrr r?rrr[r3r$r"rrs#O3IIIr$rrreturn) r& Nc|d}|d}|d}||zdkrdS||kr||krdS||krdSdS)zDetect the line ending of a string. Used by RepoCard to avoid making huge diff on newlines. Uses same implementation as in Hub server, keep it in sync. Returns: str: The detected line ending of the string. rr&rrN)count)rcrlfcrlfs r"r*r*sr t  B t  B == D Bw!||t rzzdbjjv Bwwttr$ local_pathc"t|}t|}|rQ|d}t j|}|t|tr|StddS)Nr1r2) rrr4r5r6r8r9r:r;r<)rrrBrCr,s r" metadata_loadrs:((**G  # #G , ,E [[^^ ~j)) <:dD11<KDEEEtr$r,cd}d}tj|rt|dd5}|}t |jtr|jd}n!t |jtr|j}dddn #1swxYwYt|ddd5}t|d| }t |}|rE|d| d ||d |z|| dz}n d ||d ||}|||ddddS#1swxYwYdS) a& Save the metadata dict in the upper YAML part Trying to preserve newlines as in the existing file. Docs about open() with newline="" parameter: https://docs.python.org/3/library/functions.html?highlight=open#open Does not work with "^M" linebreaks, which are replaced by r&rMutf8)rQrRrNrLF) sort_keysr(r')ospathexistsrUrar:newlinestuplerWrr4r5startr7rVclose)rr,r(rreadme data_yamlrBoutputs r" metadata_saversJG w~~j!!- *b6 : : : -fkkmmG&/511 -#_Q/ FOS11 -#_  - - - - - - - - - - - - - - - j#rF ; ; ; vde KKK  ''00  K_u{{}}_-0\j0\)0\0\PZ0\0\\_fglgpgpgrgrgtgt_uuFFJ:JyJJZJJJF V                   s%AB  B$'B$>B)E44E8;E8F)metrics_configmetrics_verifieddataset_config dataset_splitdataset_revisionmetrics_verification_tokenmodel_pretty_nametask_pretty_nametask_idmetrics_pretty_name metrics_id metrics_valuedataset_pretty_name dataset_idrrrrrrc^dt|t||||||||| | | | |  giS)u Creates a metadata dict with the result from a model evaluated on a dataset. Args: model_pretty_name (`str`): The name of the model in natural language. task_pretty_name (`str`): The name of a task in natural language. task_id (`str`): Example: automatic-speech-recognition. A task id. metrics_pretty_name (`str`): A name for the metric in natural language. Example: Test WER. metrics_id (`str`): Example: wer. A metric id from https://hf.co/metrics. metrics_value (`Any`): The value from the metric. Example: 20.0 or "20.0 ± 1.2". dataset_pretty_name (`str`): The name of the dataset in natural language. dataset_id (`str`): Example: common_voice. A dataset id from https://hf.co/datasets. metrics_config (`str`, *optional*): The name of the metric configuration used in `load_metric()`. Example: bleurt-large-512 in `load_metric("bleurt", "bleurt-large-512")`. metrics_verified (`bool`, *optional*, defaults to `False`): Indicates whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Automatically computed by Hugging Face, do not set. dataset_config (`str`, *optional*): Example: fr. The name of the dataset configuration used in `load_dataset()`. dataset_split (`str`, *optional*): Example: test. The name of the dataset split used in `load_dataset()`. dataset_revision (`str`, *optional*): Example: 5503434ddd753f426f4b38109466949a1217c2bb. The name of the dataset dataset revision used in `load_dataset()`. metrics_verification_token (`bool`, *optional*): A JSON Web Token that is used to verify whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Returns: `dict`: a metadata dict with the result from a model evaluated on a dataset. Example: ```python >>> from huggingface_hub import metadata_eval_result >>> results = metadata_eval_result( ... model_pretty_name="RoBERTa fine-tuned on ReactionGIF", ... task_pretty_name="Text Classification", ... task_id="text-classification", ... metrics_pretty_name="Accuracy", ... metrics_id="accuracy", ... metrics_value=0.2662102282047272, ... dataset_pretty_name="ReactionJPEG", ... dataset_id="julien-c/reactionjpeg", ... dataset_config="default", ... dataset_split="test", ... ) >>> results == { ... 'model-index': [ ... { ... 'name': 'RoBERTa fine-tuned on ReactionGIF', ... 'results': [ ... { ... 'task': { ... 'type': 'text-classification', ... 'name': 'Text Classification' ... }, ... 'dataset': { ... 'name': 'ReactionJPEG', ... 'type': 'julien-c/reactionjpeg', ... 'config': 'default', ... 'split': 'test' ... }, ... 'metrics': [ ... { ... 'type': 'accuracy', ... 'value': 0.2662102282047272, ... 'name': 'Accuracy', ... 'verified': False ... } ... ] ... } ... ] ... } ... ] ... } True ``` model-index) task_name task_type metric_name metric_type metric_value dataset_name dataset_type metric_configverified verify_tokenrrr) model_name eval_results)r r )rrrrrrrrrrrrrrs r"metadata_eval_resultr0seR 2(.% 3 *!.!4!+"0-!;#1"/%5    r$)r[ overwriter\rqrrrsrtrurpmetadatar[rr\rqrrrsrtruc b||nd}||dkrt} n.|dkrt} n |dkrt} ntd| | |||} nF#t $r9|dkrtd| t} YnwxYw|D]Y\} } | d krd | d vrt| d || d d <t| \}}| j j || j _ || j _ g| j j }|D]}d }|D]b}||rK||kr"|s td|jd|jdd}|j|_|jdur |j|_c|s| j j || j | 3|s1| j | | krtd| d| | j | <[| |||||||| S)a Updates the metadata in the README.md of a repository on the Hugging Face Hub. If the README.md file doesn't exist yet, a new one is created with metadata and the default ModelCard or DatasetCard template. For `space` repo, an error is thrown as a Space cannot exist without a `README.md` file. Args: repo_id (`str`): The name of the repository. metadata (`dict`): A dictionary containing the metadata to be updated. repo_type (`str`, *optional*): Set to `"dataset"` or `"space"` if updating to a dataset or space, `None` or `"model"` if updating to a model. Default is `None`. overwrite (`bool`, *optional*, defaults to `False`): If set to `True` an existing field can be overwritten, otherwise attempting to overwrite an existing field will cause an error. token (`str`, *optional*): The Hugging Face authentication token. commit_message (`str`, *optional*): The summary / title / first line of the generated commit. Defaults to `f"Update metadata with huggingface_hub"` commit_description (`str` *optional*) The description of the generated commit revision (`str`, *optional*): The git revision to commit from. Defaults to the head of the `"main"` branch. create_pr (`boolean`, *optional*): Whether or not to create a Pull Request from `revision` with that commit. Defaults to `False`. parent_commit (`str`, *optional*): The OID / SHA of the parent commit, as a hexadecimal string. Shorthands (7 first characters) are also supported. If specified and `create_pr` is `False`, the commit will fail if `revision` does not point to `parent_commit`. If specified and `create_pr` is `True`, the pull request will be created from `parent_commit`. Specifying `parent_commit` ensures the repo has not changed before committing the changes, and can be especially useful if the repo is updated / committed too concurrently. Returns: `str`: URL of the commit which updated the card metadata. Example: ```python >>> from huggingface_hub import metadata_update >>> metadata = {'model-index': [{'name': 'RoBERTa fine-tuned on ReactionGIF', ... 'results': [{'dataset': {'name': 'ReactionGIF', ... 'type': 'julien-c/reactiongif'}, ... 'metrics': [{'name': 'Recall', ... 'type': 'recall', ... 'value': 0.7762102282047272}], ... 'task': {'name': 'Text Classification', ... 'type': 'text-classification'}}]}]} >>> url = metadata_update("hf-internal-testing/reactiongif-roberta-card", metadata) ``` Nz$Update metadata with huggingface_hubrrrzUnknown repo_type: )r\r[zJCannot update metadata on a Space that doesn't contain a `README.md` file.rnamerrFz6You passed a new value for the existing metric 'name: z, type: z6'. Set `overwrite=True` to overwrite existing metrics.Tz9You passed a new value for the existing meta data field 'z7'. Set `overwrite=True` to overwrite existing metadata.)r\r[rqrrrtrsru)rrrr<rdrrritemsgetattrrr,rris_equal_except_valuerrrrrappendgetr})rprr[rr\rqrrrsrtru card_classcardkeyvaluer new_resultsexisting_results new_result result_foundexisting_results r"metadata_updatersH(6'A^^GmNI00 i   g   :y::;;;4weyII 444   ijj j'' 33 4nn&&+'+' U -  U1X%%#*4w#G#Ga &A%&H&H #J y%-)4 &'1 $$#'9#9 #.BBJ#(L+; W W%;;OLL W)_<rs  999999......rqqqqqqqqqqqqq&&&&&&HHHHHHHHHH  H % %$x../+=@WW DNN1K?B[[2:fgghhhhhhhhV Q`Q`Q`Q`Q`Q`Q`Q`hA`A`A`A`A`(A`A`A`H 1I)J* cDj TD[    cDjT"&"!% $#'-1}}}}} }  }  }}}}$J}}$J}:}Dj}!$d }  !}}}}@ !!%%) $RRR RRTz R  R : R$JRd RDjRR:R RRRRRRr$