tjdZddlmZddlmZddlmZedeGdd eZd gZd S) zOpenAI GPT-2 configuration)strict)PreTrainedConfig)auto_docstringzopenai-community/gpt2) checkpointceZdZUdZdZdgZdddddZd Zee d <d Z ee d<d Z ee d<d Z ee d<d Z ee d<dZedze d<dZee d<dZeeze d<dZeeze d<dZeeze d<dZee d<dZee d<dZee d<dZee d<dZedze d<dZee d<dZeeze d <dZee d!<dZee d"<d#Zedze d$<d#Z ee!ezdze d%<dZ"edze d&<d'Z#ee d(<d'Z$ee d)<d'Z%ee d*<dZ&ee d+<dS), GPT2ConfigaH summary_type (`string`, *optional*, defaults to `"cls_index"`): Argument used when doing sequence summary, used in the models [`GPT2DoubleHeadsModel`]. Has to be one of the following options: - `"last"`: Take the last token hidden state (like XLNet). - `"first"`: Take the first token hidden state (like BERT). - `"mean"`: Take the mean of all tokens hidden states. - `"cls_index"`: Supply a Tensor of classification token position (like GPT/GPT-2). - `"attn"`: Not implemented now, use multi-head attention. summary_use_proj (`bool`, *optional*, defaults to `True`): Argument used when doing sequence summary, used in the models [`GPT2DoubleHeadsModel`]. Whether or not to add a projection after the vector extraction. summary_activation (`str`, *optional*): Argument used when doing sequence summary. Used in for the multiple choice head in [`GPT2DoubleHeadsModel`]. Pass `"tanh"` for a tanh activation to the output, any other value will result in no activation. summary_proj_to_labels (`bool`, *optional*, defaults to `True`): Argument used when doing sequence summary, used in the models [`GPT2DoubleHeadsModel`]. Whether the projection outputs should have `config.num_labels` or `config.hidden_size` classes. summary_first_dropout (`float`, *optional*, defaults to 0.1): Argument used when doing sequence summary, used in the models [`GPT2DoubleHeadsModel`]. The dropout ratio to be used after the projection and activation. scale_attn_by_inverse_layer_idx (`bool`, *optional*, defaults to `False`): Whether to additionally scale attention weights by `1 / layer_idx + 1`. reorder_and_upcast_attn (`bool`, *optional*, defaults to `False`): Whether to scale keys (K) prior to computing attention (dot-product) and upcast attention dot-product/softmax to float() when training with mixed precision. Example: ```python >>> from transformers import GPT2Config, GPT2Model >>> # Initializing a GPT2 configuration >>> configuration = GPT2Config() >>> # Initializing a model (with random weights) from the configuration >>> model = GPT2Model(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```gpt2past_key_valuesn_embd n_positionsn_headn_layer) hidden_sizemax_position_embeddingsnum_attention_headsnum_hidden_layersiQ vocab_sizeii Nn_innergelu_newactivation_functiong? resid_pdrop embd_pdrop attn_pdropgh㈵>layer_norm_epsilong{Gz?initializer_range cls_index summary_typeTsummary_use_projsummary_activationsummary_proj_to_labelssummary_first_dropoutscale_attn_weights use_cacheiP bos_token_id eos_token_id pad_token_idFscale_attn_by_inverse_layer_idxreorder_and_upcast_attnadd_cross_attentiontie_word_embeddings)'__name__ __module__ __qualname____doc__ model_typekeys_to_ignore_at_inference attribute_maprint__annotations__r r rrrrstrrfloatrrrrrr boolr!r"r#r$r%r&r'listr(r)r*r+r,i/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/gpt2/configuration_gpt2.pyr r s ) ) VJ#4"5#0'& MJKFCGSFCGS4Z))))"K"""!J !!!!J !!! $$$$#u####L####!d!!!%)d )))#'D'''),53;,,,####It$L#*$$$+0L#S /D(000#L#*###,1#T111$)T))) %%%% $$$$$$r;r N) r0huggingface_hub.dataclassesrconfiguration_utilsrutilsrr __all__r:r;r<rAs! ......333333######2333N%N%N%N%N%!N%N%43N%b .r;