tjdZddlmZddlmZddlmZedeGdd eZd gZd S) zELECTRA model configuration)strict)PreTrainedConfig)auto_docstringz"google/electra-small-discriminator) checkpointceZdZUdZdZdZeed<dZeed<dZ eed<d Z eed <d Z eed <d Z eed<dZ eed<dZeezed<dZeezed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed <dZeezed!<d"Zed#zed$<dZeed%<d#Zeezd#zed&<d'Zeed(<d'Zeed)<d#Z ed#zed*<d#Z!ee"ezd#zed+<dZ#eed,<d#S)- ElectraConfiga summary_type (`str`, *optional*, defaults to `"first"`): Argument used when doing sequence summary. Used in the sequence classification and multiple choice models. Has to be one of the following options: - `"last"`: Take the last token hidden state (like XLNet). - `"first"`: Take the first token hidden state (like BERT). - `"mean"`: Take the mean of all tokens hidden states. - `"cls_index"`: Supply a Tensor of classification token position (like GPT/GPT-2). - `"attn"`: Not implemented now, use multi-head attention. summary_use_proj (`bool`, *optional*, defaults to `True`): Argument used when doing sequence summary. Used in the sequence classification and multiple choice models. Whether or not to add a projection after the vector extraction. summary_activation (`str`, *optional*): Argument used when doing sequence summary. Used in the sequence classification and multiple choice models. Pass `"gelu"` for a gelu activation to the output, any other value will result in no activation. summary_last_dropout (`float`, *optional*, defaults to 0.0): Argument used when doing sequence summary. Used in the sequence classification and multiple choice models. The dropout ratio to be used after the projection and activation. Examples: ```python >>> from transformers import ElectraConfig, ElectraModel >>> # Initializing a ELECTRA electra-base-uncased style configuration >>> configuration = ElectraConfig() >>> # Initializing a model (with random weights) from the electra-base-uncased style configuration >>> model = ElectraModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```electrai:w vocab_sizeembedding_size hidden_size num_hidden_layersnum_attention_headsiintermediate_sizegelu hidden_actg?hidden_dropout_probattention_probs_dropout_probimax_position_embeddingstype_vocab_sizeg{Gz?initializer_rangeg-q=layer_norm_epsfirst summary_typeTsummary_use_projsummary_activationsummary_last_dropoutrN pad_token_id use_cacheclassifier_dropoutF is_decoderadd_cross_attention bos_token_id eos_token_idtie_word_embeddings)$__name__ __module__ __qualname____doc__ model_typer int__annotations__r rrrrrstrrfloatrrrrrrr boolr!r"r#r$r%r&r'r(r)listr*o/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/electra/configuration_electra.pyr r s   DJJNCKs    !s!!!J'****03 %#+333#&S&&&OS#u###!NE!!!L#!d!!!$$$$(+%#++++ L#*   It-1 d*111J %%%%#L#*###+/L#S /D(/// $$$$$$r7r N) r.huggingface_hub.dataclassesrconfiguration_utilsrutilsrr __all__r6r7r8r=s"!......333333######?@@@=%=%=%=%=%$=%=%A@=%@  r7