tj<dZddlmZddlmZddlmZddlmZm Z e j e Z edeGd d eZ edeGd d eZd gZd S)zEvolla model configuration)strict)PreTrainedConfig)RopeParameters)auto_docstringloggingzwestlake-repl/Evolla-10B-hf) checkpointceZdZUdZdZeed<dZeed<dZeed<dZ eed <d Z eed <d Z eed <dZ eed<dZ eezed<dZeezed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed <dZeed!<dZeed"<d#S)$ SaProtConfiga mask_token_id (`int`, *optional*, defaults to 4): The id of the *mask* token in the protein sequence model. position_embedding_type (`str`, *optional*, defaults to `"rotary"`): The type of position embedding to use in the protein sequence model. Currently only `"rotary"` is supported. rope_theta (`float`, *optional*, defaults to 10000.0): The base period of the RoPE embeddings. emb_layer_norm_before (`bool`, *optional*, defaults to `False`): Whether to apply layer normalization before the position embedding in the protein sequence model. token_dropout (`bool`, *optional*, defaults to `True`): Whether to apply dropout to the tokens in the protein sequence model. i vocab_size mask_token_id pad_token_idi hidden_size!num_hidden_layersnum_attention_headsiintermediate_size皙?hidden_dropout_probattention_probs_dropout_probimax_position_embeddings{Gz?initializer_rangeh㈵>layer_norm_epsrotaryposition_embedding_typeg@ rope_thetaFemb_layer_norm_beforeT token_dropout is_decoderadd_cross_attentionN)__name__ __module__ __qualname____doc__r int__annotations__rrrrrrrfloatrrrrr strr!r"boolr#r$r%m/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/evolla/configuration_evolla.pyr r sP  JM3L#Ks!!!!!s!!!'****03 %#+333#'S'''#u###!NE!!!#+S+++J"'4'''M4J %%%%%%r0r cveZdZUdZdZdeiZdZdZe e zdze d<dZ e e d<dZe e d <d Ze e d <d Ze e d <d Ze e d<dZe dze d<dZee d<dZe e d<dZee d<dZee zdze d<dZee d<dZee zdze d<dZee d<dZe dze d<dZedze d <d!Z edze d"<dZ!e dze d#<d$Z"e dze d%<d&Z#e dze d'<dZ$e dze d(<d&Z%e dze d)<dZ&e dze d*<d+Z'ee d,<dZ(e dze d-<d.Z)e dze d/<d0Z*e e+e zdze d1<dZ,ee d2<dZ-ee d3<dZ.edze d4<dZ/edze d5<fd6Z0xZ1S)7 EvollaConfiga protein_encoder_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`SaProtConfig`]. aligner_ffn_mult (`int`, *optional*, defaults to 4): The FFN multiplier for the aligner layer. aligner_enable_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in the aligner layer. aligner_attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities in the aligner layer. aligner_num_add_layers (`int`, *optional*, defaults to 8): The number of additional layers for the aligner layer. resampler_depth (`int`, *optional*, defaults to 6): The depth of the resampler layer in the llama model. resampler_dim_head (`int`, *optional*, defaults to 64): The dimension of the heads in the resampler layer in the llama model. resampler_heads (`int`, *optional*, defaults to 8): The number of heads in the resampler layer in the llama model. resampler_num_latents (`int`, *optional*, defaults to 64): The number of latents in the resampler layer in the llama model. resampler_ff_mult (`int`, *optional*, defaults to 4): The FFN multiplier for the resampler layer. Example: ```python >>> from transformers import EvollaModel, EvollaConfig >>> # Initializing a Evolla evolla-10b style configuration >>> configuration = EvollaConfig() >>> # Initializing a model from the evolla-10b style configuration >>> model = EvollaModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```evollaprotein_encoder_configgANir iri8r rrnum_key_value_headssilu hidden_acti rr rms_norm_epsrope_parametersFattention_biasgattention_dropoutmlp_biasr aligner_ffn_multTaligner_enable_biasr$aligner_attention_probs_dropout_probaligner_num_add_layersresampler_depth@resampler_dim_headresampler_headsresampler_num_latentsresampler_ff_multrrri bos_token_idi  eos_token_id use_cachetie_word_embeddingsr$r%c |j.t|_tdn0t |jt rtdi|j|_t jdi|dS)NzX`protein_encoder_config` is `None`. Initializing the `SaProtConfig` with default values.r/)r5r loggerinfo isinstancedictsuper __post_init__)selfkwargs __class__s r1rUzEvollaConfig.__post_init__s  & .*6..D ' KKr s s s s 3T : : V*6*U*U9T*U*UD ''''''''r0)2r&r'r(r) model_typer sub_configs default_thetar5rSrr+r r*rrrrr8r:r-rr;r,r<rr=r.r>r?r@rArBrCrErGrHrIrJrrrKrLlistrMrNr$r%rU __classcell__)rXs@r1r3r3>s# # JJ+\:KM=AD#33d:AAAJK"s"""s!!!!&'t'''J#'S'''L%48O^d*T1888 ND   ,/us{T)///Hd#$cDj$$$'++++9<(%$,<<<)*C$J***"#OS4Z###%'d '''"#OS4Z###(*3:***$%sTz%%%#u####L#*###%L#*%%%+1L#S /D(111It %%%%#Jt ###',,,,(((((((((r0r3N)r)huggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrr get_loggerr&rPr r3__all__r/r0r1rds'! ......333333111111,,,,,,,,  H % %8999&&&&&#&&:9&D8999P(P(P(P(P(#P(P(:9P(f  r0