tjk'NdZddlmZddlmZddlmZddlmZm Z ddl m Z m Z e j eZeGdd eZeGd d eZeGd d eZedeGddeZdZdgZdS)zESM model configuration)Union)strict)PreTrainedConfig)auto_docstringlogging)intervalis_divisible_byc@eZdZUdZdZedzed<dZedzed<dZedzed<dZ edzed <d Z edzed <d Z edzed <dZ edzed<dZ edzed<dZedzed<dZedzed<dZedzed<dZedzed<dZedzed<dZedzed<dZedzed<dS)StructureModuleConfiga Args: sequence_dim: Single representation channel dimension pairwise_dim: Pair representation channel dimension ipa_dim: IPA hidden channel dimension resnet_dim: Angle resnet (Alg. 23 lines 11-14) hidden channel dimension num_heads_ipa: Number of IPA heads num_qk_points: Number of query/key points to generate during IPA num_v_points: Number of value points to generate during IPA dropout_rate: Dropout rate used throughout the layer num_blocks: Number of structure module blocks num_transition_layers: Number of layers in the single representation transition (Alg. 23 lines 8-9) num_resnet_blocks: Number of blocks in the angle resnet num_angles: Number of angles to generate in the angle resnet trans_scale_factor: Scale of single representation transition hidden dimension epsilon: Small number used in angle resnet normalization inf: Large number used for attention masking iN sequence_dim pairwise_dimipa_dim resnet_dim num_heads_ipa num_qk_points num_v_points皙? dropout_rate num_blocksnum_transition_layersnum_resnet_blocks num_angles trans_scale_factorg:0yE>epsilongj@inf)__name__ __module__ __qualname____doc__r int__annotations__rrrrrrrfloatrrrr!r#r$r%g/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/esm/configuration_esm.pyr r sN  D #L#*""""L#*"""GS4Z Jd    "M3:""" !M3:!!! L#*   !$L%$,$$$Jd ()3:)))$%sTz%%%Jd %'d ''' GUT\   Cr.r ceZdZUdeiZdZedzed<dZedzed<e dd Z edzed <d Z edzed <d Z edzed<d Z edzed<edd Zeezdzed<dZeezdzed<dZedzed<edd Zedzed<d Zedzed<dZeedfdzed<fdZdZxZS) TrunkConfigstructure_module0Nrisequence_state_dimr)divisorr)defaultpairwise_state_dim sequence_head_widthpairwise_head_width position_binsg?)maxdropout layer_dropFcpu_grad_checkpointr)minr max_recycles chunk_sizer c |jt|_n0t|jtrtdi|j|_t jdi|dSNr-)r2r isinstancedictsuper __post_init__selfkwargs __class__s r/rIzTrunkConfig.__post_init__bso  ($9$;$;D ! ! -t 4 4 S$9$R$RDr,r?r@boolrBrCr2rrGrIrS __classcell__rMs@r/r1r1Qs%'<=KJd %)d )))%?__Q%?%?%?%L%L%Ld LLL&(t(((&(t((( "M3:""""3((s"3"3"3C"@"@"@GUS[4 @@@%(J d"(((',,,,.xxAq999L#*999 Jd    DHeD"99:TAHHH(((((r.r1ceZdZUdeiZdZedzed<dZe dzed<dZ e dzed<dZ e dzed<dZ e dzed <d Z eezdzed <dZe dzed <dZe dzed <dZedzed<dZeedfdzed<fdZxZS) EsmFoldConfigtrunkNesm_typeTfp16_esmFuse_esm_attn_mapesm_ablate_pairwiseesm_ablate_sequencer=esm_input_dropoutembed_aa bypass_lmrlddt_head_hid_dimr1c |jt|_n0t|jtrtdi|j|_t jdi|dSrE)rZr1rFrGrHrIrJs r/rIzEsmFoldConfig.__post_init__sf : $DJJ  D ) ) 3$22tz22DJ'''''''r.)r&r'r(r1rTr[strr+r\rUr]r^r_r`r,r*rarbrcrZrrGrIrVrWs@r/rYrYs!K(KHcDj HdTk   $)dTk)))',,,,',,,,,/us{T)/// HdTk   "Itd{"""$'sTz'''/3E5}$ % ,333(((((((((r.rYzfacebook/esm-1b) checkpointc(eZdZUdZdZdeiZdZedze d<dZ edze d<dZ edze d<dZ ee d <d Z ee d <d Zee d <d Zee d<dZedze d<dZedze d<dZee d<dZee d<dZee d<dZedze d<dZedze d<dZee d<dZedze d<dZedze d <dZedze d!<dZeezdze d<dZ e!ee"ed"fzdze d#<dZ#edze d$<dZ$edze d%<dZ%ee d&<dZ&edze d'<d(Z'ee!ezdze d)<fd*Z(xZ)S)+ EsmConfiga mask_token_id (`int`, *optional*): The index of the mask token in the vocabulary. This must be included in the config because of the "mask-dropout" scaling trick, which will scale the inputs depending on the number of masked tokens. rope_theta (`float`, defaults to 10000.0): The base period of the RoPE embeddings. Only used when `position_embedding_type` is set to `"rotary"`. position_embedding_type (`str`, *optional*, defaults to `"absolute"`): Type of position embedding. Choose either `"absolute"` or "rotary"`. emb_layer_norm_before (`bool`, *optional*): Whether to apply layer normalization after embeddings but before the main stem of the network. token_dropout (`bool`, defaults to `False`): When this is enabled, masked tokens are treated as if they had been dropped out by input dropout. is_folding_model (`bool`, defaults to `False`): When this is enabled, ESMFold model will be initialized. esmfold_config (`dict`, *optional*): Configuration to initiate the ESMFold module. vocab_list (`list`, *optional*): List of the vocabulary items. Examples: ```python >>> from transformers import EsmModel, EsmConfig >>> # Initializing a ESM facebook/esm-1b style configuration >>> configuration = EsmConfig(vocab_size=33) >>> # Initializing a model from the configuration >>> model = EsmModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```esmesmfold_configN vocab_size mask_token_id pad_token_idi hidden_sizernum_hidden_layersnum_attention_headsi intermediate_sizerhidden_dropout_probattention_probs_dropout_probimax_position_embeddingsg@ rope_thetag{Gz?initializer_rangeg-q=layer_norm_epsabsoluteposition_embedding_typeT use_cacheemb_layer_norm_beforeF token_dropoutis_folding_model. vocab_list is_decoderadd_cross_attentiontie_word_embeddings bos_token_idr eos_token_idc |jr|j.tdt |_n0t |jt rt di|j|_|j-tdt|_nd|_d|_|j%t|jddrtdtj di|dS)NzCNo esmfold_config supplied for folding model, using default values.zHNo vocab_list supplied for folding model, assuming the ESM-2 vocabulary!r]FzOThe HuggingFace port of ESMFold does not support use_esm_attn_map at this time!r-)r}rjloggerinforYrFrGr~warningget_default_vocab_listgetattrrPrHrIrJs r/rIzEsmConfig.__post_init__s  #"* abbb&3oo##D/66 K&3&J&Jd6I&J&J#&ijjj"8":":"&D "DO   *wt7JL^`e/f/f *noo o'''''''r.)*r&r'r(r) model_typerYrTrkr*r+rlrmrnrorprqrrr,rsrtrurvrwryrerzrUr{r|r}rjrGr~listtuplerrrrrrIrVrWs@r/rhrhsS   DJ#]3K!Jd !!! $M3:$$$#L#*###Ks!!!!!s!!!(++++14 %$,444#'S'''J#u####(NEDL(((*4S4Z444It)-4$;---!&M4$;&&&$)dTk)))26ND=(4/66659JS E#s(O+d2999#Jt ###',,,, $$$$#L#*###+,L#S /D(,,,(((((((((r.rhcdS)N)!zzzzLAGVSERTIDPKQNFYMHWCXBUZOrO-zzr-r-r.r/rrs " " r.N)r)typingrhuggingface_hub.dataclassesrconfiguration_utilsrutilsrrutils.type_validatorsr r get_loggerr&rr r1rYrhr__all__r-r.r/rs......333333,,,,,,,,>>>>>>>>  H % %11111,111h/////"///d((((($(((,,---R(R(R(R(R( R(R(.-R(j###L -r.