tjJdZddlmZddlmZddlmZmZeje Z edeGdd eZ edeGd d eZ edeGd d eZ gdZdS)zMllama model configuration)strict)PreTrainedConfig)auto_docstringloggingzmeta-llama/Llama-3.2-11B-Vision) checkpointceZdZUdZdZdZddiZdZee d<dZ e e d <d Z ee d <d Z ee d <dZee d<dZee d<dZee d<dZee d<dZeeezeeefze d<dZeeezeeefze d<dZee d<dZee d<dZeedze d<dZeeedze d<d Zee d!<fd"Zd#Zed$efd%ZxZ S)&MllamaVisionConfiga num_global_layers (`int`, *optional*, defaults to 8): Number of global layers in the Transformer encoder. Vision model has a second transformer encoder, called global. vision_output_dim (`int`, *optional*, defaults to 7680): Dimensionality of the vision model output. Includes output of transformer encoder with intermediate layers and global transformer encoder. max_num_tiles (`int`, *optional*, defaults to 4): Maximum number of tiles for image splitting. intermediate_layers_indices (`list[int]`, *optional*, defaults to [3, 7, 15, 23, 30]): Indices of intermediate layers of transformer encoder from which to extract and output features. These output features are concatenated with final hidden state of transformer encoder. supported_aspect_ratios (`list[list[int]]`, *optional*): List of supported aspect ratios for image splitting. If not specified, the default supported aspect ratios are [[1, 1], [1, 2], [1, 3], [1, 4], [2, 1], [2, 2], [3, 1], [4, 1]] for `max_num_tiles=4`. Example: ```python >>> from transformers import MllamaVisionConfig, MllamaVisionModel >>> # Initializing a Llama config >>> config = MllamaVisionConfig() >>> # Initializing a vision model from the mllama-11b style configuration >>> model = MllamaVisionModel(config) >>> # Accessing the model configuration >>> configuration = model.config ```mllama_vision_model vision_confignum_attention_headsattention_headsi hidden_sizegelu hidden_act num_hidden_layersnum_global_layersr num_channelsiintermediate_sizeivision_output_dimi image_size patch_sizeh㈵>norm_eps max_num_tilesNintermediate_layers_indicessupported_aspect_ratios{Gz?initializer_rangec |jddgddgddgddgddgddgddgddgg|_|j gd|_tjdi|dS)Nrr)r)r"r!super __post_init__selfkwargs __class__s m/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/mllama/configuration_mllama.pyr.z MllamaVisionConfig.__post_init__Ms  ' /-.FQFQFQFQPQFUVXYTZ]^`a\befhidj+kD (  + 3/A/A/AD ,'''''''c |jddgddgddgddgddgddgddgddggkr|jdkrtddSdS)zOPart of `@strict`-powered validation. Validates the architecture of the config.r&r'rrz;max_num_tiles must be 4 for default supported aspect ratiosN)r"r ValueErrorr0s r3validate_architecturez(MllamaVisionConfig.validate_architectureUsy  (aVaVaVaVaQRVVWYZU[^_ab]cfgijek,l l l"a''Z[[ [ m l''r4returnc*t|jS)N)lenr"r7s r3max_aspect_ratio_idz&MllamaVisionConfig.max_aspect_ratio_id]s4/000r4)!__name__ __module__ __qualname____doc__ model_typebase_config_key attribute_maprint__annotations__rstrrrrrrrrlisttuplerrfloatr r!r"r$r.r8propertyr< __classcell__r2s@r3r r s  <'J%O*,=>MKJssOSL#!s!!!!s!!!47Jd3i%S/177746Jd3i%S/1666HeM348cT!18886:T$s)_t3:::#u###(((((\\\1S111X11111r4r cheZdZUdZdZdZdZdZee d<dZ ee d<d Z e e d <d Z ee d <d Zee d<dZee d<dZee d<dZedze d<dZee d<dZee d<dZee d<dZee d<dZee d<dZeedze d<d Zeeze d!<d"Zee d#<d$Zeeezdze d%<d&Zedze d'<fd(Z xZ!S))MllamaTextConfiga cross_attention_layers (`list[int]`, *optional*): Indices of the cross attention layers. If not specified, will default to [3, 8, 13, 18, 23, 28, 33, 38]. Example: ```python >>> from transformers import MllamaTextModel, MllamaTextConfig >>> # Initializing a Mllama text config >>> config = MllamaTextConfig() >>> # Initializing a model from the Mllama text configuration >>> model = MllamaTextModel(config) >>> # Accessing the model configuration >>> configuration = model.config ```mllama_text_model text_configgA vocab_sizeirsilur(rrr rnum_key_value_headsi8rNrope_parametersr rms_norm_epsimax_position_embeddingsr#r$T use_cacheFtie_word_embeddingscross_attention_layersgdropouti bos_token_idi eos_token_idi pad_token_idc Z|j gd|_tjdi|dS)N)rr r*!&r,)r[r-r.r/s r3r.zMllamaTextConfig.__post_init__s=  & .*H*H*HD ''''''''r4)"r=r>r?r@rArB default_thetarRrDrErrrFrr rUrrVdictrWrIrXr$rYboolrZr[rGr\r]r^r_r.rKrLs@r3rNrNbs  &%J#OMJKJs!!!!    #s####'OTD['''L%#*S***#u###It %%%%/3DI,333GUS[L#+1L#S /D(111%L#*%%%(((((((((r4rNc|eZdZUdZdZddiZeedZdZ e e zdze d<dZ e e zdze d<d Zee d<fd ZxZS) MllamaConfiga Example: ```python >>> from transformers import MllamaForConditionalGeneration, MllamaConfig, MllamaVisionConfig, MllamaTextConfig >>> # Initializing a CLIP-vision config >>> vision_config = MllamaVisionConfig() >>> # Initializing a Llama config >>> text_config = MllamaTextConfig() >>> # Initializing a mllama-11b style configuration >>> configuration = MllamaConfig(vision_config, text_config) >>> # Initializing a model from the mllama-11b style configuration >>> model = MllamaForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```mllamaimage_token_idimage_token_index)rPr Nr rPrQc |j.t|_tdn0t |jt rtdi|j|_|j.t|_tdn0t |jt rtdi|j|_tj di|dS)Nz9vision_config is None, using default mllama vision configz5text_config is None, using default mllama text configr,) r r loggerinfo isinstancergrPrNr-r.r/s r3r.zMllamaConfig.__post_init__s   %!3!5!5D  KKS T T T T *D 1 1 J!3!I!Id6H!I!ID    #/11D  KKO P P P P ($ / / D/CC$2BCCD '''''''r4)r=r>r?r@rArCrNr sub_configsr rgrrErPrmrDr.rKrLs@r3rjrjs  ,J-M#3EWXXK48M4**T188826K((4/666#s### ( ( ( ( ( ( ( ( (r4rj)rjrNr N)r@huggingface_hub.dataclassesrconfiguration_utilsrutilsrr get_loggerr=ror rNrj__all__r,r4r3rxsj! ......333333,,,,,,,,  H % %<===E1E1E1E1E1)E1E1>=E1P<===.(.(.(.(.('.(.(>=.(b<===.(.(.(.(.(#.(.(>=.(b E D Dr4