tj' hdZddlmZddlmZddlmZgdZeGddeZdgZ dS) z Processor class for EVOLLA. ) BatchFeature)ProcessorMixin)auto_docstring)aa_seqfoldseekmsac eZdZdfd ZddZ ddefdZe ddee e zdzd eee ee zdzd edzdedzfd Z d Z d Z dZ dZxZS)EvollaProcessorNc |td|tdt||d|j_||_||_dS)a protein_tokenizer (`EsmTokenizer`): An instance of [`EsmTokenizer`]. The protein tokenizer is a required input. protein_max_length (`int`, *optional*, defaults to 1024): The maximum length of the sequence to be generated. text_max_length (`int`, *optional*, defaults to 512): The maximum length of the text to be generated. Nz+You need to specify an `protein_tokenizer`.z"You need to specify a `tokenizer`.z<|reserved_special_token_0|>) ValueErrorsuper__init__ tokenizer pad_tokenprotein_max_lengthtext_max_length)selfprotein_tokenizerrrrkwargs __class__s j/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/evolla/processing_evolla.pyrzEvollaProcessor.__init__sj  $JKK K  ABB B *I666#A "4.c g}|D]n}|d}|d}ddt||D}||o||dd|d}|S)Nrrcfg|].\}}||z/S)upperlower).0sfs r z4EvollaProcessor.process_proteins..7s1"["["[TQ17799qwwyy#8"["["[rptT)return_tensors truncation max_lengthpadding)getjoinzipappendr) rproteinsr sa_sequencesproteinrr sa_sequence sa_tokenss rprocess_proteinsz EvollaProcessor.process_proteins2s  - -G[[**F{{:..H''"["[SQYEZEZ"["["[\\K    , , , ,** $K]gk+  rrcg}|D]4}|j|dd}||5||dddd|}|S)NFT)tokenizeadd_generation_promptr%longest)add_special_tokensr&r)r'r()rapply_chat_templater-)rtextsrpromptsmessagesprompt prompt_inputss r process_textzEvollaProcessor.process_text?s  # #H^77&*8F NN6 " " " " $& '  rr. messages_listrc ||td||n|j}||n|j}t|tr|g}t|t t fr%t|dt t fs|g}t|t t fr(td|Dstdt|t t frFtd|Ds-tddtd |t|t t fr|D]}t|t t fs td t|d td |Dstd td|Dstd|Drtd|n tdt|d | ||}|||}t|d|d|d|ddS)a proteins (`Union[List[dict], dict]`): A list of dictionaries or a single dictionary containing the following keys: - `"aa_seq"` (`str`) -- The amino acid sequence of the protein. - `"foldseek"` (`str`) -- The foldseek string of the protein. messages_list (`Union[List[List[dict]], List[dict]]`): A list of lists of dictionaries or a list of dictionaries containing the following keys: - `"role"` (`str`) -- The role of the message. - `"content"` (`str`) -- The content of the message. protein_max_length (`int`, *optional*, defaults to 1024): The maximum length of the sequence to be generated. text_max_length (`int`, *optional*, defaults to 512): The maximum length of the text. Return: a dict with following keys: - `protein_input_ids` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The input IDs for the protein sequence. - `protein_attention_mask` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The attention mask for the protein sequence. - `text_input_ids` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The input IDs for the text sequence. - `text_attention_mask` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The attention mask for the text sequence. Nz3You need to specify `messages_list` and `proteins`.c3@K|]}t|tVdSN isinstancedictr!ps r z+EvollaProcessor.__call__..s-:a:aST:a;N;N:a:a:a:a:a:arzUThe proteins should be a list of dictionaries, but not all elements are dictionaries.c3lK|]/}td|DV0dS)c3(K|] }|tvVdSrD)PROTEIN_VALID_KEYS)r!ks rrJz5EvollaProcessor.__call__...s(::A''::::::rN)allkeysrHs rrJz+EvollaProcessor.__call__..sR; ; ?@C::::: : :; ; ; ; ; ; rz2There should be a list of dictionaries with keys: z, z for each protein.But got: z;Each messages in messages_list should be a list instead of .c3@K|]}t|tVdSrDrEr!ms rrJz+EvollaProcessor.__call__..s,AA1:a..AAAAAArzfEach message in messages_list should be a list of dictionaries, but not all elements are dictionaries.c3`K|])}t|dkV*dS)N)lenrPrSs rrJz+EvollaProcessor.__call__..s5<.sSDD=>CMMfi%88DDDDDDrzlEach message in messages_list should be a list of dictionaries with two keys: 'role' and 'content'.But got: zFThe messages_list should be a list of lists of dictionaries, but it's input_idsattention_mask)protein_input_idsprotein_attention_maskr\r])data)rrrrFrGlisttuplerOr+rM TypeErrortypeanyr3r?r) rr.r@rrrr<r2 text_tokenss r__call__zEvollaProcessor.__call__Ws>  }4RSS S3E3Q//W[Wn-<-H//dNb h % % " zH mdE] 3 3 ,J}UVGWZ^`eYf"?     rc&|jj|i|SrD)r batch_decoderargsrs rrizEvollaProcessor.batch_decodes*t~*D;F;;;rc&|jj|i|SrD)rdecoderjs rrmzEvollaProcessor.decodes$t~$d5f555rc&|jj|i|SrD)rrirjs rprotein_batch_decodez$EvollaProcessor.protein_batch_decodes2t%2DCFCCCrc&|jj|i|SrD)rrmrjs rprotein_decodezEvollaProcessor.protein_decodes,t%,d=f===r)Nr r )r )r )NNNN)__name__ __module__ __qualname__rr3intr?rrarGrgrirmrorq __classcell__)rs@rr r sH//////(     #0.2>B)-&* T T t*t#d*T DJ'$t*4t;T  $J T t T T T ^T l<<<666DDD>>>>>>>rr N) __doc__feature_extraction_utilsrprocessing_utilsrutilsrrMr __all__rrrr|s544444$#####322[>[>[>[>[>n[>[>[>|  r