tjԟjdZddlmZddlZddlmZddlmZmZmZddl m Z ddl m Z dd lmZdd lmZmZmZmZmZmZdd lmZmZdd lmZdd lmZddlmZm Z m!Z!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(e"j)e*Z+ej,Z-Gddej.Z/ dDdej.dej0dej0dej0dej0dzde1de1fdZ2Gddej.Z3Gd d!ej.Z4Gd"d#ej.Z5Gd$d%ej.Z6Gd&d'ej.Z7Gd(d)eZ8Gd*d+ej.Z9Gd,d-ej.Z:Gd.d/ej.Z;Gd0d1ej.Z<Gd2d3ej.Z=e Gd4d5eZ>e Gd6d7e>Z?e Gd8d9e>Z@e d:;Gd<d=e>ZAe d>;Gd?d@e>ZBe GdAdBe>ZCgdCZDdS)EzPyTorch LayoutLM model.)CallableN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingMaskedLMOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)apply_chunking_to_forward)TransformersKwargsauto_docstringcan_return_tuplelogging)merge_with_config_defaults)capture_outputs)LayoutLMConfigc4eZdZdZfdZ ddZxZS)LayoutLMEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.cNttj|j|j|j|_tj|j|j|_ tj|j |j|_ tj|j |j|_ tj|j |j|_ tj|j |j|_tj|j|j|_t#|j|j|_tj|j|_|dt1j|jdddS)N) padding_idxeps position_idsrF) persistent)super__init__r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsmax_2d_position_embeddingsx_position_embeddingsy_position_embeddingsh_position_embeddingsw_position_embeddingstype_vocab_sizetoken_type_embeddingsLayoutLMLayerNormlayer_norm_eps LayerNormDropouthidden_dropout_probdropoutregister_buffertorcharangeexpandselfconfig __class__s l/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/layoutlm/modeling_layoutlm.pyr)zLayoutLMEmbeddings.__init__3sV !|F,=v?Q_e_rsss#%<0NPVPb#c#c %'\&2SU[Ug%h%h"%'\&2SU[Ug%h%h"%'\&2SU[Ug%h%h"%'\&2SU[Ug%h%h"%'\&2H&J\%]%]"*6+=6CXYYYz&"<==   EL)GHHOOPWXXej      Nc:||}n|dd}|d}||jn|j}||jddd|f}|!tj|tj|}|||}|} ||} ||dddddf} | |dddddf} ||dddddf} | |dddddf}n"#t$r}td|d}~wwxYw| |dddddf|dddddfz }| |dddddf|dddddfz }| |}| | z| z| z| z|z|z|z|z}||}||}|S)Nr&rdtypedevicerrz:The `bbox`coordinate values should be within 0-1000 range.)sizerKr$r?zeroslongr.r0r2r3 IndexErrorr4r5r7r:r=)rC input_idsbboxtoken_type_idsr$ inputs_embeds input_shape seq_lengthrKwords_embeddingsr0left_position_embeddingsupper_position_embeddingsright_position_embeddingslower_position_embeddingser4r5r7 embeddingss rFforwardzLayoutLMEmbeddings.forwardDs  #..**KK',,..ss3K ^ %.%:!! @T  ,QQQ  ^ J J ! "& '( ((  ( (  ( $  $$ $$ $ ^^J// \\*-- s3BD<< EEE)NNNNN)__name__ __module__ __qualname____doc__r)r^ __classcell__rEs@rFrr0scQQ     &  55555555rGrmodulequerykeyvalueattention_maskscalingr=c tj||dd|z}|||z}tj|dtj|j}tj |||j }tj||} | dd } | |fS)NrLrr&)dimrJ)ptrainingr) r?matmul transposer functionalsoftmaxfloat32torJr=ro contiguous) rfrgrhrirjrkr=kwargs attn_weights attn_outputs rFeager_attention_forwardrz}s<s}}Q':':;;gEL!#n4 =((2U](SSVVW\WbccL=((6?([[L,|U33K''1--88::K  $$rGc eZdZfdZ ddejdejdzdeede ejejdzffdZ xZ S) LayoutLMSelfAttentionct|j|jzdkr0t |ds t d|jd|jd||_|j|_t|j|jz |_|j|jz|_ tj |j|j |_ tj |j|j |_ tj |j|j |_tj|j|_|j|_|jdz|_dS)Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()g)r(r)r,num_attention_headshasattr ValueErrorrDintattention_head_size all_head_sizerLinearrgrhrir;attention_probs_dropout_probr=attention_dropoutrkrBs rFr)zLayoutLMSelfAttention.__init__s:    : :a ? ?PVXhHiHi ?8F$688 4888   #)#= #&v'9FSTTTz&"<== rGr input_tensorrc||}||}|||z}|Srrr=r:rCrrs rFr^zLayoutLMSelfOutput.forward@ =11  ]33 }|'CDD rGr_r`rar)r?rr^rcrds@rFrri>>>>> U\RWR^rGrc jeZdZfdZ ddejdejdzdeedejfdZ xZ S) LayoutLMAttentionctt||_t ||_dSr)r(r)r|rCroutputrBs rFr)zLayoutLMAttention.__init__s; )&11 (00 rGNrrjrwrc \|}|j|fd|i|\}}|||}|SNrj)rCr)rCrrjrwresidual_s rFr^zLayoutLMAttention.forwardsV !$49   )    q  M8<< rGr) r_r`rar)r?rrrrr^rcrds@rFrrs1111148  | )D0 +,          rGrcBeZdZfdZdejdejfdZxZS)LayoutLMIntermediatecttj|j|j|_t|jtrt|j|_ dS|j|_ dSr) r(r)rrr,intermediate_sizer isinstance hidden_actstrr intermediate_act_fnrBs rFr)zLayoutLMIntermediate.__init__sn Yv163KLL f' - - 9'-f.?'@D $ $ $'-'8D $ $ $rGrrcZ||}||}|Sr)rrrCrs rFr^zLayoutLMIntermediate.forwards, =11 00?? rGrrds@rFrrs^99999U\elrGrcPeZdZfdZdejdejdejfdZxZS)LayoutLMOutputcttj|j|j|_tj|j|j|_tj |j |_ dSr) r(r)rrrr,rr:r9r;r<r=rBs rFr)zLayoutLMOutput.__init__sf Yv79KLL f&8f>STTTz&"<== rGrrrc||}||}|||z}|Srrrs rFr^zLayoutLMOutput.forwardrrGrrds@rFrrrrGrc peZdZfdZ d dejdejdzdeedejfdZ dZ xZ S) LayoutLMLayerct|j|_d|_t ||_t ||_t||_ dS)Nr) r(r)chunk_size_feed_forward seq_len_dimr attentionr intermediaterrrBs rFr)zLayoutLMLayer.__init__s^ '-'E$*622088$V,, rGNrrjrwrc h|j|fd|i|}t|j|j|j|}|Sr)rrfeed_forward_chunkrr)rCrrjrws rFr^zLayoutLMLayer.forwards] '   )   2  #T%A4CSUb  rGc\||}|||}|Sr)rr)rCattention_outputintermediate_output layer_outputs rFrz LayoutLMLayer.feed_forward_chunk+s2"//0@AA{{#68HII rGr) r_r`rar)r?rrrrr^rrcrds@rFrrs-----48|)D0+,   $rGrc `eZdZfdZ ddejdejdzdeede fdZ xZ S) LayoutLMEncoderct|_tjfdt jD|_d|_dS)Nc.g|]}tS)r).0irDs rF z,LayoutLMEncoder.__init__..6s!#c#c#caM&$9$9#c#c#crGF) r(r)rDr ModuleListrangenum_hidden_layerslayergradient_checkpointingrBs `rFr)zLayoutLMEncoder.__init__3s`  ]#c#c#c#c5IaCbCb#c#c#cdd &+###rGNrrjrwrc J|jD] }|||fi|} t|S)N)last_hidden_state)rr )rCrrjrw layer_modules rFr^zLayoutLMEncoder.forward9sY !J  L(LMM +    rGr) r_r`rar)r?rrrrr r^rcrds@rFrr2s,,,,,48  | )D0 +,           rGrcBeZdZfdZdejdejfdZxZS)LayoutLMPoolercttj|j|j|_tj|_dSr)r(r)rrr,rTanh activationrBs rFr)zLayoutLMPooler.__init__MsC Yv163EFF '))rGrrcr|dddf}||}||}|S)Nr)rr)rCrfirst_token_tensor pooled_outputs rFr^zLayoutLMPooler.forwardRs@+111a40 #566  66 rGrrds@rFrrLs^$$$$$ U\elrGrcBeZdZfdZdejdejfdZxZS)LayoutLMPredictionHeadTransformcVttj|j|j|_t |jtrt|j|_ n |j|_ tj |j|j |_ dSr) r(r)rrr,rrrrr transform_act_fnr:r9rBs rFr)z(LayoutLMPredictionHeadTransform.__init__]s Yv163EFF f' - - 6$*6+<$=D ! !$*$5D !f&8f>STTTrGrrc||}||}||}|Sr)rrr:rs rFr^z'LayoutLMPredictionHeadTransform.forwardfs= =11 --m<< }55 rGrrds@rFrr\scUUUUUU\elrGrc$eZdZfdZdZxZS)LayoutLMLMPredictionHeadctt||_t j|j|jd|_t j tj |j|_ dS)NT)bias) r(r)r transformrrr,r+decoder Parameterr?rNrrBs rFr)z!LayoutLMLMPredictionHead.__init__osj 8@@y!3V5FTRRR LV->!?!?@@ rGcZ||}||}|Sr)rrrs rFr^z LayoutLMLMPredictionHead.forwardxs*}55  ]33 rG)r_r`rar)r^rcrds@rFrrnsLAAAAArGrcBeZdZfdZdejdejfdZxZS)LayoutLMOnlyMLMHeadcptt||_dSr)r(r)r predictionsrBs rFr)zLayoutLMOnlyMLMHead.__init__s/ 3F;;rGsequence_outputrc0||}|Sr)r)rCrprediction_scoress rFr^zLayoutLMOnlyMLMHead.forwards ,,_==  rGrrds@rFrrs^<<<<<!u|! !!!!!!!!rGrcfeZdZUeed<dZdZeedZ e j fdZ xZ S)LayoutLMPreTrainedModelrDlayoutlmT)r attentionscvt|t|trt j|jdSt|trQt j|j tj |j j d ddSdS)zInitialize the weightsr&r%N)r( _init_weightsrrinitzeros_rrcopy_r$r?r@rrA)rCrfrEs rFrz%LayoutLMPreTrainedModel._init_weightss f%%% f6 7 7 i K $ $ $ $ $  2 3 3 i Jv*EL9L9RSU9V,W,W,^,^_f,g,g h h h h h i irG)r_r`rar__annotations__base_model_prefixsupports_gradient_checkpointingrr|_can_record_outputsr?no_gradrrcrds@rFrrs "&*#&+ U]__iiii_iiiiirGrceZdZfdZdZdZeee dde j dzde j dzde j dzde j dzd e j dzd e j dzd e e d eezfd ZxZS) LayoutLMModelct|||_t||_t ||_t||_| dSr) r(r)rDrr]rencoderrpooler post_initrBs rFr)zLayoutLMModel.__init__sg     ,V44&v.. $V,,  rGc|jjSrr]r.rCs rFget_input_embeddingsz"LayoutLMModel.get_input_embeddingss ..rGc||j_dSrr )rCris rFset_input_embeddingsz"LayoutLMModel.set_input_embeddingss*/'''rGNrQrRrjrSr$rTrwrc <||td|+||||}n.||dd}ntd||jn|j} |t j|| }|!t j|tj| }|$t j|dztj| }|dd } | |j } d | z t j |j j z} | ||||| } |j| | fi|} | d } || }t!| |S)a bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMModel >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMModel.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> outputs = model( ... input_ids=input_ids, bbox=bbox, attention_mask=attention_mask, token_type_ids=token_type_ids ... ) >>> last_hidden_states = outputs.last_hidden_state ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer&z5You have to specify either input_ids or inputs_embeds)rKrI)rrL)rJg?)rQrRr$rSrTr)r pooler_output)r%warn_if_padding_and_no_attention_maskrMrKr?onesrNrO unsqueezerurJfinfominr]rrr )rCrQrRrjrSr$rTrwrUrKextended_attention_maskembedding_outputencoder_outputsrrs rFr^zLayoutLMModel.forwardsf  ]%>cdd d  "  6 6y. Q Q Q#..**KK  &',,..ss3KKTUU U%.%:!! @T  !"Z FCCCN  !"[EJvVVVN <;{T1FSSSD"0":":1"="="G"G"J"J"9"<"<4:"<"N"N#&)@#@EKPTPZD[D[D_"_??%)' +  '$,  #     *!, O44 )-'    rG)NNNNNN)r_r`rar)r rrrrr? LongTensorrrrrr r^rcrds@rFrrs?     ///000 .2(,37260426[ [ #d*[ %[ )D0 [ (4/ [ &- [ (4/[ +,[  + +[ [ [ ^_ [ [ [ [ [ rGrceZdZdddZfdZdZdZdZee dd e j dzd e j dzd e j dzd e j dzd e j dzde j dzde j dzde edeezfdZxZS)LayoutLMForMaskedLMzcls.predictions.biasz*layoutlm.embeddings.word_embeddings.weight)zcls.predictions.decoder.biaszcls.predictions.decoder.weightct|t||_t ||_|dSr)r(r)rrrclsrrBs rFr)zLayoutLMForMaskedLM.__init__sQ    %f-- &v.. rGc$|jjjSrrr]r.r s rFr z(LayoutLMForMaskedLM.get_input_embeddings!}'77rGc$|jjjSr)rrrr s rFget_output_embeddingsz)LayoutLMForMaskedLM.get_output_embeddings$sx#++rGcT||jj_|j|jj_dSr)rrrr)rCnew_embeddingss rFset_output_embeddingsz)LayoutLMForMaskedLM.set_output_embeddings's%'5$$2$7!!!rGNrQrRrjrSr$rTlabelsrwrc <|j||f||||d|} | d} || } d} |Kt} | | d|jj|d} t | | | j| jS)a2 bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForMaskedLM >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForMaskedLM.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "[MASK]"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> labels = tokenizer("Hello world", return_tensors="pt")["input_ids"] >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=labels, ... ) >>> loss = outputs.loss ```)rjrSr$rTrNr&losslogitsrr) rrrrrDr+rrr)rCrQrRrjrSr$rTr'rwoutputsrrmasked_lm_lossloss_fcts rFr^zLayoutLMForMaskedLM.forward+sz $-   *)%'     "!* HH_55  '))H%X!&&r4;+ABB BN $!/)     rGNNNNNNN)r_r`ra_tied_weights_keysr)r r#r&rrr?rrrrrrr^rcrds@rFrrsn)?*V 888,,,888.2(,37260426*.U U #d*U %U )D0 U (4/ U &- U (4/U  4'U +,U   U U U ^U U U U U rGrz LayoutLM Model with a sequence classification head on top (a linear layer on top of the pooled output) e.g. for document image classification tasks such as the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset. ) custom_introceZdZfdZdZee ddejdzdejdzdej dzdejdzdejdzd ej dzd ejdzd e e d e e zfd ZxZS)!LayoutLMForSequenceClassificationc6t||j|_t||_t j|j|_t j |j |j|_ | dSr r(r) num_labelsrrrr;r<r=rr, classifierrrBs rFr)z*LayoutLMForSequenceClassification.__init__y     +%f-- z&"<== )F$68IJJ rGc$|jjjSrr r s rFr z6LayoutLMForSequenceClassification.get_input_embeddingsr!rGNrQrRrjrSr$rTr'rwrc |jd ||||||d|} | d} || } || } d} |Z|jjf|jdkr d|j_nN|jdkr7|jtjks|jtj kr d|j_n d|j_|jjdkrWt} |jdkr1| | | } n| | |} n|jjdkrGt} | | d|j| d} n*|jjdkrt} | | |} t| | | j| jS) aB bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForSequenceClassification >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForSequenceClassification.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> sequence_label = torch.tensor([1]) >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=sequence_label, ... ) >>> loss = outputs.loss >>> logits = outputs.logits ```rQrRrjrSr$rTrN regressionsingle_label_classificationmulti_label_classificationr&r)r)rr=r7rD problem_typer6rJr?rOrrsqueezerrrrrr)rCrQrRrjrSr$rTr'rwr,rr+r*r.s rFr^z)LayoutLMForSequenceClassification.forwardsz $- ))%'        ]33 //  {'/?a''/;DK,,_q((flej.H.HFL\a\eLeLe/LDK,,/KDK,{'<77"99?a''#8FNN$4$4fnn6F6FGGDD#8FF33DD)-JJJ+--x B @ @&++b//RR)-III,..x//'!/)     rGr/)r_r`rar)r rrr?rrrrrrr^rcrds@rFr3r3s<888.2(,37260426*.f f #d*f %f )D0 f (4/ f &- f (4/f  4'f +,f  ) )f f f ^f f f f f rGr3a3 LayoutLM Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for sequence labeling (information extraction) tasks such as the [FUNSD](https://guillaumejaume.github.io/FUNSD/) dataset and the [SROIE](https://rrc.cvc.uab.es/?ch=13) dataset. ceZdZfdZdZee ddejdzdejdzdej dzdejdzdejdzd ej dzd ejdzd e e d e e zfd ZxZS)LayoutLMForTokenClassificationc6t||j|_t||_t j|j|_t j |j |j|_ | dSrr5rBs rFr)z'LayoutLMForTokenClassification.__init__ r8rGc$|jjjSrr r s rFr z3LayoutLMForTokenClassification.get_input_embeddingsr!rGNrQrRrjrSr$rTr'rwrc \|jd||||||d|} | d} || } || } d} |Ft} | | d|j|d} t | | | j| jS)a bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForTokenClassification >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForTokenClassification.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> token_labels = torch.tensor([1, 1, 0, 0]).unsqueeze(0) # batch size of 1 >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=token_labels, ... ) >>> loss = outputs.loss >>> logits = outputs.logits ```r;rNr&r)r) rr=r7rrr6rrr)rCrQrRrjrSr$rTr'rwr,rr+r*r.s rFr^z&LayoutLMForTokenClassification.forwardsv $- ))%'     "!*,,7711  '))H8FKKDO<>> from transformers import AutoTokenizer, LayoutLMForQuestionAnswering >>> from datasets import load_dataset >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("impira/layoutlm-document-qa", add_prefix_space=True) >>> model = LayoutLMForQuestionAnswering.from_pretrained("impira/layoutlm-document-qa", revision="1e3ebac") >>> dataset = load_dataset("nielsr/funsd", split="train") >>> example = dataset[0] >>> question = "what's his name?" >>> words = example["words"] >>> boxes = example["bboxes"] >>> encoding = tokenizer( ... question.split(), words, is_split_into_words=True, return_token_type_ids=True, return_tensors="pt" ... ) >>> bbox = [] >>> for i, s, w in zip(encoding.input_ids[0], encoding.sequence_ids(0), encoding.word_ids(0)): ... if s == 1: ... bbox.append(boxes[w]) ... elif i == tokenizer.sep_token_id: ... bbox.append([1000] * 4) ... else: ... bbox.append([0] * 4) >>> encoding["bbox"] = torch.tensor([bbox]) >>> word_ids = encoding.word_ids(0) >>> outputs = model(**encoding) >>> loss = outputs.loss >>> start_scores = outputs.start_logits >>> end_scores = outputs.end_logits >>> start, end = word_ids[start_scores.argmax(-1)], word_ids[end_scores.argmax(-1)] >>> print(" ".join(words[start : end + 1])) M. Hamann P. Harper, P. Martinez ```r;rrr&)rmN) ignore_indexrL)r* start_logits end_logitsrrr) rrIsplitr@rvlenrMclamprrrr)rCrQrRrjrSr$rTrLrMrwr,rr+rPrQ total_loss ignored_indexr. start_lossend_losss rFr^z$LayoutLMForQuestionAnswering.forwards~ $- ))%'     "!*11#)<<r<#:#: j#++B//::<< ''++6688   &=+D?''))**Q.."1"9"9""="==%%''((1,, - 5 5b 9 9 (--a00M-33A}EEO)//=AAM']CCCH!,@@Jx M::H$x/14J+%!!/)     rG)T)NNNNNNNN)r_r`rar)r rrr?rrrrrrr^rcrds@rFrGrGpsW      888.2(,372604263715e e #d*e %e )D0 e (4/ e &- e (4/e )D0e '$.e +,e  - -e e e ^e e e e e rGrG)rr3rBrGrr)re)Erbcollections.abcrr?rtorch.nnrrrr r activationsr modeling_layersr modeling_outputsr r rrrrmodeling_utilsrrprocessing_utilsr pytorch_utilsrutilsrrrr utils.genericrutils.output_capturingrconfiguration_layoutlmr get_loggerr_loggerr:r8Modulerrfloatrzr|rrrrrrrrrrrrrr3rBrG__all__rrGrFrks$$$$$$ AAAAAAAAAA&&&&&&!!!!!!999999GFFFFFFF&&&&&&666666RRRRRRRRRRRR777777555555222222  H % %LIIIIIIIIh%% I% <% % < % L4' %  %%%%%.3)3)3)3)3)BI3)3)3)n         .     29        RY   .D     bi   4     RY    bi$     ry   "!!!!!")!!!iiiiioiii&p p p p p +p p p fp p p p p 1p p p f v v v v v (?v v  v rb b b b b %<b b b Jy y y y y #:y y y x   rG