Ju00#567>r%rc\rSrSr%Sr\R \S'Sr\ \ \R S4\S'Sr \ \ \R S4\S'Sr g) DPRQuestionEncoderOutput=a pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the question representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed questions for nearest neighbors queries with context embeddings. rN.rrrrrr%r&r)r)=r'r%r)c\rSrSr%Sr\R \S'Sr\ \R \S'Sr \ \R \S'Sr \ \ \R S4\S'Sr \ \ \R S4\S 'S rg) DPRReaderOutputPa start_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the start index of the span for each passage. end_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the end index of the span for each passage. relevance_logits (`torch.FloatTensor` of shape `(n_passages, )`): Outputs of the QA classifier of the DPRReader that corresponds to the scores of each passage to answer the question, compared to all the other passages. start_logitsN end_logitsrelevance_logits.rrr)rrrrrr r!r"r/rr0rr#rr$rr%r&r,r,Ps~###.2J**+248hu0018=AM8E%"3"3S"89:A:>Ju00#567>r%r,c\rSrSrSrSrSrg)DPRPreTrainedModelhTc[U[R5(akURRR SUR RS9 URb%URRR5 gg[U[R5(axURRR SUR RS9 URb2URRURR5 gg[U[R5(aJURRR5 URRRS5 gg)zInitialize the weightsg)meanstdNg?) isinstancerLinearweightdatanormal_configinitializer_rangebiaszero_ Embedding padding_idx LayerNormfill_)selfmodules r& _init_weights DPRPreTrainedModel._init_weightsls fbii ( ( MM   & &CT[[5R5R & S{{&   &&('  - - MM   & &CT[[5R5R & S!!- ""6#5#56<<>.  - - KK   " " $ MM   $ $S ).r%rN)rrrr_supports_sdparFr$rr%r&r2r2hs N*r%r2c^\rSrSrSrS\4U4SjjrSS\S\\S\\S\\S \ S \ S \ S \ \ \ \S 444Sjjr \S \4Sj5rSrU=r$) DPREncoder} bert_modelr<c>[TU]U5 [USS9UlURRR S::a [ S5eURUlURS:aD[R"URRR UR5Ul UR5 g)NF)add_pooling_layerrz!Encoder hidden_size can't be zero) super__init__rrLr< hidden_size ValueErrorprojection_dimrr8 encode_proj post_initrDr< __class__s r&rPDPREncoder.__init__s  #FeD ?? ! ! - - 2@A A$33    "!yy)?)?)K)KVMbMbcD  r% input_idsattention_masktoken_type_ids inputs_embedsoutput_attentionsoutput_hidden_states return_dictreturn.c URUUUUUUUS9nUSn U SS2SSS24n URS:aURU 5n U(d X4USS-$[U U URUR S9$)NrYrZr[r\r]r^r_rr)last_hidden_staterrr)rLrSrTr rr) rDrYrZr[r\r]r^r_outputssequence_output pooled_outputs r&forwardDPREncoder.forwards//))'/!5#" "!*'1a0    " ,,];M#3gabkA A)-'!//))   r%cURS:aURR$URRR $)Nr)rSrT out_featuresrLr<rQ)rDs r&embeddings_sizeDPREncoder.embeddings_sizes8    "##00 0%%111r%)rLrTrS)NNNFFF)rrrrbase_model_prefixrrPrrboolrr r#rgpropertyintrkr$ __classcell__rWs@r&rJrJ}s$ y ,0+/*."'%*!! ! !(! !( !  ' !  ! #! !  )5+== >! F222r%rJc^\rSrSrSrS\4U4SjjrSS\S\S\\S\ S \ S \ S \ \ \ \S 444S jjr SrU=r$)DPRSpanPredictorencoderr<c$>[TU]U5 [U5Ul[R "URR S5Ul[R "URR S5UlUR5 g)Nrr) rOrPrJrvrr8rk qa_outputs qa_classifierrUrVs r&rPDPRSpanPredictor.__init__s_  !&) ))DLL$@$@!DYYt||'C'CQG r%rYrZr\r]r^r_r`.c FUbUR5OUR5SSupxURUUUUUUS9n U Sn URU 5n U RSSS9upU R S5R 5n U R S5R 5n UR U SS2SSS245nU RXx5n U RXx5n URU5nU(d XU4U SS-$[U U UU RU RS9$)Nr)rZr\r]r^r_rr)dim)r.r/r0rr) sizervrxsplitsqueeze contiguousryviewr,rr)rDrYrZr\r]r^r_ n_passagessequence_lengthrdrelogitsr.r/r0s r&rgDPRSpanPredictor.forwardsD;D:Oinn&6UbUgUgUijlklUm# ,, )'/!5#  "!*1#)<<r<#: #++B/::< ''+668 --oaAg.FG$((E __ZA +00< .>?'!"+M M%!-!//))   r%)rvryrx)NFFF)rrrrrmrrPrrrnrr,r#rgr$rqrrs@r&rtrts!y+/"'%*!+ + +  ' +  + # + +  fck 22 3+ + r%rtc,\rSrSr%Sr\\S'SrSrSr g)DPRPretrainedContextEncoderz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. r<N ctx_encoderr rrrrrrr"load_tf_weightsrmr$rr%r&rrs O%r%rc,\rSrSr%Sr\\S'SrSrSr g)DPRPretrainedQuestionEncoderrr<Nquestion_encoderrrrr%r&rrs O*r%rc,\rSrSr%Sr\\S'SrSrSr g)DPRPretrainedReaderi rr<Nspan_predictorrrrr%r&rr s O(r%rzf The bare DPRContextEncoder transformer outputting pooler outputs as context representations. c^\rSrSrS\4U4Sjjr\SS\\S\\S\\S\\S\\ S \\ S \\ S \ \ \ \S 444S jj5r SrU=r$)DPRContextEncoderir<cp>[TU]U5 Xl[U5UlUR 5 gN)rOrPr<rJrrUrVs r&rPDPRContextEncoder.__init__s,   %f- r%rYrZr[r\r]r^r_r`.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUbUR 5nO"UbUR 5SSnO [ S5eUb UR O UR n Uc/Uc[R"XS9OXRR:gnUc$[R"U[RU S9nURUUUUUUUS9n U(dU SS$[U RU RU R S 9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRContextEncoder, DPRContextEncoderTokenizer >>> tokenizer = DPRContextEncoderTokenizer.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> model = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ```NDYou cannot specify both input_ids and inputs_embeds at the same timer|5You have to specify either input_ids or inputs_embedsdevicedtyperrbrrrr)r<r]r^use_return_dictrRr~rr ones pad_token_idzeroslongrrrrr rDrYrZr[r\r]r^r_ input_shaperrds r&rgDPRContextEncoder.forward&slb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;6;;#;#;;   !"[[EJJvVN""))'/!5## 12; &!//w?T?Tahasas  r%)r<rNNNNNNN)rrrrrrPr rrrnrrr#rgr$rqrrs@r&rrs y'++/+/*.,0/3&*X F#X !(X !( X  ' X $D> X 'tnX d^X  &fck(:: ;X X r%rzh The bare DPRQuestionEncoder transformer outputting pooler outputs as question representations. c^\rSrSrS\4U4Sjjr\SS\\S\\S\\S\\S\\ S \\ S \\ S \ \ \ \S 444S jj5r SrU=r$)DPRQuestionEncoderir<cp>[TU]U5 Xl[U5UlUR 5 gr)rOrPr<rJrrUrVs r&rPDPRQuestionEncoder.__init__s,   *6 2 r%rYrZr[r\r]r^r_r`.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UR X5 UR 5nO"UbUR 5SSnO [ S5eUb URO URn Uc/Uc[R"XS9OXRR:gnUc$[R"U[RU S9nURUUUUUUUS9n U(dU SS$[U RU R U R"S 9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRQuestionEncoder, DPRQuestionEncoderTokenizer >>> tokenizer = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> model = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ``` Nrr|rrrrbrr)r<r]r^rrR%warn_if_padding_and_no_attention_maskr~rr rrrrrr)rrrrs r&rgDPRQuestionEncoder.forwardszb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;6;;#;#;;   !"[[EJJvVN''))'/!5#( 12; '!//w?T?Tahasas  r%)r<rr)rrrrrrPr rrrnrr)r#rgr$rqrrs@r&rrs y'++/+/*.,0/3&*Y F#Y !(Y !( Y  ' Y $D> Y 'tnY d^Y  'vs{);; <Y Y r%rzE The bare DPRReader transformer outputting span predictions. c^\rSrSrS\4U4Sjjr\SS\\S\\S\\S\\ S\\ S \\ S \ \ \ \S 444S jj5r S rU=r$) DPRReaderir<cp>[TU]U5 Xl[U5UlUR 5 gr)rOrPr<rtrrUrVs r&rPDPRReader.__init__s,   .v6 r%rYrZr\r]r^r_r`.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UR X5 UR 5nO"UbUR 5SSnO [ S5eUb URO URnUc[R"XxS9nURUUUUUUS9$)a input_ids (`tuple[torch.LongTensor]` of shapes `(n_passages, sequence_length)`): Indices of input sequence tokens in the vocabulary. It has to be a sequence triplet with 1) the question and 2) the passages titles and 3) the passages texts To match pretraining, DPR `input_ids` sequence should be formatted with [CLS] and [SEP] with the format: `[CLS] [SEP] [SEP] ` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`DPRReaderTokenizer`]. See this class documentation for more details. [What are input IDs?](../glossary#input-ids) inputs_embeds (`torch.FloatTensor` of shape `(n_passages, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. Examples: ```python >>> from transformers import DPRReader, DPRReaderTokenizer >>> tokenizer = DPRReaderTokenizer.from_pretrained("facebook/dpr-reader-single-nq-base") >>> model = DPRReader.from_pretrained("facebook/dpr-reader-single-nq-base") >>> encoded_inputs = tokenizer( ... questions=["What is love ?"], ... titles=["Haddaway"], ... texts=["'What Is Love' is a song recorded by the artist Haddaway"], ... return_tensors="pt", ... ) >>> outputs = model(**encoded_inputs) >>> start_logits = outputs.start_logits >>> end_logits = outputs.end_logits >>> relevance_logits = outputs.relevance_logits ``` Nrr|rr)r\r]r^r_) r<r]r^rrRrr~rr rr) rDrYrZr\r]r^r_rrs r&rgDPRReader.forwards`2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ CN""  '/!5# #  r%)r<r)NNNNNN)rrrrrrPr rrrnrr,r#rgr$rqrrs@r&rrs y'++/*.,0/3&*K F#K !(K  ' K $D> K 'tn K d^K  fck 22 3K K r%r)rrr2rrrr)%r dataclassesrtypingrrr rrmodeling_outputsr modeling_utilsr utilsr r r bert.modeling_bertrconfiguration_dprr get_loggerrloggerrr)r,r2rJrtrrrrrr__all__rr%r&rs<!" :- +(   H %  ?k ?  ?  ?{ ?  ?  ?k?  ?$***(52#52p6 )6 |&"4&+#5+),)  a 3a  a H b 5b  b J T #T  T n r%