tjJdZddlZddlmZddlmZmZmZddlmZ ddl m Z m Z m Z ddlmZmZmZmZmZmZmZmZddlmZdd lmZdd lmZmZmZdd lm Z m!Z!dd l"m#Z#d dl$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5ddl6m7Z7ej8e9Z:Gdde&Z;Gdde5Z<Gdde%Z=Gdde0Z>Gdde4Z?Gdde2Z@Gdde'ZAeGdd eZBGd!d"e3ZCGd#d$e,ZDGd%d&e+ZEGd'd(e1ZFGd)d*e(ZGGd+d,e*ZHGd-d.e.ZIGd/d0e)ZJGd1d2e/ZKGd3d4e-ZLgd5ZMdS)6zPyTorch ERNIE model.N)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)Cache DynamicCacheEncoderDecoderCache),BaseModelOutputWithPoolingAndCrossAttentions!CausalLMOutputWithCrossAttentionsMaskedLMOutputMultipleChoiceModelOutputNextSentencePredictorOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)PreTrainedModel)Unpack)TransformersKwargsauto_docstringlogging)can_return_tuplemerge_with_config_defaults)capture_outputs)BertCrossAttentionBertEmbeddings BertEncoderBertForMaskedLMBertForMultipleChoiceBertForNextSentencePredictionBertForPreTrainingBertForPreTrainingOutputBertForQuestionAnsweringBertForSequenceClassificationBertForTokenClassification BertLayerBertLMHeadModelBertLMPredictionHead BertModel BertPoolerBertSelfAttention) ErnieConfigceZdZdZfdZ d dejdzdejdzdejdzdejdzd ejdzd ed ej fd Z xZ S)ErnieEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.ct||j|_|jr&tj|j|j|_dSdS)N)super__init__ use_task_idnn Embeddingtask_type_vocab_size hidden_sizetask_type_embeddings)selfconfig __class__s e/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/ernie/modular_ernie.pyr3zErnieEmbeddings.__init__As_    !-   f(* V5PRXRd(e(eD % % % f fNr input_idstoken_type_ids task_type_ids position_ids inputs_embedspast_key_values_lengthreturnc||}n|dd}|\}} ||jdd|| |zf}|t|drT|j|jdd} t j| d|} | || }n+t j|t j |jj }|| |}| |} | | j }|| z} ||} | | z} |jrG|+t j|t j |jj }||}| |z } || } || } | S)Nr@rr-)dimindex)dtypedevice)sizerBhasattrr@expandshapetorchgatherzeroslongrKword_embeddingstoken_type_embeddingstoposition_embeddingsr4r9 LayerNormdropout)r:r?r@rArBrCrD input_shape batch_size seq_lengthbuffered_token_type_idsrU embeddingsrWr9s r=forwardzErnieEmbeddings.forwardHs  #..**KK',,..ss3K!, J  ,QQQ0FVlIl0l-lmL  !t-.. m*.*=*D*D\EWXYEZ\^*_*_'*/,7NTU]i*j*j*j'!8!?!? J!W!W!&[ SWSdSk!l!l!l   00;;M $ : :> J J&(()>)EFF "%:: "66|DD"55    /$ % KuzRVRcRj k k k #'#<#<]#K#K . .J^^J// \\*-- r>)NNNNNr) __name__ __module__ __qualname____doc__r3rP LongTensor FloatTensorintTensorr_ __classcell__r<s@r=r0r0>sQQfffff.226150426&'33#d*3(4/3'$. 3 &- 3 (4/ 3!$3 33333333r>r0ceZdZdS)ErnieSelfAttentionNr`rarbr>r=rkrk~Dr>rkceZdZdS)ErnieCrossAttentionNrlrmr>r=rprprnr>rpceZdZdS) ErnieLayerNrlrmr>r=rrrrrnr>rrceZdZdS) ErniePoolerNrlrmr>r=rtrtrnr>rtceZdZdS)ErnieLMPredictionHeadNrlrmr>r=rvrvrnr>rvceZdZdS) ErnieEncoderNrlrmr>r=rxrxrnr>rxcpeZdZeZdZdZdZdZdZ dZ e e e dZejfdZxZS)ErniePreTrainedModelernieT) hidden_states attentionscross_attentionsct|t|trt j|jdSt|trjt j|j tj |j j d dt j|jdSdS)zInitialize the weightsrG)r-rGN)r2 _init_weights isinstancervinitzeros_biasr0copy_rBrParangerOrNr@)r:moduler<s r=rz"ErniePreTrainedModel._init_weightss f%%% f3 4 4 / K $ $ $ $ $  0 0 / Jv*EL9L9RSU9V,W,W,^,^_f,g,g h h h K- . . . . . / /r>)r`rarbr. config_classbase_model_prefixsupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attn_supports_attention_backendrrrkrp_can_record_outputsrPno_gradrrhris@r=rzrzsL&*#N"&#(/ U]__////_/////r>rzcXeZdZdgZdfd Zeee ddej dzdej dzdej dzdej dzd ej dzd ej dzd ej dzd ej dzd e dzde dzde e deej ezfdZxZS) ErnieModelrrTct||||_d|_t ||_t ||_|rt|nd|_ | dS)NF) r2r3r;gradient_checkpointingr0r^rxencoderrtpooler post_init)r:r;add_pooling_layerr<s r=r3zErnieModel.__init__s{ v&&& &+#)&11#F++ ->Hk&)))D  r>Nr?attention_maskr@rArBrCencoder_hidden_statesencoder_attention_maskpast_key_values use_cachekwargsrEc n|du|duz rtd|jjr| | n |jj} nd} | r[| Y| |jjr6t t |jt |jnt |j} | | nd} ||||||| } | ||| || \}}|j | f|||| | |d| }|d}|j | |nd}t|||j S)  task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] Nz:You must specify exactly one of input_ids or inputs_embedsF)r;r)r?rBr@rArCrD)rrembedding_outputrr)rrrrrrB)last_hidden_state pooler_outputr) ValueErrorr; is_decoderris_encoder_decoderr r get_seq_lengthr^_create_attention_masksrrr r)r:r?rr@rArBrCrrrrrrDrencoder_outputssequence_output pooled_outputs r=r_zErnieModel.forwards0  -t"; < [YZZ Z ; ! %.%:  @UIII  0)4 8V4$L $D$D$DlZ^ZeFfFfFfggg!555  FUE`!?!?!A!A!Afg??%)''#9+  261M1M)#9-"7+ 2N2 2 ..'$,   )"7#9+%      *!,8< 8O O444UY ;-'+;    r>)T) NNNNNNNNNN)r`rarb_no_split_modulesr3rrrrPrgrboolrrtupler r_rhris@r=rrs}%       *..2.2-1,0-1596:(,!%H H <$&H  t+H  t+ H |d* H lT) H |d*H  %|d2H !& t 3H H $;H +,H  u| K KH H H ^_ H H H H H r>rceZdZdS)ErnieForPreTrainingOutputNrlrmr>r=rrrnr>rc"eZdZdddZee ddejdzdejdzdejdzdejdzd ejdzd ejdzd ejdzd ejdzd ee de eje zfdZ dS)ErnieForPreTrainingcls.predictions.bias'ernie.embeddings.word_embeddings.weightzcls.predictions.decoder.biaszcls.predictions.decoder.weightNr?rr@rArBrClabelsnext_sentence_labelrrEc |j|f|||||dd| } | dd\} } || | \} }d}||t}|| d|jj|d}||dd|d}||z}t || || j| jS)a: task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` next_sentence_label (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the next sequence prediction (classification) loss. Input should be a sequence pair (see `input_ids` docstring) Indices should be in `[0, 1]`: - 0 indicates sequence B is a continuation of sequence A, - 1 indicates sequence B is a random sequence. Example: ```python >>> from transformers import AutoTokenizer, ErnieForPreTraining >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> model = ErnieForPreTraining.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> prediction_logits = outputs.prediction_logits >>> seq_relationship_logits = outputs.seq_relationship_logits ``` Trr@rArBrC return_dictNrrG)lossprediction_logitsseq_relationship_logitsr|r}) r{clsrviewr; vocab_sizerr|r})r:r?rr@rArBrCrrroutputsrrprediction_scoresseq_relationship_score total_lossloss_fctmasked_lm_lossnext_sentence_losss r=r_zErnieForPreTraining.forwards,^$*   ))'%'      *1!&48HH_m4\4\11  "5"A'))H%X&7&<&NNNNNNNN) r`rarb_tied_weights_keysrrrPrgrrrrr_rmr>r=rrs/(>*S *..2.2-1,0-1&*37H H <$&H  t+H  t+ H |d* H lT) H |d*H  t#H #\D0H +,H  u| 8 8H H H ^H H H r>rc reZdZee ddejdzdejdzdejdzdejdzdejdzdejdzd ejdzd ejdzd ejdzd eejdzd edzde ejzde e de eje zfdZdS)ErnieForCausalLMNrr?rr@rArBrCrrrrrlogits_to_keeprrEc  n| d} |j|f|||||||| | dd | }|j}t| trt | dn| }||dd|ddf}d}| |jd|| |jjd| }t|||j |j |j |j S)a task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the left-to-right language modeling loss (next word prediction). Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels n `[0, ..., config.vocab_size]` NFT) rr@rArBrCrrrrr)logitsrr)rrrr|r}r~rm)r{rrrfslicer loss_functionr;rr rr|r}r~)r:r?rr@rArBrCrrrrrrrrr| slice_indicesrrs r=r_zErnieForCausalLM.forwardfs:  I@J  A ))'%'"7#9+ A A  A A  1 8B>SV8W8Wk~ot444]k -=!!!(;<==  %4%pVFt{OeppioppD0#3!/)$5     r>) NNNNNNNNNNNr)r`rarbrrrPrglistrrfrrrr r_rmr>r=rres[*..2.2-1,0-1596:&*59!%-.= = <$&=  t+=  t+ = |d* = lT) = |d*=  %|d2= !& t 3=  t#= el+d2= $;= el*= +,=  u| @ @= = = ^= = = r>rc8eZdZdddZee ddejdzdejdzdejdzdejdzd ejdzd ejdzd ejdzd ejdzd ejdzdee de eje zfdZ dS)ErnieForMaskedLMrrrNr?rr@rArBrCrrrrrEc B|j|f|||||||dd| } | d} || } d}| Kt}|| d|jj| d}t || | j| jS)as task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` T)rr@rArBrCrrrrNrGrrr|r}) r{rrrr;rr r|r})r:r?rr@rArBrCrrrrrrrrrs r=r_zErnieForMaskedLM.forwards4$*   ))'%'"7#9      "!* HH_55  '))H%X&7&<&) NNNNNNNNN) r`rarbrrrrPrgrrrr r_rmr>r=rrs1(>*S *..2.2-1,0-1596:&*2 2 <$&2  t+2  t+ 2 |d* 2 lT) 2 |d*2  %|d22 !& t 32  t#2 +,2  u| ~ -2 2 2 ^2 2 2 r>rceZdZee d dejdzdejdzdejdzdejdzdejdzdejdzdejdzd eed e eje zfd Z dS) ErnieForNextSentencePredictionNr?rr@rArBrCrrrEc *|j|f|||||dd|} | d} || } d} |At} | | dd|d} t | | | j| jS)a task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the next sequence prediction (classification) loss. Input should be a sequence pair (see `input_ids` docstring). Indices should be in `[0, 1]`: - 0 indicates sequence B is a continuation of sequence A, - 1 indicates sequence B is a random sequence. Example: ```python >>> from transformers import AutoTokenizer, ErnieForNextSentencePrediction >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> model = ErnieForNextSentencePrediction.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> prompt = "In Italy, pizza served in formal settings, such as at a restaurant, is presented unsliced." >>> next_sentence = "The sky is blue due to the shorter wavelength of blue light." >>> encoding = tokenizer(prompt, next_sentence, return_tensors="pt") >>> outputs = model(**encoding, labels=torch.LongTensor([1])) >>> logits = outputs.logits >>> assert logits[0, 0] < logits[0, 1] # next sentence was random ``` Trr-NrGrr)r{rrrrr|r})r:r?rr@rArBrCrrrrseq_relationship_scoresrrs r=r_z&ErnieForNextSentencePrediction.forwardsZ$*   ))'%'        "&((="9"9!  '))H!)*A*F*Fr1*M*Mv{{[]!_!_ *#*!/)     r>NNNNNNN) r`rarbrrrPrgrrrrr_rmr>r=rrs*..2.2-1,0-1&*D D <$&D  t+D  t+ D |d* D lT) D |d*D  t#D +,D  u| : :D D D ^D D D r>rceZdZee d dejdzdejdzdejdzdejdzdejdzdejdzdejdzd eed e eje zfd Z dS) ErnieForSequenceClassificationNr?rr@rArBrCrrrEc |j|f|||||dd|} | d} || } || } d} |Z|jjf|jdkr d|j_nN|jdkr7|jtjks|jtj kr d|j_n d|j_|jjdkrWt} |jdkr1| | | } n| | |} n|jjdkrGt} | | d|j| d} n*|jjdkrt} | | |} t| | | j| j S) a^ task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Trr-N regressionsingle_label_classificationmulti_label_classificationrGr)r{rY classifierr; problem_type num_labelsrJrPrSrfrsqueezerrrrr|r})r:r?rr@rArBrCrrrrrrrs r=r_z&ErnieForSequenceClassification.forward0s0$*   ))'%'         ]33 //  {'/?a''/;DK,,_q((flej.H.HFL\a\eLeLe/LDK,,/KDK,{'<77"99?a''#8FNN$4$4fnn6F6FGGDD#8FF33DD)-JJJ+--x B @ @&++b//RR)-III,..x//'!/)     r>r) r`rarbrrrPrgrrrrr_rmr>r=rr/s*..2.2-1,0-1&*B B <$&B  t+B  t+ B |d* B lT) B |d*B  t#B +,B  u| 7 7B B B ^B B B r>rceZdZee d dejdzdejdzdejdzdejdzdejdzdejdzdejdzd eed e eje zfd Z dS) ErnieForMultipleChoiceNr?rr@rArBrCrrrEc T| |jdn |jd} |)|d|dnd}|)|d|dnd}|)|d|dnd}|)|d|dnd}|=|d|d|dnd}|j|f|||||dd|} | d} || } || } | d| } d}|t }|| |}t|| | j| j S)a9 input_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) token_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) task_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] position_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_choices, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the multiple choice classification loss. Indices should be in `[0, ..., num_choices-1]` where `num_choices` is the size of the second dimension of the input tensors. (See `input_ids` above) Nr-rGTrr) rOrrLr{rYrrrr|r})r:r?rr@rArBrCrr num_choicesrrrreshaped_logitsrrs r=r_zErnieForMultipleChoice.forwardxs`-6,Aioa((}GZ[\G] >G>SINN2y~~b'9'9:::Y] M[Mg,,R1D1DR1H1HIIImqM[Mg,,R1D1DR1H1HIIImqGSG_|((\->->r-B-BCCCei (   r=#5#5b#9#9=;M;Mb;Q;Q R R R  $*   ))'%'         ]33 // ++b+66  '))H8OV44D("!/)     r>r) r`rarbrrrPrgrrrrr_rmr>r=rrws*..2.2-1,0-1&*U U <$&U  t+U  t+ U |d* U lT) U |d*U  t#U +,U  u| 8 8U U U ^U U U r>rceZdZee d dejdzdejdzdejdzdejdzdejdzdejdzdejdzd eed e eje zfd Z dS) ErnieForTokenClassificationNr?rr@rArBrCrrrEc ^|j|f|||||dd|} | d} || } || } d} |Ft} | | d|j|d} t | | | j| jS)a task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. TrrNrGr) r{rYrrrrrr|r})r:r?rr@rArBrCrrrrrrrs r=r_z#ErnieForTokenClassification.forwards,$*   ))'%'      "!*,,7711  '))H8FKKDO<r) r`rarbrrrPrgrrrrr_rmr>r=rrs*..2.2-1,0-1&*. . <$&.  t+.  t+ . |d* . lT) . |d*.  t#. +,.  u| 4 4. . . ^. . . r>rceZdZee d dejdzdejdzdejdzdejdzdejdzdejdzdejdzd ejdzd eed e eje zfd Z dS)ErnieForQuestionAnsweringNr?rr@rArBrCstart_positions end_positionsrrEc H|j|f|||||dd| } | d} || } | dd\} }| d} |d}d}||t |dkr|d}t |dkr|d}| d}|d|}|d|}t|}|| |}|||}||zd z }t|| || j | j S) rTrrr-rG)rHN) ignore_indexr)r start_logits end_logitsr|r}) r{ qa_outputssplitr contiguouslenrLclamprrr|r})r:r?rr@rArBrCrrrrrrrrr ignored_indexr start_lossend_losss r=r_z!ErnieForQuestionAnswering.forwards*$*   ))'%'      "!*11#)<<r<#:#: j#++B//::<< ''++6688   &=+D?''))**Q.."1"9"9""="==%%''((1,, - 5 5b 9 9 (--a00M-33A}EEO)//=AAM']CCCH!,@@Jx M::H$x/14J+%!!/)     r>r) r`rarbrrrPrgrrrrr_rmr>r=rrs *..2.2-1,0-1/3-1< < <$&<  t+<  t+ < |d* < lT) < |d*< ,< |d*< +,<  u| ; ;< < < ^< < < r>r) rrrrrrrrrrz)NrcrPtorch.nnr5rrrrr cache_utilsrr r modeling_outputsr r r rrrrrmodeling_utilsrprocessing_utilsrutilsrrr utils.genericrrutils.output_capturingrbert.modeling_bertrrrrr r!r"r#r$r%r&r'r(r)r*r+r,configuration_ernier. get_loggerr`loggerr0rkrprrrtrvrxrzrrrrrrrrrr__all__rmr>r=r s AAAAAAAAAA&&&&&&CCCCCCCCCC                    .-----&&&&&&@@@@@@@@@@IIIIIIII555555&-,,,,,  H % %=====n===@     *        ,                *        0        ;   /////?///2[ [ [ [ [ [ [ [ |     8   P P P P P ,P P P f@ @ @ @ @ @ @ @ F: : : : : : : : zG G G G G %BG G G TE E E E E %BE E E PX X X X X 2X X X v1 1 1 1 1 "<1 1 1 h? ? ? ? ? 8? ? ? D   r>