ktjYddlZddlZddlZddlmZddlZddlmZm Z m Z ddl m Z de de de de d ejf d Zd e de de d ejfd Zd e de fdZ d.de de de de dede d ejfdZdedeeejffdZde de de de dede de d e de de dede fdZ d/de de de de dede d e de de dede de fdZdZ d0d e d!edzd"edzd#edzd eejdzejdzejdzff d$Z d0d%ed!edzd"edzd#edzd eejdzejdzejdzff d&Zd'Zd(ed)ede de de de ded!edzd"edzd#edzd*ede dede fd+Zd,Ze d-kr edSdS)1N)Path) ModelProto TensorProto numpy_helper) OnnxModel input_ids batch_sizesequence_lengthdictionary_sizereturnc|jjjtjtjtjfvsJtj |||ftj }|jjjtjkrtj |}n3|jjjtjkrtj |}|S)a`Create input tensor based on the graph input of input_ids Args: input_ids (TensorProto): graph input of the input_ids input tensor batch_size (int): batch size sequence_length (int): sequence length dictionary_size (int): vocabulary size of dictionary Returns: np.ndarray: the input tensor created )sizedtype) type tensor_type elem_typerFLOATINT32INT64nprandomrandintint32float32int64)rr r r datas e/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/onnxruntime/transformers/bert_test_data.pyfake_input_ids_datars > % /4     9  _J3PXZX`  a aD~!+{/@@@z$  # -1B B Bx~~ K segment_idsc|jjjtjtjtjfvsJtj||ftj }|jjjtjkrtj |}n3|jjjtjkrtj |}|S)a,Create input tensor based on the graph input of segment_ids Args: segment_ids (TensorProto): graph input of the token_type_ids input tensor batch_size (int): batch size sequence_length (int): sequence length Returns: np.ndarray: the input tensor created r) rrrrrrrrzerosrrr)r r r rs rfake_segment_ids_datar$1s   ' 16     8Z1 B B BD#-1BBBz$   % /;3D D Dx~~ Krmax_sequence_lengthaverage_sequence_lengthc|dkr||ksJd|z|krtjd|z|z |Stjdd|zdz S)N)rr)r%r&s rget_random_lengthr*Lso "a ' ',CGZ,Z,Z,Z Z ""%888~a"99** f fQRUVQVQVtJNQ,>'?$q1u+'M'M\]Z!^$YZ]^Y^Y^tJNQ4F/G$qSTu+/U/UdeQ^a'!+,, f:&& 2 21Q:>** K K'(+B'BZ!^$/03J/JQ^a'!+,,", 0AAAz$  $ .+2C C Cx~~ Kr directoryinputsc btj|sL tj|t d|dn6#t $rt d|dYnwxYwt d|dt |D]\}\}}tj ||}ttj |d|dd 5}| | d d d n #1swxYwYd S) zOutput input tensors of test data to a directory Args: directory (str): path of a directory inputs (Dict[str, np.ndarray]): map from input name to value z#Successfully created the directory  zCreation of the directory z failedzWarning: directory z$ existed. Files will be overwritten.input_.pbwbN)ospathexistsmkdirprintOSError enumerateitemsr from_arrayopenjoinwriteSerializeToString)r8r9indexnamertensorfiles routput_test_datarPs 7>>) $ $U F HY    D DDD E E E E C C C AyAAA B B B B B C SISSSTTT(8833|d(t44 "',,y*=5*=*=*=>> E E 3 JJv//11 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 333s#A A)(A)/(D##D' *D' test_casesverbose random_seedc |Jtj|tj|g} t|D]} t ||||}|j|i}|rt |||||j<|rt|||| | | ||j<|r#t| dkrtd|| || S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases dictionary_size (int): vocabulary size of dictionary for input_ids verbose (bool): print more information or not random_seed (int): random seed input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value NrzExample inputs) rrseedr1rrMr$r7lenrCappend)r r rQr rRrSrr r+r&r,r- all_inputs _test_caseinput_1r9s rfake_test_datar[s D  INN; K JJ''"" %i_o^^.'*  g'<[*Ve'f'fF;# $  &:J9PRhjs''F:? #  ,s:!++ "F + + +&!!!! r'rUc ~t|||| ||||||| | } t| |krtd| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): print more information or not input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value z$Failed to create test data for test.)r[rVrC) r r rQrUrRrr r+r&r,r-r rXs rgenerate_test_datar^s`B    J :*$$ 4555 rc|t|jkrdS|j|}||}|C|||}|+|jdkr ||jd}|S)NCastr)rVinputfind_graph_input get_parentop_type) onnx_model embed_node input_indexra graph_input parent_nodes rget_graph_input_from_embed_noderj$sc**++++t  [ )E--e44K ++J DD  "{':f'D'D$55k6G6JKKK rreinput_ids_namesegment_ids_nameinput_mask_namec|}|||}|td|d}|r)||}|td|d}|r)||}|td|d|rdndz|rdndz}t||kr"td|dt||||fSt|dkrtdt||d } t| dkrw| d} t || d}t || d}t || d }|$|D]!} | j} d | vr| }"|td |||fSd}d}d}|D]/} | j} d | vr| }"d | vsd| vr| }-| }0|r |r|r|||fStd)a Find graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_model (OnnxModel): onnx model object input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Raises: ValueError: Graph does not have input named of input_ids_name or segment_ids_name or input_mask_name ValueError: Expected graph input number does not match with specified input_ids_name, segment_ids_name and input_mask_name Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask Nz Graph does not have input named r(rzExpect the graph to have z inputs. Got r/z'Expect the graph to have 3 inputs. Got EmbedLayerNormalizationmaskz#Failed to find attention mask inputtokensegmentz?Fail to assign 3 inputs. You might try rename the graph inputs.)'get_graph_inputs_excluding_initializersrb ValueErrorrVget_nodes_by_op_typerjrMlower) rerkrlrm graph_inputsrr r+expected_inputs embed_nodesrfrainput_name_lowers rfind_bert_inputsr|1s4EEGGL!//??  PPPQQ Q  X$556FGGK" !VDT!V!VWWW  W#44_EEJ! !UO!U!UVVVK6qqQ7 ;Q11PQR |   / /jjjWZ[gWhWhjjkk k+z11 <AV3|CTCTVVWWW112KLLK ;1 ^ 3J ANN 5j*aPP 4ZQOO  % ' '#(:#3#3#5#5 ---!&J  BCC C+z11IKJ   :++-- % % %JJ ' ' '98H+H+HKKII2[2Z2+z11 V W WWr onnx_filect}t|d5}||dddn #1swxYwYt |}t ||||S)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_file (str): onnx model path input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask rbN)rrHParseFromStringreadrr|)r}rkrlrmmodelrOres rget_bert_inputsrs( LLE i  +$ diikk***+++++++++++++++5!!J J8H/ Z ZZs(AAActj}|ddtd|ddtdd|d dtd d |d dtd d|ddtdd|ddtdd|ddtdd|ddtd d|ddtdd|dddd|d|dddd |d!|d"d#d$td%&|d'd(ddd)|d*|d+dtd,d-|}|S).Nz--modelTzbert onnx model path.)requiredrhelpz --output_dirFz4output test data path. Default is current directory.)rrdefaultrz --batch_sizer(zbatch size of inputz--sequence_lengthz maximum sequence length of inputz--input_ids_namezinput name for input idsz--segment_ids_namezinput name for segment idsz--input_mask_namezinput name for attention maskz --samplesz$number of test cases to be generatedz--seedr/z random seedz --verbose store_truezprint verbose information)ractionr)rRz--only_input_tensorsz-only save input tensors and no output tensors)only_input_tensorsz-az--average_sequence_lengthz)average sequence length excluding padding)rrrz-rz--random_sequence_lengthz3use uniform random instead of fixed sequence length)r,z --mask_typer)z^mask type: (1: mask index, 2: raw 2D mask, 3: key lengths, cumulated lengths of query and key))argparseArgumentParser add_argumentstrint set_defaults parse_args)parserargss rparse_argumentsrs  $ & &F  DsAXYYY   C  S!Rghhh   /    '    )    ,    3  5sAMZZZ  (   &&&  <   5111  #  8   " B  u555   m      D Krr output_dirrc t|||| \}}}t||||||||| | | }t|D]E\}}tj|dt |z}t||F| rdSddl}d| vrddgndg}| ||}d| D}t|D]\}}tj|dt |z}| ||}t|D]\}}tjtj|||}t#tj|d|d d 5}||dddn #1swxYwYdS) aICreate test data for a model, and save test data to a directory. Args: model (str): path of ONNX bert model output_dir (str): output directory batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): whether print more information input_ids_name (str): graph input name of input_ids segment_ids_name (str): graph input name of segment_ids input_mask_name (str): graph input name of input_mask only_input_tensors (bool): only save input tensors, average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type(int): mask type test_data_set_NrCUDAExecutionProviderCPUExecutionProvider) providerscg|] }|j S)rM).0outputs r z-create_and_save_test_data..NsDDDFFKDDDroutput_r=r>)rr^rEr?r@rIrrP onnxruntimeget_available_providersInferenceSession get_outputsrunrrGrasarrayrHrJrK)rrr r rQrUrRrkrlrmrr&r,r-rr r+rXr3r9r8rrsession output_namesresult output_name tensor_resultrOs rcreate_and_save_test_datar s\D*9P`bq)r)r&I{J#   Jz**,, 6GLL-=A-FGG F++++ #k&I&I&K&K K K !"899$ % **5I*FFGDDg.A.A.C.CDDDLz**>> 6GLL-=A-FGG \622' 55 > >NA{(3BJvay4I4I;WWMbgll9.>.>.>.>??FF >$ =::<<=== > > > > > > > > > > > > > > > >>>s,(G  G$ 'G$ c>t}|jdkr |j|_|j}|It |j}t j|j d|j d|j}|'t |}| ddntdt|j||j |j|j|j|j|j|j|j|j|j|j|jtd|dS)Nrbatch__seq_T)parentsexist_okz7Directory existed. test data files will be overwritten.z Test data is saved to directory:)rr&r rrrr?r@rIparentr rBrCrsamplesrUrRrkrlrmrr,r-)rrpr@s rmainrYs    D #q(('+';$J   W\\!(,aT_,a,a4K_,a,abb J 4$ //// GHHH           $ # " ,j99999r__main__)r))r\)NNN)!rr?rpathlibrnumpyronnxrrrrerrndarrayrr$r*boolr7rdictrPr[r^rjtupler|rrrr__name__rrrrs 6666666666 (+>ATWZ<{VY^`^h6B3BBBBB  FFFFF! F ! F  FZFFFFR33T#rz/-B3333.7777 7  7  7777!7!77777L!11111  1  1  111!1!111111h   "&#'"& YXYXYX$JYXDjYX4Z YX  2: bj4/d1B BC YXYXYXYX|"&#'"& [[[$J[Dj[4Z [  2: bj4/d1B BC [[[[8aaaHI> I>I>I> I>  I>  I>I>$JI>DjI>4ZI>I>!I>!I>I>I>I>I>X$:$:$:N zDFFFFFr