tjddlZddlZddlZddlmZmZddlZddlmZm Z ddl m Z m Z m Z mZddlmZddlmZmZddlmZddlmZdd lmZdd lmZdd lmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z%dd l&m'Z'm(Z(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0ddl1m2Z2m3Z3ddl4m5Z5m6Z6ddl7m7Z7ddl8m9Z9m:Z:m;Z;mm?Z?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGerddlHmIZIdZJdZKgdZLejMejNejOejPejQejRejSejTejUejVg ZWejXejYjZj[j\ej]ejYjZj^j\iZ_ dDdej`j%de)deaebejcjdfdeaebeeebefffdeaebefd ejgdzd!dfd"Zh dDdej`j%de)deaebejcjdfdeaebeeebefffdeaebefd ejgdzd!dfd#Zide)d$e(d!dfd%Zjd&ejcjdd!ekfd'Zlde'deaebefd!ekfd(Zmd)e%d*ed!dfd+Znd,ede)d$e(d!dfd-Zod.e)deaebeeebefffdeaebefd!eeebebffd/Zpde)d$e(d!dfd0Zqde)deaebejcjdfd!ejcjddzfd1Zrde)deaebejcjdfdej`j%d2ekd*edzd!df d3Zs dEde)deaebejcjdfd5etebdeaebefd*ed6ekd2ekd ejgdzd!dfd7Zude)d8e)d$e(d!dfd9Zvde)d$e(deaebejcjdfd:eaeeaefefffd;ete)d!df d<Zw dFde%d2ekd>e2eaebefzdzd?ekd@ekdAeeaebefzdzd*eeaebefzdzd6ekdBekd!e%fdCZxdS)GN)Any TYPE_CHECKING) BackendConfigget_native_backend_config)get_fused_module_classesget_pattern_to_dtype_configsget_qat_module_classes-get_root_module_to_quantized_reference_module)_is_activation_post_process)qconfig_equals QConfigAny)QConfigMapping) QuantType)_remove_qconfig) DeQuantStub) _parent_name"activation_is_statically_quantizedget_qparam_dictget_swapped_custom_module_classis_per_channelto_underlying_dtypeweight_is_quantized) GraphModule)ArgumentGraphNode)type_before_parametrizations)quantized_decomposed_lib)convert_eq_obsupdate_obs_for_equalization)ConvertCustomConfigPrepareCustomConfig)_is_observed_module_is_observed_standalone_module)lower_to_fbgemm))_compare_prepare_convert_qconfig_mappings_generate_node_name_to_qconfig&_is_qconfig_supported_by_dtype_configs_update_qconfig_for_fusion_update_qconfig_for_qat) _get_module_is_custom_module_lstm_is_custom_module_mhaassert_and_get_unique_devicecollect_producer_nodescreate_getattr_from_valueget_custom_module_class_keys graph_module_from_producer_nodesnode_arg_is_weight)Callablenumeric_debug_handlecustom)convertconvert_custom_moduleconvert_standalone_moduleconvert_weighted_modulemodelnodemodulesnode_name_to_scopenode_name_to_qconfig model_devicereturnc 4|j}|tdt|jts$tdt |jt ||\}}||j} t| dr| ||dStfdt|j t|j zD} | st| sd||5||j d||dddn #1swxYwYdS| j} d} t| dr| j} d } | t(vr| sd }d}| \}}t-| jrqt1| j}t4jjjj}t4jjjj}| j }| j!}tE| }||||||d }nyt4jjj#j}t4jjj$j}tK|}t1|}| j }| j!}tE| }|||||d }||5|j d}|g}|&D]i\}}|d vrKt|tJt0fs/tO||||z|z||}|(|T|(|j|)||tU|i}|g|ddz}|+|tU|| ||}||tX|j-vr(t\|j-tXvrt_d||ddddS#1swxYwYdS| r d }t4jjj#j0}tE| }|t4j1t4j2fvrtd| j }| j!}tg| dt4j4} tg| dt5j5t4j6j7}!|||!|d}tp| }"||5|j d}|j dgt|9z}#|)d |"tU|#i}$|)d ttj;|$dfi}%|)d ttj;|$dfi}&|d}|d}|d} |%|&||| d }dddn #1swxYwY||5|j d}|g}|&D]6\}}|d vr|}|(|!|(|7|)||tU|i}|g|ddz}t4jjj$j0}|+|tU|| ||}||t\|j-vrt_d||ddddS#1swxYwYdS| t4j<krt4jjj=j>}'||5|j d}|)d |'|t4j<fi}(|)d |'|(t4j%fi})||)||ddddS#1swxYwYdSdS)auReplace activation_post_process module call node with quantize and dequantize node working with decomposed Tensor Before: ... -> observer_0(x) -> ... After: ... -> torch.ops.quantized_decomposed.quantize_per_tensor(x, ...) -> torch.ops.quantized_decomposed.dequantize_per_tensor() -> ... or quantize_per_channel and dequantize_per_channel Nmodules must not be None*Expected node.target to be a str, but got r8c38K|]}t|VdSN_has_none_qconfig.0nr@s ^/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/ao/quantization/fx/convert.py zM_replace_observer_with_quantize_dequantize_node_decomposed..B  !122rF is_dynamicchi}d|jvr&|jdj}|tjkrd|i}|S)Nval out_dtype)metadtypetorchfloat32) dequantize_op input_nodedequantize_op_kwargs dq_out_dtypes rMadd_dequantize_op_kwargsz\_replace_observer_with_quantize_dequantize_node_decomposed..add_dequantize_op_kwargssB! JO # #%?517Lu},,(3\'B$##rP call_function)_scale_ _zero_point__axis_ _quant_min_ _quant_max__dtype_)r_r`rbrcrdr_r`rzOpt2e numeric suite has been migrated to torchao (https://github.com/pytorch/ao)zVonly uint8 and int8 are supported in reference flow for dynamic quantization right nowqschemeeps)rbrc_eps_rdrbrcrd)?graphAssertionError isinstancetargetstrtype_get_module_path_and_prefixhasattrr8alllistargsuserskeys_is_conversion_supportedinserting_beforereplace_all_uses_with erase_noderVrQSUPPORTED_QDTYPEScalculate_qparamsrrfintch_axisrWopsquantized_decomposedquantize_per_channeldefaultdequantize_per_channel quant_min quant_maxrquantize_per_tensordequantize_per_tensorfloatitemsr1append create_nodetupler^ CUSTOM_KEYrUNUMERIC_DEBUG_HANDLE_KEYNotImplementedErrortensoruint8int8getattrper_tensor_affinefinforXrg_QSCHEME_TO_CHOOSE_QPARAMS_OPvaluesoperatorgetitemfloat16convert_element_typeno_fuse)*r<r=r>r?r@rAri module_pathprefixactivation_post_processskip_replacementrVrQr] node_type quantize_opscale zero_pointr}rYrrdtype_qparamsrZquantize_op_inputskey value_or_node qparam_nodequantized_node dq_inputsdequantized_noderfrgchoose_qparams_opchoose_qparams_op_inputschoose_qparams_node scale_nodezero_point_nodedtype_convert_opconvert_fp16_nodeconvert_fp32_nodes* ` rM:_replace_observer_with_quantize_dequantize_node_decomposedras9 & KE7888 dk3 ' '  Ldk9J9J L L   6  "6K&dk2& 22''t444di4 (9(9#:#::78OPP # #D ) ) # #  & &ty| 4 4 4   T " " " # # # # # # # # # # # # # # #  $ )EJ& 558,7 $$$ !!!:!$ '+ 3EEGGz 19 : : 19::G)8MUK .EM 09I/9I(//F *!((! GG )8LTK!I:PXM%LLEZJ/9I/9I(//F *((! G # #D ) )1 #1 #1J", &-mmoo = ="]555"=5#,??6#<#f,s2%$ ##K'--k::::'--m<<<<"..;.@(A(A2N((+=abb+AAI$22i  (( CC     & &'7 8 8 8di'', *0EEE)e   T " " "c1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #1 #d l#$ i4HO %U++ %+uz2 2 2 h ,5 +5 19e>UVV-uek%-6P6P6TUU %$   :'B  # #D ) )  1J(, ! ~W^^=M=M8N8N'N $"'"3"3!2E:R4S4SUW## **!14G3KRJ$//!14G3KRO  .I .II&E% /(( G%               6 # #D ) )% #% #1J", &-mmoo = ="]555#0K&--k::::'--m<<<<"..;.@(A(A2N((+=abb+AAI"I:PWM$22i  (( CC     & &'7 8 8 8'4944)e   T " " "K% #% #% #% #% #% #% #% #% #% #% #% #% #% #% #% #% #% #L %-   99NV  # #D ) ) # #1J % 1 1!1J 3NPR!! !& 1 1!14Eu{3SUW!!   & &'8 9 9 9   T " " " # # # # # # # # # # # # # # # # # # s^6EE!$E!EP%%P),P)C W%%W),W)D\11\58\5>B`  ``c < |tdt|jts$tdt |j|j}t ||\}}||j} tfdt|j t|j zD} | st| sd| |5||j d||dddn #1swxYwYdS| j} d} t#| dr| j} | t&jt&jt&jt&jt&jfvr| sd} d}| \}}t5| jr(t9| j}|||| d }t&j}n0t?|}t9|}||| d }t&j }| |5|j d}|g}|!D]M\}}|d vr/tE||||z|z||}|#|8|#|N|$| |tK|i}|&d |f }||||ddddS#1swxYwYdS| rd} t&j'}t&j(j)j*dv}| |d}| |5|j d}|g}|+D]}|#||$| |tK|i}|&d |f }||||ddddS#1swxYwYdS| t&j,krd} d}d| i}| |5|j d}|g}|+D]}|#||$| |tK|i}|&d |f }||||ddddS#1swxYwYdSdS)zReplace activation_post_process module call node with quantize and dequantize node Before: ... -> observer_0(x) -> ... After: ... -> torch.quantize_per_tensor(x, ...) -> x.dequantize() -> ... NrDrEc38K|]}t|VdSrGrHrJs rMrNzB_replace_observer_with_quantize_dequantize_node..rOrPrFrQr^)r_r`rard)r_r`rdre dequantize)rs)fbgemmx86)rd_reduce_range_ call_methodtord)-rjrkrlrmrnrirorqrrrsrtrurvrwrxryrVrprQrWquint8qint8qint32 float8_e5m2 float8_e4m3fnr{rrfr|r}rrrrr1rrrrquantize_per_tensor_dynamicbackends quantizedenginerr)r<r=r>r?r@rArirrrrrVrQrrrrr}rrZrrrrrr reduce_rangevalues ` rM/_replace_observer_with_quantize_dequantize_noderrs 7888 dk3 ' '  Ldk9J9J L L    KE5  "6K&dk2di4 (9(9#:#::78OPP # #D ) ) # #  & &ty| 4 4 4   T " " " # # # # # # # # # # # # # # #  $ )EJ& 558,7          $ '+ 3EEGGz 19 : : 419::G *! G  4KK%LLEZJ"'PUVVG3K # #D ) ) # #1J", &-mmoo = ="]555#<#f,s2%$ ##K'--k::::'--m<<<<"..;.@(A(A2N %00^DU0VV   & &'7 8 8 8   T " " "5 # # # # # # # # # # # # # # # # # #6 '#$ 7 ~/6:KK #|DD  # #D ) ) # #1J",  )) 1 1"))%0000"..;.@(A(A2N %00^DU0VV   & &'7 8 8 8   T " " " # # # # # # # # # # # # # # # # # # %-  !  e$  # #D ) ) # #1J",  )) 1 1#))%0000"..;.@(A(A2N %00^DU0VV   & &'7 8 8 8   T " " " # # # # # # # # # # # # # # # # # # sK36D55D9<D9=CL$$L(+L(3B$P$$P(+P(B$TTTric|jd}t|tstd|||||t ||dS)Nrz@Expecting the for call custom module node to be a Node, but got )rsrkrrjrxry_insert_dequantize_node)r=ricall_custom_module_nodes rM6_replace_observer_or_dequant_stub_with_dequantize_noders#il -t 4 4  hOf h h    6777 T3U;;;;;rPrc~|j}d}t|dr|j}|tvr| p|p|tjkS)NFrQ)rVrprQrzrWr)rrVrQs rMrvrvsX # )EJ& 558,7  # # 8Z "  " EM !rPc\t|to|j|vo||jduS)z\Check if a node has a qconfig of None, i.e. user requested to not quantize the node N)rkrname)r=r@s rMrIrI+s: 4 4 I- - 4  +t 3rPobservedbackend_configc|jjD]V}|jdkr|jD]@}|r> )(>x(H(H%(0)I3**&'&((( ) ) )rPargct|tr;|jdkr0|jdkr%|jd}|||dSt|t tfr|D]}t|||dSt|tr*| D]}t|||dStj dt|ddS)zIf the arg is a dequantize Node, or a list/tuple/dict of dequantize Node, we'll recursively remove the dequantize Node rrrz6Unsupported node type in recursive remove dequantize:  stacklevelN)rkrrrlrsreplace_input_withrrr"_maybe_recursive_remove_dequantizedictrwarningswarnrn)rr=ri quantize_node arg_elements rMrrNs$#t =!8!8SZ<=W=W  ]33333 C$ ' '   I IK .{D% H H H H I I C   ::<< I IK .{D% H H H H I I  PT#YY P P      rPobs_nodec|jd}t|tstd||j|vr||jdund}|rZt |j}|r|dnd}d}|D].}|jdkr!|jtj j j ur|}n/|r|}d} n|}d} |r|j|vr||j\} } nd} | | fS)aiGiven and observer node, get the `Scope` or the fully qualified name for the submodule containing the observed node, also return a prefix of "_input" when the observed node is an input of a F.linear op, and not the output of another quantized op. TODO: this logic is hacky, we should think about how to remove it or make it more general rz.Expecting observed node to be a Node, but got Nr^_input) rsrkrrjrrrrtrrlrWnn functionallinear) rr?r@ observed_nodeis_input_observer_onlyrtfirst_linear_use_or_first_use linear_noderLrr_s rMrorods>M!$M mT * *  L] L L    !5 5 5 ]/0D88  X^$$49(Cat%   At&&18ux7J7Q+Q+Q   8,7 ))6% &  ) .2D D D+,I,NO QQ   rPc||5|d|f}t|jD]}||ur||| ddddS#1swxYwYdS)z-Inserts dequantize node for `node` in `graph`rN)inserting_afterrrrtr)r=ridequantize_node user_nodes rMrrs   t $ $DD++L4'BBdj)) D DI//,,T?CCC DDDDDDDDDDDDDDDDDDDsA A--A14A1c|jD]:}|jdkr-|t|j}t |r|cS;dS)z\ If the node is observed, return the observer instance. Otherwise, return None. call_moduleN)rtrrmrlr )r=r>maybe_obs_node maybe_obss rM_maybe_get_observer_for_nodersZ*!!   - -N$9 : :;I*955 !    4rP is_referencecr|rtjjjj}ntjjjj}|t |j}|jdj }t|j }tt|D]y} | |vrs|| } | jdkr`| jdkrU| j d} || | t| jdkr|j| z|jdj} t| dkr0| ddkrt)dt+||j|||} t-|j\}}t/|||| | |t |j<dS)aConverts a observed standalone module to a quantized standalone module by calling the fx convert api, currently using the same `is_reference` flag as parent, but we may changing this behavior in the future (e.g. separating quantization and lowering for standalone module as well) Args: - node: The call_module node of the observed standalone module - modules: named_module of original model - model: original model - is_reference: a flag from parent provided by user to decide if we want to produce a reference model or a fbgemm/qnnpack model - backend_config: backend configuration of the target backend of quantization _observed_graph_module_attrsrrrz7Currently only quantized output idxs = [0] is supported)rN)rWao quantization quantize_fxconvert_to_reference_fx convert_fxrmrlrU&standalone_module_input_quantized_idxsrrrsrangelenrrrtriry'standalone_module_output_quantized_idxsrjrrsetattr)r=r>r<rr convert_fnobserved_standalone_modulesm_input_quantized_idxsrsidxrrsm_output_quantized_idxsquantized_standalone_module parent_namers rMr:r:s*BX*6N X*6A /6c$+6F6F.G8=&,  ??DSYY00 ) ) )s)Cv&&3:+E+E # ''];;;sy>>Q&&K**3///9>& - #$$q(( #A &! + + I  ek222#-*">###%T[11K GK $(CDDD ;GC  rPFobserved_node_names is_decomposedch|t|j}|j} d} t|} t || rI|j} |}t|j\} } t|| | ||j |v}| t||s|sdSt|}| t|g}t| |sdSt| }|sdSd}|}t |t jjjjr |}|d}d|i}t |t jjr| }| }||j||jt3|}t3|}|||dn6t |t jjt jjfr|jD]}}t=||rk|drVtA||}| } | j!t j"kr | |t3| ||<~n| du}|r?| } ||}ntG|}|r| $|| }|r|r|s| |j|t3| tK|}| tM|d}|tOdtM||(||} || |d<dSt|j\} } t|| | | dS)aConvert a weighted module to reference quantized module in the model If the QConfig of a QAT module is not set, the module will still be converted to a float module. Args: - node: The call_module node of the observed standalone module - modules: named_module of original model - observed_node_names: names for the set of observed fx node, we can skip this conversion if the node is not observed Nrr) weight_ih weight_hhweightz3No reference quantized module class configured for ))rmrlqconfigr rkweight_fake_quantto_floatrrrrIrgetrnr)rrWrr intrinsic _FusedModule RNNCellBaserrrrupdateLSTMGRU_flat_weights_namesrp startswithrrVrr/rr rrj from_float)!r=r>rr@rrrrAoriginal_modulerweight_post_processqat_module_classesrr is_observedpattern_to_dtype_configs dtype_configsis_weight_quantized fused_module float_module wq_or_wq_dictweight_post_process_ihweight_post_process_hhweight_qparams_ihweight_qparams_hhwnris_ptqdeviceis_qat)root_module_to_quantized_reference_moduleref_qmodule_cls ref_qmodules! rMr;r;s6(c$+../O)1G/??/#566=.?)2244(55 T $dO<<<)22K  T#7 8 8    !+<=== y  1 $ $   Y ' ' ' ' ' %$)I)I % $rPcustom_module_class_mapping(statically_quantized_custom_module_nodesc|t|j}|j}t|r||t ||rt |jdkr>t|jdtrt |jddkstd|j\}\}} t|tstdt|tstdt| tstdt|||t|||t|| |nLt||rt |jdkrtd|j\} } } t| tstd t| tstd t| tstd t|| |t|| |t|| |nj|jd } t| tstd t|| |t||}|td||_t!|||}||}t%|j\}}t'||||dS)aConverts an observed custom module to a quantized custom module based on `custom_module_class_mapping` For static quantization, we'll also remove the previous `dequantize` node and attach the observer node for output to the module, the observer for the node will be converted to a dequantize node instead of quantize-dequantize pairs later in the graph. In the end we would have a quantized custom module that has the same interface as a default quantized module in nn.quantized namespace, i.e. quantized input and quantized output. Args: - node: The call_module node of the observed standalone module - graph: The graph containing the node - modules: named_module of original model - custom_module_class_mapping: mapping from observed custom module class to quantized custom module class, used to swap custom modules - statically_quantized_custom_module_nodes: we'll add the custom module node if we find it is statically quantized, this will be used later when converting observers to quant/dequant node pairs, if the observed node is a statically quantized custom module nodes, we'll convert the observer to a dequantize node, this is to keep the interface the same as the default quantized module. TODO: maybe we want to redesign this part to align with reference model design as well, but there has been some discussions around the interface, so we can do it later. rrzDExpected LSTM custom module inputs to be (input, (hidden0, hidden1))zExpected inputs to be a NodezExpected hidden0 to be a NodezExpected hidden1 to be a Nodez;Expected MHA custom module inputs to be (query, key, value)zExpected query to be a NodezExpected key to be a NodezExpected value to be a NoderzExpected arg to be a NodeNzIExpected activation_post_process to be present for observed custom module)rmrlrraddr-rrsrkrrjrr9r.rrr from_observedrr)r=rir>r:r;observed_custom_modulerinputshidden0hidden1queryrrrrquantized_custom_module_classquantized_custom_modulerrs rMr9r9s4>%S%5%56$,G)'228U044T::: !$ 0 06 UDI!##ty|U33$ ! %%**$Z,09 (V'gwfd++ E$%CDDDgt,, F$%DEEEgt,, F$%DEEE 8vu M M M 8w N N N 8w N N N N "4 1 1! U49~~""$Q!% E3eT** D$%BCCCc4(( B$%@AAAeT** D$%BCCC 8ue L L L 8sE J J J 8ue L L L L)A,Cc4(( B$%@AAA 8sE J J J&B4&Q&Q #&.$_>U " :%D ;W%%!<II%T[11K GK $(?@@@@@rPTconvert_custom_configis_standalone_module_remove_qconfig_flagqconfig_mappingkeep_original_weightsc  |t}t|tr0tjdt dtj|}t|tr4tjdt d|rtj|nd}tj |}|$t|tstdt|tr0tjdt dtj|}|t}t|std|jd } | j} | j} | j} | j} t|d }|r| j}tj |}| jrt-||t/||t1||t3|||j|| }| D]X\}}||vrtd |d ||4t9|||std|d|d||Y|} t;|j}|j}| j!tA||}tC|||tE||d}| j#}| j$}tK|}tM|'}tQ|}tS|}tU}tW|}tY|jj-D]} | j.dkr!|}!|dz }|!|vrt_| |j/| j.dkrta|dkrN| }"| j1d}#t|#tXtLfr"|D]}$te|#|$|"|jt|#tftfrd|vrte|#|"|jtjdti|#d| j.dkrtk| |}%|%tdtm|%rV| j1d}&|&|vrto| |j]|rtq|| || | |uts|| || | |t|%ttrto| |jtw|%rty| ||||t{|%tU|>|>|vrBt{|%|vrt{|%d|vrIt| || | ||||at{|%|vrt| |j||||jAt||j}|st|| | |}|rt||E|jFd d|S)a We will convert an observed model (a module with observer calls) to a reference quantized model, the rule is simple: 1. for each observer module call in the graph, we'll convert it to calls to quantize and dequantize functions based on the observer instance 2. for weighted operations like linear/conv, we need to convert them to reference quantized module, this requires us to know whether the dtype configured for the weight is supported in the backend, this is done in prepare step and the result is stored in observed_node_names, we can decide whether we need to swap the module based on this set Args: * `is_standalone_module`: when this flag is True, it means we are quantizing a submodule that is not inlined in parent module, and will be quantized separately as one unit. * `is_decomposed`: a boolean flag to indicate whether we want to use the quantize operator for decomposed quantized tensor (torch.ops.quantized_decomposed.quantize_per_tensor) or default/standalone quantized tensor (torch.quantize_per_tensor) Returns: a quantized standalone module, whether input/output is quantized is specified by prepare_custom_config, with input_quantized_idxs, output_quantized_idxs, please see docs for :func:`~torch.ao.quantization.prepare_fx` for details NzPassing a convert_custom_config_dict to convert is deprecated and will not be supported in a future version. Please pass in a ConvertCustomConfig instead.rrzPassing a QConfig dictionary to convert is deprecated and will not be supported in a future version. Please pass in a QConfigMapping instead.z0qconfig_mapping must be None or a QConfigMappingzPassing a backend_config_dict to prepare is deprecated and will not be supported in a future version. Please pass in a BackendConfig instead.z-incoming model must be produced by prepare_fxrF)remove_duplicatez Expected key z in convert node_name_to_qconfigz Expected k zD to have the same value in prepare and convert QConfigMappings, but z was updated to r placeholderroutputz1Unsupported node type for output_quantized_idxs: rzEExpected module for call_module node to be present in modules mapping)Gr"rkrrr FutureWarning from_dictrcopydeepcopyrjrrr$rUr?prepare_custom_configrr@ named_modulesrJr3r+r*r'r(rirr r2observed_to_quantized_mapping!equalization_node_name_to_qconfigr!r rinput_quantized_indexesoutput_quantized_indexesr rrur rsetr/rrrrrrrsrrrnr,r rrrrr%r:runionr;r9eliminate_dead_coderr&rdelete_all_unused_submodulespop)'r<rrGrHrIrJrrrKobserved_graph_module_attrsr?rTrr@r>prepare_qconfig_mapping modules_copyconvert_node_name_to_qconfigkvcustom_module_classesr:weight_eq_obs_dictplaceholder_node_seen_cntinput_quantized_idxsoutput_quantized_idxsr4root_module_classesr$fused_module_classesr;rAr=cur_placeholder_node_idx return_noderOr modrs' rMr8r8 sL$ 3 5 5'..U  Q      !4 =>S T T/4((    L      :I RN $_ 5 5 5d mO44O  #z/>'R'R #OPPP.$''A  K      '0@@244 u % %NLMMM"'*-K"L#6 $9%@$S#85&&&>>??G< ' 7 }W-- & - E #O^ D D D"5/:::1 #_   (F <o?Q( ( $ )..00  DAq444$GAGGG,A.:%a)Ea)HII(TaTT TT2Nq2QTT <8;#8"U"DP9HHug'9:::%000!"&;&S'<'U 6nEE. I N N P PQQ/??3NCC:=%%,/66LU[&''hh 7m # #'@ $ % * %'+??? (ek::: W ())Q..KYq\F&4-00 0C6s [%+FT4L11 ---6v{EKXXX VV VV W % %dG,,C{$[+3//>  $ !  $LLLJek%R! #.0( H! #.0( C--% F%+044! )'5, .c22c:M6N6N6T6T"77e())** 1559MMM4SV<rs)  %%%%%%%%  GFFFFFDDDDDDDD@@@@@@666666::::::333333! 0000000000CCCCCC211111BBBBBBBBCCCCCCCCMMMMMMMM,,,,,,                      )((((((2      L K L K J L K K    UY;JQ  > W ^!)- K#K# 8 K# K##ux& 'K#S%T "223 K# sJ/ K# ,% K# K#K#K#K#n)- S#S# 8 S# S##ux& 'S#S%T "223 S# sJ/ S# ,% S# S#S#S#S#| <  < <  < < < < eho $       *.sJ*?      )K))SW)))), C t E d    ,77S%T "2237sJ/7 38_ 7777tD$DuDDDDD   c58?23  X_t    ?< ?< #ux& '?< 8 ?< ?< "D( ?<  ?<?<?<?B<@"'bb bb/c3h?$Fb b  b $d38n4t; b"DcN2T9bb bbbbbbbrP