tjV ddlZddlZddlmZddlZddlmZddlmZm Z m Z ddl m Z m Z mZddlmZmZmZddlmZddlmZmZdd lmZdd lmZmZdd lmZmZm Z m!Z!m"Z"dd l#m$Z$dd l%m&Z&m'Z'ddl(m)Z)ddl*m+Z+m,Z,ddl-m.Z.m/Z/ddl0m1Z1m2Z2ddl3m4Z4ddl5m6Z6m7Z7m8Z8m9Z9ddl:m;Z;mm?Z?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKgdZLeMeNejOdgZPejQejRejSejTejUejVejWejXejYejZg Z[e j\ejNZ]ej^j_j`jajbej^j_j`jajbdZcej^j_j`jdjbej^j_j`jdjbdZe djdedeOdedeOdeOd eOd!eOfd"Zfd#e'd$egehejijjfd!eOfd%Zkd&e dzd!elejmdzeOffd'Znd(e)d#e'd)ed*e d+e d!eOf d,Zod#e'd)ed*e d!eOfd-Zpdd.lmqZqdd/l#m'Z'eqele'e'fe'zdfZrd#e'd$egehejijjfd0egere ffd1Zsd2e"dzd3ete'dzd)ed+e d!eOf d4Zud#e'd$egehejijjfd5e.d6ed7e dzd!eleeled8fe.e dzff d9Zvd:ejijjd;ege"ewejijjfd!dfd<Zxd3e!d=eyehfd>Zz dkd#e'd&e d?ejijjd$egehejijjfd@e&dAej{dzd!e'fdBZ|d3e!dCe'd)edDe=dzd+e d$egehejijjfdEege'eOfdFeye'd!dfdGZ}d#e'd)edDe=dzd$egehejijjfd+e dEege'eOfd!egeheffdHZ~d(e'd$egehejijjfd0egere fdIeOd!e dzf dJZd(e'd$egehejijjfd0egere fdIeOd!ejmdzf dKZd(e'd#e'd$egehejijjfd0egere fdIeOd!e dzf dLZ dld#e'ezd(e)d)ed?ejijjd$egehejijjfd@e&dDe=dzd5e.d0egere fdIeOd+e dzdAej{dzd!e)fdMZ dld#e'd)ed?ejijjd$egehejijjfd@e&dDe=dzd5e.d0egere fdIeOd+e dzdAej{dzd!dfdNZd#e'dOed?ejijjd$egehejijjfd@e&dPeOd!dfdQZd#e'd?ejijjd$egehejijjfd@e&d0egere fdIeOd!e'dzfdRZdSe'd?ejijjd$egehejijjfd@e&d0egere fdIeOd!dfdTZd#e'dUejmewzdVegehe2fd!dfdWZd@e&dVegehe2fd!dfdXZd#e'd?ejijjd$egehejijjfd!eOfdYZd#e'd?ejijjd$egehejijjffdZZd#e'd)ed$egehejijjfd5e.fd[Zd?e$dVegehe2fd\egehefd5e.d]egehefd+e d^eyehdIeOd!e'dzfd_Zd?ejijjdIeOd$egehejijjfdVed5e.d+e d!dfd`Zdae$d\egehefdbegehelehewffd5e.dcegehefddedIeOd^eyehd!dfdeZ dmd?e$ddeegehefzdIeOdbegehelehewffdfeled8fd5e.egehefzdzdgeegehefzdzd+e egehefzdzdheOd!e$fdiZdS)nN)Any) FakeTensor) ObserverBaseObserverOrFakeQuantizePlaceholderObserver) BackendConfig DTypeConfigget_native_backend_config)&get_fusion_pattern_to_root_node_getterget_module_to_qat_moduleget_pattern_to_dtype_configs)_is_activation_post_process)_is_reuse_input_qconfig QConfigAny)QConfigMapping)convertpropagate_qconfig_) _parent_nameget_qconfig_dtypesget_swapped_custom_module_class NodePatternPattern) GraphModule)GraphNode)Argument)is_equalization_observernode_supports_equalization)PrepareCustomConfigStandaloneModuleConfigEntry) _find_matches_MatchResultWithQConfig)_sorted_patterns_dict)_generate_node_name_to_qconfig_get_flattened_qconfig_dict_update_qconfig_for_fusion_update_qconfig_for_qat)_default_root_node_getter!_get_pattern_to_quantize_handlersQuantizeHandler) 3_insert_dequant_stubs_for_custom_module_lstm_output_is_custom_module_lstm+_maybe_get_custom_module_lstm_from_node_arg+_qconfig_satisfies_dtype_config_constraintsall_node_args_have_no_tensorsassert_and_get_unique_deviceget_custom_module_class_keysget_new_attr_name_with_prefix(get_non_observable_arg_indexes_and_typesnode_arg_is_biasnode_arg_is_weightNON_QUANTIZABLE_WEIGHT_OPSObservedGraphModuleAttrs)insert_observers_for_modelprepare propagate_dtypes_for_known_nodesdtypeinput_act_obs_or_fq_ctroutput_act_obs_or_fq_ctrFprev_output_dtypeprev_output_is_dynamiccur_target_dtypecur_target_is_dynamicreuse_input_obs_or_fq is_zeroth_argreturnc|r5|tvrtd||tvrtd|S|rdS|tvr|ttjgzvo||kSdS)a note: we will treat "not specified" as torch.float for now utility function that checks if we should insert an observer or fake quant node base on the requested dtype for the nodes from user is_zeroth_arg: we only dynamically quantize the first arg of the node right now this should be removed when we enable configuring dynamic quantization for a specific argument, this can be removed if we deprecate fx graph mode quantization z6Expected cur_target_dtype to be torch.float, but got: z7prev_output_dtype must not be in _DO_NOT_OBS_DTYPE_LISTF)_OBS_DTYPE_LISTAssertionError_DO_NOT_OBS_DTYPE_LISTtorchfloat)rArBrCrDrErFs ^/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/ao/quantization/fx/prepare.py_needs_obs_or_fqrOks,  ? 2 2 [IY[[   6 6 6 I u?** EK=!@ @ 6 $55 5node named_modulesct|tjjo1|jdko&t |t |jS)N call_module) isinstancerLfxroprstrtarget)rQrRs rN _is_activation_post_process_noderZsJ 4'' I G} $ I ' c$+6F6F(G H HrP obs_or_fqc:|dS|jt|ddfS)zgGiven a constructor for observer or fake quant module, returns a Tuple of dtype and is_dynamic N)NF is_dynamicF)r=getattr)r[s rN_get_dtype_and_is_dynamicr_s({ < G GGGrPargqconfig dtype_configbackend_configct|ttfrtfd|DSt|tsdSt |}t |}| o| }|rjdd}|r |nd} t| \} } j dupDj | ko9tj t| kotjS|rmjddd} | r | nd} t| \}}j}||k}tjd}|dup|o|Sjdd d}|r |nd}t|\}}j}|dup||kS) z\Check if the configured qconfig for the argument is supported by the backend or not c3>K|]}t|VdSN)(_is_input_arg_dtype_supported_by_backend).0arcrbrQras rN z;_is_input_arg_dtype_supported_by_backend..sR    5          rPTtarget_dtype_infor?Nweight_obs_or_fq_ctrF) is_activationbias_obs_or_fq_ctr)rUlisttupleallrr6r5metagetr_ input_dtypeboolr]r/input_dtype_with_constraints weight_dtypeweight_dtype_with_constraints bias_dtype)r`rQrarbrc is_weightis_biasrmr?input_act_obs_or_fq qconfig_dtypeqconfig_is_dynamicrlweight_obs_or_fqqconfig_weight_dtype_backend_config_weight_dtype dtype_matchesqconfig_satisfies_constraintsrnbias_obs_or_fqqconfig_bias_dtypebackend_config_bias_dtypes ```` rNrgrgs#e}%%                 c4 t"4--ItS))G!M1'kM- "&),?"@"D"D %# # *A J # # % % %d -F - - ) ) (D0  $ 5 \,--6H1I1II ;B   #y)<=AA "D  6JS//111t";> rPc|j}d}|jddt}|r |nd}t |\}}|r t j}||k}t||j} |dup|o| S)zZCheck if the configured qconfig for the output is supported by the backend or not Nrkr@) output_dtyperrrs_DEFAULT_FP32_OBS_OR_FQ_CTRr_rLfloat32r/output_dtype_with_constraints) rQrarbbackend_config_output_dtypeqconfig_output_dtyper@output_act_obs_or_fqqconfig_output_is_dynamicrrs rN%_is_output_dtype_supported_by_backendrs#/"; #y)<=AA"$?  '?H  """D7P7733!-$}(,GGM$O;%%! '$ . 77rP) Annotated)r obs_or_fq_mapct||||}t|jdkrRt|jdtr2|t jt jfvr|jdjdkrdSdS)zCheck if observer in same graph when the node output is not fp32 and input is 'placeholder' the input is assumed to be quantized, so it is observed in a different place rather than not observed. r placeholderFT) _get_arg_target_dtype_as_outputlenargsrUrrLquint8uint8rW)rQrRris_qatnode_output_dtypes rN_is_observer_in_same_graphr'sx8 m]F 49~~j1t<< %, !< < < ! =005 4rPpatternmatched_node_patternc ||dS|t|dkrtdt|}||g}t |}||t }||}|} |d} |D]o} d} t | jt | j zD]} | ot| | || |} | ot| || } | rdSpdS)zCheck if the dtype configuration of a pattern is supported by the backend or not, and whether the qconfig satisfies constraints specified in the corresponding dtype config. NTrz&matched_node_pattern must be non-emptyrF) rrJr rsr r)rorkwargsvaluesrgr)rrrarcpattern_to_dtype_configs dtype_configspattern_to_root_node_getterroot_node_getter root_node input_node output_noderb supportedr`s rN9_is_pattern_dtype_config_and_qconfig_supported_by_backendr>sRt#s+?'@'@1'D'DEFFF;NKK'?'C'CGR'P'PM"H"X"X266*! !566IJ&q)K%     ((4 0A0H0H0J0J+K+KK  C!&NZ,''II "G ,# #   44  5rPprepare_custom_configparent_qconfigparent_backend_config.ct|j}t||}tdddd}|j||}|j||}|jp t |}|j } |j p t}|j p|} || || fS)z Returns the standalone module QConfigMapping and PrepareCustomConfig for `node`, assuming that the module pointed to by `node` is a standalone modules. N)rXrYtyper!standalone_module_classesrsstandalone_module_namesqconfig_mappingr set_globalexample_inputsrr rc) rQrRrrr module_name module_type config_entryrrrcs rN_get_standalone_module_configsrfsdk""K}[122K.tRtDDL(BFF\L)@DD\L#2n6F6F6Q6Q77O"0N(>WBUBWBW!0I4IN ^-BN SSrProotmodule_to_qat_modulec,t||dddS)NTF)mappinginplaceremove_qconfig)r)rrs rN_qat_swap_modulesrs" D.USSSSSSrPsct|tr||jdSt|tt fr|D]}t ||dSdSrf)rUraddnamerorp_add_matched_node_name_to_set)rr maybe_nodes rNrrs~&--9 "'((((( (4- 8 89. 9 9J )*a 8 8 8 899 9 9rPmodelgraph model_devicecz|t|}|r||t|r |jdz}nd}t |}||}t ||||||<||5|d||fi} dddn #1swxYwY| S)z Attaches `obs_or_fq` to `model`, and creates a node which calls `obs_or_fq` on the output of `node`. obs_or_fq: an instance of Observer or FakeQuantize module N_equalization_process_activation_post_process_rT)r1torrr3setattrinserting_after create_node) rQr[rrRrrprefixget_new_obs_or_fq_nameobs_or_fq_namenew_obss rN_insert_obs_or_fqrs)3E:: # \""" **,55+:6BB++E22N E>9---$-M.!   t $ $PP##M>D7BOOPPPPPPPPPPPPPPP Ns B00B47B4 last_nodeqhandlercache_for_no_tensor_checkprocessed_nodesc lt|ttfr|D]}t||||||||dSt|tret|tst d|} | |vrdS|| |dSt| |||||} | | jd<dSdS)zSets the target_dtype_info for each node in matched_node_pattern Note: processed_nodes is used to ensure we only process each node once z#matched_node_pattern must be a NodeNrk) rUrorp/_set_target_dtype_info_for_matched_node_patternrrJr%_get_target_activation_dtype_for_noderr) rrrarrcrRrr node_patternrQrks rNrrs&u 66(;0  L ;)       ($ / /;.55 H !FGG G# ? " " FD!!! ? F-R      % - - *; %&&&3;;rPc Bt|||}|rdddS|lt|\}}} |tjkr|tjkr| s tjn tj} |duo|} |duo|} d} t|trC|j dkr8|j |j vr*|j |j j d} d}t|trC|j dkr8|j |j vr*|j |j j d}|j|jt!j| | ||jt%|| | d St'jt(S)a< For each op attribute in the op's input activation, output activation, weight, bias - returns the settings of dtype and is_dynamic we expect for the `quantize` call in the reference model representation, or None if there is no `quantize` call needed. For example, if we have a node corresponding to `op0` in x0 -> op0 -> x1 And we want a reference quantized representation to be x0 -> quant_static -> dequant -> op0 -> quant_dynamic -> dequant -> x1 Then this function will return { "input_act_obs_or_fq_ctr": MinMaxObserver.with_args(dtype=torch.quint8, is_dynamic=False), "output_act_obs_or_fq_ctr": MinMaxObserver.with_args(dtype=torch.quint8, is_dynamic=False), } TODO(future PR, if needed): explicitly spell out the non-Tensor dtypes. Nr> call_functionweightbiasr<) r?rlrn weight_index bias_indexr@rEinput_output_share_observers_is_standalone_module)r0rrLfloat16rMis_general_tensor_value_opis_standalone_modulerUrrWrY!_pattern_complex_format_to_config_input_type_to_indexrs activationrr with_argsrcopy+_DEFAULT_FP32_QCONFIG_FOR_TARGET_DTYPE_INFO)rQrarrRrcrargs_have_no_tensors act_dtyperwinput_act_is_dynamicryrrrrs rNrrs@9 m6 '+(,   8J78S8S5 #%%! &%rPct|tstdd|jvrt dt ||}t ||}| o| }d}|r'|jddt}n]|r5|j tvr&|jddt}n&|jddt}|r |ndS) zcGet the observer or fake quant constructor for the Argument `arg`, as input to Node `node` rrrNrkr?rlrn) rUrrJrrrr6r5rsrrYr7) r`rQrRrrrzr{rm obs_or_fq_ctrs rN_get_arg_as_input_act_obs_or_fqrs c4 31222 DI--! [   #4--ItS))G!M1'kMM   "56:: %'B    ;8 8 8 I&9:>>&(CM "56:: "=  , 5==???5rPc t|ttfrNg} |D]1} t|| |||||||| | | }| |2t || St|t s|St|t std|}|duo|}|sd|j vrtdd|j vrtd|j d}| dd}t||||| }t|\}}t|||| }t|\}}t|||||t!|jd ko||jd u }n|td t%||||| \}}}}|j}d}t)|jD] \}}||ur|}n|d}nDt+|||| }||vr t,jn t,j}||ko|t,jk}|j} | r | nd}|rd}!|jD]I}"|"jd kr<||"j}#t |#t |ur|#j|kr|#}|"}!nJ|td ||||f<|!t=|||||| }$|$}n|!}|S)zk Given a `node` and an `arg`, inserts an input observer between `node` and `arg` if necessary. rNrrrk)expected 'target_dtype_info' in node.metarEFr)rFzqconfig must not be NonerTz+arg_as_input_act_obs_or_fq must not be None)rUrorp-_maybe_insert_input_observer_for_arg_or_kwargappendrrrJrrrrrsrr_rrOrrrinput_quantized_indexes enumeraterrLrrMrusersrWrYr=r)%rQr`rarrRrrrrrrcrnew_arg_to_return inner_arg new_inner_argnew_argrrkrEarg_as_input_act_obs_or_fqarg_as_input_target_dtypearg_as_input_target_is_dynamicrrarg_as_output_target_is_dynamicneeds_obs_or_fqrsm_prepare_custom_configsm_input_quantized_idxs cur_input_idxarg_idx arg_to_checkact_post_process_ctrexisting_obs_nodemaybe_obs_node maybe_obs_mod new_obs_nodes% rNrrs(#e}%%, 4 4II%  M  $ $] 3 3 3 3tCyy*+++ c4  c4 31222G#4/SH4Q4Q4S4S N % 1 1%_  di / / !LMM M!I&9:!2 5 56Mu U U%D }mV& & " &&@ A A % *'@  v' ' # &&A B B & ++ & + % * !di..1,D ! 1D     ? !;<< <,J -!6- - )1&#;"R %.ty%9%9   !G\s"" ' #  #OO)H]M6** & !$;;; [ & +.GG=, ;  '1&: D " " " #%( "i  N M11 -n.C D ''40J+K+KKK%+/HHH1>.(6%E % - !NOO O%? sDk"  $,* L#GG'G NrPc 2g} |jD]1} t|| ||||||||| | } | | 2i}|jD]$\}}t||||||||||| | }|||<%t | |_||_dS)aK If needed, inserts observers to the input args and kwargs of `node`. Note: modifies `node` inplace. For example, if cur_node needs an observer after prev_node, we change from prev_node -> cur_node To prev_node -> obs -> cur_node Note: backend_config only needed for standalone_module node N)rrrritemsrp)rQrarrRrrrrrrcrnew_argsr`r  new_kwargskkwarg new_kwargs rN&_maybe_insert_input_observers_for_noder"xs:Hy!!?        !            JK%%''""5A        !       " 1 hDIDKKKrPequalization_qconfig is_branchc|t||sdS|rtjd|dddSg}|jD]}t |t rt ||r||=t||}|r|j n|j } | } t|| |||} || t||_dS)z If `node` needs to be equalized, find the input/weight observers it needs in `equalization_qconfig`, creates them, and inserts it into `graph`. If `node` does not need an equalization observer, returns None. NzCannot equalize z because it is part of a branch. stacklevel) rwarningswarnrrUrr5rr6rinput_activationrrp) rQr#rrRrr$rr`rzact_eq_process_ctrnew_eq_obs_modnew_eq_obs_nodes rN3_maybe_insert_input_equalization_observers_for_noder/s/#+E m,,#  Et E E ERS     Hy))#t$$ (8s(C(C  OOC &tS11  7 ' '%6  ,+--+  u   ((((hDIIIrPc|jdkrtdd}d|jvrtdd|jvrtd|jddd}|jdd }|r |nd }t |\} } d} t tjd| | | } | rd} | o| } | r|||<t|||||Sd S) a$ If `node` needs an output observer, creates it, inserts it into `graph` and returns it. If `node` does not need an output observer, returns None. Note: inserting dynamic quantization ops for output is not supported in fx graph mode quantization code path right now outputz3observer insertion for outputs is handled elsewhereFrrrkrrr@N) rWrJrrrrsr_rOrLrMr) rQrrRrrrrr@r target_dtypetarget_is_dynamicrErs rN&_maybe_insert_output_observer_for_noder4s_" w(RSSS  DI--! [   $)++HIII9%89== $y)<=AA"  '?H  """D'@@T&U&U#L#" ' UL*;=RO &D/C+CO2 d &}e   trPgraph_output_nodec dtdtjjdtt tjjfdt dtf fd fd|jD}t||_dS) z If the output needs to be quantized and there are any nodes in the output which are not already observed, inserts observers for those nodes. rrrRrrGct|trt|}d}tj}d|jvr4|jddd}||}|j}||ko|tjk}|r&|tdt||} | S|St|ttfr7fd|D} t|tr| St| St|tr/i} | D]\} } | | | <| S|dStd|)a` Navigate an arbitrary data structure of lists, tuples, dicts. For each container type, recurse on all inputs. Once any Node is found, insert an observer if needed and do not recurse further. For example, given a structure of {'foo1': [[bar1]], 'foo2': {'foo3': [[[bar3]]]}} we recurse down to bar1 and bar3, observe them if necessary, and if we inserted an observer then replace the original node with its observer. Returns the data structure with all nodes needing observation being replaced by their observers. Nrkr?z3observer_mod must not be None when need_obs is Truec,g|]}|Srr)rh inner_node&_recursive_maybe_replace_node_with_obsrrrRs rN zo_maybe_insert_observers_before_graph_output.._recursive_maybe_replace_node_with_obs..sD 76! rPz!Unhandled type for returned node:)rUrrrLrMrrrsr=rJrrorpdictr Exception)rrrRrr observer_modr  observer_clsneed_obs observer_noderesults results_dictrinner_vr:rrs ``` rNr:z[_maybe_insert_observers_before_graph_output.._recursive_maybe_replace_node_with_obsQs, j$ ' '9 )HM=&** & L(- %"jo55)/BCGG-t    +#/<>>L0<0B-+.GG=-<  "'(M!2 e]E!! %$!!  T5M 2 2 #-G*d++ &W~~%  D ) ) L(..00   7"H"HUM5## Q   43Z rPc,g|]}|Srr)rhold_argr:rrrRs rNr;z?_maybe_insert_observers_before_graph_output..s:  /.w}eTTrPN) rrLnnModuler<rXrrrp)r5rrRrrrrr:s ````` @rN+_maybe_insert_observers_before_graph_outputrICsOOxOC01O O  OOOOOOOOb(-H #8__rPr2&node_name_to_match_result_with_qconfigc&d|jdd<d|jdd<||jd\}}}}}|I|r7|jd}t |t rt|||dSdSdSdS)z Assigns `target_dtype` to `node`, setting `is_dynamic` to False. If `node` is a general tensor shape op, also call this function recursively on the first argument, to propagate the dtype to the caller. Nrkr?r@NNNNNr)rrrsrrrrUr_maybe_propagate_dtype_for_node) rQr2rJ _root_noder_patternr_qconfig prev_nodes rNrMrMsAEDI!"#<=AEDI!"#=> /22 1      C C E EIaL i & &  +<)O       rPc Z|jD]}t|}|D]}|||}|D]x}|j|}t|tt frt |}n|g}|D]7} t| t jjj rt| ||8ydS)a Currently we assume that inputs to the graph are either `torch.float` or `torch.quint8`, which is not always correct. For ops such as `x.masked_fill(mask, value)`, we know that the dtype of `mask` is a `BoolTensor`. Propagate this information throughout the graph. Note: not all dtypes in the graph will be correct after this pass, but a higher percentage of them will be correct. Hopefully in the future we can replace this with a better way to reason about dtypes of tensors. N) nodesr4rrUrprorLrVrQrrM) rrJrQnon_observable_arg_dictarg_typenon_observable_indicesindexr`arg_listcur_args rNr;r;s "J4"P"P/  H%F%>7#X/U  rPcd}tt|jD]>}t|j|tt t fr|j|}n?|dSt|t t fr |d}nt|tr|}ndSd}t||st|tsdS|jdkrdSd}tt|jD]&}|j|}t|trn'|dS|}|dz }|dkrtdt||t|tstd|j }t|tstd ||} t|t t frt|D]\} } | dkr d}t| |sRt| jdkrdS| jd} |dz }|dkrtdt| |Rt| j \} } t|| | | |jD]O}t||std t|j \} } t|| | | Pd S) a Ensures that we share an observer for all input arguments as well as the output argument. In detail, given a graph of x0 -> obs0 -> op -> x2 / x1 -> obs1 / where node obs0 points to observer instance observer0, obs1 points to observer1 and obs2 points to observer2, we make nodes obs1 and ob2 point to observer0. Returns: whether the operation succeeded or not NFrrri'z(Unable to find observer of previous nodezfirst_arg_arg must be a Nodeztarget_to_use must be a string7output_obs_node must be an activation post process nodeT)rangerrrUrrorprZrWrJrYrXrrrr)rQrrR first_argi first_arg_argiteration_guardtrace_back_node target_to_useobs_mod_to_use input_idx input_arg parent_nameroutput_obs_nodes rN(_maybe_make_input_output_share_observersrhs<&I 3ty>> " " dilT4$7 8 8  ! I E  u)dE]++!! It $ $! u O.}mLLM-.. 5  } , ,5s=-..//  A+03O/400    "5' 1 U " " !KLL L%/}mLLM( mT * *=;<<<!(M mS ) )?=>>>"=1N)dE]++F$-i$8$8 F F IyA~~O6y-PP Uy~&&** 55%N1- 1$"U**()STTT7y-PP U!-Y-= > > K M+.n E E E E :BB/OO  I ))?@@ T k*D.AAAA 4rPct|j}|D]S\}}t||st d|||j|TdS)Nr[)rorrrZrJreplace_all_uses_withr erase_node)rQrrRrrgrs rN_remove_output_observerrlWs !!## $ $E#00/OO  I  --d333 //// 00rPc||j}|j}t|||}||}t |j\}} t ||| |dSrf)rYfloat_to_observed_mappingr from_floatrr) rQrarRr custom_modulecustom_module_class_mappingobserved_custom_module_classobserved_custom_modulerfrs rN_swap_custom_module_to_observedrtdsy "$+.M"7"Q#B2G$$ :DD]SS$T[11K M+ &.DEEEEErPnode_name_to_qconfigequalization_config_mapobserved_node_namesc i}t|d} |j} |j} t } |jjD]#} tjt| j d<$d}d}i}i}|jjD],} | j dkr ||| <|dz }| j dkr ||| <|dz }-| D]1}|\}}}}}|td t|||||| || 2|jjD]} | j dkr,|| | vr"tjt| j d<9| j d vr!t| | |}|r ddd | j d<c| j dkr+|| | vr!tjt| j d<t!|j|t } | D]}|\}}}}}t#||||}|td | j dd }|r |nd}t%|\}}|sK|dt&t(t*jfvr/t||t*jjjjd|| || t7|jj}t }d}d}d} i}!t9|}"|D]} | j dkrn| j d vr|| jd\}}}}}|| j}#| j d}$d| j vr%|$duot?| j dt@}%n|$du}%|dup|% o | j dk}&t#||||}|&s>|r;t|d} | j dkr|tdtC||d}'tE| j#dkrt?| j#dtHrtE| j#dj%dkrs| j#dj%D]`}(||(jdup7|(j dko,t?| tM|(j'tP})|(| kr|)rd}'atS|}*|*|tT}+|+|},| |,u}-|-r7tW| ||| |j|||!|||" tY| |#|| |j|'| |u}.| j ddd}/| j ddd}0|.rFt[| | ||rNt]| || |j| j'|vr,|/| j'ta| || |ntc| || |j|!|}1|1t7| j%2}2|2D]}3|3|1ur|33| |1ti| | |!|}4|/r|4s|0r"tk| || stm| || |I|7r5| j'|vr,|/| j'ta| || |ntq| || |j|!|| j dkr|dz } | j dkr|dz }| } | S)a$ Inserts observers, using the following high level algorithm: For each node in the graph: 1. determine the target dtype of this node in the quantized graph, and save it for future steps 2. determine the target dtype or all args and kwargs of this node 3. if any arg or kwarg's target dtype does not match the current node's dtype, insert an observer 4. if the current node needs an output observer, insert it For example: - starting graph: x0 -> linear -> x1 - observed graph after processing x0: x0(fp32) - observed graph after processing linear: x0(fp32) -> x0_obs0(int8) -> linear(int8) -> linear_obs0(int8) - observed graph after processing x1: x0(fp32) -> x0_obs0(int8) -> linear(int8) -> linear_obs0(int8) -> x1 After a node is processed, the naive observer placement is guaranteed to be complete for that node and all of its predecessors. There can be future passes which optimize the graph by deduplicating observers, etc. Fremove_duplicaterkrrrr1Nzqhandler must not be None)rT call_methodrr>r@)rTr{rr1rLvalz%matched_node_pattern must not be NonerTTrrE)9r<rRroutput_quantized_indexessetrrSrrrrrWrrJr-_DEFAULT_QUINT8_QCONFIG_FOR_TARGET_DTYPE_INFOr0r;rr_intrMrLruao quantizationra!_default_fp32_placeholder_qconfigror1rsrrUrrrrrrrXrYrr r)r"r/r-r,rrtr4keysreplace_input_withrrhrlis_custom_modulerI)5rrJrurrvrcrwrrrRinput_quantized_idxsoutput_quantized_idxsrrQinputs_seen_counteroutputs_seen_counterplaceholder_node_to_input_indexoutput_node_to_output_indexmatch_res_with_qconfigrrrrraris_supported_by_backendoutput_act_or_fq_ctroutput_act_or_fqoutput_act_dtypernodes_before_observation#custom_module_names_already_swapped results_noderrr#this_node_dtype_infooutput_is_a_tensorskip_inserting_observersis_quantized_branchuseris_user_quantizedrrris_input_node_of_the_patternis_last_node_of_patternrrEmaybe_output_obs_node orig_users user_node_is_observer_in_same_graph_s5 rNr9r9ts x35 ,,e,DDEEM&;&S'<'U!$O !  )- 7* *  %&&79#46 !&& 7m # #4G +D 1 1 $  7h  0D ' - A % #I"O"O"Q"Q   #        !<== =7      %   !!! G} $ $/59MMM .2Y=..DI) * *WG G G#@m%>$$ $ /30422 -. Gx  +D15JJJ.2Y=..DI) *% ;"%O"H"O"O"Q"Q(( #      F-w     !<== = $y)<= & 6JS//111t78HII!& +;    J D , ,  <$%-O)   $ $EK$56658EE'L>@M/66L)G G  7m # #  WQ Q Q7:: 9  $$;#>#>ty#I#I #'9-@#A  !!%9%E&*Ie$jKK""&:%E"D;);%;((7h& %J17N $ ,P 0GP  $U%8%8%%8%P%P Q Q 7h&&+3,-TUUU1,.A+0'DI**&ty|T::+ ! 233a77$(IaL$6 ; ;D0D0H0H $ 11%)1*1!%= 8!"$.$1#dk2B2B$C\%"%" . $t||0A|6: 3?~NN0(C'F'F!:(($!1 01E F FI3793D03> #!)!K$1)"*(   L 0!)!K/ /3i.?+379=P3Q3U3U6440-1I6I,J,N,N/--)/z*1 )#$ y*$P $ % - %  ${2UUU C G G T T T ?$($+$1$9 !"!"!"!G$($)$1$)K$1$*!"!"2 5@".2$*//2C2C-D-D 1;!&!&I'04I'I'I(0$-$@$@(,.C%&%&%&%& %?(,(5(5(. %&%&!<%A!*(C!*&;!*,T(,(-(5 ,&,& %* )@,0,1,9 )*)*)*$,#7HS T T.$''A  K      '0@@ o~ 6 6IGHHH *N ; ;NLMMMmO44O=)=>>CE244"CN"S"S"78S"T"TEnUUuo666u&:;;;8IIu46K6S6S6U6UVVV A7GG%!5666@@@,,e,DDEEM)G   ))%:   ##8#P#U#U#W#WXX $7<<>>!!97, # !.0*,C,I,I,K,KOO( (V4V6J96UVVrs%  ((((((   GFFFFFMMMMMMMM@@@@@@FFFFFFFF! &&&&&&&&""""""KKKKKKKKKKKKKKKK????????000000  "   uej$7 L K L M K J K K   <1;%+NNN %x4<^i % 5 = _ j//+ %x4<`k % 5 = a l11- *** ** *  *  * ****Z #C$89  H%, H 5; t #$ H H H HH H H H  H " H  H H H H V" " "" """"JuT4Z(4/5 6  UX_,- $::;.% t^%t*t+%%" %  %%%%PT TUX_,-T/T T )4/ T  >5c?,?QUAU UV TTTT@T (/T15gtEHO?T6T1UT TTTT 9 9C9999)-  % 8?UX_,-    ,%  @5;%5;5;5;$ 5; " 5; UX_,- 5; $D$J/5;Y5; 5;5;5;5;pbB bB bB$bBUX_,- bB " bB $D$J/ bB #s(^bBbBbBbBJB B UX_,-B  $::;B   B d" B B B B J &  &UX_,- & $::; &  &  [4 & & & &%6 %6 %6UX_,-%6 $::; %6  %6 d" %6%6%6%6f,0(,hh *h hh 8? h UX_,- h  h$h/h $::;h h"D(h,%hhhhhj,0(,CC C C 8?CUX_,- C  C $ C/C $::;C C"D(C,%C CCCCL/ / /  8?/ UX_,- /  /  /  / / / / dP P 8?PUX_,-P  P  $::; P  P D[PPPPfd-d- 8?d-UX_,-d-  d-  $::; d-  d- d-d-d-d-N +$-16M1M,N ># #,06M1M,N# ####Le e 8?eUX_,-e eeeeP 0  0x 07;C9D@e"DcN2T9eeeeeeeerP