tjddlmZddlmZddlmZmZer ddlZddlm Z ej e Z GddeZ Gdd ejjZGd d e jZdd eedzfdZdS)) ConversionOps)should_convert_module)is_torch_availableloggingNc reZdZdZ ddeeeejfdedzdeeejffdZ dS) EetqQuantizec||_dSN) hf_quantizer)selfr s \/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/integrations/eetq.py__init__zEetqQuantize.__init__s(N input_dictfull_layer_namereturnc t|d\}}|d}|j}tj|}t|tj d\}}| |}| |}|||d|iS)NrF_scales) tupleitemsdevicetorcht contiguouscpueetq_kernels_hub quant_weightsint8to) r rrkwargs_value value_device int8_weightscaless rconvertzEetqQuantize.converts))++,,Q/5a| genn//115577 .<<[%*V[\\ V!nn\22 <((.I.I.I6RRrr ) __name__ __module__ __qualname__rdictstrlistrTensorr'rrr r s)))X\ S SsD$667 SJMPT* S c5<  S S S S S Srr c<eZdZeddZedZdS)EetqLinearMMFunctionNc|||||t|||}|||zn|}|Sr )save_for_backwardr w8_a16_gemm)ctxxweightr&biasoutputs rforwardzEetqLinearMMFunction.forward0sL a666!--a@@"&"2$ rc|j\}}}}tj|jd|j|j}t|||}|j drO| d | dd d}|dddfS)Nr) saved_tensorsreyeshaper rdtyperr4needs_input_gradsqueezematmul transpose unsqueeze)r5 grad_outputinputr7r&r8identity grad_inputs rbackwardzEetqLinearMMFunction.backward8s&)&7#vvt9V\!_--00??BB5;OO"--hGG   " \$,,Q//66v7G7G17M7MNNXXYZ[[J4t++rr )r(r)r* staticmethodr:rJr/rrr1r1/sM\ , ,\ , , ,rr1c4eZdZejdffd ZdZxZS) EetqLinearFcttjt j||f|d|_tjt j|tj|_|r9tjt j|tj|_ dSd|_ dS)N)r@F) requires_grad) superrnn Parameterremptyr7float16 weight_scalesr8)r in_features out_featuresr@r8 __class__s rrzEetqLinear.__init__Hs l5; \/JRW#X#X#Xhmnnn \%+|EM*Z*Z*Z[[   U[,u}%U%U%UVVDIIIDIIIrc^t||j|j|j}|Sr )r1applyr7rUr8)r rGr9s rr:zEetqLinear.forwardQs)%++E4;@RTXT]^^ r)r(r)r*rrrr: __classcell__)rXs@rrMrMGsS8= rrMFmodules_to_not_convertcddlm}|ddad}|rinddi}|D]\}}t ||st jd5t|tj r9t|j |j fd |j dui|}|||d }dddn #1swxYwY|std |S) a A helper function to replace all `torch.nn.Linear` modules by `EetqLinear` modules. Parameters: model (`torch.nn.Module`): Input model or `torch.nn.Module` as the function is run recursively. modules_to_not_convert (`list[`str`]`, *optional*, defaults to `None`): Names of the modules to not convert in `EetqLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. r<) get_kernelz#kernels-community/quantization-eetq)versionFr@Nmetar8TzYou are loading your model using eetq but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.) hub_kernelsr^r named_modulesrrr isinstancerQLinearrMrVrWr8 set_submoduleloggerwarning) modelr\ pre_quantizedr^has_been_replaced module_kwargs module_namemodule new_modules rreplace_with_eetq_linearroVsx('''''"z"GQRSSS'rxsE/.....??????////////LLL  H % %SSSSS=SSS(,,,,,5>2,,,0        %%DI