tj|dZddlmZmZmZerddlZejeZdZ dZ dZ d dZ d d Z dS) z2HQQ (Half-Quadratic Quantization) integration file)is_hqq_availableis_torch_availableloggingNcH|D] \}}||_ dSN) named_modulesname)modelr modules [/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/integrations/hqq.pyautoname_modulesrs4++-- f cfdd|dDS)N.cBg|]}|dv||S))r layers) isnumeric).0ns r z&name_to_linear_tag..!s5lll1Q>Q5Q5Q\]\g\g\i\i5QQ5Q5Q5Qr)joinsplit)r s r name_to_linear_tagr s- 88ll 3lll m mmrctrddlm}t}|D]H\}}t |t jj|fr"| t|It|S)Nr) HQQLinear) rhqq.core.quantizersetr isinstancetorchnnLinearaddrlist)r r linear_tagsr r s r get_linear_tagsr&%s0//////%%K++--66 f fux : ; ; 6 OO.t44 5 5 5   rc|D])\}}|g}||t|tjjrt |j}||vr_||W|||j|_ t||j|_ |j| dd}dD]}t||dtt|dkrt#|||\}}|d+||fS)NFT)W_qmetar patch_paramshas_been_replaced)named_childrenappendrr r!r"rr _modules quant_configtype source_clsrequires_grad_setattrlenr$children_prepare_for_hqq_linearpop) r r+r,current_key_namer r linear_tagatt_s r r8r80s[,,..!! f  #! %%% feho . . ++FK88J\)) +78DZ8PEN4(56:6llEN4(3N4(77>>> $ ' + +T**** tFOO%%&& ' '! + +#:)"3$$$ A R    # ##rFc<|gn|}t|t||j}|j}t t t |z t |z t fd|Dr0t}| |nt|}t|||\}}||j |d|j _ |std|S)a Prepares nn.Linear layers for HQQ quantization. Since each layer type can have separate quantization parameters, we need to do the following: 1- tag each module with its name via autoname_modules() 2- Extract linear_tags (e.g. ['self_attn.q_proj', ...]) 3- Map quantization parameters as a dictionary linear_tag -> quant_params as HQQLinear expects it, this is referred to as patch_params Nc3 K|]}|vV dSr)rkeyr%s r z)prepare_for_hqq_linear..ks( 6 6#3+  6 6 6 6 6 6rr*)r1 quant_method skip_moduleszY>YYZZK 6 6 6 6 6 6 666@}}[11 L))))}}[,?? 6 LrSs54AAAAAAAAAALLL  H % % nnn "$"$"$"$J,,,,,,r