tjddlmZddlmZmZddlmZmZer ddlZddl m Z ej e Z GddeZ d deedzfd ZdS) ) ConversionOps)get_module_from_nameshould_convert_module)is_torch_availableloggingNc eZdZdZ d deeeejfdej j dzdedzdeedzdeeejff dZ dS) QuantoQuantizec||_dS)N) hf_quantizer)selfr s ^/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/integrations/quanto.py__init__zQuantoQuantize.__init__s(N input_dictmodelfull_layer_name missing_keysreturnc Xt|d\}}|d}ddlm}||||t ||\} }t j| jj| _t j| j j| _ | d| j _ d| _ |ddd} || d|| d || d iS) Nrr)_load_parameter_into_modelFT.z.weightz .input_scalez .output_scale)tupleitemsmodeling_utilsrrtorchones input_scaleshape output_scalefreezeweight requires_grad_is_hf_initializedrsplitdiscard) r rrrrkwargs_valuermodule module_names rconvertzQuantoQuantize.converts0))++,,Q/5a??????""5/5AAA(@@ "Z(:(@AA#j)<)BCC &+ #$(!&,,S!44Q7  444555 999::: :::;;; r)NNN) __name__ __module__ __qualname__rdictstrlistrTensornnModuler-rrr r s))) )-&*)- d5<001x%t  3i$&  c5< rr modules_to_not_convertc ddlm}m}m}m}m}m}||||d} d||d} d} |D]\} } t| |stj d5d}t| tj rC|| j| j| jdu| jj| |j| |j}nWt| tj jr8|j1|| j| j| j| jdu| |j }|d } || |dddn #1swxYwY| st4d |S) a Public method that recursively replaces the Linear layers of the given model with Quanto quantized layers. Returns the converted model and a boolean that indicates if the conversion has been successful or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`QuantoConfig`, defaults to `None`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list`, *optional*, defaults to `None`): A list of modules to not convert. If a module name is in the list (e.g. `lm_head`), it will not be converted. r) QLayerNormQLinearqfloat8qint2qint4qint8)float8int8int4int2N)Nr@rAFmeta) in_features out_featuresbiasdtypeweights activations)rJTzYou are loading your model using quanto but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.)optimum.quantor:r;r<r=r>r? named_modulesrrdevice isinstancer5LinearrErFrGr#rHrIrJ LayerNormnormalized_shapeepselementwise_affine set_submoduleloggerwarning)rquantization_configr8r:r;r<r=r>r? w_mapping a_mappinghas_been_replacedr,r+ new_modules rreplace_with_quanto_layersr\>s $QPPPPPPPPPPPPPPP"E5%PPIw>>I$2244== V$[2HII   \& ! ! = =J&"),, $W & 2!'!4D0 --%&9&AB )*=*I J  FEH$677 rbs/.....UUUUUUUU////////LLL  H % %     ]   J/399!I,999999r