Otj-FdZddlZGddZGddZdS)aN Helper classes for working with low precision floating point types that align with the opencompute (OCP) microscaling (MX) specification. * MXFP4Tensor: 4-bit E2M1 floating point data * MXScaleTensor: 8-bit E8M0 floating point data Reference: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf Nc4eZdZddZdZdZdZdZdZdS) MXFP4TensorNc||_|Lt|tjs Jd|j|_|||_dS|!t|t r|n|f|_dStd)at Tensor class for working with four bit E2M1 floating point data as defined by the opencompute microscaling specification. Parameters: - data: A torch tensor of float32 numbers to convert to fp4e2m1 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. N%Parameter data must be a torch tensor.Either parameter data or size must be provided device isinstancetorchTensor _from_floatdatatuplesize ValueErrorselfrrr s O/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/triton/tools/mxfp.py__init__zMXFP4Tensor.__init__s  dEL11 Z Z3Z Z Z1+DK((..DIII   *4 7 7EdXDIIIMNN Ncttjdd|jtj|j}tjdd|jtj|j}tjdd|jtj|j}|dz|dzz|ztj|_|S)Nrrdtyper )r randintruint8r typer)rSEMs rrandomzMXFP4Tensor.random#s M!QTYek$+ V V V M!QTYek$+ V V V M!QTYek$+ V V V1fa(1,225;??  rc|tjks Jd|j}|dz dz|}|dz dz|}|dz|}tj|}|dk|dkz}|}|r||} ||} ||} tjd| } tj| dk| | dz } tj| dk| dzd| dzz}| tjd| z|z}|||<|||dkzxxdzcc<|tjS) z Convert fp4e2m1 data to float32. Returns: - A torch tensor of type dtype representing the fp4e2m1 data. zCCurrently only float32 is supported for fp4e2m1 to float conversionrrr??r)r float32rr zeros_likeanypowwhere)rrrr!r"r#valueis_zero non_zero_maskS_nzE_nzM_nzsignexponentmantissavalue_nzs rtozMXFP4Tensor.to+s %%%'l%%%yai3  $ $U + +ai3  $ $U + + CZ  e $ $ ##6a1f%       ,]#D]#D]#D9R&&D{419dD1H==H{419dSj#s :JKKHei8444x?H#+E-  ga !!!R'!!!zz%-(((rctj|tj}tj|}|dk}tj|tj|z}tjgdtj|j}tjddgtj|j}g}g} g} |D]} | dkrTd} |D]N} | dz}|d| zz}| || | | | O\| dz } |D]Q} d| dzz}|d| zz}| || | | | Rtj|tj |j}tj| tj|j} tj| tj|j} | d}|j d}|d}| }||| d<tj||dz }tj|dd \}}||k}|dkrT| d|d}|dktj}||d zz }tj|d }| |}| |}| |j } | |j } d| |<d| |<|d z| dzz| ztjS)a5 Convert float32 numbers to mxf4 e2m1 format. * No encodings are reserved for Inf or NaN in mxf4. * Conversion from float supports roundTiesToEven rounding mode. * If a value exceeds the mxf4 representable range after rounding, clamps to the maximum mxf4 magnitude, preserving the sign. * If a value has magnitude less than the minimum subnormal magnitude in mxf4 after rounding, converts to zero. Parameters: - values: A torch tensor of float32 numbers to convert to fp4 format. r)rrrrrr rr'rr(r&T)dimkeepdimgư>r;r)r signbitr rabsisnanisinftensorr appenditemr)viewshape unsqueezemaxminsumexpandint32argmin)rvaluesr! abs_valuesr/ is_invalidE_bitsM_bitscandidate_values candidate_E candidate_Mr"r5r# significandr. candidatesabs_values_flatNabs_values_expandedmax_candidate_valueerrors min_errors_is_tieM_bits_expanded tie_breaker best_indices E_selected M_selecteds rr zMXFP4Tensor._from_floatNs M& ! ! & &u{ 3 3Yv&& ?[((5;v+>+>> lll%+dkRRRq!fEK LLL   * *AAvv**A"#c'K'1h;7E$++E222&&q)))&&q)))) *6688a<**A"%C-K'1h;7E$++E222&&q)))&&q)))) *\"2%-PTP[\\\ l;ek$+VVV l;ek$+VVV $//"--  !! $-77::)nn..3355/B ++,.1E1Ea1H1HHII  &a>>> AJ& ::<"22U3788_D w * #  "a ' 'T{{mdi/G N3$788GCLg'Dyy%%%rNNN) __name__ __module__ __qualname__rr$r8r rxrrrrr s}OOOO*!)!)!)FV;V;V;p!!!F&&&&&rrc*eZdZddZddZdZdZdS) MXScaleTensorNc||_|Lt|tjs Jd|j|_|||_dS|!t|t r|n|f|_dStd)a6 Tensor class for working with microscaling E8M0 block scale factors. Parameters: - data: A torch tensor of float32 numbers to convert to fp8e8m0 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nrrrrs rrzMXScaleTensor.__init__s  dEL11 Z Z3Z Z Z1+DK((..DIII   *4 7 7EdXDIIIMNN Nrc d}|dnCtdttjtj||z}|dnQt dtdttjtj||z}||ks Jdtj||dz|jtj|j }||_ |S)zp Generate random E8M0 data within a specified range. * Excludes the NaN encoding (255). Nrz&Low must be less than or equal to highrr) rHintr log2rBrIrrrr r)rrurvbias min_exponent max_exponentr"s rr$zMXScaleTensor.randoms KqqSC 5+>>&A+N  zzk* K <00 1 1s7}}U[11  ;|Q<<.33EK@@ { rr)NN)rrrrr$r8r rrrrrs^OOOO&     ! ! !rr)__doc__r rrrrrrs Z&Z&Z&Z&Z&Z&Z&Z&zDDDDDDDDDDr