rtj-ddlmZddlmZddlmZddlZddlm Z m Z m Z m Z ddl mZdddd Zdd ZGd d eZd ZGddeZdZdZdddZddZGddeZdZdS))Counter)suppress) NamedTupleN)_isindevice get_namespacexpx is_scalar_nanFreturn_inverse return_countscj|jtkrt|||St|||S)aHelper function to find unique values with support for python objects. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : ndarray Values to check for unknowns. return_inverse : bool, default=False If True, also return the indices of the unique values. return_counts : bool, default=False If True, also return the number of times each unique item appears in values. Returns ------- unique : ndarray The sorted unique values. unique_inverse : ndarray The indices to reconstruct the original array from the unique array. Only provided if `return_inverse` is True. unique_counts : ndarray The number of times each of the unique values comes up in the original array. Only provided if `return_counts` is True. r )dtypeobject_unique_python _unique_np)valuesr rs S/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/sklearn/utils/_encode.py_uniquersO>|v >     ~]   c`t|\}}d\}}|r|r||\}}}}nK|r||\}}n0|r||\}}n||}|jrwt |drb|||j}|d|dz}|r ||||k<|r-| ||d||<|d|dz}|f} |r| |fz } |r| |fz } t| dkr| dn| S)zHelper function to find unique values for numpy arrays that correctly accounts for nans. See `_unique` documentation for details.)NNNr) r unique_allunique_inverse unique_counts unique_valuessizer searchsortednansumlen) rr rxp_inversecountsuniquesnan_idxrets rrr7s & ! !EB OGV+-+&(mmF&;&;#GVV +,,V44 +**622""6**|+ gbk22+//'2622-GaK-(  1)0GGg% &  + ffVGHH%566F7OMgkM*F *C z yXX]]3q66+rc.eZdZUdZeed<eed<dZdS) MissingValuesz'Data class for missing data informationr!nonecg}|jr|d|jr|tj|S)z3Convert tuple to a list where None is always first.N)r-appendr!np)selfoutputs rto_listzMissingValues.to_listdsF 9 MM$    8 " MM"& ! ! ! rN)__name__ __module__ __qualname____doc__bool__annotations__r3rrr,r,^s=11 III JJJrr,cd|D}|s|tddfSd|vr7t|dkrtdd}n#tdd}ntdd}||z }||fS)a.Extract missing values from `values`. Parameters ---------- values: set Set of values to extract missing from. Returns ------- output: set Set with missing values extracted. missing_values: MissingValues Object with missing value information. c4h|]}|t||SNr ).0values r z#_extract_missing..~s,U]mE6J6J]]]]rF)r!r-NrT)r,r#)rmissing_values_setoutput_missing_valuesr2s r_extract_missingrCns ! <}U;;;;; !!! ! " "a ' '$1e$$G$G$G ! !%2d$F$F$F ! ! -$U C C C( (F ( ((rc(eZdZdZfdZdZxZS)_nandictz!Dictionary with support for nans.ct||D]\}}t|r ||_dSdSr=)super__init__itemsr nan_value)r1mappingkeyr? __class__s rrHz_nandict.__init__sc !!!!--//  JCS!! !&   rclt|drt|r|jSt|)NrJ)hasattrr rJKeyErrorr1rLs r __missing__z_nandict.__missing__7 4 % % "-*<*< "> !smmr)r4r5r6r7rHrR __classcell__rMs@rrErEsM++rrEct||\}}tdt|D|fd|Dt |S)z,Map values based on its position in uniques.ci|]\}}|| Sr:r:)r>ivals r z#_map_to_integer..s>>>Cc1>>>rc g|] }| Sr:r:)r>vtables r z#_map_to_integer..s000AuQx000r)r)rrE enumerateasarrayr)rr(r$r%r]s @r_map_to_integerrask &' * *EB >>9W+=+=>>> ? ?E ::0000000: H HHrc" t|}t|\}}t|}||t j||j}nP#t$rCtdtd|DD}td|wxYw|f}|r|t||fz }|r|t||fz }t|dkr|dn|S)Nrc3$K|] }|jV dSr=)r6)r>ts r z!_unique_python..s$LL!q~LLLLLLrc34K|]}t|VdSr=)type)r>r\s rrfz!_unique_python..s(2K2Kq4772K2K2K2K2K2KrzPEncoders require their input argument must be uniformly strings or numbers. Got rr) setrCsortedextendr3r0arrayr TypeErrorra _get_countsr#)rr r uniques_setmissing_valuesr(typesr*s rrrs<  &kk &6{&C&C# ^%%~--//000(7&,777    LLs2K2KF2K2K2K/K/KLLLLL /', / /    *C3 0022/ FG,,..XX]]3q66+s A2A55A CT) check_unknownc@t||\}}||jds5 t||S#t$r}t d|d}~wwxYw|r$t ||}|rt d||||S)aHelper function to encode values into [0, n_uniques - 1]. Uses pure python method for object dtype, and numpy method for all other dtypes. The numpy method has the limitation that the `uniques` need to be sorted. Importantly, this is not checked but assumed to already be the case. The calling method needs to ensure this for all non-object values. Parameters ---------- values : ndarray Values to encode. uniques : ndarray The unique values in `values`. If the dtype is not object, then `uniques` needs to be sorted. check_unknown : bool, default=True If True, check for values in `values` that are not in `unique` and raise an error. This is ignored for object dtype, and treated as True in this case. This parameter is useful for _BaseEncoder._transform() to avoid calling _check_unknown() twice. Returns ------- encoded : ndarray Encoded values numericz%y contains previously unseen labels: N)risdtyperrarP ValueError_check_unknownr )rr(rrr$r%ediffs r_encoderzs: &' * *EB ::flI . . 0 J"6733 3 J J JHQHHII I J  Q!&'22D Q !O!O!OPPPw///sA A! AA!ct||\}}d}||jdst|}t |\}}t|t \|z }|joj } |joj } fd|rQ|s| s| r"|fd|D}n)|t||j }t|}| r| d| r| tjn||} tj| |d|}|rB|jrt%|||}n)|t||j }|||rV||} || r,|jr|r||} d|| <|| }t|}|r||fS|S) a Helper function to check for unknowns in values to be encoded. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : array Values to check for unknowns. known_values : array Known values. Must be unique. return_mask : bool, default=False If True, return a mask of the same shape as `values` indicating the valid values. Returns ------- diff : list The unique values present in `values` and not in `know_values`. valid_mask : boolean array Additionally returned if ``return_mask=True``. NrtcN|vp jo|dupjot|Sr=)r-r!r )r?missing_in_uniquesros ris_validz _check_unknown..is_valids>$E&+= E&*C}U/C/C rc&g|] }|Sr:r:)r>r?r~s rr^z"_check_unknown.. s!&K&K&K5xx&K&K&KrrcT) assume_uniquer$r)rrurrirCr!r-rlonesr#r8listr/r0rr setdiff1drranyisnan)r known_values return_maskr$r% valid_mask values_setmissing_in_valuesry nan_in_diff none_in_diffr diff_is_nanis_nanr~r}ros @@@rrwrws2 &, / /EBJ ::flI . .1[[ (8(D(D% %,'' *:;*G*G' 'K''+J4F4J0J (-M6H6M2M         A A{ Al AXX&K&K&K&KF&K&K&KLL WWS[[W@@ Dzz   KK     KK   ((00 }]LQSTTT  Ay A"6<<< WWS[[W@@  66"((<(( ) ) *((4..Kvvk"" *9++XXf--F)*Jv&[L)Dzz Z Krc.eZdZdZfdZdZdZxZS) _NaNCounterz$Counter with support for nan values.cpt||dSr=)rGrH_generate_items)r1rIrMs rrHz_NaNCounter.__init__Gs/ --e4455555rc#K|D]=}t|s|Vt|dsd|_|xjdz c_>dS)z>Generate items without nans. Stores the nan counts separately. nan_countrrN)r rOr)r1rIitems rrz_NaNCounter._generate_itemsJsf  D &&  4-- #!" NNa NNN   rclt|drt|r|jSt|)Nr)rOr rrPrQs rrRz_NaNCounter.__missing__TrSr)r4r5r6r7rHrrRrTrUs@rrrDs\..66666   rrc|jjdvrt|}tjt |tj}t|D]<\}}tt5||||<dddn #1swxYwY=|St|d\}}tj ||d}tj |drtj |drd|d<tj |||} tj|tj}|| ||<|S)zGet the count of each of the `uniques` in `values`. The counts will use the order passed in by `uniques`. For non-object dtypes, `uniques` is assumed to be sorted and `np.nan` is at the end. OUrcNT)r)rr)rkindrr0zerosr#int64r_rrPrisinrr zeros_like) rr(counterr2rXrrr'uniques_in_valuesunique_valid_indicess rrnrnZsy |D  f%%#g,,bh777 )) * *GAt(## * *#DMq  * * * * * * * * * * * * * * * &vTBBBM6dKKK x b!""%rx '<'<% $"?='BS:TUU ]7"( 3 3 3F &'; rs FFFFFFFFFFFF000000',5&&&&&R$,$,$,$,N     J    #)#)#)L     t    III,,,4/3(0(0(0(0(0VQQQQh',r