hzdZddlmZddlmZddlZddlmZmZddl m Z ddl m Z dd l mZdd l mZGd d e ZGddee ZGddeZGddeZGddeZGddeZGddee ZGddeZGddeZGddeZeeeedded ZdS)!zZLosses and corresponding default initial estimators for gradient boosting decision trees. )ABCMeta)abstractmethodN)expit logsumexp) TREE_LEAF)_weighted_percentileDummyClassifierDummyRegressorceZdZdZdZdZedZed dZedZ dd Z ed Z ed Z dS) LossFunctionaMAbstract base class for various loss functions. Parameters ---------- n_classes : int Number of classes. Attributes ---------- K : int The number of regression trees to be induced; 1 for regression and binary classification; ``n_classes`` for multi-class classification. Fc||_dSN)K)self n_classess W/var/www/html/Sam_Eipo/venv/lib/python3.11/site-packages/sklearn/ensemble/_gb_losses.py__init__zLossFunction.__init__#s cdS)z-Default ``init`` estimator for loss function.Nrs rinit_estimatorzLossFunction.init_estimator&rNcdS)adCompute the loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nrryraw_predictions sample_weights r__call__zLossFunction.__call__*rrc dS)NCompute the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. Nrrrr kargss rnegative_gradientzLossFunction.negative_gradient:rr皙?rc ~||} | } d| |<tj|jt kdD](} ||| | ||||dd| f|)|dd| fxx||jddddf| dzz cc<dS)azUpdate the terminal regions (=leaves) of the given tree and updates the current predictions of the model. Traverses tree and invokes template method `_update_terminal_region`. Parameters ---------- tree : tree.Tree The tree object. X : ndarray of shape (n_samples, n_features) The data array. y : ndarray of shape (n_samples,) The target labels. residual : ndarray of shape (n_samples,) The residuals (usually the negative gradient). raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n_samples,) The weight of each sample. sample_mask : ndarray of shape (n_samples,) The sample mask to be used. learning_rate : float, default=0.1 Learning rate shrinks the contribution of each tree by ``learning_rate``. k : int, default=0 The index of the estimator being updated. rNaxis) applycopynpwhere children_leftr_update_terminal_regionvaluetake) rtreeXrresidualr r! sample_mask learning_ratekterminal_regionsmasked_terminal_regionsleafs rupdate_terminal_regionsz$LossFunction.update_terminal_regionsHsR ::a==#3"7"7"9"902 -HT/9<==a@  D  ( ('1%     1AAAq!G1D1I1I 12J2 2 "  rc dS)z=Template method for updating terminal regions (i.e., leaves).Nr rr5r;r=r6rr7r r!s rr2z$LossFunction._update_terminal_regionrrcdS)aLReturn the initial raw predictions. Parameters ---------- X : ndarray of shape (n_samples, n_features) The data array. estimator : object The estimator to use to compute the predictions. Returns ------- raw_predictions : ndarray of shape (n_samples, K) The initial raw predictions. K is equal to 1 for binary classification and regression, and equal to the number of classes for multiclass classification. ``raw_predictions`` is casted into float64. Nr)rr6 estimators rget_init_raw_predictionsz%LossFunction.get_init_raw_predictionss & rrr(r) __name__ __module__ __qualname____doc__is_multi_classrrrr"r'r>r2rCrrrrrs  N<<^<    ^     ^  , ? ? ? ? B L L^ L  ^   rr) metaclassc.eZdZdZfdZdZdZxZS)RegressionLossFunctionz)Base class for regression loss functions.cLtddS)Nr)superr)r __class__s rrzRegressionLossFunction.__init__s$ 1%%%%%rcdt|drt|dstddS)zMake sure estimator has the required fit and predict methods. Parameters ---------- estimator : object The init estimator to check. fitpredictzNThe init parameter must be a valid estimator and support both fit and predict.Nhasattr ValueErrorrrBs rcheck_init_estimatorz+RegressionLossFunction.check_init_estimatorsH 5)) gi.K.K 0   rc||}|ddtjS)Nr*rN)rTreshapeastyper/float64)rr6rB predictionss rrCz/RegressionLossFunction.get_init_raw_predictionss;''** ""2q))00<<> >##%%&&1/D/D/F/F+F1*LMNNO rc 0||z S)aTCompute half of the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples,) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. rhr%s rr'z#LeastSquaresError.negative_gradients?((****rr(rc ~|dd| fxx|||zz cc<dS)aLeast squares does not need to update terminal regions. But it has to update the predictions. Parameters ---------- tree : tree.Tree The tree object. X : ndarray of shape (n_samples, n_features) The data array. y : ndarray of shape (n_samples,) The target labels. residual : ndarray of shape (n_samples,) The residuals (usually the negative gradient). raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n,) The weight of each sample. sample_mask : ndarray of shape (n,) The sample mask to be used. learning_rate : float, default=0.1 Learning rate shrinks the contribution of each tree by ``learning_rate``. k : int, default=0 The index of the estimator being updated. N)rTrh) rr5r6rr7r r!r8r9r:s rr>z)LeastSquaresError.update_terminal_regionssJP 1a1F1F1H1H!HHrc dSrrr@s rr2z)LeastSquaresError._update_terminal_region"s  rrrD) rErFrGrHrr"r'r>r2rrrrbrbs///. + + +. (I(I(I(IT      rrbc,eZdZdZdZddZdZdZdS)LeastAbsoluteErrorzLoss function for least absolute deviation (LAD) regression. Parameters ---------- n_classes : int Number of classes c$tddSNquantile?rfrrr rs rrz!LeastAbsoluteError.init_estimator9zC@@@@rNc (|;tj||z Sd|z tj|tj||z zzS)atCompute the least absolute error. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrN)r/absrhrdrirs rr"zLeastAbsoluteError.__call__<s  6!o3355566;;== =##%%&&O4I4I4K4K0K)L)LLMMN rc H|}d||z dkzdz S)aCompute the negative gradient. 1.0 if y - raw_predictions > 0.0 else -1.0 Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. rrrNrkr%s rr'z$LeastAbsoluteError.negative_gradientSs/*//11A'!+,q00rc tj||kd} || d}|| d|| dz } t| |d|j|ddf<dS)z1LAD updates terminal regions to median estimates.rr+2 percentileN)r/r0r4r r3) rr5r;r=r6rr7r r!terminal_regiondiffs rr2z*LeastAbsoluteError._update_terminal_regionds(#3t#;<eZdZdZd fd ZdZd dZdZdZxZ S) QuantileLossFunctionzLoss function for quantile regression. Quantile regression allows to estimate the percentiles of the conditional distribution of the target. Parameters ---------- alpha : float, default=0.9 The percentile. rcjt||_|dz|_dS)Nr)rPrrr|rs rrzQuantileLossFunction.__init__s/  #+rc.td|jS)Nrrrt)r rrs rrz#QuantileLossFunction.init_estimatorszDJGGGGrNc|}||z }|j}||k}|N|||zd|z ||zz |jdz }ng|t j||||zzd|z t j||||zzz |z }|S)aCompute the Quantile loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrNr)rhrrirr/)rrr r!r~rmaskrs rr"zQuantileLossFunction.__call__s*//11?" ?"  T (((AIte9J9J+JJ DD }T2T$Z?@@@u9}dU';dD5k'I J JJK!!##$D rc b|j}|}||k}||zd|z |zz S)r$rN)rrh)rrr r&rrs rr'z&QuantileLossFunction.negative_gradient"s@ )//11?" !e)u!455rc tj||kd} || d|| dz } || d}t| ||j} | |j|df<dS)Nrr+)r/r0r4r r|r3) rr5r;r=r6rr7r r!r}r~vals rr2z,QuantileLossFunction._update_terminal_region3s(#3t#;<666""""""""rrcDeZdZdZedZedZdZdS)ClassificationLossFunctionz-Base class for classification loss functions.cdS)aTemplate method to convert raw predictions into probabilities. Parameters ---------- raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. Returns ------- probas : ndarray of shape (n_samples, K) The predicted probabilities. Nrrr s r_raw_prediction_to_probaz3ClassificationLossFunction._raw_prediction_to_probaKrrcdS)aTemplate method to convert raw predictions to decisions. Parameters ---------- raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. Returns ------- encoded_predictions : ndarray of shape (n_samples, K) The predicted encoded labels. Nrrs r_raw_prediction_to_decisionz6ClassificationLossFunction._raw_prediction_to_decision[rrcdt|drt|dstddS)zMake sure estimator has fit and predict_proba methods. Parameters ---------- estimator : object The init estimator to check. rS predict_probazTThe init parameter must be a valid estimator and support both fit and predict_proba.NrUrXs rrYz/ClassificationLossFunction.check_init_estimatorksH 5)) gi.Q.Q :   rN)rErFrGrHrrrrYrrrrrHs]77   ^     ^       rrcNeZdZdZfdZdZd dZdZdZdZ d Z d Z xZ S) BinomialDevianceaBinomial deviance loss function for binary classification. Binary classification is a special case; here, we only need to fit one tree instead of ``n_classes`` trees. Parameters ---------- n_classes : int Number of classes. c|dkr-td|jj|t ddSNrz-{0:s} requires 2 classes; got {1:d} class(es)rNrOrWformatrQrErPrrrrQs rrzBinomialDeviance.__init__^ >>?FFN+Y  1%%%%%rc"tdSNpriorrer rs rrzBinomialDeviance.init_estimators0000rNc |}|0dtj||ztjd|z zSd|z tj|||ztjd|z zzS)aCompute the deviance (= 2 * negative log-likelihood). Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nr)rhr/rd logaddexprirs rr"zBinomialDeviance.__call__s *//11  _$ Q(H(HH  ##%%&&!O+r|A/O/OOQ rc J|t|z S)aPCompute half of the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. )rrhr%s rr'z"BinomialDeviance.negative_gradients#5..001111rc tj||kd} || d}|| d}|| d}tj||z} tj|||z zd|z |zz} t | dkrd|j|ddf<dS| | z |j|ddf<dS)zMake a single Newton-Raphson step. our node estimate is given by: sum(w * (y - prob)) / sum(w * prob * (1 - prob)) we take advantage that: y - prob = residual rr+rNu?j/ N)r/r0r4rirwr3 rr5r;r=r6rr7r r!r} numerator denominators rr2z(BinomialDeviance._update_terminal_regions&(#3t#;<>yQ''''rcV||}|dddf}tjtjj}tj||d|z }tj|d|z z }|ddtj S)NrNr* rr/finfofloat32epscliplogr[r\r]rr6rBprobasproba_pos_classrr s rrCz)BinomialDeviance.get_init_raw_predictionss((++ A,hrz""&'/3C@@&A4G!HII&&r1--44RZ@@@rr rErFrGrHrrr"r'r2rrrCr_r`s@rrrzs  &&&&&111 > 2 2 2===B (((AAAAAAArrcTeZdZdZdZfdZdZd dZddZd Z d Z d Z d Z xZ S)MultinomialDeviancezMultinomial deviance loss function for multi-class classification. For multi-class classification we need to fit ``n_classes`` trees at each stage. Parameters ---------- n_classes : int Number of classes. Tc|dkr,td|jjt |dS)Nz#{0:s} requires more than 2 classes.rrs rrzMultinomialDeviance.__init__sT q==5<rs********""""""......######""""""Y Y Y Y Y WY Y Y Y x=====\W====2g g g g g .g g g TG G G G G /G G G Tr r r r r .r r r jV"V"V"V"V"1V"V"V"r/////////dzAzAzAzAzA1zAzAzAzppppp4pppfqAqAqAqAqA0qAqAqAj'( $"  r