o tBh { @s*dZddlmZddlmZddlZddlmZmZddl m Z ddl m Z dd l mZdd l mZGd d d ed ZGdddeed ZGdddeZGdddeZGdddeZGdddeZGdddeed ZGdddeZGdddeZGdddeZeeeeeedded ZdS)!zZLosses and corresponding default initial estimators for gradient boosting decision trees. )ABCMeta)abstractmethodN)expit logsumexp) TREE_LEAF)_weighted_percentileDummyClassifierDummyRegressorc@sdeZdZdZdZddZddZeddd Zed d Z dddZ eddZ eddZ dS) LossFunctionaMAbstract base class for various loss functions. Parameters ---------- n_classes : int Number of classes. Attributes ---------- K : int The number of regression trees to be induced; 1 for regression and binary classification; ``n_classes`` for multi-class classification. FcCs ||_dSN)Kself n_classesrr/var/www/html/riverr-enterprise-integrations-main/venv/lib/python3.10/site-packages/sklearn/ensemble/_gb_losses.py__init__# zLossFunction.__init__cCst)z-Default ``init`` estimator for loss function.)NotImplementedErrorrrrrinit_estimator&szLossFunction.init_estimatorNcCdS)adCompute the loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nrryraw_predictions sample_weightrrr__call__*zLossFunction.__call__cKr)NCompute the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. Nrrrrkargsrrrnegative_gradient:r zLossFunction.negative_gradient皙?rc Cs||} | } d| |<t|jtkdD]} ||| | ||||dd| f|q|dd| f||jddddfj| dd7<dS)azUpdate the terminal regions (=leaves) of the given tree and updates the current predictions of the model. Traverses tree and invokes template method `_update_terminal_region`. Parameters ---------- tree : tree.Tree The tree object. X : ndarray of shape (n_samples, n_features) The data array. y : ndarray of shape (n_samples,) The target labels. residual : ndarray of shape (n_samples,) The residuals (usually the negative gradient). raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n_samples,) The weight of each sample. sample_mask : ndarray of shape (n_samples,) The sample mask to be used. learning_rate : float, default=0.1 Learning rate shrinks the contribution of each tree by ``learning_rate``. k : int, default=0 The index of the estimator being updated. rNaxis) applycopynpwhere children_leftr_update_terminal_regionvaluetake) rtreeXrresidualrr sample_mask learning_ratekterminal_regionsmasked_terminal_regionsleafrrrupdate_terminal_regionsHs" ) & z$LossFunction.update_terminal_regionsc Cr)z=Template method for updating terminal regions (i.e., leaves).Nr rr1r7r9r2rr3rrrrrr.r z$LossFunction._update_terminal_regioncCr)aLReturn the initial raw predictions. Parameters ---------- X : ndarray of shape (n_samples, n_features) The data array. estimator : object The estimator to use to compute the predictions. Returns ------- raw_predictions : ndarray of shape (n_samples, K) The initial raw predictions. K is equal to 1 for binary classification and regression, and equal to the number of classes for multiclass classification. ``raw_predictions`` is casted into float64. Nr)rr2 estimatorrrrget_init_raw_predictionssz%LossFunction.get_init_raw_predictionsrr%r) __name__ __module__ __qualname____doc__is_multi_classrrrrr$r:r.r=rrrrr s    A  r ) metaclasscs0eZdZdZfddZddZddZZS)RegressionLossFunctionz)Base class for regression loss functions.cstjdddS)Nr)superrr __class__rrrzRegressionLossFunction.__init__cC t|dr t|dstddS)zMake sure estimator has the required fit and predict methods. Parameters ---------- estimator : object The init estimator to check. fitpredictzNThe init parameter must be a valid estimator and support both fit and predict.Nhasattr ValueErrorrr<rrrcheck_init_estimator z+RegressionLossFunction.check_init_estimatorcCs||}|ddtjS)Nr&rF)rNreshapeastyper+float64)rr2r< predictionsrrrr=s z/RegressionLossFunction.get_init_raw_predictions)r?r@rArBrrSr= __classcell__rrrIrrEs  rEc@s@eZdZdZddZdddZddZ dd d Zd dZdS)LeastSquaresErrorzLoss function for least squares (LS) estimation. Terminal regions do not need to be updated for least squares. Parameters ---------- n_classes : int Number of classes. cC tddS)Nmeanstrategyr rrrrrrz LeastSquaresError.init_estimatorNcCsD|durt||dSd|t|||dS)arCompute the least squares loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrrF)r+r\ravelsumrrrrrszLeastSquaresError.__call__cKs ||S)aTCompute half of the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples,) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. r_r"rrrr$s z#LeastSquaresError.negative_gradientr%rc Cs*|dd| f|||7<dS)aLeast squares does not need to update terminal regions. But it has to update the predictions. Parameters ---------- tree : tree.Tree The tree object. X : ndarray of shape (n_samples, n_features) The data array. y : ndarray of shape (n_samples,) The target labels. residual : ndarray of shape (n_samples,) The residuals (usually the negative gradient). raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n,) The weight of each sample. sample_mask : ndarray of shape (n,) The sample mask to be used. learning_rate : float, default=0.1 Learning rate shrinks the contribution of each tree by ``learning_rate``. k : int, default=0 The index of the estimator being updated. N)rNr_) rr1r2rr3rrr4r5r6rrrr:s*(z)LeastSquaresError.update_terminal_regionsc CsdSrrr;rrrr."s z)LeastSquaresError._update_terminal_regionrr>) r?r@rArBrrr$r:r.rrrrrZs  *rZc@s2eZdZdZddZd ddZddZd d ZdS) LeastAbsoluteErrorzLoss function for least absolute deviation (LAD) regression. Parameters ---------- n_classes : int Number of classes cC tdddSNquantile?r^rer rrrrr9 z!LeastAbsoluteError.init_estimatorNc CsF|durt||Sd|t|t||S)atCompute the least absolute error. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves). sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrF)r+absr_r\r`rrrrr<szLeastAbsoluteError.__call__cKs|}d||dkdS)aCompute the negative gradient. 1.0 if y - raw_predictions > 0.0 else -1.0 Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. rrrFrar"rrrr$Ssz$LeastAbsoluteError.negative_gradientc CsZt||kd} |j| dd}|j| dd|j| dd} t| |dd|j|ddf<dS)z1LAD updates terminal regions to median estimates.rr'2 percentileN)r+r,r0rr/) rr1r7r9r2rr3rrterminal_regiondiffrrrr.ds z*LeastAbsoluteError._update_terminal_regionr)r?r@rArBrrr$r.rrrrrb0s   rbcsFeZdZdZdfdd ZddZddd Zdd d Zd d ZZ S)HuberLossFunctionahHuber loss function for robust regression. M-Regression proposed in Friedman 2001. Parameters ---------- alpha : float, default=0.9 Percentile at which to extract score. References ---------- J. Friedman, Greedy Function Approximation: A Gradient Boosting Machine, The Annals of Statistics, Vol. 29, No. 5, 2001. ?cst||_d|_dSr)rHralphagammarrqrIrrrs  zHuberLossFunction.__init__cCrcrdr rrrrrrhz HuberLossFunction.init_estimatorNc Cs|}||}|j}|dur,|dur tt||jd}n tt|||jd}t||k}|dur^td||d}t|t|||d}|||jd} | Std||||d}t|||t|||d}|||} | S)aCompute the Huber loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Ndrfrr) r_rrr+rlrirqrr`shape) rrrrrnrr gamma_masksq_losslin_losslossrrrrs2" zHuberLossFunction.__call__c Ks|}||}|durtt||jd}n tt|||jd}t||k}tj|jdftjd}||||<|t ||||<||_ |S)aCompute the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nrtrdtype) r_r+rlrirqrzerosrurWsignrr) rrrrr#rnrrrvr3rrrr$s z#HuberLossFunction.negative_gradientc  Cst||kd} |j| dd}|j} |j| dd|j| dd} t| |dd} | | } | tt| tt| | |j |df<dS)Nrr'rjrk) r+r,r0rrrr\r}minimumrir/)rr1r7r9r2rr3rrrmrrrnmediandiff_minus_medianrrrr.s z)HuberLossFunction._update_terminal_regionrpr r?r@rArBrrrr$r.rYrrrIrrozs  )rocsDeZdZdZdfdd ZddZddd Zd d Zd d ZZ S)QuantileLossFunctionzLoss function for quantile regression. Quantile regression allows to estimate the percentiles of the conditional distribution of the target. Parameters ---------- alpha : float, default=0.9 The percentile. rpcst||_|d|_dS)Nrt)rHrrqrlrsrIrrrs zQuantileLossFunction.__init__cCstd|jdS)Nrerg)r rqrrrrrsz#QuantileLossFunction.init_estimatorNcCs|}||}|j}||k}|dur-|||d||||jd}|S|t||||d|t|||||}|S)aCompute the Quantile loss. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. NrFr)r_rqr`rur+)rrrrrnrqmaskryrrrrs$ zQuantileLossFunction.__call__cKs,|j}|}||k}||d||S)r!rF)rqr_)rrrr#rqrrrrr$"s z&QuantileLossFunction.negative_gradientc Cs\t||kd} |j| dd|j| dd} |j| dd}t| ||j} | |j|df<dS)Nrr')r+r,r0rrlr/) rr1r7r9r2rr3rrrmrnvalrrrr.3s z,QuantileLossFunction._update_terminal_regionrrrrrrIrrs  rc@s,eZdZdZddZeddZddZdS) ClassificationLossFunctionz-Base class for classification loss functions.cCr)aTemplate method to convert raw predictions into probabilities. Parameters ---------- raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. Returns ------- probas : ndarray of shape (n_samples, K) The predicted probabilities. Nrrrrrr_raw_prediction_to_probaKr z3ClassificationLossFunction._raw_prediction_to_probacCr)aTemplate method to convert raw predictions to decisions. Parameters ---------- raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. Returns ------- encoded_predictions : ndarray of shape (n_samples, K) The predicted encoded labels. Nrrrrr_raw_prediction_to_decisionZr z6ClassificationLossFunction._raw_prediction_to_decisioncCrL)zMake sure estimator has fit and predict_proba methods. Parameters ---------- estimator : object The init estimator to check. rM predict_probazTThe init parameter must be a valid estimator and support both fit and predict_proba.NrOrRrrrrSjrTz/ClassificationLossFunction.check_init_estimatorN)r?r@rArBrrrrSrrrrrHs   rcZeZdZdZfddZddZdddZd d Zd d Zd dZ ddZ ddZ Z S)BinomialDevianceaBinomial deviance loss function for binary classification. Binary classification is a special case; here, we only need to fit one tree instead of ``n_classes`` trees. Parameters ---------- n_classes : int Number of classes. c.|dkrtd|jj|tjdddSNrz-{0:s} requires 2 classes; got {1:d} class(es)rFrGrQformatrJr?rHrrrIrrrzBinomialDeviance.__init__cCr[Npriorr]r rrrrrs zBinomialDeviance.init_estimatorNc CsX|}|durdt||td|Sd|t|||td|S)aCompute the deviance (= 2 * negative log-likelihood). Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. Nr)r_r+r\ logaddexpr`rrrrrszBinomialDeviance.__call__cKs|t|S)aPCompute half of the negative gradient. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. )rr_r"rrrr$s z"BinomialDeviance.negative_gradientc Cst||kd} |j| dd}|j| dd}|j| dd}t||} t|||d||} t| dkrDd|j|ddf<dS| | |j|ddf<dS)zMake a single Newton-Raphson step. our node estimate is given by: sum(w * (y - prob)) / sum(w * prob * (1 - prob)) we take advantage that: y - prob = residual rr'rFu?j/ N)r+r,r0r`rir/ rr1r7r9r2rr3rrrm numerator denominatorrrrr.s z(BinomialDeviance._update_terminal_regioncCsZtj|jddftjd}t||dddf<|dddf|dddf8<|S)NrrrzrFr+onesrurWrr_rrprobarrrrs$z)BinomialDeviance._raw_prediction_to_probacC||}tj|ddSNrFr'rr+argmaxrrrrr z,BinomialDeviance._raw_prediction_to_decisioncCs`||}|dddf}ttjj}t||d|}t|d|}|ddtj S)NrFr& rr+finfofloat32epscliplogrUrVrWrr2r<probasproba_pos_classrrrrrr=s z)BinomialDeviance.get_init_raw_predictionsr r?r@rArBrrrr$r.rrr=rYrrrIrrys  !rcs`eZdZdZdZfddZddZddd Zdd d Zd dZ ddZ ddZ ddZ Z S)MultinomialDeviancezMultinomial deviance loss function for multi-class classification. For multi-class classification we need to fit ``n_classes`` trees at each stage. Parameters ---------- n_classes : int Number of classes. Tcs*|dkr td|jjt|dS)Nz#{0:s} requires more than 2 classes.rrrIrrrs  zMultinomialDeviance.__init__cCr[rr rrrrr rz"MultinomialDeviance.init_estimatorNcCsjtj|jd|jftjd}t|jD] }||k|dd|f<qtjd||jddt|dd|dS)aCompute the Multinomial deviance. Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. rrzNr&rFr')weights) r+r|rurrWrangeaverager`r)rrrrYr6rrrrszMultinomialDeviance.__call__rc Ks,|tt|dd|ft|ddS)aCompute negative gradient for the ``k``-th class. Parameters ---------- y : ndarray of shape (n_samples,) The target labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. k : int, default=0 The index of the class. NrFr')r+ nan_to_numexpr)rrrr6kwargsrrrr$'s z%MultinomialDeviance.negative_gradientc Cst||kd} |j| dd}|j| dd}|j| dd}t||} | |jd|j9} t|||d||} t| dkrNd|j|ddf<dS| | |j|ddf<dS)z"Make a single Newton-Raphson step.rr'rFrrN)r+r,r0r`rrir/rrrrr.:s  z+MultinomialDeviance._update_terminal_regionc Cs*tt|t|ddddtjfSr)r+rrrnewaxisrrrrrVs z,MultinomialDeviance._raw_prediction_to_probacCrrrrrrrr]rz/MultinomialDeviance._raw_prediction_to_decisioncCs@||}ttjj}t||d|}t|tj}|S)NrF) rr+rrrrrrVrW)rr2r<rrrrrrr=as z,MultinomialDeviance.get_init_raw_predictionsr)r)r?r@rArBrCrrrr$r.rrr=rYrrrIrrs   rcr)ExponentialLossaExponential loss function for binary classification. Same loss as AdaBoost. Parameters ---------- n_classes : int Number of classes. References ---------- Greg Ridgeway, Generalized Boosted Models: A guide to the gbm package, 2007 crrrrrIrrrxrzExponentialLoss.__init__cCr[rr rrrrrrzExponentialLoss.init_estimatorNcCs\|}|durttd|d |Sd|t|td|d |S)aCompute the exponential loss Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble. sample_weight : ndarray of shape (n_samples,), default=None Sample weights. N@?rrF)r_r+r\rr`rrrrrs zExponentialLoss.__call__cKs$d|d}|t| |S)aUCompute the residual (= negative gradient). Parameters ---------- y : ndarray of shape (n_samples,) True labels. raw_predictions : ndarray of shape (n_samples, K) The raw predictions (i.e. values from the tree leaves) of the tree ensemble at iteration ``i - 1``. rr)r+rr_)rrrr#y_rrrr$s z!ExponentialLoss.negative_gradientc Cst||kd} |j| dd}|j| dd}|j| dd}d|d} t| |t| |} t|t| |} t| dkrPd|j|ddf<dS| | |j|ddf<dS)Nrr'rrrr)r+r,r0r`rrir/) rr1r7r9r2rr3rrrmrrrrrrr.s   z'ExponentialLoss._update_terminal_regioncCs^tj|jddftjd}td||dddf<|dddf|dddf8<|S)NrrrzrrFrrrrrrs$z(ExponentialLoss._raw_prediction_to_probacCs|dktS)Nr)r_rVintrrrrrrKz+ExponentialLoss._raw_prediction_to_decisioncCsd||}|dddf}ttjj}t||d|}dt|d|}|ddtj S)NrFrfr&rrrrrr=s z(ExponentialLoss.get_init_raw_predictionsrrrrrIrris  r) squared_errorlsabsolute_errorladhuberredeviancelog_loss exponential)rBabcrrnumpyr+ scipy.specialrr tree._treer utils.statsrdummyr r r rErZrbrorrrrrLOSS_FUNCTIONSrrrrs<      jJuY1}sv