qtj dZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z mZmZmZmZddlmZmZmZmZmZmZddlmZddlmZddlmZmZm Z dd l!m"Z"dd l#m$Z$Gd d Z%Gd de%Z&Gdde%Z'Gdde%Z(Gdde%Z)Gdde%Z*Gdde%Z+Gdde%Z,Gdde%Z-Gdde%Z.Gdd e%Z/Gd!d"e%Z0e&e'e(e)e*e+e,e.e/e0d# Z1Gd$d%Z2d&Z3Gd'd(e2e.Z4Gd)d*e2e/Z5Gd+d,e2e*Z6dS)-z This module contains loss classes suitable for fitting. It is not part of the public API. Specific losses are used for regression, binary classification or multiclass classification. Nxlogy) CyAbsoluteErrorCyExponentialLossCyHalfBinomialLossCyHalfGammaLossCyHalfMultinomialLossCyHalfPoissonLossCyHalfSquaredErrorCyHalfTweedieLossCyHalfTweedieLossIdentity CyHuberLoss CyPinballLoss) HalfLogitLink IdentityLinkInterval LogitLinkLogLinkMultinomialLogit)one_hot) check_scalar)_average _logsumexp_ravel)softmax)_weighted_percentileceZdZdZdZdZddZdZdZ dd Z dd Z dd Z dd Z dd Z ddZddZejdfdZdS)BaseLossaBase class for a loss function of 1-dimensional targets. Conventions: - y_true.shape = sample_weight.shape = (n_samples,) - y_pred.shape = raw_prediction.shape = (n_samples,) - If is_multiclass is true (multiclass classification), then y_pred.shape = raw_prediction.shape = (n_samples, n_classes) Note that this corresponds to the return value of decision_function. y_true, y_pred, sample_weight and raw_prediction must either be all float64 or all float32. gradient and hessian must be either both float64 or both float32. Note that y_pred = link.inverse(raw_prediction). Specific loss classes can inherit specific link classes to satisfy BaseLink's abstractmethods. Parameters ---------- closs: CyLossFunction For example, a CyLossFunction; hence the name "c"loss. link : BaseLink sample_weight : {None, ndarray} If sample_weight is None, the hessian might be constant. n_classes : {None, int} The number of classes for classification, else None. xp : module, default=None Array namespace module. device : device, default=None A device object (see the "Device Support" section of the array API spec). Attributes ---------- closs: CyLossFunction For example, a CyLossFunction; hence the name "c"loss. link : BaseLink n_classes : {None, int} The number of classes for classification, else None. xp : module or None Array namespace module. Ignored by the Cython implementation. device : device or None A device object. Ignored by the Cython implementation. interval_y_true : Interval Valid interval for y_true interval_y_pred : Interval Valid Interval for y_pred differentiable : bool Indicates whether or not loss function is differentiable in raw_prediction everywhere. approx_hessian : bool Indicates whether the hessian is approximated or exact. If, approximated, it should be larger or equal to the exact one. constant_hessian : bool Indicates whether the hessian is one for this loss. is_multiclass : bool Indicates whether n_classes > 2 is allowed. TFNc||_||_||_||_||_d|_d|_ttj tj dd|_ |jj |_ dS)NF) closslink n_classesxpdeviceapprox_hessianconstant_hessianrnpinfinterval_y_trueinterval_y_pred)selfr r!r"r#r$s P/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/sklearn/_loss/loss.py__init__zBaseLoss.__init__sd  " # %'FF#y8c6|j|SzuReturn True if y is in the valid range of y_true. Parameters ---------- y : ndarray )r)includesr+ys r,in_y_true_rangezBaseLoss.in_y_true_range#,,Q///r.c6|j|S)zuReturn True if y is in the valid range of y_pred. Parameters ---------- y : ndarray )r*r1r2s r,in_y_pred_rangezBaseLoss.in_y_pred_ranger5r.c|tj|}|jdkr&|jddkr|d}|j||||||S)aJCompute the pointwise loss value for each input. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. loss_out : None or C-contiguous array of shape (n_samples,) A location into which the result is stored. If None, a new array might be created. n_threads : int, default=1 Might use openmp thread parallelism. Returns ------- loss : array of shape (n_samples,) Element-wise loss function. Nr8y_trueraw_prediction sample_weightloss_out n_threads)r' empty_likendimshapesqueezer lossr+r<r=r>r?r@s r,rEz BaseLoss.losss<  }V,,H  ! # #(r? gradient_outr@)r'rArIrBrCrDr loss_gradient)r+r<r=r>r?rJr@s r,rKzBaseLoss.loss_gradients L  #=00!}^<< =|7IJJJ  !=x~NNNL  ! # #(rJr@)r'rArBrCrDr gradientr+r<r=r>rJr@s r,rMzBaseLoss.gradients>  =88L  ! # #(rJ hessian_outr@)r'rArBrCrDr gradient_hessian)r+r<r=r>rJrPr@s r,rQzBaseLoss.gradient_hessianPs0N  "!}^<<  mN;; !}[99  - 55K  ! # #(%>%--a00K ##)'%# $   [((r.c ^tj|||dd||S)a{Compute the weighted average loss. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. n_threads : int, default=1 Might use openmp thread parallelism. Returns ------- loss : float Mean or averaged loss function. Nr;weights)r'averagerE)r+r<r=r>r@s r,__call__zBaseLoss.__call__sG(z II-"#   "    r.c tj||d}dtj|jjz}|jjtj krd}n(|jjr |jj}n|jj|z}|jj tjkrd}n(|jj r |jj }n|jj |z }|||j |S|j tj |||S)a#Compute raw_prediction of an intercept-only model. This can be used as initial estimates of predictions, i.e. before the first iteration in fit. Parameters ---------- y_true : array-like of shape (n_samples,) Observed, true target values. sample_weight : None or array of shape (n_samples,) Sample weights. Returns ------- raw_prediction : numpy scalar or array of shape (n_classes,) Raw predictions of an intercept-only model. rrTaxis N) r'rUfinforIepsr*lowr( low_inclusivehighhigh_inclusiver!clip)r+r<r>y_predr\a_mina_maxs r,fit_intercept_onlyzBaseLoss.fit_intercept_onlys(FMBBB28FL))--   #w . .EE  ! / 3(,EE(,s2E   $ . .EE  ! 0 4(-EE(-3E =U]9>>&)) )9>>"'&%"?"?@@ @r.c*tj|S)a(Calculate term dropped in loss. With this term added, the loss of perfect predictions is zero. Parameters ---------- y_true : array-like of shape (n_samples,) Observed, true target values. sample_weight : None or array of shape (n_samples,), default=None Sample weights. Returns ------- constant : ndarray of shape (n_samples,) Constant value to be added to raw predictions so that the loss of perfect predictions becomes zero. )r' zeros_liker+r<r>s r,constant_to_optimal_zeroz!BaseLoss.constant_to_optimal_zeros&}V$$$r.Fc$|tjtjfvrtd|d|jr ||jf}n|f}tj|||}|jrtjd|}ntj|||}||fS)auInitialize arrays for gradients and hessians. Unless hessians are constant, arrays are initialized with undefined values. Parameters ---------- n_samples : int The number of samples, usually passed to `fit()`. dtype : {np.float64, np.float32}, default=np.float64 The dtype of the arrays gradient and hessian. order : {'C', 'F'}, default='F' Order of the arrays gradient and hessian. The default 'F' makes the arrays contiguous along samples. Returns ------- gradient : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Empty array (allocated but not initialized) to be used as argument gradient_out. hessian : C-contiguous array of shape (n_samples,), array of shape (n_samples, n_classes) or shape (1,) Empty (allocated but not initialized) array to be used as argument hessian_out. If constant_hessian is True (e.g. `HalfSquaredError`), the array is initialized to ``1``. zCValid options for 'dtype' are np.float32 and np.float64. Got dtype=z instead.)rCrIorder)r8)rCrI) r'float32float64 ValueError is_multiclassr"emptyr&ones)r+ n_samplesrIrlrCrMhessians r,init_gradient_and_hessianz"BaseLoss.init_gradient_and_hessians8 RZ0 0 0."...    !/EELE8%uEBBB   F gD666GGhU%uEEEG  r.NNNNNr8NNNr8Nr8N)__name__ __module__ __qualname____doc__differentiablerpr-r4r7rErKrMrQrVrerir'rnrur.r,rrMsE::JNM 9 9 9 9000000 ++++b=&=&=&=&F ////j@)@)@)@)D    >(A(A(A(AT%%%%*:<31!1!1!1!1!1!r.rc$eZdZdZdfd ZxZS)HalfSquaredErroraHalf squared error with identity link, for regression. Domain: y_true and y_pred all real numbers Link: y_pred = raw_prediction For a given sample x_i, half squared error is defined as:: loss(x_i) = 0.5 * (y_true_i - raw_prediction_i)**2 The factor of 0.5 simplifies the computation of gradients and results in a unit hessian (and is consistent with what is done in LightGBM). It is also half the Normal distribution deviance. Ncttt|||du|_dS)Nr r!r#r$)superr-r rr&r+r>r#r$ __class__s r,r-zHalfSquaredError.__init__6sL $&&\^^6    !. 5r.rvr{r|r}r~r- __classcell__rs@r,rr$sG"6666666666r.rc0eZdZdZdZdfd ZddZxZS) AbsoluteErroraAbsolute error with identity link, for regression. Domain: y_true and y_pred all real numbers Link: y_pred = raw_prediction For a given sample x_i, the absolute error is defined as:: loss(x_i) = |y_true_i - raw_prediction_i| Note that the exact hessian = 0 almost everywhere (except at one point, therefore differentiable = False). Optimization routines like in HGBT, however, need a hessian > 0. Therefore, we assign 1. FNcttt||d|_|du|_dS)NrT)rr-rrr%r&rs r,r-zAbsoluteError.__init__QsT !##,..R    # - 5r.cT|tj|dSt||dS)Compute raw_prediction of an intercept-only model. This is the weighted median of the target, i.e. over the samples axis=0. NrrY2)r'medianrrhs r,rez AbsoluteError.fit_intercept_onlyXs1  9V!,,, ,' rBB Br.rvrzr{r|r}r~rr-rerrs@r,rr=sj"N666666 C C C C C C C Cr.rc0eZdZdZdZdfd ZddZxZS) PinballLossaQuantile loss aka pinball loss, for regression. Domain: y_true and y_pred all real numbers quantile in (0, 1) Link: y_pred = raw_prediction For a given sample x_i, the pinball loss is defined as:: loss(x_i) = rho_{quantile}(y_true_i - raw_prediction_i) rho_{quantile}(u) = u * (quantile - 1_{u<0}) = -u *(1 - quantile) if u < 0 u * quantile if u >= 0 Note: 2 * PinballLoss(quantile=0.5) equals AbsoluteError(). Note that the exact hessian = 0 almost everywhere (except at one point, therefore differentiable = False). Optimization routines like in HGBT, however, need a hessian > 0. Therefore, we assign 1. Additional Attributes --------------------- quantile : float The quantile level of the quantile to be estimated. Must be in range (0, 1). FN?ct|dtjdddtt t |t||d|_|du|_ dS) Nquantilerr8neither target_typemin_valmax_valinclude_boundaries)rrT) rnumbersRealrr-rfloatrr%r&)r+r>rr#r$rs r,r-zPinballLoss.__init__s   (      x999    # - 5r.c|$tj|d|jjzdSt ||d|jjzS)rNdrr)r' percentiler rrrhs r,rezPinballLoss.fit_intercept_onlysN  =tz/B)BKKK K' sTZ-@'@ r.)NrNNrzrrs@r,rrdsb:N666666$        r.rc2eZdZdZdZ dfd Zd dZxZS) HuberLossaHuber loss, for regression. Domain: y_true and y_pred all real numbers quantile in (0, 1) Link: y_pred = raw_prediction For a given sample x_i, the Huber loss is defined as:: loss(x_i) = 1/2 * abserr**2 if abserr <= delta delta * (abserr - delta/2) if abserr > delta abserr = |y_true_i - raw_prediction_i| delta = quantile(abserr, self.quantile) Note: HuberLoss(quantile=1) equals HalfSquaredError and HuberLoss(quantile=0) equals delta * (AbsoluteError() - delta/2). Additional Attributes --------------------- quantile : float The quantile level which defines the breaking point `delta` to distinguish between absolute error and squared error. Must be in range (0, 1). Reference --------- .. [1] Friedman, J.H. (2001). :doi:`Greedy function approximation: A gradient boosting machine <10.1214/aos/1013203451>`. Annals of Statistics, 29, 1189-1232. FN?rct|dtjddd||_t t t|t||d|_ d |_ dS) Nrrr8rr)deltarTF) rrrrrr-rrrr%r&)r+r>rrr#r$rs r,r-zHuberLoss.__init__s    (     !  E%LL111    # %r.c |tj|dd}nt||d}||z }tj|tj|jjtj|z}|tj||zS)rNrrrrS) r'rrsignminimumr rabsrU)r+r<r>rdiffterms r,rezHuberLoss.fit_intercept_onlys  ]62A666FF)&-DDFwt}}rz$**:BF4LLIII 4?????r.)NrrNNrzrrs@r,rrsrBNLP&&&&&&*@@@@@@@@r.rc,eZdZdZdfd ZddZxZS)HalfPoissonLossaHalf Poisson deviance loss with log-link, for regression. Domain: y_true in non-negative real numbers y_pred in positive real numbers Link: y_pred = exp(raw_prediction) For a given sample x_i, half the Poisson deviance is defined as:: loss(x_i) = y_true_i * log(y_true_i/exp(raw_prediction_i)) - y_true_i + exp(raw_prediction_i) Half the Poisson deviance is actually the negative log-likelihood up to constant terms (not involving raw_prediction) and simplifies the computation of the gradients. We also skip the constant term `y_true_i * log(y_true_i) - y_true_i`. Ncttt||t dt jdd|_dS)NrrTF)rr-r rrr'r(r)rs r,r-zHalfPoissonLoss.__init__sW #%%GII"V     (264??r.c:t|||z }|||z}|Srzrr+r<r>rs r,riz(HalfPoissonLoss.constant_to_optimal_zero s+VV$$v-  $ M !D r.rvrzr{r|r}r~r-rirrs@r,rrsa(@@@@@@ r.rc,eZdZdZdfd ZddZxZS) HalfGammaLossaVHalf Gamma deviance loss with log-link, for regression. Domain: y_true and y_pred in positive real numbers Link: y_pred = exp(raw_prediction) For a given sample x_i, half Gamma deviance loss is defined as:: loss(x_i) = log(exp(raw_prediction_i)/y_true_i) + y_true/exp(raw_prediction_i) - 1 Half the Gamma deviance is actually proportional to the negative log- likelihood up to constant terms (not involving raw_prediction) and simplifies the computation of the gradients. We also skip the constant term `-log(y_true_i) - 1`. Ncttt||t dt jdd|_dS)NrrF)rr-rrrr'r(r)rs r,r-zHalfGammaLoss.__init__&sM 00wyyRPVWWW'265%@@r.cDtj| dz }|||z}|Sry)r'logrs r,riz&HalfGammaLoss.constant_to_optimal_zero*s+v"  $ M !D r.rvrzrrs@r,rrsa&AAAAAAr.rc,eZdZdZdfd ZddZxZS)HalfTweedieLossaHalf Tweedie deviance loss with log-link, for regression. Domain: y_true in real numbers for power <= 0 y_true in non-negative real numbers for 0 < power < 2 y_true in positive real numbers for 2 <= power y_pred in positive real numbers power in real numbers Link: y_pred = exp(raw_prediction) For a given sample x_i, half Tweedie deviance loss with p=power is defined as:: loss(x_i) = max(y_true_i, 0)**(2-p) / (1-p) / (2-p) - y_true_i * exp(raw_prediction_i)**(1-p) / (1-p) + exp(raw_prediction_i)**(2-p) / (2-p) Taking the limits for p=0, 1, 2 gives HalfSquaredError with a log link, HalfPoissonLoss and HalfGammaLoss. We also skip constant terms, but those are different for p=0, 1, 2. Therefore, the loss is not continuous in `power`. Note furthermore that although no Tweedie distribution exists for 0 < power < 1, it still gives a strictly consistent scoring function for the expectation. N?cttt|t |||jjdkr.ttj tj dd|_ dS|jjdkr#tdtj dd|_ dStdtj dd|_ dSN)powerrrFr:T) rr-r rrr rrr'r(r)r+r>rr#r$rs r,r-zHalfTweedieLoss.__init__Ps #%,,777    : q #+RVGRVUE#J#JD Z  ! !#+ArvtU#C#CD #+Arvue#D#DD r.c|jjdkr#t||S|jjdkr#t ||S|jjdkr#t ||S|jj}t jt j|dd|z d|z z d|z z }|||z}|S)Nr)r<r>r8r:)r rrrirrr'maximum)r+r<r>prs r,riz(HalfTweedieLoss.constant_to_optimal_zero^s : q #%%>>]? Z  " ""$$==]> Z  " " ??;;]<   A8BJvq111q599QUCq1uMD( %Kr.NrNNrzrrs@r,rr1sa< E E E E E Er.rc$eZdZdZdfd ZxZS)HalfTweedieLossIdentityanHalf Tweedie deviance loss with identity link, for regression. Domain: y_true in real numbers for power <= 0 y_true in non-negative real numbers for 0 < power < 2 y_true in positive real numbers for 2 <= power y_pred in positive real numbers for power != 0 y_pred in real numbers for power = 0 power in real numbers Link: y_pred = raw_prediction For a given sample x_i, half Tweedie deviance loss with p=power is defined as:: loss(x_i) = max(y_true_i, 0)**(2-p) / (1-p) / (2-p) - y_true_i * raw_prediction_i**(1-p) / (1-p) + raw_prediction_i**(2-p) / (2-p) Note that the minimum value of this loss is 0. Note furthermore that although no Tweedie distribution exists for 0 < power < 1, it still gives a strictly consistent scoring function for the expectation. Nrc~ttt|t |||jjdkr-ttj tj dd|_ nS|jjdkr"tdtj dd|_ n!tdtj dd|_ |jjdkr.ttj tj dd|_ dStdtj dd|_ dSr) rr-r rrr rrr'r(r)r*rs r,r-z HalfTweedieLossIdentity.__init__s  +%,,???    : q #+RVGRVUE#J#JD Z  ! !#+ArvtU#C#CD #+Arvue#D#DD : q #+RVGRVUE#J#JD #+Arvue#D#DD r.rrrs@r,rrssQ6EEEEEEEEEEr.rc2eZdZdZdfd ZddZdZxZS)HalfBinomialLossaYHalf Binomial deviance loss with logit link, for binary classification. This is also know as binary cross entropy, log-loss and logistic loss. Domain: y_true in [0, 1], i.e. regression on the unit interval y_pred in (0, 1), i.e. boundaries excluded Link: y_pred = expit(raw_prediction) For a given sample x_i, half Binomial deviance is defined as the negative log-likelihood of the Binomial/Bernoulli distribution and can be expressed as:: loss(x_i) = log(1 + exp(raw_pred_i)) - y_true_i * raw_pred_i See The Elements of Statistical Learning, by Hastie, Tibshirani, Friedman, section 4.4.1 (about logistic regression). Note that the formulation works for classification, y = {0, 1}, as well as logistic regression, y = [0, 1]. If you add `constant_to_optimal_zero` to the loss, you get half the Bernoulli/binomial deviance. More details: Inserting the predicted probability y_pred = expit(raw_prediction) in the loss gives the well known:: loss(x_i) = - y_true_i * log(y_pred_i) - (1 - y_true_i) * log(1 - y_pred_i) Nctttd||t dddd|_dSNr:r r!r"r#r$rr8T)rr-rrrr)rs r,r-zHalfBinomialLoss.__init__s[ $&&     (1dD99r.cbt||td|z d|z z}|||z}|Sryrrs r,riz)HalfBinomialLoss.constant_to_optimal_zeros=VV$$uQZV'D'DD  $ M !D r.c&|jdkr&|jddkr|d}tj|jddf|j}|j||dddf<d|dddfz |dddf<|Sa=Predict probabilities. Parameters ---------- raw_prediction : array of shape (n_samples,) or (n_samples, 1) Raw prediction values (in link space). Returns ------- proba : array of shape (n_samples, 2) Element-wise class probabilities. r:r8rrHNrBrCrDr'rqrIr!inverser+r=probas r, predict_probazHalfBinomialLoss.predict_proba  ! # #(::::::r.rcLeZdZdZdZd fd ZdZd dZdZ d d Z xZ S)HalfMultinomialLossa/Categorical cross-entropy loss, for multiclass classification. Domain: y_true in {0, 1, 2, 3, .., n_classes - 1} y_pred has n_classes elements, each element in (0, 1) Link: y_pred = softmax(raw_prediction) Note: We assume y_true to be already label encoded. The inverse link is softmax. But the full link function is the symmetric multinomial logit function. For a given sample x_i, the categorical cross-entropy loss is defined as the negative log-likelihood of the multinomial distribution, it generalizes the binary cross-entropy to more than 2 classes:: loss_i = log(sum(exp(raw_pred_{i, k}), k=0..n_classes-1)) - sum(y_true_{i, k} * raw_pred_{i, k}, k=0..n_classes-1) See [1]. Note that for the hessian, we calculate only the diagonal part in the classes: If the full hessian for classes k and l and sample i is H_i_k_l, we calculate H_i_k_k, i.e. k=l. Parameters ---------- sample_weight : {None, ndarray} If sample_weight is None, the hessian might be constant. n_classes : {None, int} The number of classes for classification, else None. xp : module or None Array namespace module. Ignored by the Cython implementation. device : device or None A device object. Ignored by the Cython implementation. References ---------- .. [1] :arxiv:`Simon, Noah, J. Friedman and T. Hastie. "A Blockwise Descent Algorithm for Group-penalized Multiresponse and Multinomial Regression". <1311.6529>` TNcttt|||t dt jdd|_t dddd|_d|_ d|_ d|_ dS)NrrTFr8) rr-r rrr'r(r)r*class_indexing_offsets y_true_inty_true_one_hotr+r>r"r#r$rs r,r-zHalfMultinomialLoss.__init__s '))!##     (264??'1eU;;'+#"r.c|j|o/tj|t |kSr0)r)r1r'allastypeintr2s r,r4z#HalfMultinomialLoss.in_y_true_range.s9#,,Q//NBF188C==A;M4N4NNr.ctj|j|j}tj|jj}t |jD]B}tj||k|d||<tj|||d|z ||<C|j |dddf dS)a-Compute raw_prediction of an intercept-only model. This is the softmax of the weighted average of the target, i.e. over the samples axis=0. Parameters ---------- y_true : array-like of shape (n_samples,) Observed, true target values. sample_weight : None or array of shape (n_samples,), default=None Sample weights. Returns ------- raw_prediction : numpy scalar or array of shape (n_classes,) Raw predictions of an intercept-only model. rHrrXr8N) r'zerosr"rIr[r\rangerUrar!reshape)r+r<r>outr\ks r,rez&HalfMultinomialLoss.fit_intercept_only7s&ht~V\:::hv|$$(t~&& 3 3AZ! ]KKKCFWSVS!c'22CFFy~~c$'l++33B777r.c6|j|S)a=Predict probabilities. Parameters ---------- raw_prediction : array of shape (n_samples, n_classes) Raw prediction values (in link space). Returns ------- proba : array of shape (n_samples, n_classes) Element-wise class probabilities. )r!r)r+r=s r,rz!HalfMultinomialLoss.predict_probaQsy  000r.r8c|@|)tj|}tj|}n+tj|}n|tj|}|j||||||||fS)aKCompute gradient and class probabilities fow raw_prediction. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. gradient_out : None or array of shape (n_samples, n_classes) A location into which the gradient is stored. If None, a new array might be created. proba_out : None or array of shape (n_samples, n_classes) A location into which the class probabilities are stored. If None, a new array might be created. n_threads : int, default=1 Might use openmp thread parallelism. Returns ------- gradient : array of shape (n_samples, n_classes) Element-wise gradients. proba : array of shape (n_samples, n_classes) Element-wise class probabilities. N)r<r=r>rJ proba_outr@)r'rAr gradient_proba)r+r<r=r>rJrr@s r,rz"HalfMultinomialLoss.gradient_proba`sH   !}^<< M.99 !}Y77   l33I !!)'% "   Y&&r.NrNNrzrx) r{r|r}r~rpr-r4rerrrrs@r,rrs..`M######"OOO88884 1 1 1&5'5'5'5'5'5'5'5'r.rc2eZdZdZdfd ZddZdZxZS)ExponentialLossa"Exponential loss with (half) logit link, for binary classification. This is also know as boosting loss. Domain: y_true in [0, 1], i.e. regression on the unit interval y_pred in (0, 1), i.e. boundaries excluded Link: y_pred = expit(2 * raw_prediction) For a given sample x_i, the exponential loss is defined as:: loss(x_i) = y_true_i * exp(-raw_pred_i)) + (1 - y_true_i) * exp(raw_pred_i) See: - J. Friedman, T. Hastie, R. Tibshirani. "Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)." Ann. Statist. 28 (2) 337 - 407, April 2000. https://doi.org/10.1214/aos/1016218223 - A. Buja, W. Stuetzle, Y. Shen. (2005). "Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications." Note that the formulation works for classification, y = {0, 1}, as well as "exponential logistic" regression, y = [0, 1]. Note that this is a proper scoring rule, but without it's canonical link. More details: Inserting the predicted probability y_pred = expit(2 * raw_prediction) in the loss gives:: loss(x_i) = y_true_i * sqrt((1 - y_pred_i) / y_pred_i) + (1 - y_true_i) * sqrt(y_pred_i / (1 - y_pred_i)) Nctttd||t dddd|_dSr)rr-rrrr)rs r,r-zExponentialLoss.__init__s[ #%%     (1dD99r.cNdtj|d|z zz}|||z}|S)Nr8)r'sqrtrs r,riz(ExponentialLoss.constant_to_optimal_zeros4BGFa&j1222  $ M !D r.c&|jdkr&|jddkr|d}tj|jddf|j}|j||dddf<d|dddfz |dddf<|Srrrs r,rzExponentialLoss.predict_probarr.rvrzrrs@r,rrsk!!F::::::r.r) squared_errorabsolute_error pinball_loss huber_loss poisson_loss gamma_loss tweedie_loss binomial_lossmultinomial_lossexponential_losscJeZdZdZ ddZ d dZ d dZ d dZdS) ArrayAPILossMixinaMixin for loss classes that are compatible with the array API. Currently this mixin redefines methods: - __call__(...) - loss(...) - loss_gradient(...) - gradient(...) such that they work according to the array API specification. It uses the attributes self.xp and self.device from BaseLoss and it assumes that methods self._compute_loss and self._compute_gradient are implemented. Nr8cz|||d}tt|||jS)aCompute the weighted average loss for the array API losses. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. n_threads : int, default=1 Ignored by the array API implementation. Returns ------- loss : float Mean or averaged loss function. Nr<r=r>)rTr#)rErrr#)r+r<r=r>r@loss_xps r,rVzArrayAPILossMixin.__call__sD4)).  Xg}IIIJJJr.c2||||S)aCompute the pointwise loss value for each input. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. loss_out : None or C-contiguous array of shape (n_samples,) Ignored by the array API implementation. n_threads : int, default=1 Ignored by the array API implementation. Returns ------- loss : array of shape (n_samples,) Element-wise loss function. r) _compute_lossrFs r,rEzArrayAPILossMixin.losss*:!!)'"   r.cj||||}||||}||fS)aGCompute loss and gradient w.r.t. raw_prediction for each input. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. loss_out : None or C-contiguous array of shape (n_samples,) Ignored by the array API implementation. gradient_out : None or C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Ignored by the array API implementation. n_threads : int, default=1 Ignored by the array API implementation. Returns ------- loss : array of shape (n_samples,) Element-wise loss function. gradient : array of shape (n_samples,) or (n_samples, n_classes) Element-wise gradients. r)r_compute_gradient) r+r<r=r>r?rJr@rErMs r,rKzArrayAPILossMixin.loss_gradientAsYH!!)'"   )))'*   X~r.c2||||S)axCompute gradient of loss w.r.t raw_prediction for each input. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. gradient_out : None or C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Ignored by the array API implementation. n_threads : int, default=1 Ignored by the array API implementation. Returns ------- gradient : array of shape (n_samples,) or (n_samples, n_classes) Element-wise gradients. r)r rNs r,rMzArrayAPILossMixin.gradientqs*<%%)'&   r.ryrwrx)r{r|r}r~rVrErKrMrr.r,rrs  " KKKKF ! ! ! ! N....h " " " " " " r.rc|j|jkrgdngd}|||dk||||dk|||||dk|d|z|||dk|d|z z|S)anNumerically stable version of log(1 + exp(x)) that is compatible with the array API. Parameters ---------- raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). raw_prediction_exp : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Exponential of the raw prediction values. xp : module, default=None Array namespace module. Returns ------- log1pexp : float Numerically stable value for log(1 + exp(raw_prediction)). )rgfffff@@)r g333333-@rr8r:g?r)rIrnwherelog1pr)r=raw_prediction_expr# constantss r, _log1pexprsj  2: - -     88)A,&  il * HH' ( ( HH)A,.s//00"il2"Q);%;;"     r.c:eZdZdZ ddZ ddZ ddZdS) HalfBinomialLossArrayAPIzHA version of the HalfBinomialLoss that is compatible with the array API.Nr8c|j|}|||||}|||||} || fSN)r<r=r>rr#exprr  r+r<r=r>r?rJr@rrErMs r,rKz&HalfBinomialLossArrayAPI.loss_gradientr"W[[88!!)'1 "   )))'1 *   X~r.c||j|}t|||j}|||zz }|||z}|S)N)r=rr#)r#rr)r+r<r=r>rlog1pexprEs r,rz&HalfBinomialLossArrayAPI._compute_losssc  %!%^!11  $ M !D r.c|j}|||}d|z }|||j|jkrdndkd|z ||zz d|zz ||z }|||z}|S)Nr8r r)r#rrrIrn)r+r<r=r>rr#neg_raw_prediction_expgrads r,r z*HalfBinomialLossArrayAPI._compute_gradientsW  %!#!7!7 !"%7!7xx ^%9RZ%G%GccS Q&jF%;; ;)) +  '     $ M !D r.rxNNr{r|r}r~rKrr rr.r,rrstRR 8 . r.rc8eZdZdZdfd Z ddZ ddZxZS) HalfMultinomialLossArrayAPIaA version of the HalfMultinomialLoss that is compatible with the array API. Parameters ---------- sample_weight : {None, ndarray} If sample_weight is None, the hessian might be constant. n_classes : {None, int} The number of classes for classification, else None. xp : module or None Array namespace module. device : device or None A device object. Nrczt|||d|_d|_d|_dS)N)r"r#r$)rr-rrrrs r,r-z$HalfMultinomialLossArrayAPI.__init__7sC 9FCCC '+##r.c||j}|j}t|d|}|j"|||j||_|j/||jd||j z|_| t||j|jz}||z }|||z}|S)Nr8)rYr#rIr$r)r$) r#r$rrasarrayint64rarangerCr"taker) r+r<r=r>r#r$ log_sum_exptrue_label_probsrEs r,rz)HalfMultinomialLossArrayAPI._compute_lossDs W aB??? ? " jjrxjOODO  & . &,q/& 99DNJ  '77 > " "DOd6Q$Q  --  $ M !D r.c"|j}|j}|jO|j"|||j||_t |j|j|j|_t|}||jz}|||dddfz}|S)Nr() num_classesrI) r#r$rrr)r*rr"rIr)r+r<r=r>r#device_r!s r,r z-HalfMultinomialLossArrayAPI._compute_gradient\s W+   &&"$**V28G*"T"T") N$*###D  ~&& ##  $ M!!!T'* *D r.rrz)r{r|r}r~r-rr rrs@r,r%r%%sy" # # # # # #" 8 r.r%c:eZdZdZ ddZ ddZ ddZdS) HalfPoissonLossArrayAPIzGA version of the HalfPoissonLoss that is compatible with the array API.Nr8c|j|}|||||}|||||} || fSrrrs r,rKz%HalfPoissonLossArrayAPI.loss_gradientrr.c\||j|}|||zz }|||z}|Srzr#r)r+r<r=r>rrEs r,rz%HalfPoissonLossArrayAPI._compute_losssB  %!%^!rr!s r,r z)HalfPoissonLossArrayAPI._compute_gradients=  %!%^!rBs*                          655555&&&&&& *)))))444444*T!T!T!T!T!T!T!T!n66666x6662$C$C$C$C$CH$C$C$CN=====(===@I@I@I@I@I@I@I@I@XhDH>?????h???D-E-E-E-E-Eh-E-E-E`DDDDDxDDDNk'k'k'k'k'(k'k'k'\HHHHHhHHHX&###%+'  b b b b b b b b JHHHVAAAAA02BAAAHUUUUU"35HUUUp55555/55555r.