h dZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z mZmZddlmZmZmZmZmZddlmZdd lmZdd lmZGd d ZGd deZGddeZGddeZGddeZ GddeZ!GddeZ"GddeZ#GddeZ$GddeZ%eeee e!e"e$e%dZ&dS) z This module contains loss classes suitable for fitting. It is not part of the public API. Specific losses are used for regression, binary classification or multiclass classification. Nxlogy) CyHalfSquaredErrorCyAbsoluteError CyPinballLossCyHalfPoissonLossCyHalfGammaLossCyHalfTweedieLossCyHalfTweedieLossIdentityCyHalfBinomialLossCyHalfMultinomialLoss)Interval IdentityLinkLogLink LogitLinkMultinomialLogit) check_scalar)ReadonlyArrayWrapper)_weighted_percentileceZdZdZdZdZdZddZdZdZ dd Z dd Z dd Z dd Z dd ZddZddZejdfdZdS)BaseLossaBase class for a loss function of 1-dimensional targets. Conventions: - y_true.shape = sample_weight.shape = (n_samples,) - y_pred.shape = raw_prediction.shape = (n_samples,) - If is_multiclass is true (multiclass classification), then y_pred.shape = raw_prediction.shape = (n_samples, n_classes) Note that this corresponds to the return value of decision_function. y_true, y_pred, sample_weight and raw_prediction must either be all float64 or all float32. gradient and hessian must be either both float64 or both float32. Note that y_pred = link.inverse(raw_prediction). Specific loss classes can inherit specific link classes to satisfy BaseLink's abstractmethods. Parameters ---------- sample_weight : {None, ndarray} If sample_weight is None, the hessian might be constant. n_classes : {None, int} The number of classes for classification, else None. Attributes ---------- closs: CyLossFunction link : BaseLink interval_y_true : Interval Valid interval for y_true interval_y_pred : Interval Valid Interval for y_pred differentiable : bool Indicates whether or not loss function is differentiable in raw_prediction everywhere. need_update_leaves_values : bool Indicates whether decision trees in gradient boosting need to uptade leave values after having been fit to the (negative) gradients. approx_hessian : bool Indicates whether the hessian is approximated or exact. If, approximated, it should be larger or equal to the exact one. constant_hessian : bool Indicates whether the hessian is one for this loss. is_multiclass : bool Indicates whether n_classes > 2 is allowed. FTNc||_||_d|_d|_||_t t j t jdd|_|jj |_ dS)NF) closslinkapprox_hessianconstant_hessian n_classesrnpinfinterval_y_trueinterval_y_pred)selfrrrs N/var/www/html/Sam_Eipo/venv/lib/python3.11/site-packages/sklearn/_loss/loss.py__init__zBaseLoss.__init__}sV  # %"'FF#y8c6|j|SzuReturn True if y is in the valid range of y_true. Parameters ---------- y : ndarray )r"includesr$ys r%in_y_true_rangezBaseLoss.in_y_true_range#,,Q///r'c6|j|S)zuReturn True if y is in the valid range of y_pred. Parameters ---------- y : ndarray )r#r*r+s r%in_y_pred_rangezBaseLoss.in_y_pred_ranger.r'rc,|tj|}|jdkr&|jddkr|d}t |}t |}|t |}|j|||||S)aJCompute the pointwise loss value for each input. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. loss_out : None or C-contiguous array of shape (n_samples,) A location into which the result is stored. If None, a new array might be created. n_threads : int, default=1 Might use openmp thread parallelism. Returns ------- loss : array of shape (n_samples,) Element-wise loss function. Nrry_trueraw_prediction sample_weightloss_out n_threads)r empty_likendimshapesqueezerrloss)r$r3r4r5r6r7s r%r<z BaseLoss.losss<  }V,,H  ! # #(  =88L  ! # #(sPN  "!}^<<  mN;; !}[99  - 55K  ! # #(%>%--a00K%f---n==  $0??Mz**)'%# +   r'c ^tj|||dd||S)a{Compute the weighted average loss. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. n_threads : int, default=1 Might use openmp thread parallelism. Returns ------- loss : float Mean or averaged loss function. Nr2)weights)r averager<)r$r3r4r5r7s r%__call__zBaseLoss.__call__sG(z II-"#   "    r'c tj||d}dtj|jjz}|jjtj krd}n(|jjr |jj}n|jj|z}|jj tjkrd}n(|jj r |jj }n|jj |z }|||j |S|j tj |||S)a#Compute raw_prediction of an intercept-only model. This can be used as initial estimates of predictions, i.e. before the first iteration in fit. Parameters ---------- y_true : array-like of shape (n_samples,) Observed, true target values. sample_weight : None or array of shape (n_samples,) Sample weights. Returns ------- raw_prediction : numpy scalar or array of shape (n_classes,) Raw predictions of an intercept-only model. rrHaxis N) r rIfinfor?epsr#lowr! low_inclusivehighhigh_inclusiverclip)r$r3r5y_predrPa_mina_maxs r%fit_intercept_onlyzBaseLoss.fit_intercept_onlys(FMBBB28FL))--   #w . .EE  ! / 3(,EE(,s2E   $ . .EE  ! 0 4(-EE(-3E =U]9>>&)) )9>>"'&%"?"?@@ @r'c*tj|S)zpCalculate term dropped in loss. With this term added, the loss of perfect predictions is zero. )r zeros_liker$r3r5s r%constant_to_optimal_zeroz!BaseLoss.constant_to_optimal_zeros }V$$$r'Fc$|tjtjfvrtd|d|jr ||jf}n|f}tj|||}|jrtjd|}ntj|||}||fS)auInitialize arrays for gradients and hessians. Unless hessians are constant, arrays are initialized with undefined values. Parameters ---------- n_samples : int The number of samples, usually passed to `fit()`. dtype : {np.float64, np.float32}, default=np.float64 The dtype of the arrays gradient and hessian. order : {'C', 'F'}, default='F' Order of the arrays gradient and hessian. The default 'F' makes the arrays contiguous along samples. Returns ------- gradient : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Empty array (allocated but not initialized) to be used as argument gradient_out. hessian : C-contiguous array of shape (n_samples,), array of shape (n_samples, n_classes) or shape (1,) Empty (allocated but not initialized) array to be used as argument hessian_out. If constant_hessian is True (e.g. `HalfSquaredError`), the array is initialized to ``1``. zCValid options for 'dtype' are np.float32 and np.float64. Got dtype=z instead.)r:r?order)r)r:r?) r float32float64 ValueError is_multiclassremptyrones)r$ n_samplesr?r`r:rChessians r%init_gradient_and_hessianz"BaseLoss.init_gradient_and_hessians8 RZ0 0 0."...    !/EELE8%uEBBB   F gD666GGhU%uEEEG  r'N)NNrNNNrNr)__name__ __module__ __qualname____doc__need_update_leaves_valuesdifferentiablerdr&r-r0r<rArCrFrJrYr]r rbrir'r%rr>sO//t!&NM9999000000 . . . . h@ @ @ @ L 2 2 2 2 pC C C C J    >(A(A(A(AT%%%%:<31!1!1!1!1!1!r'rc$eZdZdZdfd ZxZS)HalfSquaredErroraHalf squared error with identity link, for regression. Domain: y_true and y_pred all real numbers Link: y_pred = raw_prediction For a given sample x_i, half squared error is defined as:: loss(x_i) = 0.5 * (y_true_i - raw_prediction_i)**2 The factor of 0.5 simplifies the computation of gradients and results in a unit hessian (and is consistent with what is done in LightGBM). It is also half the Normal distribution deviance. Ncttt|du|_dS)Nrr)superr&rrrr$r5 __class__s r%r&zHalfSquaredError.__init__s> 133,..III - 5r'rjrmrnrorpr& __classcell__rzs@r%ruru sG"6666666666r'ruc4eZdZdZdZdZdfd ZddZxZS) AbsoluteErroraAbsolute error with identity link, for regression. Domain: y_true and y_pred all real numbers Link: y_pred = raw_prediction For a given sample x_i, the absolute error is defined as:: loss(x_i) = |y_true_i - raw_prediction_i| FTNctttd|_|du|_dS)NrwT)rxr&rrrrrys r%r&zAbsoluteError.__init__3sE 00|~~FFF" - 5r'cT|tj|dSt||dS)Compute raw_prediction of an intercept-only model. This is the weighted median of the target, i.e. over the samples axis=0. NrrM2)r medianrr\s r%rYz AbsoluteError.fit_intercept_only8s1  9V!,,, ,' rBB Br'rj rmrnrorprrrqr&rYr|r}s@r%rr"sp  N $666666 C C C C C C C Cr'rc4eZdZdZdZdZdfd Zd dZxZS) PinballLossaQuantile loss aka pinball loss, for regression. Domain: y_true and y_pred all real numbers quantile in (0, 1) Link: y_pred = raw_prediction For a given sample x_i, the pinball loss is defined as:: loss(x_i) = rho_{quantile}(y_true_i - raw_prediction_i) rho_{quantile}(u) = u * (quantile - 1_{u<0}) = -u *(1 - quantile) if u < 0 u * quantile if u >= 0 Note: 2 * PinballLoss(quantile=0.5) equals AbsoluteError(). Additional Attributes --------------------- quantile : float The quantile to be estimated. Must be in range (0, 1). FTN?ct|dtjdddtt t |td|_|du|_ dS) Nquantilerrneither) target_typemin_valmax_valinclude_boundaries)rrwT) rnumbersRealrxr&rfloatrrr)r$r5rrzs r%r&zPinballLoss.__init__as   (      x999    # - 5r'c|$tj|d|jjzdSt ||d|jjzS)rNdrr)r percentilerrrr\s r%rYzPinballLoss.fit_intercept_onlyqsN  =tz/B)BKKK K' sTZ-@'@ r')Nrrjrr}s@r%rrDsh2N $666666        r'rc,eZdZdZdfd ZddZxZS)HalfPoissonLossaHalf Poisson deviance loss with log-link, for regression. Domain: y_true in non-negative real numbers y_pred in positive real numbers Link: y_pred = exp(raw_prediction) For a given sample x_i, half the Poisson deviance is defined as:: loss(x_i) = y_true_i * log(y_true_i/exp(raw_prediction_i)) - y_true_i + exp(raw_prediction_i) Half the Poisson deviance is actually the negative log-likelihood up to constant terms (not involving raw_prediction) and simplifies the computation of the gradients. We also skip the constant term `y_true_i * log(y_true_i) - y_true_i`. Nctttt dt jdd|_dS)NrwrTF)rxr&r rrr r!r"rys r%r&zHalfPoissonLoss.__init__sI 022CCC'264??r'c:t|||z }|||z}|Srjrr$r3r5terms r%r]z(HalfPoissonLoss.constant_to_optimal_zeros+VV$$v-  $ M !D r'rjrmrnrorpr&r]r|r}s@r%rrsa(@@@@@@r'rc,eZdZdZdfd ZddZxZS) HalfGammaLossaVHalf Gamma deviance loss with log-link, for regression. Domain: y_true and y_pred in positive real numbers Link: y_pred = exp(raw_prediction) For a given sample x_i, half Gamma deviance loss is defined as:: loss(x_i) = log(exp(raw_prediction_i)/y_true_i) + y_true/exp(raw_prediction_i) - 1 Half the Gamma deviance is actually proportional to the negative log- likelihood up to constant terms (not involving raw_prediction) and simplifies the computation of the gradients. We also skip the constant term `-log(y_true_i) - 1`. Nctttt dt jdd|_dS)NrwrF)rxr&r rrr r!r"rys r%r&zHalfGammaLoss.__init__sH 00wyyAAA'265%@@r'cDtj| dz }|||z}|Srl)r logrs r%r]z&HalfGammaLoss.constant_to_optimal_zeros+v"  $ M !D r'rjrr}s@r%rrsa&AAAAAAr'rc,eZdZdZdfd ZddZxZS)HalfTweedieLossaHalf Tweedie deviance loss with log-link, for regression. Domain: y_true in real numbers for power <= 0 y_true in non-negative real numbers for 0 < power < 2 y_true in positive real numbers for 2 <= power y_pred in positive real numbers power in real numbers Link: y_pred = exp(raw_prediction) For a given sample x_i, half Tweedie deviance loss with p=power is defined as:: loss(x_i) = max(y_true_i, 0)**(2-p) / (1-p) / (2-p) - y_true_i * exp(raw_prediction_i)**(1-p) / (1-p) + exp(raw_prediction_i)**(2-p) / (2-p) Taking the limits for p=0, 1, 2 gives HalfSquaredError with a log link, HalfPoissonLoss and HalfGammaLoss. We also skip constant terms, but those are different for p=0, 1, 2. Therefore, the loss is not continuous in `power`. Note furthermore that although no Tweedie distribution exists for 0 < power < 1, it still gives a strictly consistent scoring function for the expectation. N?cttt|t |jjdkr.ttj tj dd|_ dS|jjdkr#tdtj dd|_ dStdtj dd|_ dSN)powerrwrFrT) rxr&r rrrrrr r!r"r$r5rrzs r%r&zHalfTweedieLoss.__init__s #%,,777     : q #+RVGRVUE#J#JD Z  ! !#+ArvtU#C#CD #+Arvue#D#DD r'c|jjdkr#t||S|jjdkr#t ||S|jjdkr#t ||S|jj}t jt j|dd|z d|z z d|z z }|||z}|S)Nr)r3r5rr)rrrur]rrr maximum)r$r3r5prs r%r]z(HalfTweedieLoss.constant_to_optimal_zeros : q #%%>>]? Z  " ""$$==]> Z  " " ??;;]<   A8BJvq111q599QUCq1uMD( %Kr'Nrrjrr}s@r%rrsa< E E E E E Er'rc$eZdZdZdfd ZxZS)HalfTweedieLossIdentityanHalf Tweedie deviance loss with identity link, for regression. Domain: y_true in real numbers for power <= 0 y_true in non-negative real numbers for 0 < power < 2 y_true in positive real numbers for 2 <= power y_pred in positive real numbers for power != 0 y_pred in real numbers for power = 0 power in real numbers Link: y_pred = raw_prediction For a given sample x_i, half Tweedie deviance loss with p=power is defined as:: loss(x_i) = max(y_true_i, 0)**(2-p) / (1-p) / (2-p) - y_true_i * raw_prediction_i**(1-p) / (1-p) + raw_prediction_i**(2-p) / (2-p) Note that the minimum value of this loss is 0. Note furthermore that although no Tweedie distribution exists for 0 < power < 1, it still gives a strictly consistent scoring function for the expectation. Nrczttt|t |jjdkr-ttj tj dd|_ nS|jjdkr"tdtj dd|_ n!tdtj dd|_ |jjdkr.ttj tj dd|_ dStdtj dd|_ dSr) rxr&r rrrrrr r!r"r#rs r%r&z HalfTweedieLossIdentity.__init__s +%,,???     : q #+RVGRVUE#J#JD Z  ! !#+ArvtU#C#CD #+Arvue#D#DD : q #+RVGRVUE#J#JD #+Arvue#D#DD r'rr{r}s@r%rrsQ6EEEEEEEEEEr'rc2eZdZdZdfd ZddZdZxZS)HalfBinomialLossaHalf Binomial deviance loss with logit link, for binary classification. This is also know as binary cross entropy, log-loss and logistic loss. Domain: y_true in [0, 1], i.e. regression on the unit interval y_pred in (0, 1), i.e. boundaries excluded Link: y_pred = expit(raw_prediction) For a given sample x_i, half Binomial deviance is defined as the negative log-likelihood of the Binomial/Bernoulli distribution and can be expressed as:: loss(x_i) = log(1 + exp(raw_pred_i)) - y_true_i * raw_pred_i See The Elements of Statistical Learning, by Hastie, Tibshirani, Friedman, section 4.4.1 (about logistic regression). Note that the formulation works for classification, y = {0, 1}, as well as logistic regression, y = [0, 1]. If you add `constant_to_optimal_zero` to the loss, you get half the Bernoulli/binomial deviance. Nctttdt dddd|_dS)NrrrrrrT)rxr&r rrr"rys r%r&zHalfBinomialLoss.__init__GsU $&&    (1dD99r'cbt||td|z d|z z}|||z}|Srlrrs r%r]z)HalfBinomialLoss.constant_to_optimal_zeroOs=VV$$uQZV'D'DD  $ M !D r'c&|jdkr&|jddkr|d}tj|jddf|j}|j||dddf<d|dddfz |dddf<|S)a=Predict probabilities. Parameters ---------- raw_prediction : array of shape (n_samples,) or (n_samples, 1) Raw prediction values (in link space). Returns ------- proba : array of shape (n_samples, 2) Element-wise class probabilities. rrrr>N)r9r:r;r rer?rinverse)r$r4probas r% predict_probazHalfBinomialLoss.predict_probaVs  ! # #(` TNcttt|t dt jdd|_t dddd|_dS)NrrTFr) rxr&rrrr r!r"r#)r$r5rrzs r%r&zHalfMultinomialLoss.__init__so '))!##    (264??'1eU;;r'c|j|o/tj|t |kSr))r"r*r allastypeintr+s r%r-z#HalfMultinomialLoss.in_y_true_ranges9#,,Q//NBF188C==A;M4N4NNr'ctj|j|j}tj|jj}t |jD]B}tj||k|d||<tj|||d|z ||<C|j |dddf dS)zCompute raw_prediction of an intercept-only model. This is the softmax of the weighted average of the target, i.e. over the samples axis=0. r>rrLrN) r zerosrr?rOrPrangerIrUrreshape)r$r3r5outrPks r%rYz&HalfMultinomialLoss.fit_intercept_onlys ht~V\:::hv|$$(t~&& 3 3AZ! ]KKKCFWSVS!c'22CFFy~~c$'l++33B777r'c6|j|S)a=Predict probabilities. Parameters ---------- raw_prediction : array of shape (n_samples, n_classes) Raw prediction values (in link space). Returns ------- proba : array of shape (n_samples, n_classes) Element-wise class probabilities. )rr)r$r4s r%rz!HalfMultinomialLoss.predict_probasy  000r'rcP|@|)tj|}tj|}n+tj|}n|tj|}t|}t|}|t|}|j||||||S)aKCompute gradient and class probabilities fow raw_prediction. Parameters ---------- y_true : C-contiguous array of shape (n_samples,) Observed, true target values. raw_prediction : array of shape (n_samples, n_classes) Raw prediction values (in link space). sample_weight : None or C-contiguous array of shape (n_samples,) Sample weights. gradient_out : None or array of shape (n_samples, n_classes) A location into which the gradient is stored. If None, a new array might be created. proba_out : None or array of shape (n_samples, n_classes) A location into which the class probabilities are stored. If None, a new array might be created. n_threads : int, default=1 Might use openmp thread parallelism. Returns ------- gradient : array of shape (n_samples, n_classes) Element-wise gradients. proba : array of shape (n_samples, n_classes) Element-wise class probabilities. N)r3r4r5r@ proba_outr7)r r8rrgradient_proba)r$r3r4r5r@rr7s r%rz"HalfMultinomialLoss.gradient_probasH   !}^<< M.99 !}Y77   l33I%f---n==  $0??Mz(()'% )   r')Nrrjrk) rmrnrorprdr&r-rYrrr|r}s@r%rrls  DM<<<<<<OOO 8 8 8 8 1 1 1&8 8 8 8 8 8 8 8 r'r) squared_errorabsolute_error pinball_loss poisson_loss gamma_loss tweedie_loss binomial_lossmultinomial_loss)'rprnumpyr scipy.specialr_lossrrrr r r r r rrrrrrrutilsrutils._readonly_array_wrapperr utils.statsrrrurrrrrrrr_LOSSESrsr'r%rs"                      ! @@@@@@......*F!F!F!F!F!F!F!F!Z66666x666.CCCCCHCCCD88888(888vh@H>=====h===@+E+E+E+E+Eh+E+E+E\=====x===@K K K K K (K K K ^&###%+  r'