o tBhB@s6dZddlZddlmZddlmZGdddZdS)zA Loss functions for linear models with raw_prediction = X @ coef N)sparse) squared_normc@sNeZdZdZddZddZdd d Z dd d Z dd dZ dddZ dS)LinearModelLossaGeneral class for loss functions with raw_prediction = X @ coef + intercept. Note that raw_prediction is also known as linear predictor. The loss is the sum of per sample losses and includes a term for L2 regularization:: loss = sum_i s_i loss(y_i, X_i @ coef + intercept) + 1/2 * l2_reg_strength * ||coef||_2^2 with sample weights s_i=1 if sample_weight=None. Gradient and hessian, for simplicity without intercept, are:: gradient = X.T @ loss.gradient + l2_reg_strength * coef hessian = X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity Conventions: if fit_intercept: n_dof = n_features + 1 else: n_dof = n_features if base_loss.is_multiclass: coef.shape = (n_classes, n_dof) or ravelled (n_classes * n_dof,) else: coef.shape = (n_dof,) The intercept term is at the end of the coef array: if base_loss.is_multiclass: if coef.shape (n_classes, n_dof): intercept = coef[:, -1] if coef.shape (n_classes * n_dof,) intercept = coef[n_features::n_dof] = coef[(n_dof-1)::n_dof] intercept.shape = (n_classes,) else: intercept = coef[-1] Note: If coef has shape (n_classes * n_dof,), the 2d-array can be reconstructed as coef.reshape((n_classes, -1), order="F") The option order="F" makes coef[:, i] contiguous. This, in turn, makes the coefficients without intercept, coef[:, :-1], contiguous and speeds up matrix-vector computations. Note: If the average loss per sample is wanted instead of the sum of the loss per sample, one can simply use a rescaled sample_weight such that sum(sample_weight) = 1. Parameters ---------- base_loss : instance of class BaseLoss from sklearn._loss. fit_intercept : bool cCs||_||_dS)N) base_loss fit_intercept)selfrrr x/var/www/html/riverr-enterprise-integrations-main/venv/lib/python3.10/site-packages/sklearn/linear_model/_linear_loss.py__init__Bs zLinearModelLoss.__init__cCs|jjs|jr|d}|dd}nd}|}|||}n2|jdkr.|j|jjdfdd}n|}|jrF|dddf}|ddddf}nd}||j|}|||fS)aiHelper function to get coefficients, intercept and raw_prediction. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. raw_prediction : ndarray of shape (n_samples,) or (n_samples, n_classes) NForder)r is_multiclassrndimreshape n_classesT)rcoefX interceptweightsraw_predictionr r r _w_intercept_rawFs   z LinearModelLoss._w_intercept_rawNr rc CsX|||\}}} |jj|| ||d} | } |jdkr ||nt|} | d|| S)aCompute the loss as sum over point-wise losses. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- loss : float Sum of losses per sample plus penalty. y_truer sample_weight n_threadsr?)rrlosssumrr) rrryrl2_reg_strengthr rrrr"norm2_wr r r r"sszLinearModelLoss.losscCs<|jd|jj}}|t|j} |||\} } } |jj|| ||d\} }| } |jjsZ| d|| | 7} t j || j d}|j ||| |d|<|jrV||d<| |fS| d|t | 7} t j|| f| j dd}|j ||| |ddd|f<|jr|jd d |dddf<|jdkr|jdd }| |fS) a8Computes the sum of loss and gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- loss : float Sum of losses per sample plus penalty. gradient : ndarray of shape coef.shape The gradient of the loss. rrr!dtypeNr rr(rraxisr)shaperrintrr loss_gradientr#rnp empty_liker(rremptyrravel)rrrr$rr%r n_featuresrn_dofrrrr"grad_per_samplegradr r r r.s2   "  zLinearModelLoss.loss_gradientcCs|jd|jj}}|t|j} |||\} } } |jj|| ||d} |jjsHtj || j d}|j | || |d|<|jrF| |d<|Stj || f| j dd}| j ||| |ddd|f<|jrs| j dd |dddf<|jdkr~|jdd S|S) aComputes the gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. rrr'Nr rr)rr*r)r,rrr-rrgradientrr/r0r(rr#r1rr2)rrrr$rr%r r3rr4rrrr5r6r r r r7s, "  zLinearModelLoss.gradientc sj jj\}t j \ }} jjs jj|| |d\} } tj  j d} j |  | d< jrH| | d<| t r_t j| df||fdn | ddtjf jryttj dd fdd } | | fS jj|| |d\} tjf j d d } | j  | dddf< jr| j dd| dddf< f d d } jd kr| jd d| fS| | fS)aComputes gradient and hessp (hessian product function) w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessp : callable Function that takes in a vector input of shape of gradient and and returns matrix-vector product with hessian. rr'Nr r)r,r*cst|}trj|d|d<ntjj|dg|d<|d|d7<jr]|d|d7<|d|d|d<|S)Nr )r/r0rissparserlinalg multi_dotr)sret)rhXhX_sum hessian_sumr%r3rr r hesspPs   $  z7LinearModelLoss.gradient_hessian_product..hessprr)cs |jdfdd}jr|dddf}|ddddf}nd}|j|}| |jddddtjf7}|9}durM|ddtjf9}tjf jdd}|j||dddf<jrx|jdd|dddf<jdkr|j ddS|S)Nr rrrrr*r)) rrrr#r/newaxisr1r(rr2)r; s_intercepttmp hess_prod) rrr%rr4r3probarrrr r r@s"$"  rr)r,rrr-rrrgradient_hessianr/r0r(rr#rr8 dia_matrixrAsqueezeasarraygradient_probar1rr2)rrrr$rr%r n_samplesrrr7hessianr6r@r ) rrr=r>r?r%rr4r3rErrrr gradient_hessian_productsH    M " z(LinearModelLoss.gradient_hessian_product)Nr r) __name__ __module__ __qualname____doc__r rr"r.r7rMr r r r r s8 -( > 9r)rQnumpyr/scipyr utils.extmathrrr r r r s