qtj ~dZddlZddlmZmZddlmZmZddlZ ddl m Z m Z ddl mZmZmZmZmZmZddlmZddlmZmZdd lmZmZdd lmZdd lmZm Z m!Z!gd Z"d Z# d"dZ$dZ%d#dZ&dZ'GddeeeeeeZ(Gdde(Z)Gdde(Z*Gdde(Z+Gd d!eeeZ,dS)$zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). N)ABCMetaabstractmethod)IntegralReal)pinvsvd) BaseEstimatorClassNamePrefixFeaturesOutMixinMultiOutputMixinRegressorMixinTransformerMixin _fit_context)ConvergenceWarning) check_arraycheck_consistent_length)Interval StrOptions)svd_flip) FLOAT_DTYPEScheck_is_fitted validate_data)PLSSVD PLSCanonical PLSRegressionc t|dd\}}}|jj}ddd}t j|||zt j|jz}t j||k}|ddd|f}||d|z}t j t j t j ||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssum transpose conjugatedot)ausvhtfactorcondranks ^/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/sklearn/cross_decomposition/_pls.py _pinv2_oldr4 s 1E>>>HAq"  AS ! !F 6!99vay 28A;;? 2D 6!d(  D !!!UdU( A5D5MA < RVAr%4%y%9%9:: ; ;;Aư>Fcntj|jj t fd|jD}n"#t $r}t d|d}~wwxYwd}|dkrt|t|} } t|D]r} |dkrtj | |} n0tj |j|tj ||z } | tj tj | | zz} tj || } |dkrtj | | }n5tj |j| tj | j| z }|r-|tj tj ||zz}tj ||tj ||zz }| |z }tj |||ks|j ddkrn| }t| dz}||krtj dt| ||fS)a?Return the first left and right singular vectors of X'y. Provides an alternative to the svd(X'y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3pK|]0}tjtj|k,|V1dSN)r#anyabs).0colr&s r3 z;_get_first_singular_vectors_power_method..?s?GGsRVBF3KK#4E-F-FGsGGGGGGr5y residual is constantNdBz$Maximum number of iterations reached)r#r%r r&nextT StopIterationr4ranger*sqrtshapewarningswarnr)Xymodemax_itertolnorm_y_weightsy_scoree x_weights_oldX_pinvy_pinvi x_weightsx_score y_weightsx_weights_diffn_iterr&s @r3(_get_first_singular_vectors_power_methodr^2s- (17   C=GGGGacGGGGG ===4551<=M s{{$A 1  8__"" 3;;vw//IIqsG,,rvgw/G/GGIRWRVIy99::S@@ &I&& 3;;vw//IIqsG,,rvgi/I/III  E  9!=!=>>D DI&I&&"&I*F*F*LM"]2 6.. 1 1C 7 7171:?? E! UF  <>PQQQ i ''s A A! AA!ctj|j|}t|d\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'y. Here the whole SVD is computed. FrNr)r#r*rFr)rMrNCU_Vts r3_get_first_singular_vectors_svdremsP qsAA1E***HAq" QQQT7Bq!!!tH r5Tc|d}||z}|d}||z}|rK|dd}d||dk<||z}|dd}d||dk<||z}n>tj|jd}tj|jd}||||||fS)z{Center X, y and scale if the scale parameter==True Returns ------- X, y, x_mean, y_mean, x_std, y_std raxisrD)rhddofg?)meanstdr#onesrJ)rMrNscalex_meany_meanx_stdy_stds r3_center_scale_xyrswsVVV^^FKA VVV^^FKA $11%%!esl U 11%%!esl U  ## ## a --r5ctjtj|}tj||}||z}||z}dS)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r#argmaxr=sign)r,vbiggest_abs_val_idxrvs r3 _svd_flip_1drysG)BF1II.. 71() * *DIAIAAAr5c ZeZdZUdZeedddgdgeddhged d hged d hgeedddgeed ddgdgdZe e d<e dddd d dddddZ e ddZddZddZd dZddZfdZxZS)!_PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf rDNleftclosedboolean regression canonicalr6rCrnipalsr n_componentsrndeflation_moderO algorithmrPrQcopy_parameter_constraintsTr7r8)rnrrOrrPrQrcv||_||_||_||_||_||_||_||_dSr;)rrrOrnrrPrQr) selfrrnrrOrrPrQrs r3__init__z _PLS.__init__sB),  "   r5prefer_skip_nested_validationc J t||t||tjd|jd}t |dtjd|jd}|jdkrd|_|dd}nd|_|j d }|j d}|j d}|j }|j d krt||nt|||}||krtd |d |d |j dk|_|j}t|||j\} } |_|_|_|_tj||f|_tj||f|_tj||f|_tj||f|_tj||f|_tj||f|_g|_tj| jj} tA|D]w} |j!dkrtj"tj#| d| zkd } d| dd| f< tI| | |j%|j&|j'|\}}}nD#tP$r7}tS|dkrtUj+d| Yd}~nd}~wwxYw|j,|n|j!dkrt[| | \}}t]||tj/| |}|rd}ntj/||}tj/| ||z }tj/|| tj/||z }| tj0||z} |j dkrCtj/|| tj/||z }| tj0||z} |j d krCtj/|| tj/||z }| tj0||z} ||jdd| f<||jdd| f<||jdd| f<||jdd| f<||jdd| f<||jdd| f<ytj/|jtctj/|jj2|jd|_3tj/|jtctj/|jj2|jd|_4tj/|j3|jj2|_5|j5|jzj2|jz |_5|j|_6|j3j d|_7|S)Fit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples,) or (n_samples, n_targets) Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : object Fitted model. Trr force_writeablerensure_min_samplesrNF input_namer rr ensure_2drDrr`n_components` upper bound is . Got instead. Reduce `n_components`.rr rgrjN)rOrPrQrRrAz$y residual is constant at iteration r)r)8rrr#float64rrndim _predict_1dreshaperJrrmin ValueError_norm_y_weightsrsrn_x_mean_y_mean_x_std_y_stdzeros x_weights_ y_weights_ _x_scores _y_scores x_loadings_ y_loadings_n_iter_r%r r&rHrallr=r^rOrPrQrGstrrKrLappendreryr*outerrrF x_rotations_ y_rotations_coef_ intercept__n_features_out)rrMrNnpqrrank_upper_boundrRXkyky_epskyk_maskrYr[rrTx_scoresy_ssy_scores x_loadings y_loadingss r3fitz_PLS.fits& 1%%%   *       *      6Q;;#D  "a  AA$D  GAJ GAJ GAJ( , < > >)At'7'9::4;&4<' =A#\BBBA i4+;+=>>O t{ *O t| +O"O3 3r5ct|t|||td}||jz}||jjz|jz}|jr|n|S)aLPredict targets of given samples. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples. copy : bool, default=True Whether to copy `X` or perform in-place normalization. Returns ------- y_pred : ndarray of shape (n_samples,) or (n_samples, n_targets) Returns predicted values. Notes ----- This call requires the estimation of a matrix of shape `(n_features, n_targets)`, which may be an issue in high dimensional space. Fr) rrrrrrFrrravel)rrMry_preds r3predictz _PLS.predictse,  $L N N N T\TZ\!DO3!%!1=v||~~~v=r5cV|||||S)aLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples, n_targets), default=None Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : ndarray of shape (n_samples, n_components) Return `x_scores` if `y` is not given, `(x_scores, y_scores)` otherwise. rrrrMrNs r3 fit_transformz_PLS.fit_transform&$xx1~~''1---r5cxt}d|j_d|j_|S)NTF)super__sklearn_tags__regressor_tags poor_score target_tagsrequired)rtags __class__s r3rz_PLS.__sklearn_tags__s3ww'')))-&$)! r5r)NTr;T)__name__ __module__ __qualname____doc__rrrrrdict__annotations__rrrrrrrrr __classcell__rs@r3r{r{s"(AtFCCCD%:|[&ABBCS#J''( j%!2334Xh4???@q$v6667  $ $D   #   ^*\555``65`D%%%%N****X>>>>:....(r5r{) metaclassceZdZUdZiejZeed<dD]Ze e d dddddfd Z fd Z xZ S) ra PLS regression. PLSRegression is also known as PLS2 or PLS1, depending on the number of targets. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, n_features]`. scale : bool, default=True Whether to scale `X` and `y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_target, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, y) PLSRegression() >>> y_pred = pls2.predict(X) For a comparison between PLS Regression and :class:`~sklearn.decomposition.PCA`, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_pcr_vs_pls.py`. rrrOrrTr7r8rnrPrQrc Zt||ddd|||dS)Nrr6rrrrrrrnrPrQrrs r3rzPLSRegression.__init__msH %'  r5c|t|||j|_|j|_|S)r)rrr x_scores_r y_scores_)rrMrNrs r3rzPLSRegression.fit{s4$  Aq r5r) rrrrr{rrrparampoprrrrs@r3rrseeN$Cd&A#BDBBB8**""5))))  '+cu4        r5rceZdZUdZiejZeed<dD]Ze e d dddddd fd Z xZ S) ra^ Partial Least Squares transformer and regressor. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, y) PLSCanonical() >>> X_c, y_c = plsca.transform(X, y) r)rrOrTrr7r8)rnrrPrQrc Zt||dd||||dS)Nrr6rr)rrrnrrPrQrrs r3rzPLSCanonical.__init__sH %&  r5r rrrrr{rrrrrrrrs@r3rrs``D$Cd&A#BDBBB+**""5))))              r5rceZdZUdZiejZeed<dD]Ze e d dddddfd Z xZ S) CCAa Canonical Correlation Analysis, also known as "Mode B" PLS. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, y) CCA(n_components=1) >>> X_c, y_c = cca.transform(X, y) rrrTr7r8rc Zt||ddd|||dS)NrrCrrrrs r3rz CCA.__init__xsH %&  r5rrrs@r3rrsXXt$Cd&A#BDBBB8**""5))))  '+cu4            r5rceZdZUdZeedddgdgdgdZeed<dd d d d Z e d dZ ddZ ddZ dS)raPartial Least Square SVD. This transformer simply performs an SVD on the cross-covariance matrix `X'y`. It is able to project both the training data `X` and the targets `y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, y) >>> X_c, y_c = pls.transform(X, y) >>> X_c.shape, y_c.shape ((4, 2), (4, 2)) rDNr|r}rrrnrrrT)rnrc0||_||_||_dSr;r)rrrnrs r3rzPLSSVD.__init__s(  r5rct||t||tjd|jd}t |dtjd|jd}|jdkr|dd}|j}t|j d |j d|j d}||krtd |d |d t|||j \}}|_|_|_|_tj|j|}t)|d \}}}|ddd|f}|d|}t+||\}}|j} ||_| |_|jj d|_|S)aJFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Returns ------- self : object Fitted estimator. TrrrNFrrDrrrrrr`N)rrr#rrrrrrrrJrrsrnrrrrr*rFrrrrr) rrMrNrrrarbr-rdVs r3rz PLSSVD.fits" 1%%%   *       *      6Q;; "a  A ( qwqz171:qwqzBB * * *F1AFF#FFF  FV q$*F F B1dlDL$+t{ F13NNq...1b aaa,    B2 D#4Q7 r5ct|t||tjd}||jz |jz }tj||j}|nt|ddtj}|j dkr| dd}||j z |j z }tj||j }||fS|S)a  Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `y is not None`, `(X_transformed, y_transformed)` otherwise. F)r rNrN)rrr rDr)rrr#rrrr*rrrrrrr)rrMrNXrryrrs r3rzPLSSVD.transforms&  $5 A A A$,$+ -6"do.. =A#bjQQQAv{{IIb!$$dl"dk1Bvb$/22HX% %r5cV|||||S)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `y is not None`, `(X_transformed, y_transformed)` otherwise. rrs r3rzPLSSVD.fit_transform7rr5rr;)rrrrrrrrrrrrrrr5r3rrsAAH"(AtFCCCD $$D 4 \555>>65>@@......r5r)r6r7r8Fr)-rrKabcrrnumbersrrnumpyr# scipy.linalgrr sklearn.baser r r r r rsklearn.exceptionsr sklearn.utilsrrsklearn.utils._param_validationrrsklearn.utils.extmathrsklearn.utils.validationrrr__all__r4r^rersryr{rrrrrr5r3rs''''''''""""""""""""""""211111>>>>>>>>@@@@@@@@******QQQQQQQQQQ 5 5 5<<<&=B8(8(8(8(v....4]]]]]# ]]]]@ VVVVVDVVVrB B B B B 4B B B Jk k k k k $k k k \B.B.B.B.B. ,.> B.B.B.B.B.r5