hUhdZddlZddlmZmZddlZddlmZddl m Z m Z m Z ddl mZddlmZmZmZdd lmZdd lmZmZmZd d gZd ZdZdZdZddZdZdZ dddddddddddddd dZ!Gdd e e e Z"dS) z Python implementation of the fast ICA algorithms. Reference: Tables 8.3 and 8.4 page 196 in the book: Independent Component Analysis, by Hyvarinen et al. N)IntegralReal)linalg) BaseEstimatorTransformerMixinClassNamePrefixFeaturesOutMixin)ConvergenceWarning) check_arrayas_float_arraycheck_random_state)check_is_fitted)HiddenInterval StrOptionsfasticaFastICAcz|tj||d|j|d|gz}|S)a Orthonormalize w wrt the first j rows of W. Parameters ---------- w : ndarray of shape (n,) Array to be orthogonalized W : ndarray of shape (p, n) Null space definition j : int < p The no of (from the first) rows of Null space W wrt which w is orthogonalized. Notes ----- Assumes that W is orthogonal w changed in place N)npr multi_dotT)wWjs Z/var/www/html/Sam_Eipo/venv/lib/python3.11/site-packages/sklearn/decomposition/_fastica.py_gs_decorrelationrs<*  a2A2!BQB%0 1 11A HcDtjtj||j\}}tj|tj|jjd}tj |dtj |z z|j|gS)z@Symmetric decorrelation i.e. W <- (W * W.T) ^{-1/2} * W N)a_mina_max?) reighrdotrclipfinfodtypetinyrsqrt)rsus r_sym_decorrelationr+4s} ;rva~~ & &DAq !'**/t<<rc  t|}~t|jd}t|D]}|t j|||\} } tt j| |j|z | ddtjf|zz } ~ ~ tttt j d| |dz } | }| |krntj dt||dzfS)zCParallel FastICA. Used internally by FastICA --main loop r.Nzij,ij->iz\FastICA did not converge. Consider increasing tolerance or the maximum number of iterations.)r+floatr1r3rr#rnewaxisr9r7einsumwarningswarnr ) r:r;r<r=r>r?rp_iirCrDW1rFs r_ica_parrQfs 6""A qwqz  BHoo  aq! h// e tQS 1 1B 6qqq"*}9MPQ9Q Q R R %#c")JA6677!;<<==  99 E    =    b1f9rc |dd}||z}tj||}tj|jd|j}t |D]%\}}|d|dzz z||<&||fS)Nalphar!rr-r.r)getrtanhemptyr1r& enumerater6)xr=rSgxg_xrBgx_is r_logcoshr\s LL# & &EJA AB (171:QW - - -CR==0041tQw;'--//A s7Nrctj|dz dz }||z}d|dzz |z}||dfS)Nrr.r/)rexpr6)rXr=r_rYrZs r_expr`sO &1a41  C SB q!t8s C sxxRx   rcD|dzd|dzzdfS)Nrr^r/)r6)rXr=s r_cubercs' a4!ad(b)) ))rparallelrMlogcosh-C6?svdFT) algorithmwhitenfunr=r>r;r? whiten_solver random_state return_X_meancompute_sources return_n_iterc t||||||||| |  }||| }|jdvr|j}|j}nd}d}||j|g}| r||| r||j|S)aWPerform Fast Independent Component Analysis. The implementation is based on [1]_. Read more in the :ref:`User Guide `. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vector, where `n_samples` is the number of samples and `n_features` is the number of features. n_components : int, default=None Number of components to use. If None is passed, all are used. algorithm : {'parallel', 'deflation'}, default='parallel' Specify which algorithm to use for FastICA. whiten : str or bool, default="warn" Specify the whitening strategy to use. - If 'arbitrary-variance' (default), a whitening with variance arbitrary is used. - If 'unit-variance', the whitening matrix is rescaled to ensure that each recovered source has unit variance. - If False, the data is already considered to be whitened, and no whitening is performed. .. deprecated:: 1.1 Starting in v1.3, `whiten='unit-variance'` will be used by default. `whiten=True` is deprecated from 1.1 and will raise ValueError in 1.3. Use `whiten=arbitrary-variance` instead. fun : {'logcosh', 'exp', 'cube'} or callable, default='logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example:: def my_g(x): return x ** 3, (3 * x ** 2).mean(axis=-1) fun_args : dict, default=None Arguments to send to the functional form. If empty or None and if fun='logcosh', fun_args will take value {'alpha' : 1.0}. max_iter : int, default=200 Maximum number of iterations to perform. tol : float, default=1e-4 A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged. w_init : ndarray of shape (n_components, n_components), default=None Initial un-mixing array. If `w_init=None`, then an array of values drawn from a normal distribution is used. whiten_solver : {"eigh", "svd"}, default="svd" The solver to use for whitening. - "svd" is more stable numerically if the problem is degenerate, and often faster when `n_samples <= n_features`. - "eigh" is generally more memory efficient when `n_samples >= n_features`, and can be faster when `n_samples >= 50 * n_features`. .. versionadded:: 1.2 random_state : int, RandomState instance or None, default=None Used to initialize ``w_init`` when not specified, with a normal distribution. Pass an int, for reproducible results across multiple function calls. See :term:`Glossary `. return_X_mean : bool, default=False If True, X_mean is returned too. compute_sources : bool, default=True If False, sources are not computed, but only the rotation matrix. This can save memory when working with big data. Defaults to True. return_n_iter : bool, default=False Whether or not to return the number of iterations. Returns ------- K : ndarray of shape (n_components, n_features) or None If whiten is 'True', K is the pre-whitening matrix that projects data onto the first n_components principal components. If whiten is 'False', K is 'None'. W : ndarray of shape (n_components, n_components) The square matrix that unmixes the data after whitening. The mixing matrix is the pseudo-inverse of matrix ``W K`` if K is not None, else it is the inverse of W. S : ndarray of shape (n_samples, n_components) or None Estimated source matrix. X_mean : ndarray of shape (n_features,) The mean over features. Returned only if return_X_mean is True. n_iter : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. This is returned only when return_n_iter is set to `True`. Notes ----- The data matrix X is considered to be a linear combination of non-Gaussian (independent) components i.e. X = AS where columns of S contain the independent components and A is a linear mixing matrix. In short ICA attempts to `un-mix' the data by estimating an un-mixing matrix W where ``S = W K X.`` While FastICA was proposed to estimate as many sources as features, it is possible to estimate less by setting n_components < n_features. It this case K is not a square matrix and the estimated A is the pseudo-inverse of ``W K``. This implementation was originally made for data of shape [n_features, n_samples]. Now the input is transposed before the algorithm is applied. This makes it slightly faster for Fortran-ordered input. References ---------- .. [1] A. Hyvarinen and E. Oja, "Fast Independent Component Analysis", Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430. r@rirjrkr=r>r;r?rlrmro) unit-variancearbitrary-varianceN)r_fit_transform_whiten whitening_mean_ _unmixingr8n_iter_)r:r@rirjrkr=r>r;r?rlrmrnrorpestSKX_meanreturned_valuess rrrsr !  #!   C 1o>>A {=== N #-+O'v&&&,s{+++ rc eZdZUdZeeddddgeddhgeedhed d hd gehd ege dgeedddgee d ddgddgeddhgdgd Z e e d< d"dddddddddd fd Z d#dZd"dZd"dZd$dZd$dZed Zd!ZxZS)%ra}FastICA: a fast algorithm for Independent Component Analysis. The implementation is based on [1]_. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=None Number of components to use. If None is passed, all are used. algorithm : {'parallel', 'deflation'}, default='parallel' Specify which algorithm to use for FastICA. whiten : str or bool, default="warn" Specify the whitening strategy to use. - If 'arbitrary-variance' (default), a whitening with variance arbitrary is used. - If 'unit-variance', the whitening matrix is rescaled to ensure that each recovered source has unit variance. - If False, the data is already considered to be whitened, and no whitening is performed. .. deprecated:: 1.1 Starting in v1.3, `whiten='unit-variance'` will be used by default. `whiten=True` is deprecated from 1.1 and will raise ValueError in 1.3. Use `whiten=arbitrary-variance` instead. fun : {'logcosh', 'exp', 'cube'} or callable, default='logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example:: def my_g(x): return x ** 3, (3 * x ** 2).mean(axis=-1) fun_args : dict, default=None Arguments to send to the functional form. If empty or None and if fun='logcosh', fun_args will take value {'alpha' : 1.0}. max_iter : int, default=200 Maximum number of iterations during fit. tol : float, default=1e-4 A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged. w_init : array-like of shape (n_components, n_components), default=None Initial un-mixing array. If `w_init=None`, then an array of values drawn from a normal distribution is used. whiten_solver : {"eigh", "svd"}, default="svd" The solver to use for whitening. - "svd" is more stable numerically if the problem is degenerate, and often faster when `n_samples <= n_features`. - "eigh" is generally more memory efficient when `n_samples >= n_features`, and can be faster when `n_samples >= 50 * n_features`. .. versionadded:: 1.2 random_state : int, RandomState instance or None, default=None Used to initialize ``w_init`` when not specified, with a normal distribution. Pass an int, for reproducible results across multiple function calls. See :term:`Glossary `. Attributes ---------- components_ : ndarray of shape (n_components, n_features) The linear operator to apply to the data to get the independent sources. This is equal to the unmixing matrix when ``whiten`` is False, and equal to ``np.dot(unmixing_matrix, self.whitening_)`` when ``whiten`` is True. mixing_ : ndarray of shape (n_features, n_components) The pseudo-inverse of ``components_``. It is the linear operator that maps independent sources to the data. mean_ : ndarray of shape(n_features,) The mean over features. Only set if `self.whiten` is True. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_iter_ : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. whitening_ : ndarray of shape (n_components, n_features) Only set if whiten is 'True'. This is the pre-whitening matrix that projects data onto the first `n_components` principal components. See Also -------- PCA : Principal component analysis (PCA). IncrementalPCA : Incremental principal components analysis (IPCA). KernelPCA : Kernel Principal component analysis (KPCA). MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. SparsePCA : Sparse Principal Components Analysis (SparsePCA). References ---------- .. [1] A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.decomposition import FastICA >>> X, _ = load_digits(return_X_y=True) >>> transformer = FastICA(n_components=7, ... random_state=0, ... whiten='unit-variance') >>> X_transformed = transformer.fit_transform(X) >>> X_transformed.shape (1797, 7) r.Nleft)closedrd deflationrMrurtboolean>r_cuberegz array-liker"rhrmrr_parameter_constraintsrerfrg) rirjrkr=r>r;r?rlrmc t||_||_||_||_||_||_||_||_ | |_ | |_ dSN) super__init__r@rirjrkr=r>r;r?rlrm) selfr@rirjrkr=r>r;r?rlrm __class__s rrzFastICA.__init__sj ("      *(rFcd j_jdkr!tjdtd_jdur#tjdtdd_|jt jt jgdj }j inj }tj }| d d }d |cxkrdksntd jdkrt }n?jdkrt"}n,jdkrt$}nt'jrfd}|j\}} j} js| d } tjd| t-| |} | t-| |kr't-| |} tjd| zjr|d} || d d t jfz}jdkrt5j||\} } t j| d d d}t j| jj }| |k}t j!|rtjd|| |<t j"| | | || d d |f} } n-jdkr"t5j#|ddd d\} } | t j$| dz} | | z j d | }~ ~ t j||}|t j"| z}ntK|d}j&}|2t j'|(| | f|j}n7t j'|}|j| | fkrtd d!| | fizj)||j*|d"}j+d#krtY|fi|\}}nj+d$krt[|fi|\}}~|_.|rJjr(t j/|||gj }nt j||j }nd }jrjd%krO|s't j/|||gj }t j0|dd&}||z}||j z}t j||_1| _2|_3n|_1t5j4j1d'_5|_6|S)(adFit the model. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data, where `n_samples` is the number of samples and `n_features` is the number of features. compute_sources : bool, default=False If False, sources are not computes but only the rotation matrix. This can save memory when working with big data. Defaults to False. Returns ------- S : ndarray of shape (n_samples, n_components) or None Sources matrix. `None` if `compute_sources` is `False`. rMzAStarting in v1.3, whiten='unit-variance' will be used by default.ruTzStarting in v1.3, whiten=True should be specified as whiten='arbitrary-variance' (its current behaviour). This behavior is deprecated in 1.1 and will raise ValueError in 1.3.r) stacklevel)r4r&ensure_min_samplesNrSr!r.zalpha must be in [1,2]rer_rc j|fi|Sr)rk)rXr=rs rr<z!FastICA._fit_transform..g@stx..X...rz(Ignoring n_components with whiten=False.z/n_components is too large: it will be set to %sr^r/r"zfThere are some small singular values, using whiten_solver = 'svd' might lead to more accurate results.)outrhF) full_matrices check_finiter)r4)sizer-z/w_init has invalid shape -- should be %(shape)sr1)r;r<r=r>r?rdrrt)r0keepdims)r)7rjrwrLrM FutureWarning_validate_datarfloat64float32rr=r rmrT ValueErrorrkr\r`rccallabler1r@minr6rJrlrr"r#argsortr%r&epsanyr(rhsignr r?asarraynormalr;r>rirQrGr{rstd components_ryrxpinvmixing_rz)rr:roXTr=rmrSr< n_features n_samplesr@rdr* sort_indicesrdegenerate_idxr~X1r?kwargsrrAr}S_stds` rrvzFastICA._fit_transform s${ <6 ! ! MS   0DL <4   MR     0DL  DLRZ(@UV!   .22DM)$*;<<  Wc**EQ566 6 8y AA X  AA X  AA dh   / / / / / /!# I( | F 8L MD E E E  y*55L #i44 4 4y*55L MALP    <$ 0WW"W%%F &BJ' 'B!V++{266!99--1!z!}}TTrT2 hqw''+!"S6.))M, %(.!q!!!!!!!\/(:1#u,,z"ENNNrPQrR1 1 AQ -<-(A12B "')$$ $BB ///B >Z##, )E#FFbhFF Z''F| l;;; E| <=> 8      >Z ' ' ..v..IAvv ^{ * * ..v..IAv   | $I''Ar 335F1bMMOA < !|..&: ++Q2J779Aqq4888U UW !va||D DJDOO D {4#3%HHH rcX|||dS)a5Fit the model and recover the sources from X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- X_new : ndarray of shape (n_samples, n_components) Estimated sources obtained by transforming the data with the estimated unmixing matrix. Trs_validate_paramsrvrr:ys r fit_transformzFastICA.fit_transforms/$ ""1d";;;rc\|||d|S)aFit the model to X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- self : object Returns the instance itself. Frsrrs rfitz FastICA.fits4"  Au555 rTct||||o|jtjtjgd}|jr ||jz}tj||jj S)a_Recover the sources from X (apply the unmixing matrix). Parameters ---------- X : array-like of shape (n_samples, n_features) Data to transform, where `n_samples` is the number of samples and `n_features` is the number of features. copy : bool, default=True If False, data passed to fit can be overwritten. Defaults to True. Returns ------- X_new : ndarray of shape (n_samples, n_components) Estimated sources obtained by transforming the data with the estimated unmixing matrix. F)r4r&reset) rrrwrrrryr#rrrr:r4s r transformzFastICA.transformsu$     T*dlBJ 3KSX    <  OAva)+,,,rct|t||o|jtjtjg}tj||jj}|jr ||j z }|S)a1Transform the sources back to the mixed data (apply mixing matrix). Parameters ---------- X : array-like of shape (n_samples, n_components) Sources, where `n_samples` is the number of samples and `n_components` is the number of components. copy : bool, default=True If False, data passed to fit are overwritten. Defaults to True. Returns ------- X_new : ndarray of shape (n_samples, n_features) Reconstructed data obtained with the mixing matrix. )r4r&) rr rwrrrr#rrryrs rinverse_transformzFastICA.inverse_transformse  !6$, BJ?W X X X F1dln % % <  OArc&|jjdS)z&Number of transformed output features.r)rr1rs r_n_features_outzFastICA._n_features_outs%a((rc6dtjtjgiS)Npreserves_dtype)rrrrs r _more_tagszFastICA._more_tagss!BJ #;< ? ?   55566A4LXh4???@sD8889&$*fe_556'($$D&)  )))))))4eeeeN<<<<,,----82))X)=======rr)#rrLnumbersrrnumpyrscipyrbaserrr exceptionsr utilsr r r utils.validationrutils._param_validationrrr__all__rr+rGrQr\r`rcrrrrrs""""""""SSSSSSSSSS++++++CCCCCCCCCC......BBBBBBBBBB i    2 A A A   F@    !!!*** t    tttttnC=C=C=C=C=-/?C=C=C=C=C=r