B (by@sdZddlZddlmZmZddlZddlmZm Z ddl m Z ddl m Z mZmZddlmZmZdd lmZdd lmZmZdd lmZd d dgZdddZddZd ddZGdddeee eeZGdd d eZ Gdd d eZ!Gddde eZ"dS)!zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). N)ABCMetaabstractmethod)pinv2svd)svds) BaseEstimatorRegressorMixinTransformerMixin) check_arraycheck_consistent_length)svd_flip)check_is_fitted FLOAT_DTYPES)six PLSCanonical PLSRegressionPLSSVDAư>FcCs|dddgf}d}d}d} } t|jj} xb|dkr^| dkrPt|dd} t| |} nt|j|t|j|} t| j| | kr| | 7} | tt| j| | } t|| } |dkr| dkrt|dd} t| | }nt|j| t| j| }|r$|tt|j|| }t||t|j|| }| |}t|j||ksn|jddkrpP||krt dP| }|d7}q4W| ||fS)a0Inner loop of the iterative NIPALS algorithm. Provides an alternative to the svd(X'Y); returns the first left and right singular vectors of X'Y. See PLS for the meaning of the parameters. It is similar to the Power method for determining the eigenvectors and eigenvalues of a X'Y. NrBF) check_finitez$Maximum number of iterations reached) npfinfodtypeepsrdotTsqrtshapewarningswarn)XYmodemax_itertolnorm_y_weightsZy_scoreZ x_weights_oldZiteZX_pinvZY_pinvr x_weightsZx_score y_weightsZx_weights_diffr,O/home/dcms/DCMS/lib/python3.7/site-packages/sklearn/cross_decomposition/pls_.py_nipals_twoblocks_inner_loops@    $   r.cCsNt|j|}t|dd\}}}|dddgf}|jdddgf}||fS)NF) full_matricesr)rrrr)r$r%CUsZVhuvr,r,r-_svd_cross_productTs r5TcCs|jdd}||8}|jdd}||8}|rr|jddd}d||dk<||}|jddd}d||dk<||}n t|jd}t|jd}||||||fS)z| Center X, Y and scale if the scale parameter==True Returns ------- X, Y, x_mean, y_mean, x_std, y_std r)axisr)r6Zddofg?g)ZmeanZstdronesr!)r$r%scaleZx_meanZy_meanZx_stdZy_stdr,r,r-_center_scale_xy\s     r9c @sDeZdZdZedd d Zd d ZdddZdddZdddZ dS)_PLSa Partial Least Squares (PLS) This class implements the generic PLS algorithm, constructors' parameters allow to obtain a specific implementation such as: - PLS2 regression, i.e., PLS 2 blocks, mode A, with asymmetric deflation and unnormalized y weights such as defined by [Tenenhaus 1998] p. 132. With univariate response it implements PLS1. - PLS canonical, i.e., PLS 2 blocks, mode A, with symmetric deflation and normalized y weights such as defined by [Tenenhaus 1998] (p. 132) and [Wegelin et al. 2000]. This parametrization implements the original Wold algorithm. We use the terminology defined by [Wegelin et al. 2000]. This implementation uses the PLS Wold 2 blocks algorithm based on two nested loops: (i) The outer loop iterate over components. (ii) The inner loop estimates the weights vectors. This can be done with two algo. (a) the inner loop of the original NIPALS algo. or (b) a SVD on residuals cross-covariance matrices. n_components : int, number of components to keep. (default 2). scale : boolean, scale data? (default True) deflation_mode : str, "canonical" or "regression". See notes. mode : "A" classical PLS and "B" CCA. See notes. norm_y_weights : boolean, normalize Y weights to one? (default False) algorithm : string, "nipals" or "svd" The algorithm used to estimate the weights. It will be called n_components times, i.e. once for each iteration of the outer loop. max_iter : an integer, the maximum number of iterations (default 500) of the NIPALS inner loop (used only if algorithm="nipals") tol : non-negative real, default 1e-06 The tolerance used in the iterative algorithm. copy : boolean, default True Whether the deflation should be done on a copy. Let the default value to True unless you don't care about side effects. Attributes ---------- x_weights_ : array, [p, n_components] X block weights vectors. y_weights_ : array, [q, n_components] Y block weights vectors. x_loadings_ : array, [p, n_components] X block loadings vectors. y_loadings_ : array, [q, n_components] Y block loadings vectors. x_scores_ : array, [n_samples, n_components] X scores. y_scores_ : array, [n_samples, n_components] Y scores. x_rotations_ : array, [p, n_components] X block to latents rotations. y_rotations_ : array, [q, n_components] Y block to latents rotations. coef_ : array, [p, q] The coefficients of the linear model: ``Y = X coef_ + Err`` n_iter_ : array-like Number of iterations of the NIPALS inner loop for each component. Not useful if the algorithm given is "svd". References ---------- Jacob A. Wegelin. A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case. Technical Report 371, Department of Statistics, University of Washington, Seattle, 2000. In French but still a reference: Tenenhaus, M. (1998). La regression PLS: theorie et pratique. Paris: Editions Technic. See also -------- PLSCanonical PLSRegression CCA PLS_SVD rT regressionrnipalsFư>c Cs:||_||_||_||_||_||_||_||_| |_dS)N) n_componentsdeflation_moder&r)r8 algorithmr'r(copy) selfr?r8r@r&rAr)r'r(rBr,r,r-__init__sz _PLS.__init__c CsRt||t|tj|jd}t|tj|jdd}|jdkrF|dd}|jd}|jd}|jd}|jdksx|j|krt d|j|j dkrt d |j |j d kr|j d krt d |j d krt dt |||j\}}|_|_|_|_|}|}t||jf|_t||jf|_t||jf|_t||jf|_t||jf|_t||jf|_g|_xDt|jD]4}tt|j|ttj j!krt"#d|P|j dkrt$|||j |j%|j&|j'd\} } } |j(| n|j d krt)||d\} } t*| | j\} } | j} t|| } |j'r8d} nt| j| } t|| | }t| j| ttj j!krt"#d|Pt|j| t| j| }|t| |j8}|j dkrt|j|t|j|}|t||j8}|j dkr&t|j| t| j| }|t| |j8}| +|jdd|f<|+|jdd|f<| +|jdd|f<| +|jdd|f<|+|jdd|f<|+|jdd|f<qvWt|jt,t|jj|jdd|_-|jddkrt|jt,t|jj|jdd|_.n t/d|_.ds,|j dkrNt|j-|jj|_0|j0|j|_0|S)aFit model to data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. Y : array-like, shape = [n_samples, n_targets] Target vectors, where n_samples is the number of samples and n_targets is the number of response variables. )rrBF)rrB ensure_2drrz Invalid number of components: %d)rr<z7Got algorithm %s when only 'svd' and 'nipals' are knownrrzHIncompatible configuration: mode B is not implemented with svd algorithm) canonicalr;zThe deflation mode is unknownz#Y residual constant at iteration %sr<)r$r%r&r'r(r))r$r%z!X scores are null at iteration %srGr;N)rT)1r r rfloat64rBndimreshaper!r? ValueErrorrAr&r@r9r8x_mean_y_mean_x_std_y_std_zeros x_scores_ y_scores_ x_weights_ y_weights_Z x_loadings_Z y_loadings_n_iter_rangeallrrrdoublerr"r#r.r'r(r)appendr5r Zravelr x_rotations_ y_rotations_r7coef_)rCr$r%npqZXkZYkkr*r+rUx_scoresZy_ssy_scoresZ x_loadingsZ y_loadingsr,r,r-fits          "$           z_PLS.fitNcCst|dt||td}||j8}||j}t||j}|dk rt|d|td}|jdkrh| dd}||j 8}||j }t||j }||fS|S)aApply the dimension reduction learned on the train data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. Y : array-like, shape = [n_samples, n_targets] Target vectors, where n_samples is the number of samples and n_targets is the number of response variables. copy : boolean, default True Whether to copy X and Y, or perform in-place normalization. Returns ------- x_scores if Y is not given, (x_scores, y_scores) otherwise. rL)rBrNF)rErBrrrF) rr rrLrNrrrZrIrJrMrOr[)rCr$r%rBrarbr,r,r- transformts       z_PLS.transformcCsDt|dt||td}||j8}||j}t||j}||jS)a Apply the dimension reduction learned on the train data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. copy : boolean, default True Whether to copy X and Y, or perform in-place normalization. Notes ----- This call requires the estimation of a p x q matrix, which may be an issue in high dimensional space. rL)rBr) rr rrLrNrrr\rM)rCr$rBZYpredr,r,r-predicts    z _PLS.predictcCs|||||S)aBLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. y : array-like, shape = [n_samples, n_targets] Target vectors, where n_samples is the number of samples and n_targets is the number of response variables. Returns ------- x_scores if Y is not given, (x_scores, y_scores) otherwise. )rcrd)rCr$yr,r,r- fit_transformsz_PLS.fit_transform) rTr;rr<Fr=r>T)NT)T)N) __name__ __module__ __qualname____doc__rrDrcrdrergr,r,r,r-r:vsb  & r:cs"eZdZdZdfdd ZZS) raPLS regression PLSRegression implements the PLS 2 blocks regression known as PLS2 or PLS1 in case of one dimensional response. This class inherits from _PLS with mode="A", deflation_mode="regression", norm_y_weights=False and algorithm="nipals". Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, (default 2) Number of components to keep. scale : boolean, (default True) whether to scale the data max_iter : an integer, (default 500) the maximum number of iterations of the NIPALS inner loop (used only if algorithm="nipals") tol : non-negative real Tolerance used in the iterative algorithm default 1e-06. copy : boolean, default True Whether the deflation should be done on a copy. Let the default value to True unless you don't care about side effect Attributes ---------- x_weights_ : array, [p, n_components] X block weights vectors. y_weights_ : array, [q, n_components] Y block weights vectors. x_loadings_ : array, [p, n_components] X block loadings vectors. y_loadings_ : array, [q, n_components] Y block loadings vectors. x_scores_ : array, [n_samples, n_components] X scores. y_scores_ : array, [n_samples, n_components] Y scores. x_rotations_ : array, [p, n_components] X block to latents rotations. y_rotations_ : array, [q, n_components] Y block to latents rotations. coef_ : array, [p, q] The coefficients of the linear model: ``Y = X coef_ + Err`` n_iter_ : array-like Number of iterations of the NIPALS inner loop for each component. Notes ----- Matrices:: T: x_scores_ U: y_scores_ W: x_weights_ C: y_weights_ P: x_loadings_ Q: y_loadings__ Are computed such that:: X = T P.T + Err and Y = U Q.T + Err T[:, k] = Xk W[:, k] for k in range(n_components) U[:, k] = Yk C[:, k] for k in range(n_components) x_rotations_ = W (P.T W)^(-1) y_rotations_ = C (Q.T C)^(-1) where Xk and Yk are residual matrices at iteration k. `Slides explaining PLS `_ For each component k, find weights u, v that optimizes: ``max corr(Xk u, Yk v) * std(Xk u) std(Yk u)``, such that ``|u| = 1`` Note that it maximizes both the correlations between the scores and the intra-block variances. The residual matrix of X (Xk+1) block is obtained by the deflation on the current X score: x_score. The residual matrix of Y (Yk+1) block is obtained by deflation on the current X score. This performs the PLS regression known as PLS2. This mode is prediction oriented. This implementation provides the same results that 3 PLS packages provided in the R language (R-project): - "mixOmics" with function pls(X, Y, mode = "regression") - "plspm " with function plsreg2(X, Y) - "pls" with function oscorespls.fit(X, Y) Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, Y) ... # doctest: +NORMALIZE_WHITESPACE PLSRegression(copy=True, max_iter=500, n_components=2, scale=True, tol=1e-06) >>> Y_pred = pls2.predict(X) References ---------- Jacob A. Wegelin. A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case. Technical Report 371, Department of Statistics, University of Washington, Seattle, 2000. In french but still a reference: Tenenhaus, M. (1998). La regression PLS: theorie et pratique. Paris: Editions Technic. rTư>c s$tt|j||ddd|||ddS)Nr;rF)r?r8r@r&r)r'r(rB)superrrD)rCr?r8r'r(rB) __class__r,r-rDJs  zPLSRegression.__init__)rTrlrmT)rhrirjrkrD __classcell__r,r,)ror-rscs"eZdZdZd fdd ZZS) ra PLSCanonical implements the 2 blocks canonical PLS of the original Wold algorithm [Tenenhaus 1998] p.204, referred as PLS-C2A in [Wegelin 2000]. This class inherits from PLS with mode="A" and deflation_mode="canonical", norm_y_weights=True and algorithm="nipals", but svd should provide similar results up to numerical errors. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, (default 2). Number of components to keep scale : boolean, (default True) Option to scale data algorithm : string, "nipals" or "svd" The algorithm used to estimate the weights. It will be called n_components times, i.e. once for each iteration of the outer loop. max_iter : an integer, (default 500) the maximum number of iterations of the NIPALS inner loop (used only if algorithm="nipals") tol : non-negative real, default 1e-06 the tolerance used in the iterative algorithm copy : boolean, default True Whether the deflation should be done on a copy. Let the default value to True unless you don't care about side effect Attributes ---------- x_weights_ : array, shape = [p, n_components] X block weights vectors. y_weights_ : array, shape = [q, n_components] Y block weights vectors. x_loadings_ : array, shape = [p, n_components] X block loadings vectors. y_loadings_ : array, shape = [q, n_components] Y block loadings vectors. x_scores_ : array, shape = [n_samples, n_components] X scores. y_scores_ : array, shape = [n_samples, n_components] Y scores. x_rotations_ : array, shape = [p, n_components] X block to latents rotations. y_rotations_ : array, shape = [q, n_components] Y block to latents rotations. n_iter_ : array-like Number of iterations of the NIPALS inner loop for each component. Not useful if the algorithm provided is "svd". Notes ----- Matrices:: T: x_scores_ U: y_scores_ W: x_weights_ C: y_weights_ P: x_loadings_ Q: y_loadings__ Are computed such that:: X = T P.T + Err and Y = U Q.T + Err T[:, k] = Xk W[:, k] for k in range(n_components) U[:, k] = Yk C[:, k] for k in range(n_components) x_rotations_ = W (P.T W)^(-1) y_rotations_ = C (Q.T C)^(-1) where Xk and Yk are residual matrices at iteration k. `Slides explaining PLS `_ For each component k, find weights u, v that optimize:: max corr(Xk u, Yk v) * std(Xk u) std(Yk u), such that ``|u| = |v| = 1`` Note that it maximizes both the correlations between the scores and the intra-block variances. The residual matrix of X (Xk+1) block is obtained by the deflation on the current X score: x_score. The residual matrix of Y (Yk+1) block is obtained by deflation on the current Y score. This performs a canonical symmetric version of the PLS regression. But slightly different than the CCA. This is mostly used for modeling. This implementation provides the same results that the "plspm" package provided in the R language (R-project), using the function plsca(X, Y). Results are equal or collinear with the function ``pls(..., mode = "canonical")`` of the "mixOmics" package. The difference relies in the fact that mixOmics implementation does not exactly implement the Wold algorithm since it does not normalize y_weights to one. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, Y) ... # doctest: +NORMALIZE_WHITESPACE PLSCanonical(algorithm='nipals', copy=True, max_iter=500, n_components=2, scale=True, tol=1e-06) >>> X_c, Y_c = plsca.transform(X, Y) References ---------- Jacob A. Wegelin. A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case. Technical Report 371, Department of Statistics, University of Washington, Seattle, 2000. Tenenhaus, M. (1998). La regression PLS: theorie et pratique. Paris: Editions Technic. See also -------- CCA PLSSVD rTr<ư>c s&tt|j||ddd||||d dS)NrGrT) r?r8r@r&r)rAr'r(rB)rnrrD)rCr?r8rAr'r(rB)ror,r-rDs  zPLSCanonical.__init__)rTr<rqrrT)rhrirjrkrDrpr,r,)ror-rSsc@s6eZdZdZd ddZddZdd d Zdd d ZdS)raGPartial Least Square SVD Simply perform a svd on the crosscovariance matrix: X'Y There are no iterative deflation here. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default 2 Number of components to keep. scale : boolean, default True Whether to scale X and Y. copy : boolean, default True Whether to copy X and Y, or perform in-place computations. Attributes ---------- x_weights_ : array, [p, n_components] X block weights vectors. y_weights_ : array, [q, n_components] Y block weights vectors. x_scores_ : array, [n_samples, n_components] X scores. y_scores_ : array, [n_samples, n_components] Y scores. See also -------- PLSCanonical CCA rTcCs||_||_||_dS)N)r?r8rB)rCr?r8rBr,r,r-rD szPLSSVD.__init__cCs,t||t|tj|jd}t|tj|jdd}|jdkrF|dd}|jt|j d|j dkrt d|jt |j t |j ft |||j \}}|_|_|_|_t|j|}|jt|j krt|dd\}}}nt||jd\}}}t||\}}|j}t|||_t|||_||_||_|S) aFit model to data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. Y : array-like, shape = [n_samples, n_targets] Target vectors, where n_samples is the number of samples and n_targets is the number of response variables. )rrBF)rrBrErrFzRInvalid number of components n_components=%d with X of shape %s and Y of shape %s.)r/)r`)r r rrHrBrIrJr?maxr!rKstrr9r8rLrMrNrOrrminrrr rQrRrSrT)rCr$r%r0r1r2Vr,r,r-rcs(   "z PLSSVD.fitNcCs~t|dt|tjd}||j|j}t||j}|dk rz|jdkrT| dd}||j |j }t||j }||fS|S)a Apply the dimension reduction learned on the train data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. Y : array-like, shape = [n_samples, n_targets] Target vectors, where n_samples is the number of samples and n_targets is the number of response variables. rL)rNrrF) rr rrHrLrNrrSrIrJrMrOrT)rCr$r%ZXrraZYrrbr,r,r-rdAs   zPLSSVD.transformcCs|||||S)aBLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vectors, where n_samples is the number of samples and n_features is the number of predictors. y : array-like, shape = [n_samples, n_targets] Target vectors, where n_samples is the number of samples and n_targets is the number of response variables. Returns ------- x_scores if Y is not given, (x_scores, y_scores) otherwise. )rcrd)rCr$rfr,r,r-rg[szPLSSVD.fit_transform)rTT)N)N)rhrirjrkrDrcrdrgr,r,r,r-rs % 0 )rrrF)T)#rkr"abcrrnumpyrZ scipy.linalgrrZscipy.sparse.linalgrbaserr r utilsr r Z utils.extmathr Zutils.validationrrZ externalsr__all__r.r5r9with_metaclassr:rrrr,r,r,r-s0     ; R