How to cite this WIREs title:
WIREs Comp Stat

Partial least squares algorithms and methods

Advanced Review

Vincenzo Esposito Vinzi, Giorgio Russolillo

Published Online: Dec 26 2012

DOI: 10.1002/wics.1239

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Partial least squares (PLS) refers to a set of iterative algorithms based on least squares that implement a broad spectrum of both explanatory and exploratory multivariate techniques, from regression to path modeling, and from principal component to multi‐block data analysis. This article focuses on PLS regression and PLS path modeling, which are PLS approaches to regularized regression and to predictive path modeling. The computational flows and the optimization criteria of these methods are reviewed in detail, as well as the tools for the assessment and interpretation of PLS models. The most recent developments and some of the most promising on going researches are enhanced. WIREs Comput Stat 2013, 5:1–19. doi: 10.1002/wics.1239 This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Statistical Models > Linear Models Algorithms and Computational Methods > Least Squares

PLS path model representation. The inner model is painted in blue gray, the outer model in sky blue.

[ Normal View | Magnified View ]

HBAT inner model.

[ Normal View | Magnified View ]

HBAT path model.

[ Normal View | Magnified View ]

An example of confirmatory path model with four reflective blocks.

[ Normal View | Magnified View ]

An example of hierarchical path model with three reflective blocks.

[ Normal View | Magnified View ]

References

Wold, H. Non linear estimation by iterative least squares procedure, Research paper in Statistics: Festschift for J. Neyman. F. David, 1966.
Wold, H. Estimation of principal component and related models by iterative least squares. In: Krishnaiah, PR, ed. Multivariate Analysis. New York: Academic Press; 1966, 391–420.
Wold, H. Path models with latent variables: the non‐linear iterative partial least squares (NIPALS) approach. In: Blalock, HM, Aganbegian, A, Borodkin, FM, Boudon, Raymond, Capecchi, Vittorio, eds. Quantitative Sociology: Intentional Perspective on Mathematical and Statistical Modeling. New York: Academic Press; 1975, 307–357.
Wold, H. Causal flows with latent variables. Partings of the ways in the light of NIPALS modelling. Eur Econ Rev 1984, 5:67–86.
Wold, H. Soft modeling by latent variables: the nonlinear iterative partial least squares approach. In: Gani, J, ed. Perspectives in Probability and Statistics, Papers in Honor of M.S. Bartlett. London: Academic Press; 1975, 117–142.
Wold, H. Soft modeling: the basic design and some extensions. In: Jöreskog, KG, Wold, H, eds. Systems under Indirect Observation, Part II. Amsterdam: North‐Holland; 1982, 1–54.
Tenenhaus, M, Esposito Vinzi, V, Chatelin, Y‐M, Lauro, CN. PLS path modeling. Comput Stat Data Anal 2005, 48:159–205.
Jöreskog, KJ. A general method for analysis of covariance structure. Biometrika 1970, 57:239–251.
Wold, S, Martens, H, Wold, H. The multivariate calibration problem in chemistry solved by the PLS method. In: Ruhe, A, Kagstrom, B, eds. Proceedings of the Conference on Matrix Pencils. Lectures Notes in Mathematics. Heidelberg: Springer; 1983.
Wold, S, Ruhe, A, Wold, H, Dunn, W. The collinearity problem in linear regression. the PLS approach to generalised inverses. J Sci Stat Comput, SIAM 1984, 5:735–743.
Wold, S, Sjöström, M, Eriksson, L. PLS regression: a basic tool of chemometrics. Chemometr Intell Lab Syst 2001, 58:109–130.
Esposito Vinzi, V, Trinchera, L, Amato, S. PLS path modeling: Recent developments and open issues for model assessment and improvement. In: Esposito Vinzi, V, Chin, W, Henseler, J, Wang, H, eds. Handbook of Partial Least Squares (PLS): Concepts, Methods and Applications. Berlin, Heidelberg, New York: Springer; 2010.
Sánchez, G. PATHMOX approach: segmentation trees in partial least squares path modeling, PhD Thesis, Universitat Politècnica de Catalunya, Barcelona, Spain, 2009.
Tenenhaus, M. La Régression PLS: Théorie et Pratique. Paris: Technip; 1998.
Abdi, H. Partial least squares regression, projection on latent structures, PLS‐regression. WIRES: Comput Stat 2010, 2:97–106.
De Jong, S. PLS shrinks. J Chemometr 1995, 9:323–326.
Höskuldsson, A. PLS regression methods. J Chemometr 1988, 2:211–228.
Frazer, RA, Duncan, WJ, Collar, AR. Elementary Matrices and Some Applications to Dynamics and Differential Equations. Cambridge: Cambridge University Press; 1938, 1963 printing.
Frank, IE, Friedman, JH. A statistical view of some chemometrics regression tools. Technometrics 1993, 35:109–135.
Garthwaite, PH. An interpretation of partial least squares. J Am Stat Assoc 1994, 89:122–127.
Hoerl, AE, Kennard, RW. Ridge regression: biased estimation of non‐orthogonal components. Technometrics 1970, 12:55–67.
Jolliffe, IT. A note on the use of principal components in regression. J R Stat Soc [Ser C](Appl Stat) 1982, 31:300–303.
Ball, RJ. The significance of simultaneous methods of parameter estimation in econometric models. Appl Stat 1963, 12:14–25.
Wold, S. PLS for multivariate linear modeling. In: van de Waterbeemd, H, ed. QSAR: Chemometric Methods in Molecular Design. Methods and Principles in Medicinal Chemistry. Weinheim, Germany: Verlag‐Chemie; 2009.
Wold, H. Partial least squares. In: Kotzand, S, Johnson, NL, eds. Encyclopedia of Statistical Sciences, vol. 6, New York: Wiley; 1985, 581–591.
Sampson, PD, Streissguth, AP, Barr, HM, Bookstein, FL. Neurobehavioral effects of prenatal alcohol: Part ii. partial least squares analysis. Neurotoxicol Teratol 1989, 11:477–491.
Rosipal, R, Krämer, N. Overview and recent advances in partial least squares. In: Saunders, C, Grobelnik, M, Gunn, S, Shawe‐Taylor, J, eds. Subspace, Latent Structure and Feature Selection Techniques. Springer; 2006, 34–51.
Wegelin, JA. A survey on partial least squares (PLS) methods, with emphasis on the two‐block case, Technical Report No 371, Department of Statistics, University of Washington, USA: Seattle, 2000.
Tishler, A, Dvir, D, Shenhar, A, Lipovetsky, S. Identifying critical success factors in defense development projects: a multivariate analysis. Technol Forecast Soc Change 1996, 51:151–171.
Tucker, LR. An inter‐battery method of factor analysis. Psychometrika 1958, 23:111–136.
Fornell, C, Bookstein, FL. Two structural equation models: LISREL and PLS applied to consumer exit‐voice theory. J Mark Res 1982, XIX:440–452.
Addinsoft. XLSTAT 2012. 2012. Paris. Available at: http:www.xlstat.com/en/products/xlstat‐plspm/. (Accessed November 29, 2012).
Lyttkens, E, Areskoug, B, Wold, H. The convergence of NIPALS estimation procedures for six path models with one or two latent variables, Technical Report, University of Göteborg, 1975.
Krämer, N. Analysis of high‐dimensional data with partial least squares and boosting, PhD Thesis, Technische Universität Berlin, Berlin, Germany, 2007.
Henseler, J. On the convergence of the partial least squares path modeling algorithm. Comput Stat 2010, 25:107–120.
Lohmöller, JB. LVPLS Program Manual, version 1.8. Technical Report, Zentralarchiv für Empirische Sozialforschung, Universität Zu Köln, Köln, 1987.
Lohmöller, JB. Latent Variable Path Modeling with Partial Least Squares. Heildelberg: Physica‐Verlag; 1989.
Hanafi, M. PLS path modeling: computation of latent variables with the estimation mode B. Comput Stat 2007, 22:275–292.
Fornell, CC, Larcker, DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 1981, 48:39–50.
Esposito Vinzi, V, Tenenhaus, M, Amato, S. A global goodness‐of‐fit index for PLS structural equation modelling. Proceedings of the XLII SIS Scientific Meeting Vol. Contributed Papers, Padova: CLEUP; 2004, 739–742.
Stone, M. Cross‐validatory choice and assessment of statistical predictions. J R Stat Soc 1974, 36:111–147.
Geisser, S. A predictive approach to the random effect model. Biometrika 1974, 61:101–107.
Efron, B. The Jackknife, the Bootstrap and Other Resampling Plans. Philadelphia, PA: SIAM; 1982.
van Belle, G. Statistical Rules of Thumb. New York: John Wiley %26 Sons; 2002.
Hotelling, H. Relations between two sets of variates. Biometrika 1936, 28:321–377.
Rao, CR. The use and interpretation of principal component analysis in applied research. Sankhya 1964, A 26:329–359.
Van de Wollenberg, AL. Redundancy analysis. an alternative for canonical correlation analysis. Psychometrika 1977, 42:207–219.
Horst, P. Relations among m sets of measures. Psychometrika 1961, 26:129–149.
Carroll, JD. Generalization of canonical analysis to three or more sets of variables, Proceedings of the 76th Convention of the American Psychological Association, vol. 3, 1968; 227–228.
Escofier, B, Pagés, J. Multiple factor analysis (AFMULT package). Comput Stat Data Anal 1994, 18:121–140.
Kettenring, JR. Canonical analysis of several sets of variables. Biometrika 1971, 58:433–451.
Tenenhaus, M, Hanafi, M. A bridge between PLS path modeling and multi‐block data analysis. In: Esposito Vinzi, V, Chin, W, Henseler, J, Wang, H, eds. Handbook of Partial Least Squares (PLS): Concepts, Methods and Applications. Heidelberg, Germany: Springer Verlag; 2010.
Glang, M. Maximierung der Summe erklärter Varianzen in linearrekursiven Strukturgleichungsmodellen mit multiple Indikatoren: Eine Alternative zum Schätzmodus B des Partial‐Least‐Squares‐Verfahren, PhD Thesis, Universität Hamburg, Hamburg, Germany, 1988.
Mathes, H. Global optimisation criteria of the pls‐algorithm in recursive path models with latent variables. In: Haagen, K, Bartholomew, DJ, Deister, M, eds. Statistical Modelling and Latent Variables. Amsterdam: Elsevier Science; 1993.
Tenenhaus, A, Tenenhaus, M. Regularized generalized canonical correlation analysis. Psychometrika 2011, 76:257–284.
Hair, JF, Black, WC, Babin, BJ, Anderson, RE. Multivariate Data Analysis. A Global Perspective. Upper Saddle River, New Jersey: Pearson Education Inc.; 2010.
Lê Cao, KA, Rossouw, D, Robert‐Granié, C, Besse, P. Sparse pls: Variable selection when integrating omics data. Stat Appl Mol Biol 2008, 7.
Chun, H, Keleş, S. Sparse partial least squares regression for simultaneous dimension reduction and variable selection. J R Stat Soc [Ser B] (Stat Methodol) 2010, 72:3–25.
Krämer, N, Sugiyama, M. The degrees of freedom of partial least squares regression. J Am Stat Assoc 2011, 106:697–705.
Russolillo, G. Non‐metric partial least squares. Electron J Statistics 2012, 6:1641–1669.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts