How to cite this WIREs title:
WIREs Comp Stat

Factor analysis: Latent variable, matrix decomposition, and constrained uniqueness formulations

Advanced Review

Kohei Adachi

Published Online: Mar 19 2019

DOI: 10.1002/wics.1458

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Factor analysis (FA) is a time‐honored multivariate analysis procedure for exploring the factors that underlie observed multiple variables to explain their variations. According to how the factors are treated, the formulations of FA can be classified into three approaches. One of them can be called a latent variable (LV) formulation, while the remaining two can be named matrix decomposition (MD) ones. The factors are regarded as random latent variables in the LV formulation, but treated as fixed parameter matrices in the MD ones. Here, the latter MD formulations are subdivided into the one simply called MD and the constrained uniqueness (CU) formulation with a special constraint. As compared to the LV formulation which is classic, MD and CU ones are recently established. Thus, it is useful to review the formulations now. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Multivariate Analysis

Graphical representation of FA with p = 5 and m = 2

[ Normal View | Magnified View ]

MRFA algorithm

[ Normal View | Magnified View ]

CUFA algorithm for obtaining Λ

[ Normal View | Magnified View ]

Adachi and Trendafilov's cone of common‐unique factor scores (Adachi & Trendafilov, )

[ Normal View | Magnified View ]

MDFA algorithm for obtaining B

[ Normal View | Magnified View ]

MLEM algorithm

[ Normal View | Magnified View ]

MLGR algorithm

[ Normal View | Magnified View ]

LSCF algorithm

[ Normal View | Magnified View ]

References

Adachi,, K. (2012). Some contributions to data‐fitting factor analysis with empirical comparisons to covariance‐fitting factor analysis. Journal of the Japanese Society of Computational Statistics, 25, 25–38.
Adachi,, K. (2013). Factor analysis with EM algorithm never gives improper solutions when sample covariance and initial parameter matrices are proper. Psychometrika, 78, 380–394.
Adachi,, K. (2015a). A new algorithm for generalized least squares factor analysis with a majorization technique. Open Journal of Statistics, 5, 165–172.
Adachi,, K. (2015b). A matrix‐intensive approach to factor analysis. Journal of the Japan Statistical Society, Japanese Issue, 44, 363–382 (in Japanese).
Adachi,, K. (2016). Matrix‐based introduction to multivariate data analysis. Singapore: Springer.
Adachi,, K., & Trendafilov,, N. T. (2015). Sparse orthogonal factor analysis. In E. Carpita,, E. Brentari,, & E. M. Qannari, (Eds.), Advances in latent variable: Methods, models, and applications (pp. 227–239). Cham, Switzerland: Springer.
Adachi,, K., & Trendafilov,, N. T. (2018). Some mathematical properties of the matrix decomposition solution in factor analysis. Psychometrika, 83, 407–424.
Akaike,, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723.
Anderson,, T. W. (2003). An introduction to multivariate statistical analysis (3rd ed.). New York, NY: Wiley.
Anderson,, T. W., & Rubin,, H. (1956). Statistical inference in factor analysis. In J. Neyman, (Ed.), Proceedings of the third Berkeley symposium on mathematical statistics and probability (Vol. 5, pp. 111–150). Berkeley: University of California Press.
Bartholomew,, D., Knott,, M., & Moustaki,, I. (2011). Latent variable models and factor analysis: A unified approach (3rd ed.). Chichester, England: Wiley.
Bertsimas,, D., Copenhaver,, M. S., & Mazumder,, R. (2017). Certifiably optimal low rank factor analysis. Journal of Machine Learning Research, 18, 1–53.
Boyd,, S., Parikh,, N., Chu,, B., Peleato,, B., & Eckstein,, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3, 1042–1068.
Browne,, M. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150.
de Leeuw,, J. (2004). Least squares optimal scaling of partially observed linear systems. In b. K. van Montfort,, J. Oud,, & A. Satorra, (Eds.), Recent developments of structural equation models: Theory and applications (pp. 121–134). Dordrecht, the Netherlands: Kluwer Academic.
Dempster,, A. P., Laird,, N. M., & Rubin,, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38.
Eckart,, C., & Young,, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1, 211–218.
Guttman,, L. (1955). The determinacy of factor score matrices with implications for five other basic problems of common factor theory. British Journal of Statistical Psychology, 8, 65–81.
Harman,, H. H. (1976). Modern factor analysis (3rd ed.). Chicago, IL: The University of Chicago Press.
Harman,, H. H., & Jones,, W. H. (1966). Factor analysis by minimizing residuals (Minres). Psychomerika, 31, 351–369.
Hirose,, K., & Yamamoto,, M. (2014). Estimation of an oblique structure via penalized likelihood factor analysis. Computational Statistics and Data Analysis, 79, 120–132.
Izenman,, A. J. (2008). Modern multivariate statistical techniques: Regression, classification, and manifold learning. New York, NY: Springer.
Jennrich,, R. I., & Robinson,, S. M. (1969). A Newton‐Raphson algorithm for maximum likelihood factor analysis. Psychometrika, 34, 111–123.
Jöreskog,, K. G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, 443–482.
Jöreskog,, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.
Kaiser,, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187–200.
Kano,, Y. (1998). Causes and treatment of improper solutions: Exploratory factor analysis. Bulletin of the Department of Human Sciences, Osaka University, 24, 303–327 (in Japanese).
Koch,, I. (2014). Analysis of multivariate and high‐dimensional data. New York, NY: Cambridge University Press.
Kodama,, S., Shinagawa,, F., & Mogi,, M. (1978). Nihongo‐ban WISC‐R chino‐kensa‐ho (Japanese version of WISC‐R intelligence test). Tokyo, Japan: Nihon Bunka Kagaku Sha (in Japanese).
Lawley,, D. N. (1941). Further investigation of factor estimation. Proceedings of the Royal Society of Edinburgh, A61, 176–185.
Lee,, S. Y. (1978). The gauss‐Newton algorithm for the weighted least squares factor analysis. Journal of the Royal Statistical Society: Series D (The Statistician), 27, 103–114.
Mulaik,, S. A. (2010). Foundations of factor analysis (2nd ed.). Boca Raton, FL: CRC Press.
R Development Core Team. (2008). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, ISBN: 3–900051–07‐0. Retrieved from http://www.R-project.org
Rao,, C. R. (1955). Estimation and tests of significance in factor analysis. Psychometrika, 20, 93–111.
Rubin,, D. B., & Thayer,, D. T. (1982). EM algorithms for ML factor analysis. Psychometrika, 47, 69–76.
SAS Institute Inc. (2009). SAS/STAT® 9.2 users guide, version (2nd ed.). Cary, NC: SAS Institute Inc.
Schwarz,, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
Shapiro,, A., & ten Berge,, J. M. F. (2002). Statistical inference of minimum rank factor analysis. Psychometrika, 67, 79–94.
Sočan,, G. (2003). The incremental value of minimum rank factor analysis. In PhD thesis. Groningen, the Netherlands: University of Groningen.
Spearman,, C. (1904). “General intelligence”, objectively determined and measured. American Journal of Psychology, 15, 201–293.
SPSS. (1997). SPSS® 7.5 statistical algorithms. Chicago, IL: Author.
Stegeman,, A. (2016). A new method for simultaneous estimation of the factor model parameters, factor scores, and unique parts. Computational Statistics and Data Analysis, 99, 189–203.
ten Berge,, J. M. F. (1983). A generalization of Kristof`s theorem on the trace of certain matrix products. Psychometrika, 48, 519–523.
ten Berge,, J. M. F. (2000). Linking reliability and factor analysis: Recent developments in some classic psychometric problems. In S. E. Hampson, (Ed.), Advances in personality psychology (Vol. 1, pp. 138–156). London, England: Routledge.
ten Berge,, J. M. F., & Kiers,, H. A. L. (1991). A numerical approach to the exact and the approximate minimum rank of a covarianxce matrix. Psychometrika, 56, 309–315.
Thurstone,, L. L. (1935). The vectors of mind. Chicago, IL: University of Chicago Press.
Tipping,, M. E., & Bishop,, C. M. (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61, 611–622.
Trendafilov,, N. T., Fontanella,, S., & Adachi,, K. (2017). Sparse exploratory factor analysis. Psychometrika, 82, 778–794.
Unkel,, S., & Trendafilov,, N. T. (2010). Simultaneous parameter estimation in exploratory factor analysis: An expository review. International Statistical Review, 78, 363–382.
Yanai,, H., & Ichikawa,, M. (2007). Factor analysis. In C. R. Rao, & S. Sinharay, (Eds.), Handbook of statistics vol. 26: Psychometrics (pp. 257–296). Amsterdam, the Netherlands: Elsevier.

Further Reading

Finesso,, L., & Spreij,, P. (2016). Factor analysis models via I‐divergence optimization. Psychometrika, 81, 702–726.
Rowe,, D. B. (2003). Multivariate Bayesian statistics: Models for source separation and signal unmixing. Boca Raton, FL: CRC Press.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts