How to cite this WIREs title:
WIREs Comp Stat

High‐dimensional covariance matrix estimation

Advanced Review

Clifford Lam

Published Online: Oct 23 2019

DOI: 10.1002/wics.1485

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Covariance matrix estimation plays an important role in statistical analysis in many fields, including (but not limited to) portfolio allocation and risk management in finance, graphical modeling, and clustering for genes discovery in bioinformatics, Kalman filtering and factor analysis in economics. In this paper, we give a selective review of covariance and precision matrix estimation when the matrix dimension can be diverging with, or even larger than the sample size. Two broad categories of regularization methods are presented. The first category exploits an assumed structure of the covariance or precision matrix for consistent estimation. The second category shrinks the eigenvalues of a sample covariance matrix, knowing from random matrix theory that such eigenvalues are biased from the population counterparts when the matrix dimension grows at the same rate as the sample size. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Statistical and Graphical Methods of Data Analysis > Nonparametric Methods

Upper left: ∑, p = 120. Upper right: ∣∑∣, in log‐scale. Lower left: ∑⁻¹. Lower right: ∣∑⁻¹∣, in log‐scale

[ Normal View | Magnified View ]

References

Abadir,, K. M., Distaso,, W., & Žikeš,, F. (2014). Design‐free estimation of variance matrices. Journal of Econometrics, 181(2), 165–180.
Abdi,, H., & Williams,, L. J. (2010). Principal component analysis. WIREs Computational Statistics, 2(4), 433–459.
Bai,, J., & Ng,, S. (2002). Determining the number of factors in approximate factor models. Econometrica, 70(1), 191–221.
Bai,, Z. D., & Yin,, Y. Q. (1993). Limit of the smallest eigenvalue of a large dimensional sample covariance matrix. The Annals of Probability, 21(3), 1275–1294.
Banerjee,, O., El Ghaoui,, L., & d`Aspremont,, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9, 485–516.
Bickel,, P. J., & Levina,, E. (2008a). Covariance regularization by thresholding. The Annals of Statistics, 36(6), 2577–2604.
Bickel,, P. J., & Levina,, E. (2008b). Regularized estimation of large covariance matrices. The Annals of Statistics, 36(1), 199–227.
Bien,, J. (2019). Graph‐guided banding of the covariance matrix. Journal of the American Statistical Association, 114(526), 782–792.
Bien,, J., Bunea,, F., & Xiao,, L. (2016). Convex banding of the covariance matrix. Journal of the American Statistical Association, 111(514), 834–845.
Bien,, J., & Tibshirani,, R. J. (2011). Sparse estimation of a covariance matrix. Biometrika, 98(4), 807–820.
Cai,, T., & Liu,, W. (2011). Adaptive thresholding for sparse covariance matrix estimation. Journal of the American Statistical Association, 106(494), 672–684.
Cai,, T., Liu,, W., & Luo,, X. (2011). A constrained ℓ₁ minimization approach to sparse precision matrix estimation. Journal of the American Statistical Association, 106(494), 594–607.
Cai,, T. T., Liu,, W., & Zhou,, H. H. (2016). Estimating sparse precision matrix: Optimal rates of convergence and adaptive estimation. The Annals of Statistics, 44(2), 455–488.
Cai,, T. T., Ren,, Z., & Zhou,, H. H. (2016). Estimating structured high‐dimensional covariance and precision matrices: Optimal rates and adaptive estimation. Electronic Journal of Statistics, 10(1), 1–59.
Cai,, T. T., & Yuan,, M. (2012). Adaptive covariance matrix estimation through block thresholding. The Annals of Statistics, 40(4), 2014–2042.
Cai,, T. T., Zhang,, C.‐H., & Zhou,, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. The Annals of Statistics, 38(4), 2118–2144.
Chamberlain,, G., & Rothschild,, M. (1983). Arbitrage, factor structure, and mean‐variance analysis on large asset markets. Econometrica, 51(5), 1281–1304.
Chandrasekaran,, V., Parrilo,, P. A., & Willsky,, A. S. (2012). Latent variable graphical model selection via convex optimization. The Annals of Statistics, 40(4), 1935–1967.
Chen,, M., Gao,, C., & Ren,, Z. (2018). Robust covariance and scatter matrix estimation under Huber`s contamination model. The Annals of Statistics, 46(5), 1932–1960.
Daniels,, M. J., & Kass,, R. E. (2001). Shrinkage estimators for covariance matrices. Biometrics, 57(4), 1173–1184.
Dao,, C., Lu,, K., & Xiu,, D. (2017). Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data (Chicago Booth Research Paper No. 17‐02).
DeMiguel,, V., Garlappi,, L., Nogales,, F. J., & Uppal,, R. (2009). A generalized approach to portfolio optimization: Improving performance by constraining portfolio norms. Management Science, 55(5), 798–812.
Donoho,, D., Gavish,, M., & Johnstone,, I. (2018). Optimal shrinkage of eigenvalues in the spiked covariance model. The Annals of Statistics, 46(4), 1742–1778.
Engle,, R. F., Ledoit,, O., & Wolf,, M. (2019). Large dynamic covariance matrices. Journal of Business %26 Economic Statistics, 37(2), 363–375.
Fan,, J., Fan,, Y., & Lv,, J. (2008). High dimensional covariance matrix estimation using a factor model. Journal of Econometrics, 147(1), 186–197.
Fan,, J., & Kim,, D. (2018). Robust high‐dimensional volatility matrix estimation for high‐frequency factor model. Journal of the American Statistical Association, 113(523), 1268–1283.
Fan,, J., & Li,, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.
Fan,, J., Li,, Y., & Yu,, K. (2012). Vast volatility matrix estimation using high‐frequency data for portfolio selection. Journal of the American Statistical Association, 107(497), 412–428.
Fan,, J., Liao,, Y., & Liu,, H. (2016). An overview of the estimation of large covariance and precision matrices. The Econometrics Journal, 19(1), C1–C32.
Fan,, J., Liao,, Y., & Mincheva,, M. (2013). Large covariance estimation by thresholding principal orthogonal complements. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 75(4), 603–680.
Fan,, J., Liu,, H., & Wang,, W. (2018). Large covariance estimation through elliptical factor models. The Annals of Statistics, 46(4), 1383–1414.
Fan,, J., Wang,, W., & Zhong,, Y. (2019). Robust covariance estimation for approximate factor models. Journal of Econometrics, 208(1), 5–22.
Friedman,, J., Hastie,, T., & Tibshirani,, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432–441.
Furrer,, R., Genton,, M. G., & Nychka,, D. (2006). Covariance tapering for interpolation of large spatial datasets. Journal of Computational and Graphical Statistics, 15(3), 502–523.
Guo,, S., Box,, J. L., & Zhang,, W. (2017). A dynamic structure for high‐dimensional covariance matrices and its application in portfolio allocation. Journal of the American Statistical Association, 112(517), 235–253.
Huang,, J. Z., Liu,, N., Pourahmadi,, M., & Liu,, L. (2006). Covariance matrix selection and estimation via penalised normal likelihood. Biometrika, 93(1), 85–98.
Huang,, N., & Fryzlewicz,, P. (2019). Novelist estimator of large correlation and covariance matrices and their inverses. TEST, 28(3), 694–727.
Joachimi,, B. (2016). Non‐linear shrinkage estimation of large‐scale structure covariance. Monthly Notices of the Royal Astronomical Society: Letters, 466(1), L83–L87.
Johnstone,, I. M., & Lu,, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. Journal of the American Statistical Association, 104(486), 682–693.
Johnstone,, I. M., & Paul,, D. (2018). Pca in high dimensions: An orientation. Proceedings of the IEEE, 106(8), 1277–1292.
Kendall,, M. (1948). Rank correlation methods. London, England: Griffin.
Lam,, C. (2016). Nonparametric eigenvalue‐regularized precision or covariance matrix estimator. The Annals of Statistics, 44(3), 928–953.
Lam,, C., & Fan,, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. The Annals of Statistics, 37(6B), 4254–4278.
Lam,, C., & Feng,, P. (2018). A nonparametric eigenvalue‐regularized integrated covariance matrix estimator for asset return data. Journal of Econometrics, 206(1), 226–257.
Lam,, C., Feng,, P., & Hu,, C. (2017). Nonlinear shrinkage estimation of large integrated covariance matrices. Biometrika, 104(2), 481–488.
Lam,, C., Yao,, Q., & Bathia,, N. (2011). Estimation of latent factors for high‐dimensional time series. Biometrika, 98(4), 901–918.
Ledoit,, O., & Péché,, S. (2011). Eigenvectors of some large sample covariance matrix ensembles. Probability Theory and Related Fields, 151(1–2), 233–264.
Ledoit,, O., & Wolf,, M. (2004). A well‐conditioned estimator for large‐dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 365–411.
Ledoit,, O., & Wolf,, M. (2012). Nonlinear shrinkage estimation of large‐dimensional covariance matrices. The Annals of Statistics, 40(2), 1024–1060.
Ledoit,, O., & Wolf,, M. (2015). Spectrum estimation: A unified framework for covariance matrix estimation and PCA in large dimensions. Journal of Multivariate Analysis, 139, 360–384.
Ledoit,, O., & Wolf,, M. (2017). Nonlinear shrinkage of the covariance matrix for portfolio selection: Markowitz meets goldilocks. The Review of Financial Studies, 30(12), 4349–4388.
Li,, D., Xue,, L., & Zou,, H. (2018). Applications of peter Hall`s martingale limit theory to estimating and testing high dimensional covariance matrices. Statistica Sinica, 28, 2657–2670.
Li,, D., & Zou,, H. (2016). Sure information criteria for large covariance matrix estimation and their asymptotic properties. IEEE Transactions on Information Theory, 62(4), 2153–2169.
Liu,, H., Han,, F., Yuan,, M., Lafferty,, J., & Wasserman,, L. (2012). High‐dimensional semiparametric gaussian copula graphical models. The Annals of Statistics, 40(4), 2293–2326.
Liu,, H., Lafferty,, J., & Wasserman,, L. (2009). The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 10, 2295–2328.
Ma,, S., Xue,, L., & Zou,, H. (2013). Alternating direction methods for latent variable gaussian graphical model selection. Neural Computation, 25(8), 2172–2198.
Marčenko,, V., & Pastur,, L. (1967). Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR‐Sbornik, 1, 457–483.
Mazumder,, R., & Hastie,, T. (2012). The graphical lasso: New insights and alternatives. Electronic Journal of Statistics, 6, 2125–2149.
Meinshausen,, N., & Bühlmann,, P. (2006). High‐dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34(3), 1436–1462.
Pan,, J., & Mackenzie,, G. (2003). On modelling mean covariance structures in longitudinal studies. Biometrika, 90(1), 239–244.
Paul,, D., & Aue,, A. (2014). Random matrix theory in statistics: A review. Journal of Statistical Planning and Inference, 150, 1–29.
Pourahmadi,, M. (2007). Cholesky decompositions and estimation of a covariance matrix: Orthogonality of variance‐correlation parameters. Biometrika, 94(4), 1006–1013.
Pourahmadi,, M. (2013). High‐dimensional covariance estimation with high‐dimensional data. In Wiley Series in Probability and Statistics. Hoboken, NJ: Wiley Interscience.
Qiu,, Y., & Chen,, S. X. (2012). Test for bandedness of high‐dimensional covariance matrices and bandwidth estimation. The Annals of Statistics, 40(3), 1285–1314.
Ravikumar,, P., Wainwright,, M. J., Raskutti,, G., & Yu,, B. (2011). High‐dimensional covariance estimation by minimizing 1‐penalized log‐determinant divergence. Electronic Journal of Statistics, 5, 935–980.
Ross,, S. A. (1976). The arbitrage theory of capital asset pricing. Journal of Economic Theory, 13(3), 341–360.
Rothman,, A. J., Bickel,, P. J., Levina,, E., & Zhu,, J. (2008). Sparse permutation invariant covariance estimation. Electronic Journal of Statistics, 2, 494–515.
Rothman,, A. J., Levina,, E., & Zhu,, J. (2009). Generalized thresholding of large covariance matrices. Journal of the American Statistical Association, 104(485), 177–186.
Rothman,, A. J., Levina,, E., & Zhu,, J. (2010). A new approach to Cholesky‐based covariance regularization in high dimensions. Biometrika, 97(3), 539–550.
Schäfer,, J., & Strimmer,, K. (2005). A shrinkage approach to large‐scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 4(1), Article 32.
Shen,, D., Shen,, H., & Marron,, J. S. (2016). A general framework for consistency of principal component analysis. Journal of Machine Learning Research, 17, 1–29.
Stein,, C. (1975). Estimation of a covariance matrix. Paper presented at Rietz Lecture, 39th Annual Meeting IMS. Atlanta, GA.
Stein,, C. (1986). Lectures on the theory of estimation of many parameters. Journal of Soviet mathematics, 34(1), 1373–1403.
Wang,, Y., & Zou,, J. (2010). Vast volatility matrix estimation for high‐frequency financial data. The Annals of Statistics, 38(2), 943–978.
Warton,, D. I. (2008). Penalized normal likelihood and ridge regularization of correlation and covariance matrices. Journal of the American Statistical Association, 103(481), 340–349.
Won,, J.‐H., Lim,, J., Kim,, S.‐J., & Rajaratnam,, B. (2013). Condition‐number‐regularized covariance estimation. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 75(3), 427–450.
Xue,, L., Ma,, S., & Zou,, H. (2012). Positive‐definite 1‐penalized estimation of large covariance matrices. Journal of the American Statistical Association, 107(500), 1480–1491.
Xue,, L., & Zou,, H. (2012). Regularized rank‐based estimation of high‐dimensional nonparanormal graphical models. The Annals of Statistics, 40(5), 2541–2571.
Xue,, L., & Zou,, H. (2014). Rank‐based tapering estimation of bandable correlation matrices. Statistica Sinica, 24, 83–100.
Yuan,, M., & Lin,, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1), 19–35.
Zou,, H., & Xue,, L. (2018). A selective overview of sparse principal component analysis. Proceedings of the IEEE, 106(8), 1311–1320.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

High‐dimensional covariance matrix estimation

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox