How to cite this WIREs title:
WIREs Comp Stat

Minimum covariance determinant and extensions

Advanced Review

Mia Hubert, Michiel Debruyne, Peter J. Rousseeuw

Published Online: Dec 22 2017

DOI: 10.1002/wics.1421

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

The minimum covariance determinant (MCD) method is a highly robust estimator of multivariate location and scatter, for which a fast algorithm is available. Since estimating the covariance matrix is the cornerstone of many multivariate statistical methods, the MCD is an important building block when developing robust multivariate techniques. It also serves as a convenient and efficient tool for outlier detection. The MCD estimator is reviewed, along with its main properties such as affine equivariance, breakdown value, and influence function. We discuss its computation, and list applications and extensions of the MCD in applied and methodological multivariate statistics. Two recent extensions of the MCD are described. The first one is a fast deterministic algorithm which inherits the robustness of the MCD while being almost affine equivariant. The second is tailored to high‐dimensional data, possibly with more dimensions than cases, and incorporates regularization to prevent singular matrices. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Statistical and Graphical Methods of Data Analysis > Robust Methods Statistical Learning and Exploratory Methods of the Data Sciences > Knowledge Discovery

Bivariate wine data: tolerance ellipse of the classical mean and covariance matrix (red), and that of the robust location and scatter matrix (blue)

[ Normal View | Magnified View ]

Distance‐distance plot of the full 13‐dimensional wine data set

[ Normal View | Magnified View ]

(a) Mahalanobis distances and (b) robust distances for the bivariate wine data

[ Normal View | Magnified View ]

References

Agulló,, J., Croux,, C., & Van Aelst,, S. (2008). The multivariate least trimmed squares estimator. Journal of Multivariate Analysis, 99, 311–318.
Billor,, N., Hadi,, A., & Velleman,, P. (2000). BACON: Blocked adaptive computationally efficient outlier nominators. Computational Statistics %26 Data Analysis, 34, 279–298.
Boudt,, K., Rousseeuw,, P., Vanduffel,, S., & Verdonck,, T. (2017). The minimum regularized covariance determinant estimator. arXiv: 1701.07086.
Butler,, R., Davies,, P., & Jhun,, M. (1993). Asymptotics for the minimum covariance determinant estimator. The Annals of Statistics, 21(3), 1385–1400.
Cator,, E., & Lopuhaä,, H. (2010). Asymptotic expansion of the minimum covariance determinant estimators. Journal of Multivariate Analysis, 101, 2372–2388.
Cator,, E., & Lopuhaä,, H. (2012). Central limit theorem and influence function for the MCD estimators at general multivariate distributions. Bernouilli, 18, 520–551.
Cerioli,, A. (2010). Multivariate outlier detection with high‐breakdown estimators. Journal of the American Statistical Association, 105(489), 147–156.
Ceulemans,, E., Hubert,, M., & Rousseeuw,, P. (2013). Robust multilevel simultaneous component analysis. Chemometrics and Intelligent Laboratory Systems, 129, 33–39.
Cheng,, T.‐C., & Victoria‐Feser,, M.‐P. (2002). High breakdown estimation of multivariate location and scale with missing observations. British Journal of Mathematical and Statistical Psychology, 55, 317–335.
Christmann,, A., & Van Aelst,, S. (2006). Robust estimation of Cronbach`s alpha. Journal of Multivariate Analysis, 97(7), 1660–1674.
Čıžek,, P. (2008). Robust and efficient adaptive estimation of binary‐choice regression models. Journal of the American Statistical Association, 103(482), 687–696.
Coakley,, C., & Hettmansperger,, T. (1993). A bounded influence, high breakdown, efficient regression estimator. Journal of the American Statistical Association, 88, 872–880.
Copt,, S., & Victoria‐Feser,, M.‐P. (2004). Fast algorithms for computing high breakdown covariance matrices with missing data. In M.Hubert,, G.Pison,, A.Struyf,, & S. Van Aelst, (Eds.), Theory and applications of recent robust methods (pp. 71–82). Basel: Birkhäuser.
Croux,, C., & Dehon,, C. (2002). Analyse canonique basée sur des estimateurs robustes de la matrice de covariance. La Revue de Statistique Appliquée, 2, 5–26.
Croux,, C., & Haesbroeck,, G. (1999). Influence function and efficiency of the minimum covariance determinant scatter matrix estimator. Journal of Multivariate Analysis, 71, 161–190.
Croux,, C., & Haesbroeck,, G. (2000). Principal components analysis based on robust estimators of the covariance or correlation matrix: Influence functions and efficiencies. Biometrika, 87, 603–618.
Croux,, C., & Haesbroeck,, G. (2001). Maxbias curves of robust scale estimators based on subranges. Metrika, 53, 101–122.
Croux,, C., & Haesbroeck,, G. (2002). Maxbias curves of location estimators based on subranges. Journal of Nonparametric Statistics, 14, 295–306.
Croux,, C., & Haesbroeck,, G. (2003). Implementing the Bianco and Yohai estimator for logistic regression. Computational Statistics %26 Data Analysis, 44, 273–295.
Croux,, C., & Rousseeuw,, P. (1992). A class of high‐breakdown scale estimators based on subranges. Communications in Statistics, 21, 1935–1951.
Cuesta‐Albertos,, J., Gordaliza,, A., & Matrán,, C. (1997). Trimmed k‐means: An attempt 19 to robustify quantizers. The Annals of Statistics, 25, 553–576.
Cuesta‐Albertos,, J., Matrán,, C., & Mayo‐Iscar,, A. (2008). Robust estimation in the normal mixture model based on robust clustering. Journal of the Royal Statistical Society, 70, 779–802.
Davies,, L. (1987). Asymptotic behavior of S‐estimators of multivariate location parameters and dispersion matrices. The Annals of Statistics, 15, 1269–1292.
Debruyne,, M., & Hubert,, M. (2009). The influence function of the Stahel‐Donoho covariance estimator of smallest outlyingness. Statistics %26 Probability Letters, 79, 275–282.
Donoho,, D., & Gasko,, M. (1992). Breakdown properties of location estimates based on halfspace depth and projected outlyingness. The Annals of Statistics, 20(4), 1803–1827.
Engelen,, S., & Hubert,, M. (2011). Detecting outlying samples in a parallel factor analysis model. Analytica Chimica Acta, 705, 155–165.
Fekri,, M., & Ruiz‐Gazen,, A. (2004). Robust weighted orthogonal regression in the errors‐in‐variables model. Journal of Multivariate Analysis, 88(1), 89–108.
Filzmoser,, P., Garrett,, R., & Reimann,, C. (2005). Multivariate outlier detection in exploration geochemistry. Computers and Geosciences, 31, 579–587.
Gallegos,, M., & Ritter,, G. (2005). A robust method for cluster analysis. The Annals of Statistics, 33, 347–380.
Gambacciani,, M., & Paolella,, M. S. (2017). Robust normal mixtures for financial portfolio allocation. Econometrics and Statistics, 3, 91–111.
García‐Escudero,, L., Gordaliza,, A., San Martín,, R., Van Aelst,, S., & Zamar,, R. (2009). Robust linear clustering. Journal of the Royal Statistical Society, 71, 1–18.
Hadi,, A., & Luceño,, A. (1997). Maximum trimmed likelihood estimators: A unified approach, examples and algorithms. Computational Statistics %26 Data Analysis, 25, 251–272.
Hampel,, F., Ronchetti,, E., Rousseeuw,, P., & Stahel,, W. (1986). Robust statistics: The approach based on influence functions. New York, NY: Wiley.
Hardin,, J., & Rocke,, D. (2004). Outlier detection in the multiple cluster setting using the minimum covariance determinant estimator. Computational Statistics %26 Data Analysis, 44, 625–638.
Hardin,, J., & Rocke,, D. M. (2005). The distribution of robust distances. Journal of Computational and Graphical Statistics, 14(4), 928–946.
Hawkins,, D., & McLachlan,, G. (1997). High‐breakdown linear discriminant analysis. Journal of the American Statistical Association, 92, 136–143.
Hettich,, S., & Bay,, S. (1999). The UCI KDD archive [Computer software manual]. Irvine, CA: University of California, Department of Information and Computer Science. Retrieved from http://kdd.ics.uci.edu
Hubert,, M., & Engelen,, S. (2007). Fast cross‐validation for high‐breakdown resampling algorithms for PCA. Computational Statistics %26 Data Analysis, 51, 5013–5024.
Hubert,, M., & Rousseeuw,, P. (1996). Robust regression with both continuous and binary regressors. Journal of Statistical Planning and Inference, 57, 153–163.
Hubert,, M., Rousseeuw,, P., & Van Aelst,, S. (2008). High breakdown robust multivariate methods. Statistical Science, 23, 92–119.
Hubert,, M., Rousseeuw,, P., & Vanden Branden,, K. (2005). ROBPCA: A new approach to robust principal components analysis. Technometrics, 47, 64–79.
Hubert,, M., Rousseeuw,, P., Vanpaemel,, D., & Verdonck,, T. (2015). The DetS and DetMM estimators for multivariate location and scatter. Computational Statistics %26 Data Analysis, 81, 64–75.
Hubert,, M., Rousseeuw,, P., & Verdonck,, T. (2012). A deterministic algorithm for robust location and scatter. Journal of Computational and Graphical Statistics, 21, 618–637.
Hubert,, M., & Van Driessen,, K. (2004). Fast and robust discriminant analysis. Computational Statistics %26 Data Analysis, 45, 301–320.
Hubert,, M., & Vanden Branden,, K. (2003). Robust methods for partial least squares regression. Journal of Chemometrics, 17, 537–549.
Hubert,, M., & Verboven,, S. (2003). A robust PCR method for high‐dimensional regressors. Journal of Chemometrics, 17, 438–452.
Jensen,, W., Birch,, J., & Woodal,, W. (2007). High breakdown estimation methods for phase I multivariate control charts. Quality and Reliability Engineering International, 23(5), 615–629.
Lopuhaä,, H. (1999). Asymptotics of reweighted estimators of multivariate location and scatter. The Annals of Statistics, 27, 1638–1665.
Lopuhaä,, H., & Rousseeuw,, P. (1991). Breakdown points of affine equivariant estimators of multivariate location and covariance matrices. The Annals of Statistics, 19, 229–248.
Lu,, Y., Wang,, J., Kong,, J., Zhang,, B., & Zhang,, J. (2006). An integrated algorithm for MRI brain images segmentation. Computer Vision Approaches to Medical Image Analysis, 4241, 132–1342.
Maronna,, R. (2005). Principal components and orthogonal regression based on robust scales. Technometrics, 47, 264–273.
Maronna,, R., Martin,, D., & Yohai,, V. (2006). Robust statistics: Theory and methods. New York, NY: Wiley.
Maronna,, R., & Zamar,, R. (2002). Robust estimates of location and dispersion for high‐dimensional data sets. Technometrics, 44, 307–317.
Müller,, C., & Neykov,, N. (2003). Breakdown points of trimmed likelihood estimators and related estimators in generalized linear models. Journal of Statistical Planning and Inference, 116, 503–519.
Neykov,, N., Neytchev,, P., Van Gelder,, P., & Todorov,, V. (2007). Robust detection of discordant sites in regional frequency analysis. Water Resources Research, 43(6).
Pison,, G., Rousseeuw,, P., Filzmoser,, P., & Croux,, C. (2003). Robust factor analysis. Journal of Multivariate Analysis, 84, 145–172.
Pison,, G., Van Aelst,, S., & Willems,, G. (2002). Small sample corrections for LTS and MCD. Metrika, 55, 111–123.
Prastawa,, M., Bullitt,, E., Ho,, S., & Gerig,, G. (2004). A brain tumor segmentation framework based on outlier detection. Medical Image Analysis, 8, 275–283.
Rocke,, D., & Woodruff,, D. (1999). A synthesis of outlier detection and cluster identification. Technical report.
Roelant,, E., Van Aelst,, S., & Willems,, G. (2009). The minimum weighted covariance determinant estimator. Metrika, 70, 177–204.
Rousseeuw,, P. (1984). Least median of squares regression. Journal of the American Statistical Association, 79, 871–880.
Rousseeuw,, P. (1985). Multivariate estimation with high breakdown point. In W.Grossmann,, G.Pflug,, I.Vincze,, & W.Wertz, (Eds.), Mathematical statistics and applications (Vol. B, pp. 283–297). Dordrecht: Reidel.
Rousseeuw,, P. (2005). Discussion on ‘Breakdown and groups’. Annals of Statistics, 33, 1004–1009.
Rousseeuw,, P., & Christmann,, A. (2003). Robustness against separation and outliers in logistic regression. Computational Statistics %26 Data Analysis, 43, 315–332.
Rousseeuw,, P., & Croux,, C. (1993). Alternatives to the median absolute deviation. Journal of the American Statistical Association, 88, 1273–1283.
Rousseeuw,, P., & Hubert,, M. (2017). Anomaly detection by robust statistics. WIREs Data Mining and Knowledge Discovery. https://doi.org/10.1002/widm.1236
Rousseeuw,, P., & Leroy,, A. (1987). Robust regression and outlier detection. New York, NY: Wiley.
Rousseeuw,, P., Van Aelst,, S., Van Driessen,, K., & Agulló,, J. (2004). Robust multivariate regression. Technometrics, 46, 293–305.
Rousseeuw,, P., & Van Driessen,, K. (1999). A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41, 212–223.
Rousseeuw,, P., & Van Driessen,, K. (2006). Computing LTS regression for large data sets. Data Mining and Knowledge Discovery, 12, 29–45.
Rousseeuw,, P., & van Zomeren,, B. (1990). Unmasking multivariate outliers and leverage points. Journal of the American Statistical Association, 85, 633–651.
Salibian‐Barrera,, M., & Yohai,, V. (2006). A fast algorithm for S‐regression estimates. Journal of Computational and Graphical Statistics, 15, 414–427.
Serneels,, S., & Verdonck,, T. (2008). Principal component analysis for data containing outliers and missing elements. Computational Statistics %26 Data Analysis, 52, 1712–1727.
Simpson,, D., Ruppert,, D., & Carroll,, R. (1992). On one‐step GM‐estimates and stability of inferences in linear regression. Journal of the American Statistical Association, 87, 439–450.
Stahel,, W. (1981). Robuste Schätzungen: infinitesimale Optimalität und Schätzungen von Kovarianzmatrizen. (Unpublished doctoral dissertation). ETH Zürich, Zürich.
Tyler,, D. E., Critchley,, F., Dümbgen,, L., & Oja,, H. (2009). Invariant co‐ordinate selection. Journal of the Royal Statistical Society, 71(3), 549–592.
van Helvoort,, P., Filzmoser,, P., & van Gaans,, P. (2005). Sequential factor analysis as a new approach to multivariate analysis of heterogeneous geochemical datasets: An application to a bulk chemical characterization of fluvial deposits (Rhine‐Meuse delta, The Netherlands). Applied Geochemistry, 20(12), 2233–2251.
Vanden Branden,, K., & Hubert,, M. (2004). Robustness properties of a robust PLS regression method. Analytica Chimica Acta, 515, 229–241.
Vanden Branden,, K., & Hubert,, M. (2005). Robust classification in high dimensions based on the SIMCA method. Chemometrics and Intelligent Laboratory Systems, 79, 10–21.
Vandev,, D., & Neykov,, N. (1998). About regression estimators with high breakdown point. Statistics, 32, 111–129.
Verboven,, S., & Hubert,, M. (2005). LIBRA: A Matlab library for robust analysis. Chemometrics and Intelligent Laboratory Systems, 75, 127–136.
Verboven,, S., & Hubert,, M. (2010). MATLAB library LIBRA. WIREs Computational Statistics, 2, 509–515.
Víšek,, J. (2002). The least weighted squares I. The asymptotic linearity of normal equations. Bulletin of the Czech Econometric Society, 9, 31–58.
Visuri,, S., Koivunen,, V., & Oja,, H. (2000). Sign and rank covariance matrices. Journal of Statistical Planning and Inference, 91, 557–575.
Vogler,, C., Goldenstein,, S., Stolfi,, J., Pavlovic,, V., & Metaxas,, D. (2007). Outlier rejection in high‐dimensional deformable models. Image and Vision Computing, 25(3), 274–284.
Welsh,, R., & Zhou,, X. (2007). Application of robust statistics to asset allocation models. Revstat, 5, 97–114.
Willems,, G., Pison,, G., Rousseeuw,, P., & Van Aelst,, S. (2002). A robust Hotelling test. Metrika, 55, 125–138.
Willems,, G., & Van Aelst,, S. (2004). A fast bootstrap method for the MCD estimator. In J.Antoch, (Ed.), Proceedings in computational statistics (pp. 1979–1986). Heidelberg: Springer.
Zaman,, A., Rousseeuw,, P., & Orhan,, M. (2001). Econometric applications of high‐breakdown robust regression techniques. Economics Letters, 71, 1–8.

Other Versions

Previous Versions

Minimum covariance determinant
Michiel Debruyne,Mia Hubert
Published Online: December 31 2009
DOI: 10.1002/wics.61

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Minimum covariance determinant and extensions

Related Articles

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox