How to cite this WIREs title:
WIREs Comp Stat

Diagnostic tools for hierarchical linear models

Advanced Review

Adam Loy, Heike Hofmann

Published Online: Dec 26 2012

DOI: 10.1002/wics.1238

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Hierarchical structures are omnipresent in today's society—this is reflected in the data that we collect on all aspects of this society. Hierarchical linear models allow a representation of structural levels in a statistical modeling framework. Diagnostic tools are used to assess the quality of model estimation and explore features of the data not well described by the model. Residual and influence diagnostics are familiar tools for the classical regression model with independent observations. For hierarchical linear models, these diagnostic tools must be adjusted to reflect the dependence introduced by the nested data structure. Residual analysis now includes the assessment of distributional assumptions at each level of the model. This requires the use of level‐dependent residual quantities. Similarly, the parameter estimates may be influenced at each level of the model, requiring influence diagnostics that can pinpoint specific levels of the model, as well as specific aspects of the model. We present an overview of the diagnostic tools available for hierarchical linear models that are familiar from linear models. Additionally, we discuss the utility of the lineup protocol for residual analysis with complex models. WIREs Comput Stat 2013, 5:48–61. doi: 10.1002/wics.1238 This article is categorized under: Statistical Models > Bayesian Models Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical and Graphical Methods of Data Analysis > Statistical Graphics and Visualization

Plot of serum bilirubin level by week for the placebo group (left) and the treatment group (right).

[ Normal View | Magnified View ]

Modified dotplot of Cook's distance for the fixed effects for the methylprednisolone study. The plot shows that subject 55 may have a large effect on the fixed effects estimates.

[ Normal View | Magnified View ]

Dotplot (left) and modified dotplot (right) of Cook's distance for the variance components for the methylprednisolone study. The modified dotplot collapses all subjects ‘within’ a specified cutoff for the statistic. Here, the boxplot criterion for outliers was used as a measure of internal standing to form the cutoff (vertical red line). The plots show that subject 41, indicating this subject has a large effect on the variance component estimates.

[ Normal View | Magnified View ]

A lineup of 20 plots of EB level‐2 residuals against baseline serum bilirubin from the log transformed model. One of the plots is constructed from the observed methylprednisolone study data while the others are simulated residuals. Can you spot the observed residuals?

[ Normal View | Magnified View ]

A lineup of 20 boxplots (ordered by IQR) of LS level‐1 residuals of the log transformed model. One of the plots is constructed from the observed methylprednisolone study data while the others are simulated residuals. Can you spot the observed residuals?

[ Normal View | Magnified View ]

A normal quantile plot of the standardized LS level‐1 residuals. The large deviations from the reference line indicate that we are dealing with a very heavy‐tailed distribution.

[ Normal View | Magnified View ]

A lineup of 20 boxplots (ordered by IQR) of LS level‐1 residuals. One of the plots is constructed from the observed methylprednisolone study data while the others are simulated residuals. Can you spot the observed residuals?

[ Normal View | Magnified View ]

References

Box, GP, Jenkins, GM. Time Series Analysis: Forecasting and Control, revised. Oakland: Holden‐Day; 1976.
Tukey, JW. Exploratory Data Analysis. Reading: Addison‐Wesley; 1977.
Bryk, AS, Raudenbush, SW. Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park: Sage Publications; 1992.
Goldstein, H. Multilevel Statistical Models. London: Edward Arnold; 1995.
Pinheiro, JC, Bates, DM. Mixed‐Effects Models in S and S‐Plus. New York: Springer; 2000.
Carithers, RL, Franklin Herlong, H, Mae Diehl, A, Shaw, EW, Combes, B, Fallon, HJ, Maddrey, WC. Methylprednisolone therapy in patients with severe alcoholic hepatitis. Ann Int Med 1989, 110:685–690.
Vonesh, EF, Chinchilli, VM. Linear and Nonlinear Models for the Analysis of Repeated Measurements. New York: Marcel Dekker; 1997.
Diggle, PJ, Heagerty, P, Liang, K‐Y, Zeger, SL. Analysis of Longitudinal Data. 2nd ed. Oxford: Oxford University Press; 2002.
Hilden‐Minton, JA. Multilevel Diagnostics for Mixed and Hierarchical Linear Models, PhD Thesis, University of California Los Angeles, 1995.
Loy, A, Hofmann, H. Minimally confounded residuals for mixed/hierarchical linear models, in preparation.
Buja, A, Cook, D, Hofmann, H, Lawrence, M, Lee, E‐K, Swayne, DF, Wickham, H. Statistical inference for exploratory data analysis and model diagnostics. R Soc Phil Trans A 2009, 367:4361–4383.
Fraccaro, R, Hyndman, RJ, Veevers, A. Residual diagnostic plots for checking for model mis‐specification in time series regression. Austr New Zeal J Stat 2000, 42:463–477.
Laird, NM, Ware, JH. Random‐effects models for longitudinal data. Biometrics 1982, 38:963–974.
Lange, N, Ryan, L. Assessing normality in random effects models. Ann Stat 1989, 17:624–642.
Eberly, LE, Thackeray, LM. On Lange and Ryan`s plotting technique for diagnosing non‐normality of random effects. Stat Prob Lett 2005, 75:77–85.
Haslett, J, Haslett, SJ. The three basic types of residuals for a linear model. Int Stat Rev 2007, 75:1–24.
Haslett, J, Hayes, K. Residuals for the linear model with general covariance structure. J R Stat Soc Ser B 1998, 60:201–215.
Haslett, J. A simple derivation of deletion diagnostic results for the general linear model with correlated errors. J R Stat Soc Ser B 1999, 61:603–609.
Haslett, J, Dillane, D. Application of ‘delete = replace’ to deletion diagnostics for variance component estimation in the linear mixed model. J R Stat Soc Ser B 2004, 66:131–143.
Dillane, D. Deletion Diagnostics for the Linear Mixed Model, PhD Thesis, Trinity College Dublin, September 2005.
Belsley, DA, Kuh, E, Welsch, RE. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: John Wiley %26 Sons; 1980.
Dennis Cook, R, Weisberg, S. Residuals and Influence in Regression. New York: Chapman and Hall; 1982.
Chatterjee, S, Hadi, AS. Influential observations, high leverage points, and outliers in linear regression. Stat Sci 1986, 1:379–393.
Schabenberger, O. Mixed model influence diagnostics. In Proceedings of the Twenty‐Ninth Annual SAS^® Users Group International Conference, Cary, NC, 2004. SAS Users Group International, SAS Institute Inc.
Rabe‐Hesketh, S, Skrondal, A. Diagnostics for generalised linear mixed models, United Kingdom Stata Users` Group Meetings 2003 05, Stata Users Group, 2003.
Rabash, J, Steele, F, Browne, WJ, Goldstein, H. A User`s Guide to MLwiN, v2.26, Centre for Multilevel Modelling, University of Bristol, 2012.
Loy, A. HLMdiag: Diagnostic tools for two‐level normal mixed/hierarchical linear models, R package version 0.1.5, 2012.
R Core Team, R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, 2012. ISBN 3‐900051‐07‐0. Available from: http://www.R‐project.org/.
West, BT, Galecki, AT. An overview of current software procedures for fitting linear mixed models. The American Statistician 2011, 65:274–282.
Demidenko, E, Stukel, TA. Influence analysis for linear mixed‐effects models. Stat Med 2005, 24:893–909.
Nobre, JS, Singer, JM. Leverage analysis for linear mixed models. J Appl Stat 2011, 38:1063–1072.
Dennis Cook, R. Detection of influential observation in linear regression. Technometrics 1977, 19:15–18.
Longford, NT. Simulation‐based diagnostics in random‐coefficient models. J R Stat Soc Ser A 2001, 164:259–273.
Christensen, R, Pearson, LM, Johnson, W. Case‐deletion diagnostics for mixed models. Technometrics 1992, 34:38–45.
Banerjee, M, Frees, E. Influence diagnostics for linear longitudinal models. J Am Stat Assoc 1997, 92:999–1005.
Zewotir, T, Galpin, JS. Influence diagnostics for linear mixed models. J Data Sci 2005, 3:153–177.
Zewotir, T. Multiple cases deletion diagnostics for linear mixed models. Commun Stat—Theory Meth 2008, 37:1071–1084.
Shi, L, Chen, G. Deletion, replacement and mean‐shift for diagnostics in linear mixed models. Computational Statistics %26 Data Analysis 2012, 56:202–208.
Beckman, RJ, Nachtsheim, CJ, Dennis Cook, R. Diagnostics for mixed‐model analysis of variance. Technometrics 1987, 29:413–426.
Lesaffre, E, Verbeke, G. Local influence in linear mixed models. Biometrics 1998, 54:570–582.
Shi, L, Miguel Ojeda, M. Local influence in multilevel regression for growth curves. J Multivariate Anal 2004, 91:282–304.
Shi, L, Chen, G. Case deletion diagnostics in multilevel models. J Multivariate Anal 2008, 99:1860–1877.
Langford, IH, Lewis, T. Outliers in multilevel data. J R Stat Soc Ser A 1998, 161:121–160.
Longford, NT. Discussion on ‘Outliers in multilevel data’ (by I.H. Langford and T. Lewis). J R Stat Soc Ser A 1998, 161:154–155.
Shi, L, Chen, G. Detection of outliers in multilevel models. J Stat Plann Infer 2008, 138:3189–3199.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts