This Title All WIREs
How to cite this WIREs title:
WIREs Comp Stat

# Statistical nonparametric mapping: Multivariate permutation tests for location, correlation, and regression problems in neuroimaging

Can't access this content? Tell your librarian.

Nonparametric statistical inference via permutation testing is on the rise in neuroimaging research. This rise in popularity is likely in response to recent studies that have demonstrated limitations of parametric inference in certain situations. Nonparametric tests have the appeal of requiring fewer assumptions than their parametric counterparts, and are often touted as being more flexible and more useful for small samples. Furthermore, recent studies have demonstrated the robustness of nonparametric methods in situations when parametric inference fails. As a result, many nonstatistical neuroimaging researchers are likely to believe that nonparametric permutation tests are always a “safe choice” because the results do not depend on distributional assumptions and/or large sample approximations. Alas, this commonly held belief is not entirely accurate, given that nonparametric tests still do rely on assumptions and/or approximations for valid statistical inference. When these assumptions are met, nonparametric permutation tests have the potential to produce valid inferential results for the intended hypotheses. However, as I demonstrate, when these assumptions are violated, nonparametric permutation tests can produce invalid and/or misleading results, which have important implications for the use of such methods in neuroimaging research. All hope is not lost though, as recent theoretical developments in nonparametric statistics can improve current implementations of permutation tests in neuroimaging research. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical and Graphical Methods of Data Analysis > Bootstrap and Resampling Data: Types and Structure > Image and Spatial Data
The XY slice with Z = 0 (top) and the ZY slice with X = 0 (bottom) for three trivariate distributions with μj = 0 and for j ∈ {X, Y, Z}. The XZ slice with Y = 0 is identical to the top row. Note that ρXZ = ρXY = 0 for all three distributions, ρZY = 0 for the spherical uniform, and ρZY = 1/2 for the multivariate normal and multivariate t
[ Normal View | Magnified View ]
Correlation test. Type I error rate calculated across 10,000 replications for the parametric tests (red squares) and the permutation tests (black circles). The unfilled points denote the results using the (classic) T statistic, whereas the filled points denote the results using the (studentized) T* statistic. The vertical bars denote approximate 99% confidence intervals, and the dotted horizontal lines denote the nominal rate of α = 0.05
[ Normal View | Magnified View ]
Three bivariate distributions with μX = μY = 0, , and ρXY = 0
[ Normal View | Magnified View ]
Two‐sample location test. Type I error rate calculated across 10,000 replications for the t‐test (red squares) and the permutation test (black circles). The unfilled points denote results using the (Student) T statistic, whereas the filled points denote results using the (Welch) T* statistic. The vertical bars denote approximate 99% confidence intervals, and the dotted horizontal lines denote the nominal rate of α = 0.05. Note that the second group is twice as large as the first group, that is, n2 = 2n1
[ Normal View | Magnified View ]
One‐sample location test. Type I error rate calculated across 10,000 replications for the t‐test (red squares) and the permutation test (black circles). The vertical bars denote approximate 99% confidence intervals, and the dotted horizontal lines denote the nominal rate of α = 0.05
[ Normal View | Magnified View ]
Six univariate distributions with mean μ = 0 and variance σ2 = 1
[ Normal View | Magnified View ]
Timeline of the number of publications on permutation (black solid line) and bootstrap (gray dashed line) resampling methods for neuroimaging data. Obtained from a Scopus search for publications with titles, abstracts, and/or keywords containing the given term in combination with one or more of the terms—EEG, MEG, PET, CT, MRI, fMRI, DTI, Neuroimage, Neuroimaging, Brain image, Brain imaging
[ Normal View | Magnified View ]
Regression test. Type I error rate calculated across 10,000 replications for the parametric tests (red squares) and the permutation tests (black circles). The unfilled points denote the results using the (classic) F‐test statistic, whereas the filled points denote the results using the (Wald) W‐test statistic. The vertical bars denote approximate 99% confidence intervals, and the dotted horizontal lines denote the nominal rate of α = 0.05
[ Normal View | Magnified View ]