How to cite this WIREs title:
WIREs Comp Stat

Orthogonal series density estimation

Advanced Review

Sam Efromovich

Published Online: Jul 15 2010

DOI: 10.1002/wics.97

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Orthogonal series density estimation is a powerful nonparametric estimation methodology that allows one to analyze and present data at hand without any prior opinion about shape of an underlying density. The idea of construction of an adaptive orthogonal series density estimator is explained on the classical example of a direct sample from a univariate density. Data‐driven estimators, which have been used for years, as well as recently proposed procedures, are reviewed. Orthogonal series estimation is also known for its sharp minimax properties which are explained. Furthermore, applications of the orthogonal series methodology to more complicated settings, including censored and biased data as well as estimation of the density of regression errors and the conditional density, are also presented. Copyright © 2010 John Wiley & Sons, Inc. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Density Estimation

Performance of the default R (and S‐PLUS) histogram is shown in the top diagrams and performance of the orthogonal series Universal estimator (dashed lines) is shown in the bottom diagrams for the same simulated samples of size n = 50. The underlying density in the left diagrams is uniform, and in other diagrams it is shown by the solid line. Bottom diagrams exhibit underlying samples using histograms with smaller bin width.

[ Normal View | Magnified View ]

References

Anderson, GL, de Figueiredo, RJP. An adaptive orthogonal series estimator for probability density functions. Ann Stat 1980; 8: 347–376.
Čencov, NN. Statistical Decision Rules and Optimum Inference. New York: Springer‐Verlag; 1980.
Devroye, L, Györfi, L. Nonparametric Density Estimation: The L₁ View. New York: John Wiley %26 Sons; 1985.
Devroye, L, Lugosi, G. Combinatorial Methods in Density Estimation. New York: Springer; 2001.
Efromovich, S. Nonparametric Curve Estimation: Methods, Theory and Applications. New York: Springer; 1999.
Ibragimov, IA, Khasminskii, RZ. Statistical Estimation: Asymptotic Theory. New York: Springer; 1981.
Massart, P. Concentration Inequalities and Model Selection, vol. 1896, Lecture Notes in Mathematics, New York: Springer; 2007.
Rosenblatt, M. Curve estimates. Ann Math Stat 1971; 42: 1815–1842.
Schwartz, S. Estimation of probability density by an orthogonal series. Ann Math Stat 1967; 38: 1262–1265.
Scott, DW. Multivariate Density Estimation: Theory, Practice, and Visualization. New York: John Wiley %26 Sons; 1992.
Silverman, BW. Density Estimation for Statistics and Data Analysis. London: Chapman %26 Hall; 1986.
Tapia, RA, Thompson, JR. Nonparametric Probability Density Estimation. Baltimore: Johns Hopkins University Press; 1978.
Wegman, EJ. Nonparametric probability density estimation: I. A summary of available methods. Technometrics 1972; 14: 533–546.
Scott, DW. Histogram. WIREs Comp Stat 2010; 2(1): 44–48.
Čencov, NN. Evaluation of an unknown distribution density from observations. Soviet Math. Dokl. 1962; 3: 1559–1562.
Hart, JD. Nonparametric Smoothing and Lack‐of‐Fit Tests. New York: Springer; 1997.
Walter,, GG. Wavelets and other Orthogonal Systems with Applications. London: CRC Press; 1994.
Hall, P. Orthogonal series methods for both qualitative and quantitative data. Ann Stat 1983; 11: 1004–1007.
Hendriks, H. Nonparametric estimation of a probability density on a Riemannian manifold using Fourier expansions. Ann Stat 1990; 18: 832–849.
Wahba, G. Optimal convergence properties of variable knot, kernel, and orthogonal series methods for density estimation. Ann Stat 1975; 3: 15–29.
Watson, GS. Density estimation by orthogonal series. Ann Math Stat 1969; 38: 1262–1265.
Rosenblatt, M. Remarks on some non‐parametric estimates of a density function. Ann Math Stat 1956; 27: 832–837.
Hall, P. Estimating a density on the positive half line by the method of orthogonal series. Ann Inst Stat Math 1980; 32: 351–362.
Walter, GG. Properties of Hermite series estimation of probability density. Ann Stat 1977; 5: 1258–1264.
Hall, P. On trigonometric series estimates of densities. Ann Stat 1981; 9: 683–685.
Tarter, ME, Lock, MD. Model‐Free Curve Estimation. New York: Chapman %26 Hall; 1993.
Buckland, ST. Fitting density functions with polynomials. J R Stat Soc [Ser A] 1992; 41: 63–76.
Rudzkis, R, Radavicius, M. Adaptive estimation of distribution density in the basis of algebraic polynomials. Theory Probab Appl 2005; 49: 93–109.
Chicken, E, Cai, T. Block thresholding for density estimation: local and global adaptivity. J Multivar Anal 2005; 95: 75–106.
Donoho, D, Johnstone, I, Kerkyacharian, G, Picard, D. Density estimation by wavelet thresholding. Ann Stat 1996; 24: 508–539.
Härdle, W, Kerkyacharian, G, Picard, D, Tsybakov, A. Wavelets, Approximation, and Statistical Applications. New York: Springer; 1998.
Good, IJ, Gaskins, RA. Density estimation and bump‐hunting by penalized likelihood method exemplified by scattering and meteorite data, with discussion. J Am Stat Assoc 1980; 75: 42–73.
Wahba, G. Data‐based optimal smoothing of orthogonal series density estimates. Ann Stat 1981; 9: 146–156.
Tarter, ME, Kronmal, RA. An introduction to the implementation and theory of nonparametric density estimation. Am Stat 1976; 30: 105–112.
Diggle, PJ, Hall, P. The selection of terms in an orthogonal series density estimator. J Am Stat Assoc 1986; 81: 230–233.
Hart, JD. On the choice of truncation point in Fourier series density estimation. J Stat Comput Simul 1985; 21: 95–116.
Hall, P. Cross‐validation and the smoothing of orthogonal series density estimators. J Multivar Anal 1987; 21: 189–206.
Scott, DW, Terrell, GR. Biased and unbiased cross‐validation in density estimation. J Am Stat Assoc 1987; 82: 1131–1146.
Kronmal, RA, Tarter, ME. The estimation of probability densities and cumulatives by Fourier series methods. J Am Stat Assoc 1968; 63: 925–952.
Crain, BB. A note on density estimation using orthogonal expansions. Ann Stat 1973; 2: 454–463.
Efromovich, S. Nonparametric estimation of a density with unknown smoothness. Theory Probab Appl 1985; 30: 557–568.
Hall, P, Kerkyacharian, G, Picard, D. Block threshold rules for curve estimation using Kernel and wavelet methods. Ann Stat 1998; 26: 922–942.
Rigollet, P. Adaptive density estimation using the blockwise Stein method. Bernoulli (Andover) 2006; 12: 351–370.
Efromovich, S. Density estimation for the case of supersmooth measurement error. J Am Stat Assoc 1997; 92: 526–535.
Efromovich, S. Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Ann Stat 2008; 36: 1127–1155.
Efromovich, S. Adaptive orthogonal series density estimation for small samples. Comput Stat Data Anal 1996; 22: 599–617.
Catoni, O. Statistical Learning Theory and Stochastic Optimization, vol. 1851, Lecture Notes in Mathematics. New York: Springer; 2004.
Nemirovski, A. Topics in Non‐Parametric Statistics, vol. 1738, Lecture Notes in Mathematics, New York: Springer; 2000.
Rigollet, P, Tsybakov, A. Linear and convex aggregation of density estimators. Math Methods Stat 2007; 16: 260–280.
Samarov, A, Tsybakov, A. Aggregation of density estimators and dimension reduction. In: Nair, V, ed. Advances in Statistical Models and Inference, Essays in Honor of Kjell Doksum. Singapore: World Scientific; 233–251, 2007.
Yang, Y. Mixing strategies for density estimation. Ann Stat 2000; 28: 75–87.
Efromovich, S. Density estimation under random censorship and order restrictions: from asymptotic to small samples. J Am Stat Assoc 2001; 96: 667–685.
Gajek, L. On improving density estimators which are not bona fide functions. Ann Stat 1986; 14: 1612–1618.
Efromovich, S. Lower bound for estimation of Sobolev densities of order less 1/2. J Stat Plan Inference 2009; 139: 2261–2268.
Golubev, GK. Nonparametric estimation of smooth probability densities in L₂. Prob Inform Transm 1992; 28: 44–54.
Efromovich, S. Adaptive estimation of error density in nonparametric regression with small sample size. J Stat Plan Inference 2007; 137: 363–378.
Marron, JS, Wand, MP. Exact mean integrated squared error. Ann Stat 1992; 20: 712–736.
Efromovich, S. Density estimation for biased data. Ann Stat 2004; 32: 1137–1161.
Efromovich, S. Estimation of the density of regression errors. Ann Stat 2005; 33: 2194–2227.
Efromovich, S. Optimal nonparametric estimation of the density of regression errors with finite support. Ann Inst Stat Math 2007; 59: 617–654.
Efromovich, S. Conditional density estimation. Ann Stat 2007; 35: 2504–2535.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Orthogonal series density estimation

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox