How to cite this WIREs title:
WIREs Comp Stat

Differential equations in data analysis

Advanced Review

Itai Dattner

Published Online: Dec 01 2020

DOI: 10.1002/wics.1534

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract Differential equations have proven to be a powerful mathematical tool in science and engineering, leading to better understanding, prediction, and control of dynamic processes. In this paper, we review the role played by differential equations in data analysis. More specifically, we consider the intersection between differential equations and data analysis in the light of modern statistical learning methodologies. This article is categorized under: Data: Types and Structure > Time Series, Stochastic Processes, and Functional Data Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical Models > Nonlinear Models

References

Akaike,, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723.
Anderson,, R. M., & May,, R. M. (1992). Infectious diseases of humans: Dynamics and control. Great Clarendon Street Oxford, UK: Oxford University Press.
Anderson,, W. (2018). The compatibility of differential equations and causal models reconsidered. Erkenntnis, 85(2), 317–332. https://doi.org/10.1007/s10670-018-0029-1.
Arnold,, V. (1977). Ordinary differential equations. Berlin, Germany: Springer‐Verlag Berlin and Heidelberg GmbH %26 Co. KG.
Atkinson,, A. C., & Donev,, A. N. (1992). Optimum experimental designs. Oxford statistical science series. Wotton‐under‐Edge Gloucestershire, UK: Clarendon Press.
Audoly,, S., Bellu,, G., D`Angio,, L., Saccomani,, M. P., & Cobelli,, C. (2001). Global identifiability of nonlinear models of biological systems. Biomedical Engineering, IEEE Transactions on, 48(1), 55–65.
Baker,, R. E., Pena,, J.‐M., Jayamohan,, J., & Jérusalem,, A. (2018). Mechanistic models versus machine learning, a fight worth fighting for the biological community? Biology Letters, 14(5), 20170660.
Barrett,, A. B., & Barnett,, L. (2013). Granger causality is designed to measure effect, not mechanism. Frontiers in Neuroinformatics, 7, 6.
Bates,, D. M., & Watts,, D. G. (1988). Nonlinear regression analysis and its applications. Hoboken, NJ: John Wiley %26 Sons, Inc. https://www.wiley.com/en-us/globallocations.
Bellman,, R., & Åström,, K. J. (1970). On structural identifiability. Mathematical Biosciences, 7(3), 329–339.
Bellman,, R., & Roth,, R. S. (1971). The use of splines with unknown end points in the identification of systems. Journal of Mathematical Analysis and Applications, 34(1), 26–33.
Bellman,, R. E., & Kalaba,, R. E. (1965). Quasilinearization and nonlinear boundary‐value problems. Santa Monica, CA: Rand Corporation.
Bennett,, D., Silverstein,, S. M., & Niv,, Y. (2019). The two cultures of computational psychiatry. JAMA Psychiatry, 76(6), 563–564.
Bhaumik,, P., & Ghosal,, S. (2015). Bayesian two‐step estimation in differential equation models. Electronic Journal of Statistics, 9(2), 3124–3154.
Bhaumik,, P., & Ghosal,, S. (2017). Efficient Bayesian estimation and uncertainty quantification in ordinary differential equation models. Bernoulli, 23(4B), 3537–3570.
Bickel,, P. J., & Ritov,, Y. (2003). Nonparametric estimators which can be “plugged‐in”. The Annals of Statistics, 31(4), 1033–1053.
Blom,, T., Bongers,, S., & Mooij,, J. M. (2019). Beyond structural causal models: Causal constraints models. Proceedings of the Conference on Uncertainty in Artificial Intelligence, 1805, 06539.
Bolker,, B. M. (2008). Ecological models and data in R. Princeton, New Jersey: Princeton University Press.
Box,, G. E. (1976). Science and statistics. Journal of the American Statistical Association, 71(356), 791–799.
Breiman,, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16(3), 199–231.
Brewer,, D., Barenco,, M., Callard,, R., Hubank,, M., & Stark,, J. (2008). Fitting ordinary differential equations to short time course data. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1865), 519–544.
Brunel,, N. J., & Clairon,, Q. (2015). A tracking approach to parameter estimation in linear ordinary differential equations. Electronic Journal of Statistics, 9(2), 2903–2949.
Brunel,, N. J., Clairon,, Q., & d`Alché Buc,, F. (2014). Parametric estimation of ordinary differential equations with orthogonality conditions. Journal of the American Statistical Association, 109(505), 173–185.
Brunel,, N. J. B. (2008). Parameter estimation of ODE`s via nonparametric estimators. Electronic Journal of Statistics, 2, 1242–1267.
Brunton,, S. L., Proctor,, J. L., & Kutz,, J. N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15), 3932–3937.
Busetto,, A. G., Hauser,, A., Krummenacher,, G., Sunnåker,, M., Dimopoulos,, S., Ong,, C. S., … Buhmann,, J. M. (2013). Near‐optimal experimental design for model selection in systems biology. Bioinformatics, 29(20), 2625–2632.
Calderhead,, B., Girolami,, M., & Lawrence,, N. D. (2009). Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes. Advances in Neural Information Processing Systems, 21(NIPS 2008), 217–224.
Campbell,, D., & Chkrebtii,, O. (2013). Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates. Mathematical Biosciences, 246(2), 283–292.
Campbell,, D., & Lele,, S. (2014). An ANOVA test for parameter estimability using data cloning with application to statistical inference for dynamic systems. Computational Statistics and Data Analysis, 70, 257–267.
Campbell,, D., & Steele,, R. J. (2012). Smooth functional tempering for nonlinear differential equation models. Statistics and Computing, 22(2), 429–443.
Campbell,, D. A., Hooker,, G., & McAuley,, K. B. (2012). Parameter estimation in differential equation models with constrained states. Journal of Chemometrics, 26(6), 322–332.
Cao,, J., Fussmann,, G. F., & Ramsay,, J. O. (2008). Estimating a predator‐prey dynamical model with the parameter cascades method. Biometrics, 64(3), 959–967.
Cao,, J., Huang,, J. Z., & Wu,, H. (2012). Penalized nonlinear least squares estimation of time‐varying parameters in ordinary differential equations. Journal of Computational and Graphical Statistics, 21(1), 42–56.
Cao,, J., Wang,, L., & Xu,, J. (2011). Robust estimation for ordinary differential equation models. Biometrics, 67(4), 1305–1313.
Chakraborty,, A., Bingham,, D., Dhavala,, S. S., Kuranz,, C. C., Drake,, P. R., Grosskopf,, M. J., … Mallick,, B. K. (2017). Emulation of numerical models with over‐specified basis functions. Technometrics, 59(2), 153–164.others
Chang,, J.‐S., Li,, C.‐C., Liu,, W.‐L., & Deng,, J.‐H. (2015). Two‐stage parameter estimation applied to ordinary differential equation models. Journal of the Taiwan Institute of Chemical Engineers, 57, 26–35.
Chen,, J., & Wu,, H. (2008a). Efficient local estimation for time‐varying coefficients in deterministic dynamic models with applications to HIV‐1 dynamics. Journal of the American Statistical Association, 103(481), 369–384.
Chen,, J., & Wu,, H. (2008b). Estimation of time‐varying parameters in deterministic dynamic models. Statistica Sinica, 18(3), 987–1006.
Chen,, J.‐F. (2013). State space models and differential equations for dynamic gene regulatory network identification. (Doctoral dissertation). University of Rochester, New York. Retrieved from http://hdl.handle.net/1802/27874.
Chen,, S., Shojaie,, A., & Witten,, D. M. (2017). Network reconstruction from high‐dimensional ordinary differential equations. Journal of the American Statistical Association, 112(520), 1697–1707.
Chen,, T. Q., Rubanova,, Y., Bettencourt,, J., & Duvenaud,, D. K. (2018). Neural ordinary differential equations. Advances in Neural Information Processing Systems, 31(NIPS 2018), 6571–6583.
Chkrebtii,, O. A., Campbell,, D. A., Calderhead,, B., & Girolami,, M. A. (2016). Bayesian solution uncertainty quantification for differential equations. Bayesian Analysis, 11(4), 1239–1267.
Chou,, I.‐C., Martens,, H., & Voit,, E. O. (2006). Parameter estimation in biochemical systems models with alternating regression. Theoretical Biology and Medical Modelling, 3(1), 25.
Chou,, I.‐C., & Voit,, E. O. (2009). Recent developments in parameter estimation and structure identification of biochemical and genomic systems. Mathematical Biosciences, 219(2), 57–83.
Chow,, S.‐M., Bendezú,, J. J., Cole,, P. M., & Ram,, N. (2016). A comparison of two‐stage approaches for fitting nonlinear ordinary differential equation models with mixed effects. Multivariate Behavioral Research, 51(2–3), 154–184.
Claeskens,, G., & Hjort,, N. L. (2008). Model selection and model averaging. Cambridge, UK: Cambridge University Press.
Clairon,, Q., & Brunel,, N. J.‐B. (2018). Optimal control and additive perturbations help in estimating ill‐posed and uncertain dynamical systems. Journal of the American Statistical Association, 113(523), 1195–1209.
Clairon,, Q., & Brunel,, N. J.‐B. (2019). Tracking for parameter and state estimation in possibly misspecified partially observed linear ordinary differential equations. Journal of Statistical Planning and Inference, 199, 188–206.
Cobelli,, C., & Distefano,, J. J. (1980). Parameter and structural identifiability concepts and ambiguities: A critical review and analysis. American Journal of Physiology. Regulatory, Integrative and Comparative Physiology, 239(1), R7–R24.
Coddington,, E. A. (1989). An introduction to ordinary differential equations. Wellman Avenue, Chelmsford, MA: Courier Corporation.
Coddington,, E. A., & Levinson,, N. (1955). Theory of ordinary differential equations. New Delhi: Tata McGraw‐Hill Publishing Company Limited.
Conrad,, P. R., Girolami,, M., Särkkä,, S., Stuart,, A., & Zygalakis,, K. (2017). Statistical analysis of differential equations: Introducing probability measures on numerical solutions. Statistics and Computing, 27(4), 1065–1082.
Daniels,, B. C., & Nemenman,, I. (2015). Automated adaptive inference of phenomenological dynamical models. Nature Communications, 6, 8133.
Dass,, S. C., Lee,, J., Lee,, K., & Park,, J. (2017). Laplace based approximate posterior inference for differential equation models. Statistics and Computing, 27(3), 679–698.
Dattner,, I. (2015). A model‐based initial guess for estimating parameters in systems of ordinary differential equations. Biometrics, 71(4), 1176–1184.
Dattner,, I., & Gugushvili,, S. (2018). Application of one‐step method to parameter estimation in ODE models. Statistica Neerlandica, 72(2), 126–156.
Dattner,, I., & Huppert,, A. (2018). Modern statistical tools for inference and prediction of infectious diseases using mathematical models. Statistical Methods in Medical Research, 27(7), 1927–1929.
Dattner,, I., & Klaassen,, C. A. J. (2015). Optimal rate of direct estimators in systems of ordinary differential equations linear in functions of the parameters. Electronic Journal of Statistics, 9(2), 1939–1973.
Dattner,, I., Miller,, E., Petrenko,, M., Kadouri,, D. E., Jurkevitch,, E., & Huppert,, A. (2017). Modelling and parameter inference of predator–prey dynamics in heterogeneous environments using the direct integral approach. Journal of the Royal Society Interface, 14(126), 20160525.
Dattner, I., Ship, H., Voit, E. O. (2020). Separable nonlinear least‐squares parameter estimation for complex dynamic systems. Complexity, 2020, 1–11. https://doi.org/10.1155/2020/6403641.
de Bazelaire,, C., Siauve,, N., Fournier,, L., Frouin,, F., Robert,, P., Clement,, O., … Cuenod,, C. A. (2005). Comprehensive model for simultaneous MRI determination of perfusion and permeability using a blood‐pool agent in rats rhabdomyosarcoma. European Radiology, 15(12), 2497–2505.
Dean,, J., & Ghemawat,, S. (2004). MapReduce: Simplified data processing on large clusters. OSDI`04: Sixth symposium on operating system design and implementation (pp. 137–150).
Denis‐Vidal,, L., Joly‐Blanchard,, G., & Noiret,, C. (2003). System identifiability (symbolic computation) and parameter estimation (numerical computation). Numerical Algorithms, 34(2–4), 283–292.
Ding,, A. A., & Wu,, H. (2014). Estimation of ordinary differential equation parameters using constrained local polynomial regression. Statistica Sinica, 24(4), 1613–1631.
Dondelinger,, F., Filippone,, M., Rogers,, S., & Husmeier,, D. (2013). ODE parameter inference using adaptive gradient matching with Gaussian processes. Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, 31, 216–228.
Dony,, L., He,, F., & Stumpf,, M. P. (2019). Parametric and non‐parametric gradient matching for network inference: A comparison. BMC Bioinformatics, 20(1), 52.
Douc,, R., Moulines,, E., & Stoffer,, D. (2014). Nonlinear time series: Theory, methods and applications with R examples. Boca Raton, FL: Chapman and Hall/CRC.
Durbin,, J., & Koopman,, S. (2001). Time series analysis by state space methods. Oxford, UK: Oxford University Press.
Duriez,, T., Brunton,, S. L., & Noack,, B. R. (2017). Machine learning control—Taming nonlinear dynamics and turbulence (Vol. 116). Springer Nature Switzerland AG: Springer.
Edelstein‐Keshet,, L. (2005). Mathematical models in biology (Vol. 46). Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM).
Fan,, J., & Yao,, Q. (2008). Nonlinear time series: Nonparametric and parametric methods. New York: Springer Science %26 Business Media.
Fang,, Y., Wu,, H., & Zhu,, L.‐X. (2011). A two‐stage estimation method for random coefficient differential equation models with application to longitudinal HIV dynamic data. Statistica Sinica, 21(3), 1145–1170.
Faraji,, M., & Voit,, E. O. (2017). Nonparametric dynamic modeling. Mathematical Biosciences, 287, 130–146.
FitzHugh,, R. (1961). Impulses and physiological states in theoretical models of nerve membrane. Biophysical Journal, 1(6), 445–466.
Franceschini,, G., & Macchietto,, S. (2008). Model‐based design of experiments for parameter precision: State of the art. Chemical Engineering Science, 63(19), 4846–4872.
Frasso,, G., Jaeger,, J., & Lambert,, P. (2016). Inference in dynamic systems using B‐splines and quasilinearized ODE penalties. Biometrical Journal, 58(3), 691–714.
Gábor,, A., & Banga,, J. R. (2014). Improved parameter estimation in kinetic models: Selection and tuning of regularization methods. In: P. Mendes, J. O. Dada, K. Smallbone (eds) Computational Methods in Systems Biology. CMSB 2014. Lecture Notes in Computer Science, vol 8859. Springer, Cham. https://doi.org/10.1007/978-3-319-12982-2_4
Ghosh,, S., Dasmahapatra,, S., & Maharatna,, K. (2017). Fast approximate Bayesian computation for estimating parameters in differential equations. Statistics and Computing, 27(1), 19–38.
Girolami,, M. (2008). Bayesian inference for differential equations. Theoretical Computer Science, 408(1), 4–16.
Goldstein,, L., & Messer,, K. (1992). Optimal plug‐in estimators for nonparametric functional estimation. The Annals of Statistics, 20, 1306–1328.
Golub,, G., & Pereyra,, V. (2003). Separable nonlinear least squares: The variable projection method and its applications. Inverse Problems, 19(2), 1–26.
Golub,, G. H., & Pereyra,, V. (1973). The differentiation of pseudo‐inverses and nonlinear least squares problems whose variables separate. SIAM Journal on Numerical Analysis, 10(2), 413–432.
González,, J., Vujačić,, I., & Wit,, E. (2014). Reproducing kernel Hilbert space based estimation of systems of ordinary differential equations. Pattern Recognition Letters, 45, 26–32.
Goodfellow,, I., Bengio,, Y., & Courville,, A. (2016). Deep learning. Cambridge, Massachusetts; London, England: MIT Press.
Granger,, C. W. (1969). Investigating causal relations by econometric models and cross‐spectral methods. Econometrica: Journal of the Econometric Society, 37(3), 424–438.
Grant,, M. J., & Booth,, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108.
Gugushvili,, S., & Klaassen,, C. A. J. (2012). $\sqrt{n}$ ‐consistent parameter estimation for systems of ordinary differential equations: Bypassing numerical integration via smoothing. Bernoulli, 18(3), 1061–1098.
Guha,, N., Wu,, X., Efendiev,, Y., Jin,, B., & Mallick,, B. K. (2015). A variational Bayesian approach for inverse problems with skew‐t error distributions. Journal of Computational Physics, 301, 377–393.
Hass,, H., Kreutz,, C., Timmer,, J., & Kaschek,, D. (2015). Fast integration‐based prediction bands for ordinary differential equation models. Bioinformatics, 32(8), 1204–1210.
Hemker,, P. (1972). Numerical methods for differential equations in system simulation and in parameter estimation. Analysis and Simulation of Biochemical Systems, 28, 59–80.
Henderson,, J., & Michailidis,, G. (2014). Network reconstruction using nonparametric additive ODE models. PLoS One, 9(4), e94003.
Himmelblau,, D., Jones,, C., & Bischoff,, K. (1967). Determination of rate constants for complex kinetics models. Industrial and Engineering Chemistry Fundamentals, 6(4), 539–543.
Hockin,, M. F., Jones,, K. C., Everse,, S. J., & Mann,, K. G. (2002). A model for the stoichiometric regulation of blood coagulation. Journal of Biological Chemistry, 277(21), 18322–18333.
Holder,, A. B., & Rodrigo,, M. R. (2013). An integration‐based method for estimating parameters in a system of differential equations. Applied Mathematics and Computation, 219(18), 9700–9708.
Hong,, Z., & Lian,, H. (2012). Time‐varying coefficient estimation in differential equation models with noisy time‐varying covariates. Journal of Multivariate Analysis, 103(1), 58–67.
Hooker,, G. (2009). Forcing function diagnostics for nonlinear dynamics. Biometrics, 65(3), 928–936.
Hooker,, G., & Ellner,, S. P. (2015). Goodness of fit in nonlinear dynamics: Misspecified rates or misspecified states? The Annals of Applied Statistics, 9(2), 754–776.
Hooker,, G., Ellner,, S. P., Roditi,, L. D. V., & Earn,, D. J. (2011). Parameterizing state–space models for infectious disease dynamics by generalized profiling: Measles in Ontario. Journal of the Royal Society Interface, 8(60), 961–974.
Hooker,, G., Ramsay,, J., & Xiao,, L. (2016). CollocInfer: Collocation inference in differential equation models. Journal of Statistical Software, 75(2), 1–52. Retreived from https://www.jstatsoft.org/v075/i02
Hu,, T., Qiu,, Y., & Cui,, H. (2015). Robust estimation of constant and time‐varying parameters in nonlinear ordinary differential equation models. Journal of Nonparametric Statistics, 27(3), 349–371.
Huang,, H., Handel,, A., & Song,, X. (2020). A Bayesian approach to estimate parameters of ordinary differential equation. Computational Statistics, 35, 1481–1499. https://doi.org/10.1007/s00180-020-00962-8
Huber,, P. J. (1979). Robust smoothing. In R. L. Launer, & G. N. Wilkinson, (Eds.), Robustness in statistics (pp. 33–47). New York: Academic Press.
Huber,, P. J. (1981). Robust statistics. Hoboken, New Jersey: Wiley.
Ilya,, G. (2006). Optimal control and forecasting of complex dynamical systems. Singapore: World Scientific.
Jaeger,, J., & Lambert,, P. (2014). Bayesian penalized smoothing approaches in models specified using differential equations with unknown error distributions. Journal of Applied Statistics, 41(12), 2709–2726.
Jeffrey,, A. M., & Xia,, X. (2005). Identifiability of HIV/AIDS model. In W. Y. Tan, & H. Wu, (Eds.), Deterministic and stochastic models of AIDS epidemics and HIV infections with intervention (pp. 255–286). Singapore: World Scientific.
Jennrich,, R. I. (1969). Asymptotic properties of non‐linear least squares estimators. The Annals of Mathematical Statistics, 40(2), 633–643.
Jia,, G., Stephanopoulos,, G. N., & Gunawan,, R. (2011). Parameter estimation of kinetic models from metabolic profiles: Two‐phase dynamic decoupling method. Bioinformatics, 27(14), 1964–1970.
Jin,, B. (2012). A variational Bayesian method to inverse problems with impulsive noise. Journal of Computational Physics, 231(2), 423–435.
Jin,, B., & Zou,, J. (2010). Hierarchical Bayesian inference for ill‐posed problems via variational method. Journal of Computational Physics, 229(19), 7317–7343.
Jordan,, M. I. (2018). Dynamical, symplectic and stochastic perspectives on gradient‐based optimization. Proceedings of the International Congress of Mathematicians, 2018(1), 523–550.
Kalman,, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1), 35–45.
Keeling,, M. J., & Rohani,, P. (2008). Modeling infectious diseases in humans and animals. Princeton, NJ: Princeton University Press.
Kennedy,, M. C., Anderson,, C. W., Conti,, S., & O`Hagan,, A. (2006). Case studies in Gaussian process modelling of computer codes. Reliability Engineering %26 System Safety, 91(10–11), 1301–1309. https://www.jstatsoft.org/v069/i12
King,, A., Nguyen,, D., & Ionides,, E. (2016). Statistical inference for partially observed Markov processes via the R package pomp. Journal of Statistical Software, 69(12), 1–43. Retreived from https://www.jstatsoft.org/v069/i12
Kirk,, P., Thorne,, T., & Stumpf,, M. P. (2013). Model selection in systems and synthetic biology. Current Opinion in Biotechnology, 24(4), 767–774.
Koopman,, B. O. (1931). Hamiltonian systems and transformation in Hilbert spaces. Proceedings of the National Academy of Sciences of the United States of America, 17(5), 315–318.
Kreutz,, C. (2018). An easy and efficient approach for testing identifiability. Bioinformatics, 34(11), 1913–1921.
Kreutz,, C., Raue,, A., Kaschek,, D., & Timmer,, J. (2013). Profile likelihood in systems biology. The FEBS Journal, 280(11), 2564–2571.
Kreutz,, C., Raue,, A., & Timmer,, J. (2012). Likelihood based observability analysis and confidence intervals for predictions of dynamic models. BMC Systems Biology, 6(1), 120.
Kreutz,, C., & Timmer,, J. (2009). Systems biology: Experimental design. The FEBS Journal, 276(4), 923–942.
Lawton,, W. H., & Sylvestre,, E. A. (1971). Elimination of linear parameters in nonlinear regression. Technometrics, 13(3), 461–467.
Leander,, J., Lundh,, T., & Jirstrand,, M. (2014). Stochastic differential equations as a tool to regularize the parameter estimation problem for continuous time dynamical systems given discrete time measurements. Mathematical Biosciences, 251, 54–62.
Lee,, K., Lee,, J., & Dass,, S. C. (2018). Inference for differential equation models using relaxation via dynamical systems. Computational Statistics and Data Analysis, 127, 116–134.
Letham,, B., Letham,, P. A., Rudin,, C., & Browne,, E. P. (2016). Prediction uncertainty and optimal experimental design for learning dynamical systems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 26(6), 063110.
Letham,, B., Letham,, P. A., Rudin,, C., & Browne,, E. P. (2017). Erratum: “Prediction uncertainty and optimal experimental design for learning dynamical systems” [Chaos 26, 063110 (2016)]. Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(6), 069901.
Li,, Y., Zhu,, J., & Wang,, N. (2015). Regularized semiparametric estimation for ordinary differential equations. Technometrics, 57(3), 341–350.
Liang,, H., Miao,, H., & Wu,, H. (2010). Estimation of constant and time‐varying dynamic parameters of HIV infection in a nonlinear differential equation model. The Annals of Applied Statistics, 4(1), 460–483.
Liang,, H., & Wu,, H. (2008). Parameter estimation for differential equation models using a framework of measurement error in regression models. Journal of the American Statistical Association, 103(484), 1570–1583.
Liepe,, J., Filippi,, S., Komorowski,, M., & Stumpf,, M. P. (2013). Maximizing the information content of experiments in systems biology. PLoS Computational Biology, 9(1), Article e1002888.
Liepe,, J., Kirk,, P., Filippi,, S., Toni,, T., Barnes,, C. P., & Stumpf,, M. P. (2014). A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation. Nature Protocols, 9(2), 439–456.
Lill,, D., Timmer,, J., & Kaschek,, D. (2019). Local Riemannian geometry of model manifolds and its implications for practical parameter identifiability. PLoS One, 14(6), e0217837.
Liu,, B., Wang,, L., Nie,, Y., & Cao,, J. (2019). Bayesian inference of mixed‐effects ordinary differential equations models using heavy‐tailed distributions. Computational Statistics and Data Analysis, 137, 233–246.
Ljung,, L., & Glad,, T. (1994). On global identifiability for arbitrary model parametrizations. Automatica, 30(2), 265–276.
Long,, Z., Lu,, Y., Ma,, X., & Dong,, B. (2017). PDE‐net: Learning PDEs from data. arXiv preprint arXiv:1710.09668.
Loskot,, P., Atitey,, K., & Mihaylova,, L. (2019). Comprehensive review of models and methods for inferences in bio‐chemical reaction networks. Frontiers in Genetics, 10, 549.
Lotka,, A. J. (1958). Elements of mathematical biology. New York: Dover Publications, Inc.
Lu,, T., Liang,, H., Li,, H., & Wu,, H. (2011). High‐dimensional ODEs coupled with mixed‐effects modeling techniques for dynamic gene regulatory network identification. Journal of the American Statistical Association, 106(496), 1242–1258.
Lusch,, B., Kutz,, J. N., & Brunton,, S. L. (2018). Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9(1), 1–10.
Macdonald,, B., Higham,, C., & Husmeier,, D. (2015). Controversy in mechanistic modelling with Gaussian processes. Proceedings of Machine Learning Research, 37, 1539–1547.
Macdonald,, B., Niu,, M., Rogers,, S., Filippone,, M., & Husmeier,, D. (2016). Approximate parameter inference in systems biology using gradient matching: A comparative evaluation. Biomedical Engineering Online, 15(1), 80.
Mangan,, N. M., Kutz,, J. N., Brunton,, S. L., & Proctor,, J. L. (2017). Model selection for dynamical systems via sparse regression and information criteria. Proceedings of the Royal Society A, 473(2204), 20170009.
Martin,, B. T., Munch,, S. B., & Hein,, A. M. (2018). Reverse‐engineering ecological theory from data. Proceedings of the Royal Society B, 285, 20180422.
May,, R. M. (2001). Stability and complexity in model ecosystems, with a new introduction by the author (2nd ed.). Princeton, NJ: Princeton University Press.
McGoff,, K., Mukherjee,, S., & Pillai,, N. (2015). Statistical inference for dynamical systems: A review. Statistics Surveys, 9, 209–252.
McLean,, K. A., Wu,, S., & McAuley,, K. B. (2012). Mean‐squared‐error methods for selecting optimal parameter subsets for estimation. Industrial %26 Engineering Chemistry Research, 51(17), 6105–6115.
Meng,, L., Zhang,, J., Zhang,, X., & Feng,, G. (2019). Bayesian estimation of time‐varying parameters in ordinary differential equation models with noisy time‐varying covariates. Communications in Statistics: Simulation and Computation, 1–16. https://doi.org/10.1080/03610918.2019.1565584
Miao,, H., Dykes,, C., Demeter,, L. M., Cavenaugh,, J., Park,, S. Y., Perelson,, A. S., & Wu,, H. (2008). Modeling and estimation of kinetic parameters and replicative fitness of HIV‐1 from flow‐cytometry‐based growth competition experiments. Bulletin of Mathematical Biology, 70(6), 1749–1771.
Miao,, H., Dykes,, C., Demeter,, L. M., & Wu,, H. (2009). Differential equation modeling of HIV viral fitness experiments: Model identification, model selection, and multimodel inference. Biometrics, 65(1), 292–300.
Miao,, H., Wu,, H., & Xue,, H. (2014). Generalized ordinary differential equation models. Journal of the American Statistical Association, 109(508), 1672–1682.
Miao,, H., Xia,, X., Perelson,, A. S., & Wu,, H. (2011). On identifiability of nonlinear ODE models and applications in viral dynamics. SIAM Review, 53(1), 3–39.
Mikkelsen,, F. V., & Hansen,, N. R. (2017). Learning large scale ordinary differential equation systems. arXiv preprint arXiv:1710.09308.
Muehlebach, M. %26 Jordan, M. (2019). A Dynamical Systems Perspective on Nesterov Acceleration. Proceedings of the 36th International Conference on Machine Learning, in PMLR, 97, 4656–4662
Muino,, J., Voit,, E. O., & Sorribas,, A. (2006). GS‐distributions: A new family of distributions for continuous unimodal variables. Computational Statistics and Data Analysis, 50(10), 2769–2798.
Murphy,, S. A., & Van der Vaart,, A. W. (2000). On profile likelihood. Journal of the American Statistical Association, 95(450), 449–465.
Nagumo,, J., Arimoto,, S., & Yoshizawa,, S. (1962). An active pulse transmission line simulating nerve axon. Proceedings of the IRE, 50(10), 2061–2070.
Niu,, M., Macdonald,, B., Rogers,, S., Filippone,, M., & Husmeier,, D. (2018). Statistical inference in mechanistic models: Time warping for improved gradient matching. Computational Statistics, 33(2), 1091–1123.
Niu,, M., Rogers,, S., Filippone,, M., & Husmeier,, D. (2016). Fast inference in nonlinear dynamical systems using gradient matching. Proceedings of Machine Learning Research, 48, 1699–1707.
Nowak,, M. A., & May,, R. M. (2000). Virus dynamics: Mathematical principles of immunology and virology. Oxford, UK: Oxford University Press.
Overstall,, A. M., Woods,, D. C., & Parker,, B. M. (2019). Bayesian optimal design for ordinary differential equation models with application in biological science. Journal of the American Statistical Association, 115(530), 583–598.
Pantazis,, Y., & Tsamardinos,, I. (2019). A unified approach for sparse dynamical system inference from temporal measurements. Bioinformatics, 35(18), 3387–3396.
Pearl,, J. (2009). Causality. Cambridge, UK: Cambridge University Press.
Pearson,, K. (1895). X. contributions to the mathematical theory of evolution.—II. Skew variation in homogeneous material. Philosophical Transactions of the Royal Society of London A, 186, 343–414.
Perretti,, C. T., Munch,, S. B., & Sugihara,, G. (2013). Reply to Hartig and Dormann: The true model myth. Proceedings of the National Academy of Sciences, 110(42), E3976–E3977. Retrevied from http://www.pnas.org/content/110/42/E3976.short
Peschel,, M., & Mende,, W. (1986). The predator‐prey model: Do we live in a Volterra world? Wien, New York: Springer‐Verlag.
Qi,, X., & Zhao,, H. (2010). Asymptotic efficiency and finite‐sample properties of the generalized profiling estimation of parameters in ordinary differential equations. The Annals of Statistics, 38(1), 435–481.
Qiu,, Y., Hu,, T., Liang,, B., & Cui,, H. (2016). Robust estimation of parameters in nonlinear ordinary differential equation models. Journal of Systems Science and Complexity, 29(1), 41–60.
Quaiser,, T., & Mönnigmann,, M. (2009). Systematic identifiability testing for unambiguous mechanistic modeling–application to JAK‐STAT, MAP kinase, and NF‐κ B signaling pathway models. BMC Systems Biology, 3(1), 50.
Raissi,, M., & Karniadakis,, G. E. (2018). Hidden physics models: Machine learning of nonlinear partial differential equations. Journal of Computational Physics, 357, 125–141.
Ramsay,, J., & Hooker,, G. (2017). Dynamic data analysis. New York: Springer‐Verlag.
Ramsay,, J., Ramsay,, J., & Silverman,, B. (2005). Functional data analysis. New York: Springer Science %26 Business Media.
Ramsay,, J. O., Hooker,, G., Campbell,, D., & Cao,, J. (2007). Parameter estimation for differential equations: A generalized smoothing approach. Journal of the Royal Statistical Society, Series B (Methodology), 69(5), 741–796.
Ranciati,, S., Wit,, E. C., & Viroli,, C. (2019). Bayesian smooth‐and‐match inference for ordinary differential equations models linear in the parameters. Statistica Neerlandica, 74(2), 125–144.
Raue,, A., Becker,, V., Klingmüller,, U., & Timmer,, J. (2010). Identifiability and observability analysis for experimental design in nonlinear dynamical models. Chaos: An Interdisciplinary Journal of Nonlinear Science, 20(4), 045105.
Raue,, A., Kreutz,, C., Maiwald,, T., Bachmann,, J., Schilling,, M., Klingmüller,, U., & Timmer,, J. (2009). Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics, 25(15), 1923–1929.
Raue,, A., Kreutz,, C., Maiwald,, T., Klingmüller,, U., & Timmer,, J. (2011). Addressing parameter identifiability by model‐based experimentation. IET Systems Biology, 5(2), 120–130.
Raue,, A., Kreutz,, C., Theis,, F. J., & Timmer,, J. (2013). Joining forces of Bayesian and frequentist methodology: A study for inference in the presence of non‐identifiability. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371, 20110544. https://doi.org/10.1098/rsta.2011.0544
Raue,, A., Steiert,, B., Schelker,, M., Kreutz,, C., Maiwald,, T., Hass,, H., … Timmer,, J. (2015). Data2Dynamics: A modeling environment tailored to parameter estimation in dynamical systems. Bioinformatics, 31(21), 3558–3560.
Ripley,, B. D. (2004). Selecting amongst large classes of models. In N. Adams,, M. Crowder,, D. J. Hand,, & D. Stephens, (Eds.), Methods and models in statistics (pp. 155–170). Covent Garden, London, UK: Imperial College Press. https://doi.org/10.1142/9781860945410_0007
Rubenstein,, P. K., Bongers,, S., Schölkopf,, B., & Mooij,, J. M. (2016). From deterministic ODEs to dynamic structural causal models. arXiv preprint arXiv:1608.08028.
Rust,, P. F., & Voit,, E. O. (1990). Statistical densities, cumulatives, quantiles, and power obtained by S‐system differential equations. Journal of the American Statistical Association, 85(410), 572–578.
Savageau,, M. A. (1982). A suprasystem of probability distributions. Biometrical Journal, 24(4), 323–330.
Schaeffer,, H. (2017). Learning partial differential equations via data discovery and sparse optimization. Proceedings of the Royal Society A, 473(2197), 20160446.
Schaeffer,, H., Tran,, G., & Ward,, R. (2018). Extracting sparse high‐dimensional dynamics from limited data. SIAM Journal on Applied Mathematics, 78(6), 3279–3295.
Schittkowski,, K. (2002). Numerical data fitting in dynamical systems: A practical introduction with applications and software. Dordrecht, The Netherlands: Kluwer Academic Publishers.
Schmidt,, M., & Lipson,, H. (2009). Distilling free‐form natural laws from experimental data. Science, 324(5923), 81–85.
Seber,, G. A. F., & Wild,, C. J. (2003). Nonlinear regression. Hoboken, New Jersey: John Wiley %26 Sons.
Shmueli,, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310. https://doi.org/10.1214/10-STS330
Shulkind,, G., Horesh,, L., & Avron,, H. (2018). Experimental design for nonparametric correction of misspecified dynamical models. SIAM/ASA Journal on Uncertainty Quantification, 6(2), 880–906.
Silverman,, B. W. (1982). On the estimation of a probability density function by the maximum penalized likelihood method. The Annals of Statistics, 10(3), 795–810.
Silverman,, B. W. (1986). Density estimation for statistics and data analysis (Vol. 26). Boca Raton, FL: Chapman and Hall/CRC.
Srinath,, S., & Gunawan,, R. (2010). Parameter identifiability of power‐law biochemical system models. Journal of Biotechnology, 149(3), 132–140.
Strebel,, O. (2013). A preprocessing method for parameter estimation in ordinary differential equations. Chaos, Solitons %26 Fractals, 57, 93–104.
Strogatz,, S. H. (2018). Nonlinear dynamics and chaos: With applications to physics, biology, chemistry, and engineering. Boca Raton, FL: Chapman and Hall/CRC.
Sugihara,, G., May,, R., Ye,, H., Hsieh,, C.‐h., Deyle,, E., Fogarty,, M., & Munch,, S. (2012). Detecting causality in complex ecosystems. Science, 338(6106), 496–500.
Sun,, J., Garibaldi,, J. M., & Hodgman,, C. (2011). Parameter estimation using metaheuristics in systems biology: A comprehensive review. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 9(1), 185–202.
Swartz,, J., & Bremermann,, H. (1975). Discussion of parameter estimation in biological modelling: Algorithms for estimation and evaluation of the estimates. Journal of Mathematical Biology, 1(3), 241–257.
Tan,, Q., & Ghosal,, S. (2019). Bayesian analysis of mixed‐effect regression models driven by ordinary differential equations. Sankhya B. https://doi.org/10.1007/s13571-019-00199-6
Tank,, D., Regehr,, W., & Delaney,, K. (1995). A quantitative analysis of presynaptic calcium dynamics that contribute to short‐term enhancement. The Journal of Neuroscience, 15(12), 7940–7952.
Teijeiro,, D., Pardo,, X. C., Penas,, D. R., González,, P., Banga,, J. R., & Doallo,, R. (2017). A cloud‐based enhanced differential evolution algorithm for parameter estimation problems in computational systems biology. Cluster Computing, 20(3), 1937–1950.
Tenenbaun,, M., & Pollard,, H. (1963). Ordinary differential equations. Garden City, NY: Dover Publications.
Tong,, H. (1990). Non‐linear time series: A dynamical system approach. Oxford, UK: Oxford University Press.
Toni,, T., & Stumpf,, M. P. (2010). Simulation‐based model selection for dynamical systems in systems and population biology. Bioinformatics, 26(1), 104–110.
Toni,, T., Welch,, D., Strelkowa,, N., Ipsen,, A., & Stumpf,, M. P. (2009). Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of the Royal Society Interface, 6(31), 187–202.
Tönsing,, C., Timmer,, J., & Kreutz,, C. (2018). Profile likelihood‐based analyses of infectious disease models. Statistical Methods in Medical Research, 27(7), 1979–1998.
Tönsing,, C., Timmer,, J., & Kreutz,, C. (2019). Optimal paths between parameter estimates in non‐linear ODE systems using the nudged elastic band method. Frontiers in Physics, 7, 149.
Transtrum,, M. K., & Qiu,, P. (2016). Bridging mechanistic and phenomenological models of complex biological systems. PLoS Computational Biology, 12(5), e1004915.
Van der Vaart,, A. W. (2000). Asymptotic statistics (Vol. 3). Cambridge, UK: Cambridge University Press.
Vanlier,, J., Tiemann,, C., Hilbers,, P., & Van Riel,, N. (2013). Parameter uncertainty in biochemical models described by ordinary differential equations. Mathematical Biosciences, 246(2), 305–314.
Varah,, J. (1982). A spline least squares method for numerical parameter estimation in differential equations. SIAM Journal on Scientific and Statistical Computing, 3(1), 28–46.
Varanasi,, S. K., & Jampana,, P. (2018). Identification of parsimonious continuous time LTI models with applications. Journal of Process Control, 69, 128–137.
Velten,, K. (2009). Mathematical modeling and simulation: Introduction for scientists and engineers. Weinheim, Germany: Wiley‐VCH Verlag GmbH %26 Co. KGaA.
Voit,, E. O. (1992). The S‐distribution. A tool for approximation and classification of univariate, unimodal probability distributions. Biometrical Journal, 34(7), 855–878.
Voit,, E. O. (2000). Computational analysis of biochemical systems: A practical guide for biochemists and molecular biologists. Cambridge, UK: Cambridge University Press.
Voit,, E. O., & Almeida,, J. (2004). Decoupling dynamical systems for pathway identification from metabolic profiles. Bioinformatics, 20(11), 1670–1681.
Volterra,, V. (1926). Fluctuations in the abundance of a species considered mathematically. Nature, 118, 558–560.
Voss,, H. U., Timmer,, J., & Kurths,, J. (2004). Nonlinear dynamical system identification from uncertain and indirect measurements. International Journal of Bifurcation and Chaos, 14(06), 1905–1933.
Vujačić,, I., & Dattner,, I. (2018). Consistency of direct integral estimator for partially observed systems of ordinary differential equations. Statistics %26 Probability Letters, 132, 40–45.
Vujačić,, I., Dattner,, I., González,, J., & Wit,, E. (2015). Time‐course window estimator for ordinary differential equations linear in the parameters. Statistics and Computing, 25(6), 1057–1070.
Vujačić,, I., Mahmoudi,, S. M., & Wit,, E. (2016). Generalized Tikhonov regularization in estimation of ordinary differential equations models. Stat, 5(1), 132–143.
Wang,, L., Cao,, J., Ramsay,, J., Burger,, D., Laporte,, C., & Rockstroh,, J. (2014). Estimating mixed‐effects differential equation models. Statistics and Computing, 24(1), 111–121.
Wang,, W.‐X., Lai,, Y.‐C., & Grebogi,, C. (2016). Data based identification and prediction of nonlinear and complex dynamical systems. Physics Reports, 644, 1–76.
Wasserman,, L. (2013). All of statistics: A concise course in statistical inference. New York: Springer Science %26 Business Media.
Weber,, M. (2016). On the incompatibility of dynamical biological mechanisms and causal graphs. Philosophy of Science, 83(5), 959–971.
Wei,, B., Xie,, N., & Yang,, L. (2020). Understanding cumulative sum operator in grey prediction model with integral matching. Communications in Nonlinear Science and Numerical Simulation, 82, 105076.
White,, C. R., & Marshall,, D. J. (2019). Should we care if models are phenomenological or mechanistic? Trends in Ecology %26 Evolution, 34(4), 276–278.
Wibisono,, A., Wilson,, A. C., & Jordan,, M. I. (2016). A variational perspective on accelerated methods in optimization. Proceedings of the National Academy of Sciences, 113(47), E7351–E7358.
Wilkinson,, D. (2011). Stochastic Modelling for Systems Biology, second edition. 6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487‐2742: Chapman %26 Hall/CRC.
Wit,, E., van den Heuvel,, E., & Romeijn,, J.‐W. (2012). ‘All models are wrong…’: An introduction to model uncertainty. Statistica Neerlandica, 66(3), 217–236.
Wood,, S. N. (2010). Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466(7310), 1102–1104.
Wu, C., Miao, O., Warnes, G. R., Wu, C., LeBlanc, A., Dykes, C., %26 Demeter, L. M. (2008). DEDiscover: A computation and simulation tool for HIV viral fitness research, in 2008 International Conference on Biomedical Engineering and Informatics (BMEI 2008), Sanya, (pp. 687–694). https://doi.org/10.1109/BMEI.2008.288
Wu,, C.‐F. (1981). Asymptotic theory of nonlinear least squares estimation. The Annals of Statistics, 9(3), 501–513.
Wu,, H., Lu,, T., Xue,, H., & Liang,, H. (2014). Sparse additive ordinary differential equations for dynamic gene regulatory network modeling. Journal of the American Statistical Association, 109(506), 700–716.
Wu,, H., Xue,, H., & Kumar,, A. (2012). Numerical discretization‐based estimation methods for ordinary differential equation models via penalized spline smoothing with applications in biomedical research. Biometrics, 68(2), 344–352.
Wu,, H., Zhu,, H., Miao,, H., & Perelson,, A. S. (2008). Parameter identifiability and estimation of HIV/AIDS dynamic models. Bulletin of Mathematical Biology, 70(3), 785–799.
Wu,, L., Qiu,, X., Yuan,, Y.‐x., & Wu,, H. (2019). Parameter estimation and variable selection for big systems of linear ordinary differential equations: A matrix‐based approach. Journal of the American Statistical Association, 114(526), 657–667.
Xia,, X., & Moog,, C. (2003). Identifiability of nonlinear systems with application to HIV/AIDS models. IEEE Transactions on Automatic Control, 48(2), 330–336.
Xue,, H., Miao,, H., & Wu,, H. (2010). Sieve estimation of constant and time‐varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. The Annals of Statistics, 38(4), 2351–2387.
Xun,, X., Cao,, J., Mallick,, B., Maity,, A., & Carroll,, R. J. (2013). Parameter estimation of partial differential equation models. Journal of the American Statistical Association, 108(503), 1009–1020.
Yaari,, R., & Dattner,, I. (2019). Simode: R package for statistical inference of ordinary differential equations using separable integral‐matching. Journal of Open Source Software, 4(44), 1850.
Yaari,, R., Dattner,, I., & Huppert,, A. (2018). A two‐stage approach for estimating the parameters of an age‐group epidemic model from incidence data. Statistical Methods in Medical Research, 27(7), 1999–2014.
Yaari,, R., Huppert,, A., & Dattner,, I. (2019). A statistical methodology for data‐driven partitioning of infectious disease incidence into age‐groups. arXiv preprint arXiv:1907.03441.
Yeung,, E., Kundu,, S., & Hodas,, N. (2019). Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. Paper presented at 2019 American control conference (ACC) (pp. 4832–4839). New York, NY: IEEE.
Yu,, L., & Voit,, E. O. (2006). Construction of bivariate S‐distributions with copulas. Computational Statistics and Data Analysis, 51(3), 1822–1839.
Zaharia,, M., Chowdhury,, M., Das,, T., Dave,, A., Ma,, J., McCauley,, M., Franklin,, M. J., Shenker,, S., & Stoica,, I. (2012). Resilient distributed datasets: A fault‐tolerant abstraction for in‐memory cluster computing. Paper presented at Proceedings of the 9th USENIX conference on networked systems design and implementation. Retrieved from https://www.usenix.org/system/files/conference/nsdi12/nsdi12-final138.pdf.
Zhang,, X., Cao,, J., & Carroll,, R. J. (2015). On the selection of ordinary differential equation models with application to predator‐prey dynamical models. Biometrics, 71(1), 131–138.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Differential equations in data analysis

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox