Home
This Title All WIREs
WIREs RSS Feed
How to cite this WIREs title:
WIREs Data Mining Knowl Discov
Impact Factor: 2.541

An introduction to algorithmic differentiation

Full article on Wiley Online Library:   HTML PDF

Can't access this content? Tell your librarian.

Abstract Algorithmic differentiation (AD), also known as automatic differentiation, is a technology for accurate and efficient evaluation of derivatives of a function given as a computer model. The evaluations of such models are essential building blocks in numerous scientific computing and data analysis applications, including optimization, parameter identification, sensitivity analysis, uncertainty quantification, nonlinear equation solving, and integration of differential equations. We provide an introduction to AD and present its basic ideas and techniques, some of its most important results, the implementation paradigms it relies on, the connection it has to other domains including machine learning and parallel computing, and a few of the major open problems in the area. Topics we discuss include: forward mode and reverse mode of AD, higher‐order derivatives, operator overloading and source transformation, sparsity exploitation, checkpointing, cross‐country mode, and differentiating iterative processes. This article is categorized under: Algorithmic Development > Scalable Statistical Methods Technologies > Data Preprocessing
Differentiated code generated by Tapenade for the function in Example . Left: Forward mode, right: Reverse mode
[ Normal View | Magnified View ]
Idea of reverse mode of algorithmic differentiation: Mapping of variable x to variable y by function F and normal to normal by function
[ Normal View | Magnified View ]
Idea of forward mode algorithmic differentiation: Mapping of variable x to variable y by function F and tangent to tangent by function
[ Normal View | Magnified View ]
Computational graph for Example . The vertices v1 through v6 corresponding to intermediate variables have been annotated by elemental functions (operations). The edges are annotated with weights indicating local partial derivatives (i.e., for an edge (vj,vi), the edge weight is )
[ Normal View | Magnified View ]
Evaluation procedure and real C code for the function in Example
[ Normal View | Magnified View ]
A framework for computing a sparse derivative matrix. Step 1 and Step 3 need to be done by an algorithmic differentiation tool; Steps 2 and 4 can be done outside an AD tool
[ Normal View | Magnified View ]
Calculation of derivatives of the function in Example using ADOL‐C
[ Normal View | Magnified View ]

Browse by Topic

Technologies > Data Preprocessing
Algorithmic Development > Scalable Statistical Methods

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts