Numerical modeling and simulation of complex physical systems are becoming increasingly important in physics and engineering. Many problems are formulated in the form of ordinary or partial differential equations, which require numerical solution schemes such as finite difference, finite element or integral methods. A particular problem can have multiple input parameters, and one often wants to optimize an objective function based on the computed solution. In these cases, the systems of ordinary or partial differential equations serve as the constraints, and the objective function evaluates the numerical solution of the differential equations. Examples of ode/pde constrained optimization include weather forecasting, airfoil shape design and design of micro-electrical-mechanical devices. A characteristic of this type of optimization problems is the high cost of constraint evaluation, since solving partial differential equations is a numerically costly task. We introduce the adjoint sensitivity method, which computes objective function gradient with the same computational cost as solving the differential equations once.
We present the adjoint sensitivity method for a partial differential equation that can be semi-discretized in space and solved as an ode in time. Heat or wave equations are examples of this type of equations. The optimization problem can be formulated as
We can interpret as the governing ordinary differential equation in time, where is the input parameters, is time and is the solution to the ordinary differential equation. The function is the constraint on initial conditions. The objective function assigns a value to the input parameter and solution pair . Solving this optimization problem usually involves finding the gradient function and applying a gradient based algorithm such as conjugate gradient descent. We derive the solution to through sensitivity method.
We first define the Lagrangian
,where and are Lagrangian multipliers. Since ,we take the derivative of the Lagrangian
Note that is difficult to compute, so we need to eliminate this term using integration by parts. We also need to eliminate costly terms such as by choosing appropriate Lagrange multipliers. In this case, we can let
Finally, gradient function simplifies to
We demonstrate a very simple example of using the adjoint method to compute the objective function gradient. We want to compute the gradient of the system
where are the input parameters. Firstly, using the approach discussed in the previous section, we need to evaluate the function once: . We then compute the Lagrange multiplier by solving
This leads to . Finally, we can compute the objective function gradient as
The adjoint method can be applied to solve more difficult problems such as inverse Schrodinger equation and address optimization of fluid flow that exhibits chaotic behavior. More examples can be found at http://engineer-chaos.blogspot.com/p/what-is-chaos.html and http://math.mit.edu/~stevenj/18.336/adjoint.pdf.
R. M. Errico, “What is an adjoint model?,” Bul- letin Am. Meteorological Soc. , vol. 78, pp. 2577– 2591, 1997.
Y Cao, S. Li, L. Petzold, and R. Serban, “Adjoint sensitivity analysis for differential- algebraic equations: The adjoint DAE system and its numerical solution,” SIAM J. Sci. Com- put. , vol. 24, no. 3, pp. 1076–1089, 2003.
Optimization With PDE Constraints ,Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S. 2009
Written by Yufeng (Kevin) Chen