Syllabus  |   Lectures  |   Downloads  |   FAQ  |   Ask a question  |  
Course Co-ordinated by IIT Madras
Prof. S. Lakshmivarahan
IIT Madras(USA)


Download Syllabus in PDF format

Untitled Document

Much of the human activity is controlled by prediction of various phenomena that affect our life - be it of the man-made or natural type. Examples include prediction of weather – hurricanes, tornados, snow/ice showers, heat waves, short-term climate scenarios, prediction of revenue by local/state and national governments to develop budget priorities for the next fiscal year, prediction of the growth in GDP, prediction of IBM stock price on the day of equinox, to name a few. These predictions are generated by running a relevant class of models which may be either causality based (as in hurricane prediction) or empirically derived (as in the prediction of revenue or unemployment or GDP growth, etc).

Besides being causality based or empirically derived, models, in general, occur in various shapes and forms. A model can be (a) static or dynamic (b) deterministic or stochastic (c) operate in discrete/continuous time and (d) may be defined on continuous space or discrete space. Irrespective of its origin, models in general have several unknowns – initial conditions (IC), boundary conditions (BC), and parameters. The solutions of these models, in general, constitute the basis for generating predictions.

However, to compute the solution we need to know or estimate the values of the unknown IC, BC and/or parameters. This estimation is enabled by using the observations of the phenomenon in question – using the observed pressure distribution around the eye of the hurricane, data from satellites or ground based radars, from the time series of data on unemployment, etc. Data assimilation in large measure relates to the process of “fusing” data with the model for the singular purpose of estimating the unknowns. Once these estimates are available, we obtain an instantiation of the model which is then run forward in time to generate the requisite forecast products for public consumption.

Data assimilation as stated is intimately related to the "inverse" problems in mathematics or regression problems in statistics or retrieval problems in Geosciences. Finding the sea surface temperature distribution near the equatorial Pacific from Satellite measurements or distribution of the aerosol from satellite observations or estimating the amount of rain from radar observations are but a few of the examples of inverse problems that lie at the heart of data assimilation.

Some phenomena are intrinsically predictable (lunar/solar eclipses for the next 50 years) but certain other phenomena are only predictable for a short horizon – weather at most for a week. The length of the predictable horizon is related to predictability limit of the model. This is closely related to understanding the growth of prediction errors. This topic is closely related to analysis of chaotic systems.

Our aim is to provide a broad based background on the mathematical principles and tools from linear algebra, multivariate calculus and finite dimensional optimization theory, estimation theory, non-linear dynamics and chaos that constitute the basis for dynamic data assimilation as we know today. Our aim is to present the ideas at the level of a first year graduate/final year undergraduate student aspiring to enter this exciting area.

Module No:



Module I:


Data Mining, Data Assimilation, Inverse problems and Prediction


Static vs. dynamic and deterministic vs. stochastic problems- formulation & classification

Module II: Mathematical tools


Finite dimensional vector space – basic concepts


Overview of properties and operations on matrices


Special classes of matrices, Eigen decomposition, and matrix square root


Gradient, Jacobian, Hessian, Quadratic forms and their properties

Module III: Static, deterministic models: least Squares method – formulation and properties


Linear least squares (LLS) - over determined case, weighted and unweighted formulation, orthogonal and oblique projections


LLS- Underdetermined case -Lagrangian multiplier


Nonlinear least squares problem (NLS) - formulation


Approximation – first and second-order methods for solving NLS.


Examples of LLS and NLS - satellite retrieval

Module IV:
Matrix methods solving LLS


Normal equations – symmetric positive definite (SPD) systems – multiplicative matrix decomposition


Cholesky decomposition- matrix square root


Gramm-Schmidt orthogonalization process


QR decomposition


Singular value decomposition (SVD)


Solution of retrieval problems

Module V: Direct minimization methods for solving LLS


LLS as a quadratic minimization problem


Gradient method, its properties


Convergence and speed of convergence of gradient method


Conjugate gradient and Quasi-Newton methods


Practice problems and programming exercises

Module VI:Deterministic, dynamic models – adjoint method


Dynamic models, role of observations, and least squares objective function, estimation of initial condition (IC) and parameters, adjoint sensitivity


A straight line problem – a warm up


Linear model, first-order adjoint dynamics and computation of the gradient of the least squares objective function


Nonlinear model and first-order adjoint dynamics'


Illustrative examples and practice problems, programming exercises

Module VII: Deterministic, Dynamic models – Other methods


Forward sensitivity method for estimation of IC and parameters, forward sensitivity dynamics - Example of Carbon dynamics


Relation between adjoint and forward sensitivity, Predictability, Lyapunov index


Method of nudging and overview of nudging methods

Module VIII: Static, stochastic models – Bayesian framework


Bayesian method – linear, Gaussian case


Linear minimum variance estimation (LMVE) and prelude to Kalman filter


Model space vs. observation space formulation -Duality between Bayesian and LMVE

Module IX: Dynamic, Stochastic models - Kalman filtering


Derivation of the Kalman filter equations


Derivation of Nonlinear filter


Computational requirements


Ensemble Kalman filtering

Module X: Dynamic stochastic models – Other methods


Unscented Kalman filtering


Particle filtering


An overview and assessment of methods


Good facility with Calculus, Linear Algebra, basic Probability Theory and Statistics


  1. J.M. Lewis, S. Lakshmivarahan and S. K. Dhall (2006) Dynamic Data Assimilation: a least squares approach, Cambridge University Press, 654 pages + Appendices A through F
  2. E. Kalnay (2003) Atmospheric Modeling, Data Assimilation, and Predictability, Cambridge University Press

Papers (more will be added)

  1. S. Lakshmivarahan and D. Stensrud (2009) Ensemble Kalman Filter: An innovative approach for meteorological data assimilation”, IEEE Control System Society, Special Issue, Vol 29, pp 34-46,
  2. S. Lakshmivarahan and John M. Lewis (2010) “Forward Sensitivity Based Approach to Dynamic Data Assimilation”, Advances in Meteorology, Volume 2010, Article ID 375615, doi 1155/2010/375615
  3. C. Gao, H. Wang, W. Weng, S. Lakshmivarahan, Y. Zhang and Y. Luo (2011) " Assimilation of multiple data sets with the ensemble Kalman filter to improve forecasts of forest carbon dynamics", Ecological Applications, Vol 21, pp 1461-1473
  4. S. Lakshmivarahan and J. M. Lewis (2013) "Nudging Methods: A Critical Overview"
    a chapter in Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications Vol II, edited bySeon Ki Park and L. Liang,Springer Verlag, pp 27-58
  5. S. Lakshmivarahan, J. M. Lewis and Dung Phan (2013) "Data assimilation as a problem in optimal tracking: Application of Pontryagin's minimum principle", Journal of Atmospheric Sciences, Vol 70, pp 1257-1277
  6. J. M. Lewis and S. Lakshmivarahan (2013) "A Question of Adequacy of observations in Variational DataAssimilation" a chapter in Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications Vol II, edited by Seon Ki Park and L. Liang,Springer Verlag, pp 111-124

E. Kalnay (2003)Atmospheric Modeling, Data Assimilation, and Predictability, Cambridge University Press

Important: Please enable javascript in your browser and download Adobe Flash player to view this site
Site Maintained by Web Studio, IIT Madras. Contact Webmaster: