 Coordinators IIT Madras

Untitled Document

This is a graduate level class on probability theory, geared towards students who are interested in a rigorous development of the subject. It is likely to be useful for students specializing in communications, networks, signal processing, stochastic control, machine learning, and related areas. In general, the course is not so much about computing probabilities, expectations, densities etc. Instead, we will focus on the 'nuts and bolts' of probability theory, and aim to develop a more intricate understanding of the subject. For example, emphasis will be placed on deriving and proving fundamental results, starting from the basic axioms.

Course Contents:

• Probability Spaces, σ-algebras, events, probability measures
• Borel Sets and Lebesgue measure
• Conditioning, Bayes' rule
• Independence
• Borel-Cantelli Lemmas
• Measurable functions, random variables
• Distribution functions, types of random variables
• Joint distributions, transformation of random variables
• Integration, expectation, covariance, correlation
• Conditional expectation and MMSE estimation
• Monotone convergence theorem, Dominated convergence theorem, Fatou's lemma
• Transforms (Moment generating fufunction, characteristic function)
• Concentration Inequalities
• Jointly Gaussian random variables
• Convergence of random variables, various notions of convergence
• Central limit theorem
• The laws of large numbers (the weak and strong laws)

 Module no. Module Name Lecture Names 0 Preliminaries Set Theory Real Analysis basics Cardinality and Countability 1 Probability Measures Probability Spaces Properties of Probability Measures Discrete Probability Spaces Borel Sets and Lebesgue Measure Infinite Coin Toss Model Conditional Probability and Independence Borel-Cantelli Lemmas 2 Random Variables RVs as measurable functions Probability law, types of RVs, and CDF Multiple Random Variables and Independence Jointly Continuous Random Variables, Conditional Distributions Sums of Random Variables General Transformations of Random Variables, Jacobian formula 3 Integration and Expectation Abstract Integration Properties of Abstract Integrals Monotone Convergence Theorem Integration over Different Spaces Integration of Continuous Random Variables, Radon-Nikodym theorem Fatou's Lemma and Dominated Convergence Theorem Variance and Covariance Conditional Expectation and MMSE estimate 4 Transforms Probability Generating Functions Moment Generating Functions Characteristic Functions Inversion Theorem and Uniqueness of the Inversion Concentration Inequalities 5 Limit theorems Convergence of Random Variables and related theorems Weak Law of Large Numbers Strong Law of Large Numbers Central limit theorem, Multi-variate Gaussian Distribution

There will be no official pre-requisites. Although the course will build up from the basics, it will be taught at a fairly sophisticated level. Familiarity with concepts from real analysis will also be useful. Perhaps the most important prerequisite for this class is an intangible one, namely mathematical maturity.

1. Probability and Random Processes by Geoffrey R. Grimmett and David R. Stirzaker. Oxford University Press, 3rd edition, 2001.
2. MIT OCW Notes (Course 6.436, hyperlink below)

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-436j-fundamentals-of-probability-fall-2008/lecture-notes/

1. Probability with Martigales by D. Williams, Cambridge University Press, 1991.
2. A First Look at Rigorous Probability Theory by J. Rosenthal, World Scientific Publishing Co Pte Ltd; 2nd Revised edition, 2006.