Schedule
- Definition and basic properties, the transition matrix. Calculation of n-step transition probabilities. Communicating classes, closed classes, absorption, irreducibility. Calculation of hitting probabilities and mean hitting times; survival probability for birth and death chains. Stopping times and statement of the strong Markov property.
- Recurrence and transience; equivalence of transience and summability of n-step transition probabilities; equivalence of recurrence and certainty of return. Recurrence as a class property, relation with closed classes. Simple random walks in dimensions one, two and three.
- Invariant distributions, statement of existence and uniqueness. Mean return time, positive recurrence; equivalence of positive recurrence and the existence of an invariant distribution. Convergence to equilibrium for irreducible, positive recurrent, aperiodic chains and proof by coupling. *Long-run proportion of time spent in given state*.
- Time reversal, detailed balance, reversibility; random walk on a graph.
Example Sheets
The example sheets will be posted here during the course of the term.
- Example Sheet 1 (to be posted October 10)
- Example Sheet 2 (to be posted October 31)
References
- J.R. Norris. Markov Chains. CUP 1997. Available here (from university network).
- G.R. Grimmett and D. Welsh. Probability, An Introduction. OUP, 2nd edition, 2014.