Schedule
- Definition and basic properties, the transition matrix. Calculation of n-step transition probabilities. Communicating classes, closed classes, absorption, irreducibility. Calculation of hitting probabilities and mean hitting times; survival probability for birth and death chains. Stopping times and statement of the strong Markov property.
- Recurrence and transience; equivalence of transience and summability of n-step transition probabilities; equivalence of recurrence and certainty of return. Recurrence as a class property, relation with closed classes. Simple random walks in dimensions one, two and three.
- Invariant distributions, statement of existence and uniqueness. Mean return time, positive recurrence; equivalence of positive recurrence and the existence of an invariant distribution. Convergence to equilibrium for irreducible, positive recurrent, aperiodic chains and proof by coupling. *Long-run proportion of time spent in given state*.
- Time reversal, detailed balance, reversibility; random walk on a graph.
Videos available on Moodle
Example Sheets
References
- J.R. Norris. Markov Chains. CUP 1997. Available here (from university network).
- G.R. Grimmett and D. Welsh. Probability, An Introduction. OUP, 2nd edition, 2014.