Schedule
- Finite-state continuous-time Markov chains: basic properties. Q-matrix (or generator), backward and forward equations. The homogeneous Poisson process and its properties (thinning, superposition). Birth and death processes.
- General continuous-time Markov chains. Jump chains. Explosion. Minimal Chains. Communicating classes. Hitting times and probabilities. Recurrence and transience. Positive and null recurrence. Convergence to equilibrium. Reversibility.
- Applications: the M/M/1 and M/M/∞ queues. Burke’s theorem. Jackson’s theorem for queueing networks. The M/G/1 queue.
- Renewal theory: renewal theorems, equilibrium theory (proof of convergence only in discrete time). Renewal-reward processes. Little’s formula.
- Spatial Poisson processes in d dimensions. The superposition, mapping, and colouring theorems. Renyi’s theorem. Applications including Olbers’ paradox.
Lecture notes (updated throughout the term)
Videos available on Moodle
Example Sheets
References
- N. Berestyki and P. Sousi, Lecture notes from 2017, without Section 5. Available here.
- J.R. Norris. Markov Chains, Chapters 2 and 3. CUP 1997. Available here (from university network).
- G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes, Chapter 6. OUP 2020.