Markov Chains

Checkpoints

• Markov Chains Essential A Markov chain is a sequence of random variables (X_n) for which the conditional distribution of the next state given the entire past depends only on the present: P(X_{n+1} = j | X_0, …, X_n) = P(X_{n+1} = j | X_n). This checkpoint defines discrete-time Markov chains on a countable state space, introduces the transition matrix and the Chapman–Kolmogorov equation P^(m+n) = P^m P^n, and discusses initial distributions, n-step transitions, and the basic vocabulary of states (transient, recurrent, periodic) — leaving the spectral analysis of long-run behaviour for later.