Dynamic programming. Markov decision problems. Dynamic programming over the infinite horizon. Positive and negative programming. Optimal stopping problems. Bandit processes and the Gittins index. Average-cost programming. Continuous-time Markov decision processes. LQ regulation. Controllability. Observability. Imperfect observation. Dynamic programming in continuous time. Pontryagin’s maximum principle.