Discrete Time Markov Chains (DTMCs)
1.1. Introduction to Markov Chains in Discrete Time
1.2. Classification of states
1.3. The steady state
1.4. Analysis of transient states
1.5. Analysis of recurring states
1.6. Chains with reward
1.7. Strings in continuous time
Markov Decision Processes (MDPs)
2.1. Types of MDPs problems
2.2. Terminated MDPs (SSPs)
23. Evaluating the cost of a policy
2.4. Policy improvement
2.5. The Policy Iteration algorithm
2.6. Discounted MDPs
2.7. Medium cost MDPs
Simulation
3. Introduction to systems simulation.
4. Random number generation.
5. Random variable sample generators.
6. Parametric estimation.
7. Population contrasts.
