MA485 STOCHASTIC MODELS IN OPERATIONS RESEARCH (3 Cr.)

COURSE DESCRIPTION

Prerequisite:  MA 371 (Applied Probability and Statistics) MA 381 (Integer Programming and Network Flows)

General Introduction And Goals
Survey of stochastic models in operations research with emphasis on dynamic programming, Markovian decision processes, queuing, inventory control, production planning, and simulation models.

Course Outline

1. Probabilistic Dynamic Programming
   1. Finite-Stage Models
   2. Infinite-Stage Models
   3. Discounted Dynamic Programming
   4. Negative Dynamic Programming
   5. Positive Dynamic Programming
2. Markovian Decision Processes
   1. Introduction to Stochastic Processes
   2. Markov Chains
      1. Random Walk Models
      2. Urn Models
      3. Diffusion Models
      4. Queuing Models
      5. Inventory Models
      6. Gambler's Ruin Models
      7. Genetics Models
      8. Branching Processes
      9. Random Placement Models
      10. Water-Resource Models
      11. Work Force Planning Models . . .
   3. Discrete-State Discrete-Transition Markov Processes
   4. Discrete-State Continuous-Transition Markov Processes
   5. n-Step Transition Probabilities
   6. Classification of States in a Markov Chain
   7. Steady State Probabilities
   8. Markovian Decision Models
   9. Dynamic Programming Formulation of Markovian Decision Processes
   10. Finite-Stage Dynamic Programming Models
   11. Infinite-Stage Dynamic Programming Models
   12. Exhaustive Enumeration Methods
   13. Policy Iteration Methods without Discounting
   14. Linear Programming Formulation and Solution of Markovian Decision Problems
3. Queuing Theory
   1. Basic Definitions and Terminology
   2. Basic Structure of Queuing Models
   3. Examples of Real Queuing Systems
   4. Modeling Arrival and Service Processes
   5. Birth-and-Death Processes
   6. Queuing Models Based on Birth-and-Death Processes
   7. The M/M/1/GD/∞/∞ Queuing System
   8. The M/M/1/GD/c/∞ Queuing System
   9. The M/M/s/GD/∞/∞ Queuing System
   10. The M/M/∞/GD/∞/∞ Queuing System
   11. The M/G/1/GD/∞/∞ Queuing System
   12. Priority Queuing Models
   13. Queuing Networks
   14. Queuing Optimization Models
   15. Applications of Queuing Theory
4. Simulation
   1. Basic Definitions and Concepts
   2. Formulation and Implementation of Simulation Models
   3. Random Numbers and Monte Carlo Simulation
   4. Simulation with Discrete Random Variables
   5. Simulation with Continuous Random Variables
   6. Statistical Analysis in Simulation
   7. Simulation Languages