Definition and classification of stochastic processes, Poisson process, Flow of events, telegraph signal, Birth and death processes, application in queues, Random walk, Markov chains: classification of states, hitting times and absorption probabilities, Analysis of finite transient discrete Markov chain,finite and infinite ergodic chains, Continuous-time Markov chains, The Kolmogorov differential equations and transition probabilities, Martingales, Stopping times, Martingales to analyze random walks, application to GI/G/1 queues and ruin problems in risk theory.
Renewal processes, key renewal theorem, renewal reward process, Regenerative processes, stationary point processes, Branching process,Condititional limit laws of branching process, Continuous time Markov branching process, General branching process, Elements of continuous time and continuous state space processes: Brownian motion, Diffusion processes, variations of Brownian motions and drift, backward and forward diffusion equations, applications of Kolmogorov’s equations for obtaining limiting distributions of semi-Markov processes, queues |