Markov Processes for Stochastic Modeling

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems.

Covering a wide range of  areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader.

Presents both the theory and applications of the different aspects of Markov processesIncludes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presentedDiscusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis.

1. Basic Concepts
2. Introduction to Markov Processes 
3. Discrete-Time Markov Chains
4. Continuous-Time Markov Chains 
5. Markovian Queueing Systems 
6. Markov Renewal Processes
7. Markovian Arrival Processes 
8. Random Walk 
9. Brownian Motion and Diffusion Processes 
10. Controlled Markov  Processes
11. Hidden Markov Models
12. Markov Point Processes