Markov Chains: Analytic and Monte Carlo Computations

Markov Chains: Analytic and Monte Carlo Computations introduces the main notions related to Markov chains and provides explanations on how to characterize, simulate, and recognize them. Starting with basic notions, this book leads progressively to advanced and recent topics in the field, allowing the reader to master the main aspects of the classical theory. This book also features:

• Numerous exercises with solutions as well as extended case studies.
• A detailed and rigorous presentation of Markov chains with discrete time and state space.
• An appendix presenting probabilistic notions that are necessary to the reader, as well as giving more advanced measure-theoretic notions.

Table of Contents

Chapter 1 First steps
Chapter 2 Past, present, and future
Chapter 3 Transience and recurrence
Chapter 4 Long-time behavior
Chapter 5 Monte Carlo methods