Description : The theory of Markov chains, although a special case of Markov processes, is here developed for its own sake and presented on its own merits. In general, the hypothesis of a denumerable state space, which is the defining hypothesis of what we call a "chain" here, generates more clear-cut questions and demands more precise and definitive an swers. For example, the principal limit theorem (§§ 1. 6, II. 10), still the object of research for general Markov processes, is here in its neat final form; and the strong Markov property (§ 11. 9) is here always applicable. While probability theory has advanced far enough that a degree of sophistication is needed even in the limited context of this book, it is still possible here to keep the proportion of definitions to theorems relatively low. . From the standpoint of the general theory of stochastic processes, a continuous parameter Markov chain appears to be the first essentially discontinuous process that has been studied in some detail. It is common that the sample functions of such a chain have discontinuities worse than jumps, and these baser discontinuities play a central role in the theory, of which the mystery remains to be completely unraveled. In this connection the basic concepts of separability and measurability, which are usually applied only at an early stage of the discussion to establish a certain smoothness of the sample functions, are here applied constantly as indispensable tools.
Description : Contents. Part 1. Discrete parameter. Fundamental definitions. Tran- sition probabilities. Classification of states. Recurrence. Criteria and examples. The main limit theorem. Various complements. Repetitive pattern and renewal process. Taboo probabilities. The generating function. The moments of first entrance time distributions. A random walk example. System theorems. Functionals and associated random va- riables. Ergodic theorems. Futher limit theorems. Almost closed and sojourn sets. Part 2. Continous parameter. Transition matrix: Basic properties. Standard transition matrix. Differentiability. Definiti- ons and measure-theoretic foundations. The set of constancy. Continu- ity properties of sample functions. Futher specifications of the pro- cess. Optional variable. Strong markov property. Classification of states. Taboo probability functions. Last exit time. Ratio limit the- orems, discrete approximations. Functionals. Post-exit process. Im- bedded renewal process. The two systems of differential equations. The minimal solution. The first infinity. Examples.
Description : Stochastic processes are necessary ingredients for building models of a wide variety of phenomena exhibiting time varying randomness. In a lively and imaginative presentation, studded with examples, exercises, and applications, and supported by inclusion of computational procedures, the author has created a textbook that provides easy access to this fundamental topic for many students of applied sciences at many levels. With its carefully modularized discussion and crystal clear differentiation between rigorous proof and plausibility argument, it is accessible to beginners but flexible enough to serve as well those who come to the course with strong backgrounds. The prerequisite background for reading the book is a graduate level pre-measure theoretic probability course. No knowledge of measure theory is presumed and advanced notions of conditioning are scrupulously avoided until the later chapters of the book. The book can be used for either a one or two semester course as given in departments of mathematics, statistics, operation research, business and management, or a number of engineering departments. Its approach to exercises and applications is practical and serious. Some underlying principles of complex problems and computations are cleanly and quickly delineated through rich vignettes of whimsically imagined Happy Harry and his Optima Street gang’s adventures in a world whose randomness is a never-ending source of both wonder and scientific insight. The tools of applied probability---discrete spaces, Markov chains, renewal theory, point processes, branching processes, random walks, Brownian motion---are presented to the reader in illuminating discussion. Applications include such topics as queuing, storage, risk analysis, genetics, inventory, choice, economics, sociology, and other. Because of the conviction that analysts who build models should know how to build them for each class of process studied, the author has included such constructions.
Description : In this rigorous account the author studies both discrete-time and continuous-time chains. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials, in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and a careful selection of exercises and examples drawn both from theory and practice. This is an ideal text for seminars on random processes or for those that are more oriented towards applications, for advanced undergraduates or graduate students with some background in basic probability theory.
Description : Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and have had an especially profound impact on the practice of Bayesian statistics. Furthermore, MCMC methods have enabled the development and use of intricate models in an astonishing array of disciplines as diverse as fisheries science and economics. The wide-ranging practical importance of MCMC has sparked an expansive and deep investigation into fundamental Markov chain theory. The Handbook of Markov Chain Monte Carlo provides a reference for the broad audience of developers and users of MCMC methodology interested in keeping up with cutting-edge theory and applications. The first half of the book covers MCMC foundations, methodology, and algorithms. The second half considers the use of MCMC in a variety of practical applications including in educational research, astrophysics, brain imaging, ecology, and sociology. The in-depth introductory section of the book allows graduate students and practicing scientists new to MCMC to become thoroughly acquainted with the basic theory, algorithms, and applications. The book supplies detailed examples and case studies of realistic scientific problems presenting the diversity of methods used by the wide-ranging MCMC community. Those familiar with MCMC methods will find this book a useful refresher of current theory and recent developments.
Description : Fundamental concepts of Markov chains; The classical approach to markov chains; The algebraic approach to Markov chains; Nonstationary Markov chains and the ergodic coeficient; Analysis of a markov chain on a computer; Continuous time Markov chains.
Description : This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mathematics, including students from engineering, economics, physics, and biology. Topics covered are: Doeblin's theory, general ergodic properties, and continuous time processes. Applications are dispersed throughout the book. In addition, a whole chapter is devoted to reversible processes and the use of their associated Dirichlet forms to estimate the rate of convergence to equilibrium. These results are then applied to the analysis of the Metropolis (a.k.a simulated annealing) algorithm. The corrected and enlarged 2nd edition contains a new chapter in which the author develops computational methods for Markov chains on a finite state space. Most intriguing is the section with a new technique for computing stationary measures, which is applied to derivations of Wilson's algorithm and Kirchoff's formula for spanning trees in a connected graph.
Description : Papers presented at a workshop held January 1990 (location unspecified) cover just about all aspects of solving Markov models numerically. There are papers on matrix generation techniques and generalized stochastic Petri nets; the computation of stationary distributions, including aggregation/disagg