Description : The theory of Markov chains, although a special case of Markov processes, is here developed for its own sake and presented on its own merits. In general, the hypothesis of a denumerable state space, which is the defining hypothesis of what we call a "chain" here, generates more clear-cut questions and demands more precise and definitive an swers. For example, the principal limit theorem (§§ 1. 6, II. 10), still the object of research for general Markov processes, is here in its neat final form; and the strong Markov property (§ 11. 9) is here always applicable. While probability theory has advanced far enough that a degree of sophistication is needed even in the limited context of this book, it is still possible here to keep the proportion of definitions to theorems relatively low. . From the standpoint of the general theory of stochastic processes, a continuous parameter Markov chain appears to be the first essentially discontinuous process that has been studied in some detail. It is common that the sample functions of such a chain have discontinuities worse than jumps, and these baser discontinuities play a central role in the theory, of which the mystery remains to be completely unraveled. In this connection the basic concepts of separability and measurability, which are usually applied only at an early stage of the discussion to establish a certain smoothness of the sample functions, are here applied constantly as indispensable tools.
Description : Contents. Part 1. Discrete parameter. Fundamental definitions. Tran- sition probabilities. Classification of states. Recurrence. Criteria and examples. The main limit theorem. Various complements. Repetitive pattern and renewal process. Taboo probabilities. The generating function. The moments of first entrance time distributions. A random walk example. System theorems. Functionals and associated random va- riables. Ergodic theorems. Futher limit theorems. Almost closed and sojourn sets. Part 2. Continous parameter. Transition matrix: Basic properties. Standard transition matrix. Differentiability. Definiti- ons and measure-theoretic foundations. The set of constancy. Continu- ity properties of sample functions. Futher specifications of the pro- cess. Optional variable. Strong markov property. Classification of states. Taboo probability functions. Last exit time. Ratio limit the- orems, discrete approximations. Functionals. Post-exit process. Im- bedded renewal process. The two systems of differential equations. The minimal solution. The first infinity. Examples.
Description : In this rigorous account the author studies both discrete-time and continuous-time chains. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials, in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and a careful selection of exercises and examples drawn both from theory and practice. This is an ideal text for seminars on random processes or for those that are more oriented towards applications, for advanced undergraduates or graduate students with some background in basic probability theory.
Description : Stochastic processes are necessary ingredients for building models of a wide variety of phenomena exhibiting time varying randomness. This text offers easy access to this fundamental topic for many students of applied sciences at many levels. It includes examples, exercises, applications, and computational procedures. It is uniquely useful for beginners and non-beginners in the field. No knowledge of measure theory is presumed.
Description : Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and have had an especially profound impact on the practice of Bayesian statistics. Furthermore, MCMC methods have enabled the development and use of intricate models in an astonishing array of disciplines as diverse as fisheries science and economics. The wide-ranging practical importance of MCMC has sparked an expansive and deep investigation into fundamental Markov chain theory. The Handbook of Markov Chain Monte Carlo provides a reference for the broad audience of developers and users of MCMC methodology interested in keeping up with cutting-edge theory and applications. The first half of the book covers MCMC foundations, methodology, and algorithms. The second half considers the use of MCMC in a variety of practical applications including in educational research, astrophysics, brain imaging, ecology, and sociology. The in-depth introductory section of the book allows graduate students and practicing scientists new to MCMC to become thoroughly acquainted with the basic theory, algorithms, and applications. The book supplies detailed examples and case studies of realistic scientific problems presenting the diversity of methods used by the wide-ranging MCMC community. Those familiar with MCMC methods will find this book a useful refresher of current theory and recent developments.
Description : This 3rd edition of the successful Elements of Applied Stochastic Processes improves on the last edition by condensing the material and organising it into a more teachable format. It provides more in-depth coverage of Markov chains and simple Markov process and gives added emphasis to statistical inference in stochastic processes. Integration of theory and application offers improved teachability Provides a comprehensive introduction to stationary processes and time series analysis Integrates a broad set of applications into the text Utilizes a wealth of examples from research papers and monographs
Description : John Walsh, one of the great masters of the subject, has written a superb book on probability. It covers at a leisurely pace all the important topics that students need to know, and provides excellent examples. I regret his book was not available when I taught such a course myself, a few years ago. --Ioannis Karatzas, Columbia University In this wonderful book, John Walsh presents a panoramic view of Probability Theory, starting from basic facts on mean, median and mode, continuing with an excellent account of Markov chains and martingales, and culminating with Brownian motion. Throughout, the author's personal style is apparent; he manages to combine rigor with an emphasis on the key ideas so the reader never loses sight of the forest by being surrounded by too many trees. As noted in the preface, ``To teach a course with pleasure, one should learn at the same time.'' Indeed, almost all instructors will learn something new from the book (e.g. the potential-theoretic proof of Skorokhod embedding) and at the same time, it is attractive and approachable for students. --Yuval Peres, Microsoft With many examples in each section that enhance the presentation, this book is a welcome addition to the collection of books that serve the needs of advanced undergraduate as well as first year graduate students. The pace is leisurely which makes it more attractive as a text. --Srinivasa Varadhan, Courant Institute, New York This book covers in a leisurely manner all the standard material that one would want in a full year probability course with a slant towards applications in financial analysis at the graduate or senior undergraduate honors level. It contains a fair amount of measure theory and real analysis built in but it introduces sigma-fields, measure theory, and expectation in an especially elementary and intuitive way. A large variety of examples and exercises in each chapter enrich the presentation in the text.
Description : This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mathematics, including students from engineering, economics, physics, and biology. Topics covered are: Doeblin's theory, general ergodic properties, and continuous time processes. Applications are dispersed throughout the book. In addition, a whole chapter is devoted to reversible processes and the use of their associated Dirichlet forms to estimate the rate of convergence to equilibrium. These results are then applied to the analysis of the Metropolis (a.k.a simulated annealing) algorithm. The corrected and enlarged 2nd edition contains a new chapter in which the author develops computational methods for Markov chains on a finite state space. Most intriguing is the section with a new technique for computing stationary measures, which is applied to derivations of Wilson's algorithm and Kirchoff's formula for spanning trees in a connected graph.
Description : Self-contained treatment covers both theory and applications. Topics include the fundamental role of homogeneous infinite Markov chains in the mathematical modeling of psychology and genetics. 1980 edition.