Description : This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mathematics, including students from engineering, economics, physics, and biology. Topics covered are: Doeblin's theory, general ergodic properties, and continuous time processes. Applications are dispersed throughout the book. In addition, a whole chapter is devoted to reversible processes and the use of their associated Dirichlet forms to estimate the rate of convergence to equilibrium. These results are then applied to the analysis of the Metropolis (a.k.a simulated annealing) algorithm. The corrected and enlarged 2nd edition contains a new chapter in which the author develops computational methods for Markov chains on a finite state space. Most intriguing is the section with a new technique for computing stationary measures, which is applied to derivations of Wilson's algorithm and Kirchoff's formula for spanning trees in a connected graph.
Description : Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader. Presents both the theory and applications of the different aspects of Markov processes Includes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presented Discusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis.
Description : Markov process theory is basically an extension of ordinary calculus to accommodate functions whos time evolutions are not entirely deterministic. It is a subject that is becoming increasingly important for many fields of science. This book develops the single-variable theory of both continuous and jump Markov processes in a way that should appeal especially to physicists and chemists at the senior and graduate level. A self-contained, prgamatic exposition of the needed elements of random variable theory Logically integrated derviations of the Chapman-Kolmogorov equation, the Kramers-Moyal equations, the Fokker-Planck equations, the Langevin equation, the master equations, and the moment equations Detailed exposition of Monte Carlo simulation methods, with plots of many numerical examples Clear treatments of first passages, first exits, and stable state fluctuations and transitions Carefully drawn applications to Brownian motion, molecular diffusion, and chemical kinetics
Description : Recursive Estimation and Control for Stochastic Systems Han-Fu Chen This self-contained volume presents both the discrete-time and continuous-time systems, and incorporates not only well-known results in these fields but also many of the latest research findings. It shows how to analyze the convergence of recursive estimates through a combination of the probabilistic and ordinary differential equation methods and establishes the connection between the Gauss-Markov estimate and the Kalman filter through stochastic observability, and more. 1985 (0 471-81566-7) 378 pp. Nonparametric Density Estimation The L1 View Luc Devroye and Laszlo Gyorfi The first systematic, single-source examination that develops from first principles the "natural" theory for density estimation and shows why the classical L2 theory masks some fundamental properties of density estimates. Linking different subareas of statistics, including simulation, pattern recognition, detection theory, and minimax theory, it shows how to construct, use, and analyze density estimates. Relevant recent literature is tied in with the classical works of Parzen, Rosenblatt, and others. 1985 (0 471-81646-9) 368 pp. Elements of Applied Stochastic Processes Second Edition U. Narayan Bhat An applied introduction to stochastic models, this expanded and revised account develops basic concepts and techniques and applies them to problems arising in queueing, reliability, inventory and computer communications, social and behavioral processes, business management, and time series analysis. 1984 (0 471-87826-X) 736 pp.
Description : From the reviews of the First Edition: "This excellent book is based on several sets of lecture notes written over a decade and has its origin in a one-semester course given by the author at the ETH, Zürich, in the spring of 1970. The author's aim was to present some of the best features of Markov processes and, in particular, of Brownian motion with a minimum of prerequisites and technicalities. The reader who becomes acquainted with the volume cannot but agree with the reviewer that the author was very successful in accomplishing this goal...The volume is very useful for people who wish to learn Markov processes but it seems to the reviewer that it is also of great interest to specialists in this area who could derive much stimulus from it. One can be convinced that it will receive wide circulation." (Mathematical Reviews) This new edition contains 9 new chapters which include new exercises, references, and multiple corrections throughout the original text.
Description : This work offers a highly useful, well developed reference on Markov processes, the universal model for random processes and evolutions. The wide range of applications, in exact sciences as well as in other areas like social studies, require a volume that offers a refresher on fundamentals before conveying the Markov processes and examples for applications. This work does just that, and with the necessary mathematical rigor.
Description : Theory of Markov Processes provides information pertinent to the logical foundations of the theory of Markov random processes. This book discusses the properties of the trajectories of Markov processes and their infinitesimal operators. Organized into six chapters, this book begins with an overview of the necessary concepts and theorems from measure theory. This text then provides a general definition of Markov process and investigates the operations that make possible an inspection of the class of Markov processes corresponding to a given transition function. Other chapters consider the more complicated operation of generating a subprocess. This book discusses as well the construction of Markov processes with given transition functions. The final chapter deals with the conditions to be imposed on the transition function so that among the Markov processes corresponding to this function, there should be at least one. This book is a valuable resource for mathematicians, students, and research workers.
Description : The modem theory of Markov processes has its origins in the studies of A. A. MARKOV (1906-1907) on sequences of experiments "connected in a chain" and in the attempts to describe mathematically the physical phenomenon known as Brownian motion (L. BACHELlER 1900, A. EIN STEIN 1905). The first correct mathematical construction of a Markov process with continuous trajectories was given by N. WIENER in 1923. (This process is often called the Wiener process.) The general theory of Markov processes was developed in the 1930's and 1940's by A. N. KOL MOGOROV, W. FELLER, W. DOEBLlN, P. LEVY, J. L. DOOB, and others. During the past ten years the theory of Markov processes has entered a new period of intensive development. The methods of the theory of semigroups of linear operators made possible further progress in the classification of Markov processes by their infinitesimal characteristics. The broad classes of Markov processes with continuous trajectories be came the main object of study. The connections between Markov pro cesses and classical analysis were further developed. It has become possible not only to apply the results and methods of analysis to the problems of probability theory, but also to investigate analytic problems using probabilistic methods. Remarkable new connections between Markov processes and potential theory were revealed. The foundations of the theory were reviewed critically: the new concept of strong Markov process acquired for the whole theory of Markov processes great importance.