E. B. Dynkin

The modem theory of Markov processes has its origins in the studies of A. A. MARKOV (1906-1907) on sequences of experiments "connected in a chain" and in the attempts to describe mathematically the physical phenomenon known as Brownian motion (L. BACHELlER 1900, A. EIN- STEIN 1905). The first correct mathematical construction of a Markov process with continuous trajectories was given by N. WIENER in 1923. (This process is often called the Wiener process.) The general theory of Markov processes was developed in the 1930's and 1940's by A. N. KOL- MOGOROV, W. FELLER, W. DOEBLlN, P. LEVY, J. L. DOOB, and others. During the past ten years the theory of Markov processes has entered a new period of intensive development. The methods of the theory of semigroups of linear operators made possible further progress in the classification of Markov processes by their infinitesimal characteristics. The broad classes of Markov processes with continuous trajectories be- came the main object of study. The connections between Markov pro- cesses and classical analysis were further developed. It has become possible not only to apply the results and methods of analysis to the problems of probability theory, but also to investigate analytic problems using probabilistic methods. Remarkable new connections between Markov processes and potential theory were revealed. The foundations of the theory were reviewed critically: the new concept of strong Markov process acquired for the whole theory of Markov processes great importance.

This book is devoted to the systematic exposition of the contemporary theory of controlled Markov processes with discrete time parameter or in another termi­ nology multistage Markovian decision processes. We discuss the applications of this theory to various concrete problems. Particular attention is paid to mathe­ matical models of economic planning, taking account of stochastic factors. The authors strove to construct the exposition in such a way that a reader interested in the applications can get through the book with a minimal mathe­ matical apparatus. On the other hand, a mathematician will find, in the appropriate chapters, a rigorous theory of general control models, based on advanced measure theory, analytic set theory, measurable selection theorems, and so forth. We have abstained from the manner of presentation of many mathematical monographs, in which one presents immediately the most general situation and only then discusses simpler special cases and examples. Wishing to separate out difficulties, we introduce new concepts and ideas in the simplest setting, where they already begin to work. Thus, before considering control problems on an infinite time interval, we investigate in detail the case of the finite interval. Here we first study in detail models with finite state and action spaces-a case not requiring a departure from the realm of elementary mathematics, and at the same time illustrating the most important principles of the theory.

Eugene Dynkin is a rare example of a contemporary mathematician who has achieved outstanding results in two quite different areas of research: algebra and probability. In both areas, his ideas constitute an essential part of modern mathematical knowledge and form a basis for further development. Although his last work in algebra was published in 1955, his contributions continue to influence current research in algebra and in the physics of elementary particles. His work in probability is part of both the historical and the modern development of the topic. This volume presents Dynkin's scientific contributions in both areas. Included are commentary by recognized experts in the corresponding fields who describe the time, place, role, and impact of Dynkin's research and achievements. Biographical notes and the recollections of his students are also featured.

This book is devoted to the applications of probability theory to the theory of nonlinear partial differential equations. More precisely, it is shown that all positive solutions for a class of nonlinear elliptic equations in a domain are described in terms of their traces on the boundary of the domain. The main probabilistic tool is the theory of superdiffusions, which describes a random evolution of a cloud of particles. A substantial enhancement of this theory is presented that will be of interest to anyone who works on applications of probabilistic methods to mathematical analysis. The book is suitable for graduate students and research mathematicians interested in probability theory and its applications to differential equations. Also of interest by this author is "Diffusions, Superdiffusions and Partial Differential Equations" in the "AMS" series, Colloquium Publications.

Interactions between the theory of partial differential equations of elliptic and parabolic types and the theory of stochastic processes are beneficial for both probability theory and analysis. At the beginning, mostly analytic results were used by probabilists. More recently, analysts (and physicists) took inspiration from the probabilistic approach. Of course, the development of analysis in general and of the theory of partial differential equations in particular, was motivated to a great extent by problems in physics. A difference between physics and probability is that the latter provides not only an intuition, but also rigorous mathematical tools for proving theorems. The subject of this book is connections between linear and semilinear differential equations and the corresponding Markov processes called diffusions and superdiffusions. Most of the book is devoted to a systematic presentation (in a more general setting, with simplified proofs) of the results obtained since 1988 in a series of papers of Dynkin and Dynkin and Kuznetsov. Many results obtained originally by using superdiffusions are extended in the book to more general equations by applying a combination of diffusions with purely analytic methods. Almost all chapters involve a mixture of probability and analysis. Similar to the other books by Dynkin, "Markov Processes" (Springer-Verlag), "Controlled Markov Processes" (Springer-Verlag), and "An Introduction to Branching Measure-Valued Processes" (American Mathematical Society), this book can become a classical account of the presented topics.

The theory of Markov Processes has become a powerful tool in partial differential equations and potential theory with important applications to physics. Professor Dynkin has made many profound contributions to the subject and in this volume are collected several of his most important expository and survey articles. The content of these articles has not been covered in any monograph as yet. This account is accessible to graduate students in mathematics and operations research and will be welcomed by all those interested in stochastic processes and their applications.