Albert N. Shiryaev

This textbook is the second volume of a pair that presents the latest English edition of the author’s classic, Probability. Building on the foundations established in the preceding Probability-1, this volume guides the reader on to the theory of random processes. The new edition includes expanded material on financial mathematics and financial engineering; new problems, exercises, and proofs throughout; and a Historical Review charting the development of the mathematical theory of probability. Suitable for an advanced undergraduate or beginning graduate student with a course in probability theory, this volume forms the natural sequel to Probability-1. Probability-2 opens with classical results related to sequences and sums of independent random variables, such as the zero–one laws, convergence of series, strong law of large numbers, and the law of the iterated logarithm. The subsequent chapters go on to develop the theory of random processes with discrete time: stationary processes, martingales, and Markov processes. The Historical Review illustrates the growth from intuitive notions of randomness in history through to modern day probability theory and theory of random processes. Along with its companion volume, this textbook presents a systematic treatment of probability from the ground up, starting with intuitive ideas and gradually developing more sophisticated subjects, such as random walks, martingales, Markov chains, the measure-theoretic foundations of probability theory, weak convergence of probability measures, and the central limit theorem. Many examples are discussed in detail, and there are a large number of exercises throughout.

This two-volume textbook set presents a systematic treatment of probability from the ground up. It begins with intuitive ideas and gradually develops more sophisticated subjects, such as random walks, martingales, Markov chains, the measure-theoretic foundations of probability theory, weak convergence of probability measures, and the central limit theorem. Numerous detailed examples and exercises are provided throughout. In addition to the thorough presentation of probability, a historical exposition is presented that documents how the mathematical theory of probability has developed. The two volumes that compose this set together constitute the third English edition of the author’s classic, Probability. The first volume is particularly suitable for a graduate course on probability aimed at mathematics, statistics, or engineering students, and the second volume would be ideal for a graduate course on random process.

This textbook is the second volume of a pair that presents the latest English edition of the author’s classic, Probability. Building on the foundations established in the preceding Probability-1, this volume guides the reader on to the theory of random processes. The new edition includes expanded material on financial mathematics and financial engineering; new problems, exercises, and proofs throughout; and a Historical Review charting the development of the mathematical theory of probability. Suitable for an advanced undergraduate or beginning graduate student with a course in probability theory, this volume forms the natural sequel to Probability-1. Probability-2 opens with classical results related to sequences and sums of independent random variables, such as the zero–one laws, convergence of series, strong law of large numbers, and the law of the iterated logarithm. The subsequent chapters go on to develop the theory of random processes with discrete time: stationary processes, martingales, and Markov processes. The Historical Review illustrates the growth from intuitive notions of randomness in history through to modern day probability theory and theory of random processes. Along with its companion volume, this textbook presents a systematic treatment of probability from the ground up, starting with intuitive ideas and gradually developing more sophisticated subjects, such as random walks, martingales, Markov chains, the measure-theoretic foundations of probability theory, weak convergence of probability measures, and the central limit theorem. Many examples are discussed in detail, and there are a large number of exercises throughout.

This monograph focuses on those stochastic quickest detection tasks in disorder problems that arise in the dynamical analysis of statistical data. These include quickest detection of randomly appearing targets, of spontaneously arising effects, and of arbitrage (in financial mathematics). There is also currently great interest in quickest detection methods for randomly occurring intrusions in information systems and in the design of defense methods against cyber-attacks. The author shows that the majority of quickest detection problems can be reformulated as optimal stopping problems where the stopping time is the moment the occurrence of disorder is signaled. Thus, considerable attention is devoted to the general theory of optimal stopping rules, and to its concrete problem-solving methods. The exposition covers both the discrete time case, which is in principle relatively simple and allows step-by-step considerations, and the continuous-time case, which often requires more technical machinery such as martingales, supermartingales, and stochastic integrals. There is a focus on the well-developed apparatus of Brownian motion, which enables the exact solution of many problems. The last chapter presents applications to financial markets. Researchers and graduate students interested in probability, decision theory and statistical sequential analysis will find this book useful.

This book contains a systematic treatment of probability from the ground up, starting with intuitive ideas and gradually developing more sophisticated subjects, such as random walks, martingales, Markov chains, the measure-theoretic foundations of probability theory, weak convergence of probability measures, and the central limit theorem. Many examples are discussed in detail, and there are a large number of exercises. The book is accessible to advanced undergraduates and can be used as a text for independent study. To accommodate the greatly expanded material in the third edition of Probability, the book is now divided into two volumes. This first volume contains updated references and substantial revisions of the first three chapters of the second edition. In particular, new material has been added on generating functions, the inclusion-exclusion principle, theorems on monotonic classes (relying on a detailed treatment of “p-?” systems), and the fundamental theorems of mathematical statistics.

This book contains a systematic treatment of probability from the ground up, starting with intuitive ideas and gradually developing more sophisticated subjects, such as random walks, martingales, Markov chains, the measure-theoretic foundations of probability theory, weak convergence of probability measures, and the central limit theorem. Many examples are discussed in detail, and there are a large number of exercises. The book is accessible to advanced undergraduates and can be used as a text for independent study. To accommodate the greatly expanded material in the third edition of Probability, the book is now divided into two volumes. This first volume contains updated references and substantial revisions of the first three chapters of the second edition. In particular, new material has been added on generating functions, the inclusion-exclusion principle, theorems on monotonic classes (relying on a detailed treatment of “p-?” systems), and the fundamental theorems of mathematical statistics.

Change of Time and Change of Measure provides a comprehensive account of two topics that are of particular significance in both theoretical and applied stochastics: random change of time and change of probability law. Random change of time is key to understanding the nature of various stochastic processes, and gives rise to interesting mathematical results and insights of importance for the modeling and interpretation of empirically observed dynamic processes. Change of probability law is a technique for solving central questions in mathematical finance, and also has a considerable role in insurance mathematics, large deviation theory, and other fields. The book comprehensively collects and integrates results from a number of scattered sources in the literature and discusses the importance of the results relative to the existing literature, particularly with regard to mathematical finance. In this Second Edition a Chapter 13 entitled 'A Wider View' has been added. This outlines some of the developments that have taken place in the area of Change of Time and Change of Measure since the publication of the First Edition. Most of these developments have their root in the study of the Statistical Theory of Turbulence rather than in Financial Mathematics and Econometrics, and they form part of the new research area termed 'Ambit Stochastics'.

For the first two editions of the book Probability (GTM 95), each chapter included a comprehensive and diverse set of relevant exercises. While the work on the third edition was still in progress, it was decided that it would be more appropriate to publish a separate book that would comprise all of the exercises from previous editions, in addition to many new exercises. Most of the material in this book consists of exercises created by Shiryaev, collected and compiled over the course of many years while working on many interesting topics. Many of the exercises resulted from discussions that took place during special seminars for graduate and undergraduate students. Many of the exercises included in the book contain helpful hints and other relevant information. Lastly, the author has included an appendix at the end of the book that contains a summary of the main results, notation and terminology from Probability Theory that are used throughout the present book. This Appendix also contains additional material from Combinatorics, Potential Theory and Markov Chains, which is not covered in the book, but is nevertheless needed for many of the exercises included here.

For the first two editions of the book Probability (GTM 95), each chapter included a comprehensive and diverse set of relevant exercises. While the work on the third edition was still in progress, it was decided that it would be more appropriate to publish a separate book that would comprise all of the exercises from previous editions, in addition to many new exercises. Most of the material in this book consists of exercises created by Shiryaev, collected and compiled over the course of many years while working on many interesting topics. Many of the exercises resulted from discussions that took place during special seminars for graduate and undergraduate students. Many of the exercises included in the book contain helpful hints and other relevant information. Lastly, the author has included an appendix at the end of the book that contains a summary of the main results, notation and terminology from Probability Theory that are used throughout the present book. This Appendix also contains additional material from Combinatorics, Potential Theory and Markov Chains, which is not covered in the book, but is nevertheless needed for many of the exercises included here.

At the end of 1960s and the beginning of 1970s, when the Russian version of this book was written, the 'general theory of random processes' did not operate widely with such notions as semimartingale, stochastic integral with respect to semimartingale, the Ito formula for semimartingales, etc. At that time in stochastic calculus (theory of martingales), the main object was the square integrable martingale. In a short time, this theory was applied to such areas as nonlinear filtering, optimal stochastic control, statistics for diffusion­ type processes. In the first edition of these volumes, the stochastic calculus, based on square integrable martingale theory, was presented in detail with the proof of the Doob-Meyer decomposition for submartingales and the description of a structure for stochastic integrals. In the first volume ('General Theory') these results were used for a presentation of further important facts such as the Girsanov theorem and its generalizations, theorems on the innovation pro­ cesses, structure of the densities (Radon-Nikodym derivatives) for absolutely continuous measures being distributions of diffusion and ItO-type processes, and existence theorems for weak and strong solutions of stochastic differential equations. All the results and facts mentioned above have played a key role in the derivation of 'general equations' for nonlinear filtering, prediction, and smoothing of random processes.

At the end of 1960s and the beginning of 1970s, when the Russian version of this book was written, the 'general theory of random processes' did not operate widely with such notions as semimartingale, stochastic integral with respect to semimartingale, the ItO formula for semimartingales, etc. At that time in stochastic calculus (theory of martingales), the main object was the square integrable martingale. In a short time, this theory was applied to such areas as nonlinear filtering, optimal stochastic control, statistics for diffusion­ type processes. In the first edition of these volumes, the stochastic calculus, based on square integrable martingale theory, was presented in detail with the proof of the Doob-Meyer decomposition for submartingales and the description of a structure for stochastic integrals. In the first volume ('General Theory') these results were used for a presentation of further important facts such as the Girsanov theorem and its generalizations, theorems on the innovation pro­ cesses, structure of the densities (Radon-Nikodym derivatives) for absolutely continuous measures being distributions of diffusion and Itô-type processes, and existence theorems for weak and strong solutions of stochastic differential equations. All the results and facts mentioned above have played a key role in the derivation of 'general equations' for nonlinear filtering, prediction, and smoothing of random processes.

Change of Time and Change of Measure provides a comprehensive account of two topics that are of particular significance in both theoretical and applied stochastics: random change of time and change of probability law. Random change of time is key to understanding the nature of various stochastic processes, and gives rise to interesting mathematical results and insights of importance for the modeling and interpretation of empirically observed dynamic processes. Change of probability law is a technique for solving central questions in mathematical finance, and also has a considerable role in insurance mathematics, large deviation theory, and other fields. The book comprehensively collects and integrates results from a number of scattered sources in the literature and discusses the importance of the results relative to the existing literature, particularly with regard to mathematical finance. It is invaluable as a textbook for graduate-level courses and students or a handy reference for researchers and practitioners in financial mathematics and econometrics.

Along with conventional problems of statistics and probability, the - vestigation of problems occurring in what is now referred to as stochastic theory of optimal control also started in the 1940s and 1950s. One of the most advanced aspects of this theory is the theory of optimal stopping rules, the development of which was considerably stimulated by A. Wald, whose Sequential ~nal~sis' was published in 1947. In contrast to the classical methods of mathematical statistics, according to which the number of observations is fixed in advance, the methods of sequential analysis are characterized by the fact that the time at which the observations are terminated (stopping time) is random and is defined by the observer based on the data observed. A. Wald showed the advantage of sequential methods in the problem of testing (from independent obser- tions) two simple hypotheses. He proved that such methods yield on the average a smaller number of observations than any other method using fixed sample size (and the same probabilities of wrong decisions). Furth- more, Wald described a specific sequential procedure based on his sequ- tial probability ratio criterion which proved to be optimal in the class of all sequential methods. By the sequential method, as applied to the problem of testing two simple hypotheses, we mean a rule according to which the time at which the observations are terminated is prescribed as well as the terminal decision as to which of the two hypotheses is true.

At the end of 1960s and the beginning of 1970s, when the Russian version of this book was written, the 'general theory of random processes' did not operate widely with such notions as semimartingale, stochastic integral with respect to semimartingale, the Ito formula for semimartingales, etc. At that time in stochastic calculus (theory of martingales), the main object was the square integrable martingale. In a short time, this theory was applied to such areas as nonlinear filtering, optimal stochastic control, statistics for diffusion­ type processes. In the first edition of these volumes, the stochastic calculus, based on square integrable martingale theory, was presented in detail with the proof of the Doob-Meyer decomposition for submartingales and the description of a structure for stochastic integrals. In the first volume ('General Theory') these results were used for a presentation of further important facts such as the Girsanov theorem and its generalizations, theorems on the innovation pro­ cesses, structure of the densities (Radon-Nikodym derivatives) for absolutely continuous measures being distributions of diffusion and ItO-type processes, and existence theorems for weak and strong solutions of stochastic differential equations. All the results and facts mentioned above have played a key role in the derivation of 'general equations' for nonlinear filtering, prediction, and smoothing of random processes.

The subject of these two volumes is non-linear filtering (prediction and smoothing) theory and its application to the problem of optimal estimation, control with incomplete data, information theory and sequential testing of hypothesis.

This volume provides an exposition of some fundamental aspects of the asymptotic theory of statistical experiments. The most important of them is “how to construct asymptotically optimal decisions if we know the structure of optimal decisions for the limit experiment”.

This important book provides information necessary for those dealing with stochastic calculus and pricing in the models of financial markets operating under uncertainty; introduces the reader to the main concepts, notions and results of stochastic financial mathematics; and develops applications of these results to various kinds of calculations required in financial engineering. It also answers the requests of teachers of financial mathematics and engineering by making a bias towards probabilistic and statistical ideas and the methods of stochastic calculus in the analysis of market risks.