Bert Fristedt

Our purpose in writing this monograph is to give a comprehensive treatment of the subject. We define bandit problems and give the necessary foundations in Chapter 2. Many of the important results that have appeared in the literature are presented in later chapters; these are interspersed with new results. We give proofs unless they are very easy or the result is not used in the sequel. We have simplified a number of arguments so many of the proofs given tend to be conceptual rather than calculational. All results given have been incorporated into our style and notation. The exposition is aimed at a variety of types of readers. Bandit problems and the associated mathematical and technical issues are developed from first principles. Since we have tried to be comprehens­ ive the mathematical level is sometimes advanced; for example, we use measure-theoretic notions freely in Chapter 2. But the mathema­ tically uninitiated reader can easily sidestep such discussion when it occurs in Chapter 2 and elsewhere. We have tried to appeal to graduate students and professionals in engineering, biometry, econ­ omics, management science, and operations research, as well as those in mathematics and statistics. The monograph could serve as a reference for professionals or as a telA in a semester or year-long graduate level course.

Filtering and prediction is about observing moving objects when the observations are corrupted by random errors. The main focus is then on filtering out the errors and extracting from the observations the most precise information about the object, which itself may or may not be moving in a somewhat random fashion. Next comes the prediction step where, using information about the past behavior of the object, one tries to predict its future path. The first three chapters of the book deal with discrete probability spaces, random variables, conditioning, Markov chains, and filtering of discrete Markov chains. The next three chapters deal with the more sophisticated notions of conditioning in nondiscrete situations, filtering of continuous-space Markov chains, and of Wiener process. Filtering and prediction of stationary sequences is discussed in the last two chapters. The authors believe that they have succeeded in presenting necessary ideas in an elementary manner without sacrificing the rigor too much. Such rigorous treatment is lacking at this level in the literature. In the past few years the material in the book was offered as a one-semester undergraduate/beginning graduate course at the University of Minnesota. Some of the many problems suggested in the text were used in homework assignments.