Filip Radlinski

Over the years, information retrieval and search systems have become more conversational. The last few years have seen a tremendous acceleration of this evolution driven by progress in machine learning. Whereas the possibility of a conversational information seeking (CIS) system robustly understanding conversational input from a person was previously limited, it can now almost be taken for granted. Consumer hardware that supports and encourages conversation is now common, raising awareness of--and the expectation of--conversational support in information retrieval systems. From the research community, this has been accompanied by significant progress in defining more natural CIS tasks, metrics, challenges, and benchmarks. As these developments allowed the field to expand rapidly, this monograph aims to summarise the current state-of-the-art of CIS research, and provide an introduction to new researchers as well as a reference for established researchers in this area. This monograph provides a thorough overview of CIS definitions, applications, interactions, interfaces, design, implementation, and evaluation. The authors view CIS applications as including conversational search, conversational question answering, and conversational recommendation. Their aim is to provide an overview of past research related to CIS, introduce the current state-of-the-art in CIS, highlight the challenges still being faced in the community, and suggest future directions.

Online evaluation is one of the most common approaches to measure the effectiveness of an information retrieval system. It involves fielding the information retrieval system to real users, and observing these users' interactions in situ while they engage with the system. This allows actual users with real world information needs to play an important part in assessing retrieval quality. Online Evaluation for Information Retrieval provides the reader with a comprehensive overview of the topic. It shows how online evaluation is used for controlled experiments, segmenting them into experiment designs that allow absolute or relative quality assessments. The presentation of different metrics further partitions online evaluation based on different sized experimental units commonly of interest: documents, lists, and sessions. It also includes an extensive discussion of recent work on data re-use, and experiment estimation based on historical data. This book pays particular attention to practical issues: How to run evaluations in practice, how to select experimental parameters, how to take into account ethical considerations inherent in online evaluations, and limitations that experimenters should be aware of. While most published work on online experimentation today is on a large scale in systems with millions of users, this monograph also emphasizes that the same techniques can be applied on a small scale. To this end, it highlights recent work that makes it easier to use at smaller scales and encourages studying real-world information seeking in a wide range of scenarios. The monograph concludes with a summary of the most recent work in the area, and outlines some open problems, as well as postulating future directions.