Maosong Fu

This book provides both a basic understanding of stream processing in general, and practical guidance for development and research with Apache Heron in particular. It delivers to developers of streaming applications basic and systematic knowledge about Heron, which is today only scattered across project documents, technique blogs and code snippets on the Web. The book is organized in four parts: Part I describes basic knowledge about stream processing, Apache Storm, and Apache Heron (Incubating), and also introduces the Heron source repository. Part II then goes into details and describes two data models to write Heron topologies and often used topology features, including stateful processing. This part is especially targeted at software developers who write topologies using Heron APIs. Next, part III describes Heron tools, including the command-line interface and the user interface, needed to manage a single topology or multiple topologies in a data center. This part is particularlyaimed at operators who deploy and manage running jobs. Eventually, part IV describes the Heron source code and how to customize or extend Heron. This part is especially suggested for software engineers who would like to contribute code to the Heron repository and who are curious about Heron insights. Overall, this book aims at professionals who want to process streaming data based on Apache Heron. A basic knowledge of Java and Bash commands for Linux is assumed.

For data engineers and researchers new to stream processing, this practical book provides a professional reference that focuses on Apache Heron. Authors Huijun Wu and Maosong Fu from Twitter provide the basic knowledge you need to get started with this real-time processing engine. Learn how Heron serves as a general-purpose, modular, and extensible platform that you can use to support common real-time analytics use cases. Through the course of this book, you'll discover approaches for tackling challenges in stream processing systems and applications. You'll also understand how to build streaming applications that can benefit from Heron's robustness, high performance, adaptability to cloud environments, and ease of use. With this book, you'll examine: A complete study path that shows you how to develop stream processing systemsHeron's data model, system, topology submission process, architecture, and componentsHow to compile the Heron source codeMethods for migrating Apache Storm's topology to HeronHeron components, including state manager, scheduler, topology master, stream manager, instance, metrics manager, and metrics cacheHeron tools, including tracker, UI, and explorerNew features, such as health manager, Python topology, delivery semantics, and API server