Wes McKinney

From fundamental techniques to advanced strategies for handling big data, visualization, and more, this book equips you with skills to excel in real-world data analysis projects. Key Features This book targets features in pandas 2.x and beyond Practical, easy to implement recipes for quick solutions to common problems in data using pandas Master the fundamentals of pandas to quickly begin exploring any dataset Book DescriptionUnlock the full power of pandas 2.x with this hands-on cookbook, designed for Python developers, data analysts, and data scientists who need fast, efficient solutions for real-world data challenges. This book provides practical, ready-to-use recipes to streamline your workflow. With step-by-step guidance, you'll master data wrangling, visualization, performance optimization, and scalable data analysis using pandas’ most powerful features. From importing and merging large datasets to advanced time series analysis and SQL-like operations, this cookbook equips you with the tools to analyze, manipulate, and visualize data like a pro. Learn how to boost efficiency, optimize memory usage, and seamlessly integrate pandas with NumPy, PyArrow, and databases. This book will help you transform raw data into actionable insights with ease. What you will learn The pandas type system and how to best navigate it Import/export DataFrames to/from common data formats Data exploration in pandas through dozens of practice problems Grouping, aggregation, transformation, reshaping, and filtering data Merge data from different sources through pandas SQL-like operations Leverage the robust pandas time series functionality in advanced analyses Scale pandas operations to get the most out of your system The large ecosystem that pandas can coordinate with and supplement Who this book is forThis book is for Python developers, data scientists, engineers, and analysts. pandas is the ideal tool for manipulating structured data with Python and this book provides ample instruction and examples. Not only does it cover the basics required to be proficient, but it goes into the details of idiomatic pandas

Harness the power of Apache Arrow to optimize tabular data processing and develop robust, high-performance data systems with its standardized, language-independent columnar memory format Key Features Explore Apache Arrow's data types and integration with pandas, Polars, and Parquet Work with Arrow libraries such as Flight SQL, Acero compute engine, and Dataset APIs for tabular data Enhance and accelerate machine learning data pipelines using Apache Arrow and its subprojects Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionApache Arrow is an open source, columnar in-memory data format designed for efficient data processing and analytics. This book harnesses the author’s 15 years of experience to show you a standardized way to work with tabular data across various programming languages and environments, enabling high-performance data processing and exchange. This updated second edition gives you an overview of the Arrow format, highlighting its versatility and benefits through real-world use cases. It guides you through enhancing data science workflows, optimizing performance with Apache Parquet and Spark, and ensuring seamless data translation. You’ll explore data interchange and storage formats, and Arrow's relationships with Parquet, Protocol Buffers, FlatBuffers, JSON, and CSV. You’ll also discover Apache Arrow subprojects, including Flight, SQL, Database Connectivity, and nanoarrow. You’ll learn to streamline machine learning workflows, use Arrow Dataset APIs, and integrate with popular analytical data systems such as Snowflake, Dremio, and DuckDB. The latter chapters provide real-world examples and case studies of products powered by Apache Arrow, providing practical insights into its applications. By the end of this book, you’ll have all the building blocks to create efficient and powerful analytical services and utilities with Apache Arrow. What you will learn Use Apache Arrow libraries to access data files, both locally and in the cloud Understand the zero-copy elements of the Apache Arrow format Improve the read performance of data pipelines by memory-mapping Arrow files Produce and consume Apache Arrow data efficiently by sharing memory with the C API Leverage the Arrow compute engine, Acero, to perform complex operations Create Arrow Flight servers and clients for transferring data quickly Build the Arrow libraries locally and contribute to the community Who this book is forThis book is for developers, data engineers, and data scientists looking to explore the capabilities of Apache Arrow from the ground up. Whether you’re building utilities for data analytics and query engines, or building full pipelines with tabular data, this book can help you out regardless of your preferred programming language. A basic understanding of data analysis concepts is needed, but not necessary. Code examples are provided using C++, Python, and Go throughout the book.

Get the definitive handbook for manipulating, processing, cleaning, and crunching datasets in Python. Updated for Python 3.10 and pandas 1.4, the third edition of this hands-on guide is packed with practical case studies that show you how to solve a broad set of data analysis problems effectively. You'll learn the latest versions of pandas, NumPy, and Jupyter in the process. Written by Wes McKinney, the creator of the Python pandas project, this book is a practical, modern introduction to data science tools in Python. It's ideal for analysts new to Python and for Python programmers new to data science and scientific computing. Data files and related material are available on GitHub. Use the Jupyter notebook and IPython shell for exploratory computing Learn basic and advanced features in NumPy Get started with data analysis tools in the pandas library Use flexible tools to load, clean, transform, merge, and reshape data Create informative visualizations with matplotlib Apply the pandas groupby facility to slice, dice, and summarize datasets Analyze and manipulate regular and irregular time series data Learn how to solve real-world data analysis problems with thorough, detailed examples

Process tabular data and build high-performance query engines on modern CPUs and GPUs using Apache Arrow, a standardized language-independent memory format, for optimal performance Key Features Learn about Apache Arrow's data types and interoperability with pandas and Parquet Work with Apache Arrow Flight RPC, Compute, and Dataset APIs to produce and consume tabular data Reviewed, contributed, and supported by Dremio, the co-creator of Apache Arrow Book DescriptionApache Arrow is designed to accelerate analytics and allow the exchange of data across big data systems easily. In-Memory Analytics with Apache Arrow begins with a quick overview of the Apache Arrow format, before moving on to helping you to understand Arrow’s versatility and benefits as you walk through a variety of real-world use cases. You'll cover key tasks such as enhancing data science workflows with Arrow, using Arrow and Apache Parquet with Apache Spark and Jupyter for better performance and hassle-free data translation, as well as working with Perspective, an open source interactive graphical and tabular analysis tool for browsers. As you advance, you'll explore the different data interchange and storage formats and become well-versed with the relationships between Arrow, Parquet, Feather, Protobuf, Flatbuffers, JSON, and CSV. In addition to understanding the basic structure of the Arrow Flight and Flight SQL protocols, you'll learn about Dremio’s usage of Apache Arrow to enhance SQL analytics and discover how Arrow can be used in web-based browser apps. Finally, you'll get to grips with the upcoming features of Arrow to help you stay ahead of the curve. By the end of this book, you will have all the building blocks to create useful, efficient, and powerful analytical services and utilities with Apache Arrow. What you will learn Use Apache Arrow libraries to access data files both locally and in the cloud Understand the zero-copy elements of the Apache Arrow format Improve read performance by memory-mapping files with Apache Arrow Produce or consume Apache Arrow data efficiently using a C API Use the Apache Arrow Compute APIs to perform complex operations Create Arrow Flight servers and clients for transferring data quickly Build the Arrow libraries locally and contribute back to the community Who this book is forThis book is for developers, data analysts, and data scientists looking to explore the capabilities of Apache Arrow from the ground up. This book will also be useful for any engineers who are working on building utilities for data analytics and query engines, or otherwise working with tabular data, regardless of the programming language. Some familiarity with basic concepts of data analysis will help you to get the most out of this book but isn't required. Code examples are provided in the C++, Go, and Python programming languages.