Learning ELK Stack

Build mesmerizing visualizations, analytics, and logs from your data using Elasticsearch, Logstash, and Kibana

About This Book

• Solve all your data analytics problems with the ELK stack
• Explore the power of Kibana4 search and visualizations built over Elasticsearch queries and learn about the features and plugins of Logstash
• Develop a complete data pipeline using the ELK stack

Who This Book Is For

If you are a developer or DevOps engineer interested in building a system that provides amazing insights and business metrics out of data sources, of various formats and types, using the open source technology stack that ELK provides, then this book is for you. Basic knowledge of Unix or any programming language will be helpful to make the most out of this book.

What You Will Learn

• Install, configure, and run Elasticsearch, Logstash, and Kibana
• Understand the need for log analytics and the current challenges in log analysis
• Build your own data pipeline using the ELK stack
• Familiarize yourself with the key features of Logstash and the variety of input, filter, and output plugins it provides
• Build your own custom Logstash plugin
• Create actionable insights using charts, histograms, and quick search features in Kibana4
• Understand the role of Elasticsearch in the ELK stack

In Detail

The ELK stack―Elasticsearch, Logstash, and Kibana, is a powerful combination of open source tools. Elasticsearch is for deep search and data analytics. Logstash is for centralized logging, log enrichment, and parsing. Kibana is for powerful and beautiful data visualizations. In short, the Elasticsearch ELK stack makes searching and analyzing data easier than ever before.

This book will introduce you to the ELK (Elasticsearch, Logstash, and Kibana) stack, starting by showing you how to set up the stack by installing the tools, and basic configuration. You’ll move on to building a basic data pipeline using the ELK stack.

Next, you’ll explore the key features of Logstash and its role in the ELK stack, including creating Logstash plugins, which will enable you to use your own customized plugins. The importance of Elasticsearch and Kibana in the ELK stack is also covered, along with various types of advanced data analysis, and a variety of charts, tables ,and maps.

Finally, by the end of the book you will be able to develop full-fledged data pipeline using the ELK stack and have a solid understanding of the role of each of the components.

Style and approach

This book is a step-by-step guide, complete with various examples to solve your data analytics problems by using the ELK stack to explore and visualize data.

Table of Contents

Chapter 1: Introduction to ELK Stack
Chapter 2: Building Your First Data Pipeline with ELK
Chapter 3: Collect, Parse and Transform Data with Logstash
Chapter 4: Creating Custom Logstash Plugins
Chapter 5: Why Do We Need Elasticsearch in ELK?
Chapter 6: Finding Insights with Kibana
Chapter 7: Kibana – Visualization and Dashboard
Chapter 8: Putting It All Together
Chapter 9: ELK Stack in Production
Chapter 10: Expanding Horizons with ELK