Kafka Development and Best Practices

This course provides a technical introduction to Kafka development. The course describes the design goals underpinning Kafka, and explains key Kafka concepts such as partitions, replication, and consumer groups. The course then leads you on a detailed tour of the Kafka APIs in Java, covering the Producer API, the Consumer API, and the Streams API.

The course also describes Kafka design patterns and best practices, so you use Kafka as part of an event-driven architecture and avoid common pitfalls.

Duration

2 days

 

Prerequisites

  • Solid programming experience in Java or a similar language

What you'll learn

  • Understanding Kafka design goals and architecture
  • Setting up a development environment for Kafka
  • Using the Kafka Producer and Consumer APIs
  • Using the Streams API
  • Kafka design patterns and best practices

Course details

Introduction to Kafka

  • What is Kafka
  • Kafka design goals and architecture
  • Getting Kafka

Understanding Kafka Concepts and Terminology

  • Topics and partitions
  • Partitions and replication
  • Consumers
  • Kafka commands

Using the Kafka Producer API

  • Overview
  • Defining a simple producer
  • Defining a custom partitioner
  • Defining a custom serializer
  • Defining a custom interceptor
  • Producer patterns and best practices

Using the Kafka Consumer API

  • Overview
  • Defining a single consumer
  • Defining multiple consumers and groups
  • Defining a custom deserializer
  • Defining a custom interceptor
  • Consumer patterns and best practices

Using the Streams API

  • Overview
  • Defining a topology
  • Special types of processors in a topology
  • Understanding the Kafka Streams API
  • Defining a minimal topology
  • Additional techniques

Design Patterns and Best Practices

  • Data schema governance
  • Types of messages
  • Event-driven design
  • CQRS
  • Eventual consistency