Apache Kafka For Beginners

Start Here - Learn Apache Kafka Ecosystem, Architecture, and Client API Programming in Java

Last updated 2022-01-10 | 4.6

- Apache Kafka Ecosystem
- Components and Big Picture
- Kafka Architecture and Core Concepts
- Multi-node Confluent Community Kafka

What you'll learn

Apache Kafka Ecosystem
Components and Big Picture
Kafka Architecture and Core Concepts
Multi-node Confluent Community Kafka
Kafka Storage Architecture in depth
Kafka Cluster Architecture in depth
Kafka Producer API Programming in Java
Kafka Consumer API Programming in Java
Idempotence and Transactions in Kafka
JSON and AVRO Serialization
Exactly Once Processing in Kafka

* Requirements

* You must be familiar with Java Programming

Description

I designed this course for absolute beginners and intermediate Kafka professionals. This course will deliver What? Why? And How? of Apache Kafka and take you deep into Client API programming in Java.

About the Course

I am creating Apache Kafka for absolute beginners course to help you understand the Apache Kafka Stack, the architecture of Kafka components, Kafka Client APIs (Producers and Consumers) and apply that knowledge to create Kafka programs in Java.

Who should take this course?

This course is designed for software engineers, solution architects, and managers willing to implement Kafka and solve real-time stream processing problems. I am also creating this course for data architects and data engineers who are responsible for designing and building the organization’s data-centric infrastructure. Another group of people is the managers and architects who do not directly work with Kafka implementation. Still, they work with the people who implement Kafka Streams at the ground level.

Who this course is for:

  • Anyone who wants to become familiar with Kafka Technology Ecosystem and its application
  • Software Engineers and Architects who are willing to design and develop Kafka Solutions

Course content

10 sections • 55 lectures

What is Apache Kafka Preview 10:00

Apache Kafka Core Concepts Preview 17:49

Kafka Connect Core Concepts Preview 20:52

Kafka Streams Core Concepts Preview 13:36

Kafka SQL Core Concepts Preview 04:37

When to use What? Preview 07:33

Source Code and Other Resources Preview 00:11

Kafka Quick Start Preview 01:21

Installing Single Node Kafka Cluster Preview 09:18

Using Command-Line Producer and Consumer Preview 06:33

Installing a Multi-Node Kafka Cluster Preview 07:30

Using Consumer Groups Preview 08:04

Configuring your development IDE Preview 06:10

Understanding Kafka Storage Architecture Preview 03:10

Kafka Topics and Partitions Preview 06:00

Kafka Topic Replication Preview 02:21

Partition Leaders and Followers Preview 02:04

Kafka Log Segments Preview 02:49

Kafka Message Offsets Preview 02:20

Kafka Message Index Preview 04:49

Understanding Kafka Cluster Preview 02:38

Zookeeper in Kafka Preview 03:18

Kafka Cluster Controller Preview 04:24

Partition Allocation and Fault Tolerance Preview 10:15

Partition Leader Vs Partition Follower Preview 04:53

The ISR List - In Sync Replica Preview 04:34

Committed Vs Un-Committed Records Preview 03:40

Minimum ISR List Preview 02:55

Introducing Kafka Producers Preview 01:22

Creating your first Kafka Producer Preview 10:42

Producer Record Preview 02:29

Producer Serializer Preview 02:12

Producer Partitioner Preview 04:11

Message Timestamp Preview 03:16

Producer Message Buffer Preview 02:40

Producer IO Thread and Retires Preview 01:13

Summarizing Producer Internals Preview 01:58

Horizontal Vs. Vertical Scalability Preview 03:10

Producer Multi-Threading Scenario Preview 01:52

Creating Multi-Threaded Kafka Producer Preview 11:46

At Least Once Vs. At Most Once Preview 03:41

Exactly Once - Producer Idempotence Preview 03:32

Transactions in Kafka Producer Preview 12:50

Working with Types and Serialization Preview 03:22

Using JSON Schema Preview 08:38

Using AVRO Schema Preview 07:38

POS Simulator using JSON Serialized Invoices Preview 09:42

POS Simulator using AVRO Serialized Invoices Preview 10:28

Introducing Kafka Consumers Preview 01:28

Creating Kafka Consume - Transform - Produce Pipeline Preview 11:24

Consumer Group and Scalability Preview 05:02

Consumer Positions - Current Offset Vs. Committed Offset Preview 04:25

Challenges with Kafka Consumers? Preview 03:39

Where to go Next? Preview 01:18

Bonus Lecture : Get Extra Preview 00:27