Change Data Capture (CDC) with PostgreSQL and Debezium

This post briefly documents the process of Change Data Capture (CDC) with PostgreSQL and Debezium open source tools.

Introduction

Change data capture (CDC) refers to the process of identifying and capturing changes made to data in a database and then delivering those changes in real-time to a downstream process or system

PostgreSQL, also known as Postgres, is a free and open-source relational database management system emphasizing extensibility and SQL compliance.

Debezium is an open source distributed platform for change data capture.

Minimum Software Requirements

• Docker (PostgreSQL, Zookeeper, Kafka, Debezium, schema-registry)
• Docker Compose

Getting Started

Setup

DB Configuration

CREATE TABLE STUDENT(id INTEGER PRIMARY KEY, name VARCHAR);

ALTER TABLE public.student REPLICA IDENTITY FULL;

SELECT * FROM STUDENT;

Setup debezium Connector

Tail kafka CDC topic

Add, Update, Delete Records in the Table and see it in the kafka topic.

INSERT INTO STUDENT(id,name) VALUES (1,'JOHN');
INSERT INTO STUDENT(id,name) VALUES (2,'JANE');

UPDATE STUDENT SET name='JOHNNY' WHERE ID=1;