Part 2: Installing Kafka Connect and Streaming Data from SQL Server with the Debezium Connector

In the previous part, we saw how to use Strimzi to bring up a Kafka cluster and a way to inspect the broker and topics using Kafka-UI.

In this part, we will install Kafka connect with the Debezium SQL Server connector and stream data changes to Kafka topics.

Please pull down the github repo here to get access to the files used in this blog post.

Create SQL server with the AdventureWorks DB

For this example, I'm running a simple SQL server instance in the Kubernetes cluster with the adventure works database from Microsoft. An image with the adventure works DB packaged with SQL server can be found here. Here is the yaml used,

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sqlserver
  namespace: kafka
  labels:
    app: sqlserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sqlserver
  template:
    metadata:
      labels:
        app: sqlserver
    spec:
      containers:
      - name: sqlserver
        image: chriseaton/adventureworks:latest
        ports:
        - containerPort: 1433
        env:
        - name: SA_PASSWORD
          value: "Your-SA-Password"
        - name: ACCEPT_EULA
          value: "Y"
        - name: MSSQL_AGENT_ENABLED
          value…