Skip to main content
Skip table of contents

Apache Kafka Setup

Important: Apache Kafka is only supported for Germain UX 2024.2 and up.

Here are setup steps and details for Apache Kafka.

Windows

  1. Install WSL2

Open up Powershell as admin:

CODE
wsl --install
  1. Install JDK

Open up Ubuntu command line:

CODE
sudo -i
apt update
apt install openjdk-17-jdk
  1. Install Kafka

CODE
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
  1. Update Kafka log directory

CODE
- Edit /opt/kafka/config/kraft/server.properties
- Update the log directory as follows: log.dirs=/opt/kafka/data
- Save the changes 
- Create the /opt/kafka/data directory if it doesn't exist.
  1. Configure Kafka storage

CODE
cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties
echo $KAFKA_CLUSTER_ID

Note down the KAFKA_CLUSTER_ID printed by the previous command. You will need to specify it each time you start your cluster.

  1. Start Kafka

CODE
export KAFKA_CLUSTER_ID=<SUBSTITUTE YOUR KAFKA CLUSTER ID>
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=True"
bin/kafka-server-start.sh config/kraft/server.properties

The Kafka cluster can be reached via localhost:9092.

Linux

  1. Install JDK

CODE
sudo -i
apt update
apt install openjdk-17-jdk
  1. Install Kafka

CODE
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
  1. Update Kafka log directory

CODE
- Edit /opt/kafka/config/kraft/server.properties
- Update the log directory as follows: log.dirs=/opt/kafka/data
- Save the changes 
- Create the /opt/kafka/data directory if it doesn't exist.
  1. Configure Kafka storage

CODE
cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties
echo $KAFKA_CLUSTER_ID

Note down the KAFKA_CLUSTER_ID printed by the previous command. You will need to specify it each time you start your cluster.

  1. Start Kafka

CODE
export KAFKA_CLUSTER_ID=<SUBSTITUTE YOUR KAFKA CLUSTER ID>
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=True"
bin/kafka-server-start.sh config/kraft/server.properties

The Kafka cluster can be reached via localhost:9092.

Modify the advertised listeners hostname ( Optional )

If the Kafka cluster cannot be reached via localhost:9092, you must modify the advertised listeners. property in /opt/kafka/config/kraft/server.properties

CODE
- Edit /opt/kafka/config/kraft/server.properties
- Update the advertised.listeners as follows: advertised.listeners=PLAINTEXT://your_preferred_hostname:9092
- Save the changes 
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.