Apache Kafka Setup
Important: Apache Kafka is only supported for Germain UX 2024.2 and up.
Here are setup steps and details for Apache Kafka.
Windows
Install WSL2
Open up Powershell as admin:
wsl --install
Install JDK
Open up Ubuntu command line:
sudo -i
apt update
apt install openjdk-17-jdk
Install Kafka
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
Update Kafka log directory
- Edit /opt/kafka/config/kraft/server.properties
- Update the log directory as follows: log.dirs=/opt/kafka/data
- Save the changes
- Create the /opt/kafka/data directory if it doesn't exist.
Configure Kafka storage
cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties
echo $KAFKA_CLUSTER_ID
Note down the KAFKA_CLUSTER_ID printed by the previous command. You will need to specify it each time you start your cluster.
Start Kafka
export KAFKA_CLUSTER_ID=<SUBSTITUTE YOUR KAFKA CLUSTER ID>
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=True"
bin/kafka-server-start.sh config/kraft/server.properties
The Kafka cluster can be reached via localhost:9092.
Linux
Install JDK
sudo -i
apt update
apt install openjdk-17-jdk
Install Kafka
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
Update Kafka log directory
- Edit /opt/kafka/config/kraft/server.properties
- Update the log directory as follows: log.dirs=/opt/kafka/data
- Save the changes
- Create the /opt/kafka/data directory if it doesn't exist.
Configure Kafka storage
cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties
echo $KAFKA_CLUSTER_ID
Note down the KAFKA_CLUSTER_ID printed by the previous command. You will need to specify it each time you start your cluster.
Start Kafka
export KAFKA_CLUSTER_ID=<SUBSTITUTE YOUR KAFKA CLUSTER ID>
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=True"
bin/kafka-server-start.sh config/kraft/server.properties
The Kafka cluster can be reached via localhost:9092.
Modify the advertised listeners hostname ( Optional )
If the Kafka cluster cannot be reached via localhost:9092, you must modify the advertised listeners. property in /opt/kafka/config/kraft/server.properties
- Edit /opt/kafka/config/kraft/server.properties
- Update the advertised.listeners as follows: advertised.listeners=PLAINTEXT://your_preferred_hostname:9092
- Save the changes