Apache Kafka Setup

⚙️ Apache Kafka Setup

Apache Kafka provides messaging services for GermainUX Enterprise and its nodes. Kafka replaces Apache ActiveMQ in GermainUX 2024.2 and later.

The procedure below creates a single-node Kafka cluster in KRaft mode. For production environments requiring high availability, security, monitoring, or multiple brokers, follow your organization’s Kafka deployment standards.

📋 Prerequisites

Before installing Kafka, confirm that:

💻 Windows installation

Kafka runs on Windows through Windows Subsystem for Linux 2 (WSL2).

1️⃣ Install WSL2

Open PowerShell as an administrator:

wsl --install

Restart Windows if prompted, and then open the installed Ubuntu environment.

2️⃣ Install JDK 17

Run the following commands in Ubuntu:

sudo -i
apt update
apt install openjdk-17-jdk

Verify the installation:

java -version

3️⃣ Download and install Kafka

The example below uses Kafka 3.7.1. Use the version approved for your GermainUX release.

cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka

🐧 Linux installation

1️⃣ Install JDK 17

The following example applies to Ubuntu and other Debian-based distributions:

sudo -i
apt update
apt install openjdk-17-jdk

Verify the installation:

java -version

For other Linux distributions, install JDK 17 using the appropriate package manager.

2️⃣ Download and install Kafka

cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka

🔧 Configure Kafka

The following configuration steps apply to both Linux and WSL2 installations.

💾 1. Configure the data directory

Open the KRaft server configuration:

vi /opt/kafka/config/kraft/server.properties

Set the Kafka data directory:

log.dirs=/opt/kafka/data

Create the directory:

mkdir -p /opt/kafka/data

Ensure that the account running Kafka owns or can write to this directory.

📡 2. Configure advertised listeners

Kafka returns the address configured in advertised.listeners to its clients. The address must therefore be resolvable and reachable from Germain Enterprise and every Germain node.

For a local, single-host deployment:

advertised.listeners=PLAINTEXT://localhost:9092

For a remote deployment, replace localhost with the Kafka server’s DNS name or reachable IP address:

advertised.listeners=PLAINTEXT://kafka.example.com:9092

Do not advertise localhost when GermainUX components connect from other hosts. Remote clients would interpret it as their own local machine.

If needed, also verify the listener binding:

listeners=PLAINTEXT://:9092,CONTROLLER://:9093

📁 Initialize Kafka storage

Kafka storage must be formatted once before the broker starts for the first time.

Generate a cluster ID:

cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
echo "$KAFKA_CLUSTER_ID"

Record the cluster ID for recovery and administration purposes.

Format the storage directory:

bin/kafka-storage.sh format \
  -t "$KAFKA_CLUSTER_ID" \
  -c config/kraft/server.properties

The cluster ID is required when formatting the storage—not each time Kafka starts.

🚀 Start Kafka

Start Kafka in the foreground:

cd /opt/kafka
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=true"
bin/kafka-server-start.sh config/kraft/server.properties

A local broker is available at:

localhost:9092

For production use, configure Kafka as an operating-system service so it starts automatically after the host restarts.

✅ Validate the installation

📱 Confirm that Kafka is listening

ss -ltn | grep 9092

🔗 Test connectivity from each GermainUX host

nc -vz <kafka-host> 9092

Replace <kafka-host> with the hostname configured in advertised.listeners.

🛠️ Test Kafka operations

Create a temporary topic:

cd /opt/kafka

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create \
  --topic germainux-test \
  --partitions 1 \
  --replication-factor 1

List the available topics:

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --list

Delete the test topic when validation is complete:

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --delete \
  --topic germainux-test

▶️ Start GermainUX

Use the following startup order:

1. Start Kafka and wait until it is ready to accept connections.

2. Start Germain Enterprise.

3. Start the Germain nodes.

4. Confirm that the GermainUX components connect to Kafka without errors.

Production considerations

The preceding procedure uses an unencrypted PLAINTEXT listener and a single Kafka broker. Before using this configuration in production, consider:

Do not expose an unsecured Kafka listener to the public internet.

🔍 Troubleshooting

If GermainUX cannot connect to Kafka, verify:

Review the Kafka console output or service logs for configuration, listener, storage, and authentication errors.

Source configuration: Apache Kafka Setup.

sl --install

2. Install JDK

Open up Ubuntu command line:

sudo -i
apt update
apt install openjdk-17-jdk

3. Install Kafka

cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka

ℹ️