Apache Kafka Setup - GermainUX Docs

ServiceEnterprise

Availability2024.1

Windows installation

Install through WSL2

Configure Kafka

Steps for Linux and WSL2

Validate the installation

Check the Kafka setup

Troubleshooting

Fix Kafka connection problems

Apache Kafka Setup Apache Kafka provides messaging services for GermainUX Enterprise and its nodes. Kafka replaces Apache ActiveMQ in GermainUX 2024.2 and later.

CompatibilityUse Kafka with GermainUX 2024.2 or later. Earlier GermainUX releases use Apache ActiveMQ.

The procedure below creates a single-node Kafka cluster in KRaft mode. For production environments requiring high availability, security, monitoring, or multiple brokers, follow your organization’s Kafka deployment standards.

Prerequisites

Before installing Kafka, confirm that:

	Requirement
1	JDK 17 is installed.
2	Port `9092` is available.
3	The Kafka host is reachable from Germain Enterprise and every Germain node.
4	The Kafka data directory has sufficient storage.
5	The operating-system account running Kafka can read and write the Kafka installation and data directories.

Windows installation

Kafka runs on Windows through Windows Subsystem for Linux 2 (WSL2).

1 Install WSL2

Open PowerShell as an administrator:

wsl --install

Restart Windows if prompted, and then open the installed Ubuntu environment.

2 Install JDK 17

Run the following commands in Ubuntu:

sudo -i
apt update
apt install openjdk-17-jdk

Verify the installation:

java -version

3 Download and install Kafka

The example below uses Kafka 3.7.1. Use the version approved for your GermainUX release.

cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka

Linux installation

1 Install JDK 17

The following example applies to Ubuntu and other Debian-based distributions:

sudo -i
apt update
apt install openjdk-17-jdk

Verify the installation:

java -version

For other Linux distributions, install JDK 17 using the appropriate package manager.

2 Download and install Kafka

cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka

Configure Kafka

The following configuration steps apply to both Linux and WSL2 installations.

1. Configure the data directory

Open the KRaft server configuration:

vi /opt/kafka/config/kraft/server.properties

Set the Kafka data directory:

log.dirs=/opt/kafka/data

Create the directory:

mkdir -p /opt/kafka/data

Ensure that the account running Kafka owns or can write to this directory.

2. Configure advertised listeners

Kafka returns the address configured in advertised.listeners to its clients. The address must therefore be resolvable and reachable from Germain Enterprise and every Germain node. For a local, single-host deployment:

advertised.listeners=PLAINTEXT://localhost:9092

For a remote deployment, replace localhost with the Kafka server’s DNS name or reachable IP address:

advertised.listeners=PLAINTEXT://kafka.example.com:9092

Do not advertise localhost when GermainUX components connect from other hosts. Remote clients would interpret it as their own local machine. If needed, also verify the listener binding:

listeners=PLAINTEXT://:9092,CONTROLLER://:9093

Initialize Kafka storage

Kafka storage must be formatted once before the broker starts for the first time. Generate a cluster ID:

cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
echo "$KAFKA_CLUSTER_ID"

Record the cluster ID for recovery and administration purposes. Format the storage directory:

bin/kafka-storage.sh format \
  -t "$KAFKA_CLUSTER_ID" \
  -c config/kraft/server.properties

ImportantDo not run the format command again after Kafka begins storing data. Reformatting the storage directory can make existing data unavailable.

The cluster ID is required when formatting the storage—not each time Kafka starts.

Start Kafka

Start Kafka in the foreground:

cd /opt/kafka
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=true"
bin/kafka-server-start.sh config/kraft/server.properties

A local broker is available at:

localhost:9092

For production use, configure Kafka as an operating-system service so it starts automatically after the host restarts.

Validate the installation

Confirm that Kafka is listening

ss -ltn | grep 9092

Test connectivity from each GermainUX host

nc -vz <kafka-host> 9092

Replace <kafka-host> with the hostname configured in advertised.listeners.

Test Kafka operations

Create a temporary topic:

cd /opt/kafka

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create \
  --topic germainux-test \
  --partitions 1 \
  --replication-factor 1

List the available topics:

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --list

Delete the test topic when validation is complete:

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --delete \
  --topic germainux-test

Start GermainUX

Use the following startup order:

Start Kafka and wait until it is ready to accept connections

Start Germain Enterprise

Start the Germain nodes

Confirm that the GermainUX components connect to Kafka without errors

Production considerations

The preceding procedure uses an unencrypted PLAINTEXT listener and a single Kafka broker. Before using this configuration in production, consider:

TLS encryption
Client authentication and authorization
Multiple brokers and replication
Disk-capacity monitoring
Topic retention policies
JVM and operating-system monitoring

Show all 10

Automated startup and restart
Configuration and data backups
Firewall restrictions
Log rotation

Do not expose an unsecured Kafka listener to the public internet.

Troubleshooting

If GermainUX cannot connect to Kafka, verify:

Kafka is running.
Port 9092 is listening and accessible.
advertised.listeners contains a hostname reachable by every GermainUX host.
DNS resolves the advertised hostname correctly.
No firewall blocks the connection.
Kafka and GermainUX use compatible security settings.
The Kafka data directory is writable and has sufficient space.
Kafka started before Germain Enterprise and the nodes.

Review the Kafka console output or service logs for configuration, listener, storage, and authentication errors. Source configuration: Apache Kafka Setup.

sl --install

Install JDK

Open up Ubuntu command line:

sudo -i
apt update
apt install openjdk-17-jdk

Install Kafka

cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka

Get Help

Contact GermainUX Support

The GermainUX team helps you get up and running faster, empower your team, turn data into actionable insights, and optimize performance to deliver measurable business outcomes.

Can't find what you need?

Search all GermainUX docs with Ctrl + K, or use the feedback buttons at the bottom of this page to tell us what is missing.

The content of this document is proprietary to GERMAIN SOFTWARE.

Windows installation

Configure Kafka

Validate the installation

Troubleshooting

​Prerequisites

​Windows installation

​1 Install WSL2

​2 Install JDK 17

​3 Download and install Kafka

​Linux installation

​1 Install JDK 17

​2 Download and install Kafka

​Configure Kafka

​1. Configure the data directory

​2. Configure advertised listeners

​Initialize Kafka storage

​Start Kafka

​Validate the installation

​Confirm that Kafka is listening

​Test connectivity from each GermainUX host

​Test Kafka operations

​Start GermainUX

​Production considerations

​Troubleshooting

​Get Help

Contact GermainUX Support

Can't find what you need?

Prerequisites

Windows installation

1 Install WSL2

2 Install JDK 17

3 Download and install Kafka

Linux installation

1 Install JDK 17

2 Download and install Kafka

Configure Kafka

1. Configure the data directory

2. Configure advertised listeners

Initialize Kafka storage

Start Kafka

Validate the installation

Confirm that Kafka is listening

Test connectivity from each GermainUX host

Test Kafka operations

Start GermainUX

Production considerations

Troubleshooting

Get Help