> ## Documentation Index
> Fetch the complete documentation index at: https://docs.germainux.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Apache Kafka Setup

> Apache Kafka provides messaging services for GermainUX Enterprise and its nodes. Kafka replaces Apache ActiveMQ in GermainUX 2024.2 and later.

Service
`Enterprise`

Availability
2024.1

> [card] [Windows installation](#windows-installation)
>
> Install through WSL2

> [card] [Configure Kafka](#configure-kafka)
>
> Steps for Linux and WSL2

> [card] [Validate the installation](#validate-the-installation)
>
> Check the Kafka setup

> [card] [Troubleshooting](#troubleshooting)
>
> Fix Kafka connection problems

**Apache Kafka Setup**

Apache Kafka provides messaging services for GermainUX Enterprise and its nodes. Kafka replaces Apache ActiveMQ in GermainUX 2024.2 and later.

> [!NOTE]
>
> **Compatibility**
>
> Use Kafka with GermainUX 2024.2 or later. Earlier GermainUX releases use Apache ActiveMQ.

The procedure below creates a single-node Kafka cluster in KRaft mode. For production environments requiring high availability, security, monitoring, or multiple brokers, follow your organization’s Kafka deployment standards.

## Prerequisites

Before installing Kafka, confirm that:

| | Requirement |
| - | - |
| 1 | JDK 17 is installed. |
| 2 | Port `9092` is available. |
| 3 | The Kafka host is reachable from Germain Enterprise and every Germain node. |
| 4 | The Kafka data directory has sufficient storage. |
| 5 | The operating-system account running Kafka can read and write the Kafka installation and data directories. |

## Windows installation

Kafka runs on Windows through Windows Subsystem for Linux 2 (WSL2).

### 1 Install WSL2

Open PowerShell as an administrator:

```text theme={null}
wsl --install
```

Restart Windows if prompted, and then open the installed Ubuntu environment.

### 2 Install JDK 17

Run the following commands in Ubuntu:

```text theme={null}
sudo -i
apt update
apt install openjdk-17-jdk
```

Verify the installation:

```text theme={null}
java -version
```

### 3 Download and install Kafka

The example below uses Kafka 3.7.1. Use the version approved for your GermainUX release.

```text theme={null}
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
```

## Linux installation

### 1 Install JDK 17

The following example applies to Ubuntu and other Debian-based distributions:

```text theme={null}
sudo -i
apt update
apt install openjdk-17-jdk
```

Verify the installation:

```text theme={null}
java -version
```

For other Linux distributions, install JDK 17 using the appropriate package manager.

### 2 Download and install Kafka

```text theme={null}
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
```

## Configure Kafka

The following configuration steps apply to both Linux and WSL2 installations.

### 1. Configure the data directory

Open the KRaft server configuration:

```text theme={null}
vi /opt/kafka/config/kraft/server.properties
```

Set the Kafka data directory:

```text theme={null}
log.dirs=/opt/kafka/data
```

Create the directory:

```text theme={null}
mkdir -p /opt/kafka/data
```

Ensure that the account running Kafka owns or can write to this directory.

### 2. Configure advertised listeners

Kafka returns the address configured in `advertised.listeners` to its clients. The address must therefore be resolvable and reachable from Germain Enterprise and every Germain node.

For a local, single-host deployment:

```text theme={null}
advertised.listeners=PLAINTEXT://localhost:9092
```

For a remote deployment, replace `localhost` with the Kafka server’s DNS name or reachable IP address:

```text theme={null}
advertised.listeners=PLAINTEXT://kafka.example.com:9092
```

Do not advertise `localhost` when GermainUX components connect from other hosts. Remote clients would interpret it as their own local machine.

If needed, also verify the listener binding:

```text theme={null}
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
```

## Initialize Kafka storage

Kafka storage must be formatted once before the broker starts for the first time.

Generate a cluster ID:

```bash theme={null}
cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
echo "$KAFKA_CLUSTER_ID"
```

Record the cluster ID for recovery and administration purposes.

Format the storage directory:

```text theme={null}
bin/kafka-storage.sh format \
  -t "$KAFKA_CLUSTER_ID" \
  -c config/kraft/server.properties
```

> [!WARNING]
>
> **Important**
>
> Do not run the format command again after Kafka begins storing data. Reformatting the storage directory can make existing data unavailable.

The cluster ID is required when formatting the storage—not each time Kafka starts.

## Start Kafka

Start Kafka in the foreground:

```text theme={null}
cd /opt/kafka
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=true"
bin/kafka-server-start.sh config/kraft/server.properties
```

A local broker is available at:

```text theme={null}
localhost:9092
```

For production use, configure Kafka as an operating-system service so it starts automatically after the host restarts.

## Validate the installation

### Confirm that Kafka is listening

```text theme={null}
ss -ltn | grep 9092
```

### Test connectivity from each GermainUX host

```text theme={null}
nc -vz <kafka-host> 9092
```

Replace `<kafka-host>` with the hostname configured in `advertised.listeners`.

### Test Kafka operations

Create a temporary topic:

```text theme={null}
cd /opt/kafka

bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create \
  --topic germainux-test \
  --partitions 1 \
  --replication-factor 1
```

List the available topics:

```text theme={null}
bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --list
```

Delete the test topic when validation is complete:

```text theme={null}
bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --delete \
  --topic germainux-test
```

## Start GermainUX

Use the following startup order:

**1. Start Kafka and wait until it is ready to accept connections**

**2. Start Germain Enterprise**

**3. Start the Germain nodes**

**4. Confirm that the GermainUX components connect to Kafka without errors**

## Production considerations

The preceding procedure uses an unencrypted `PLAINTEXT` listener and a single Kafka broker. Before using this configuration in production, consider:

* TLS encryption
* Client authentication and authorization
* Multiple brokers and replication
* Disk-capacity monitoring
* Topic retention policies
* JVM and operating-system monitoring

> [accordion] Show all 10
>
> * Automated startup and restart
> * Configuration and data backups
> * Firewall restrictions
> * Log rotation

Do not expose an unsecured Kafka listener to the public internet.

## Troubleshooting

If GermainUX cannot connect to Kafka, verify:

* Kafka is running.
* Port `9092` is listening and accessible.
* `advertised.listeners` contains a hostname reachable by every GermainUX host.
* DNS resolves the advertised hostname correctly.
* No firewall blocks the connection.
* Kafka and GermainUX use compatible security settings.
* The Kafka data directory is writable and has sufficient space.
* Kafka started before Germain Enterprise and the nodes.

Review the Kafka console output or service logs for configuration, listener, storage, and authentication errors.

Source configuration: [Apache Kafka Setup](https://docs.germainux.com/main/apache-kafka-setup.md).

```text theme={null}
sl --install
```

1. Install JDK

Open up Ubuntu command line:

```text theme={null}
sudo -i
apt update
apt install openjdk-17-jdk
```

1. Install Kafka

```text theme={null}
cd /opt
wget https://downloads.apache.org/kafka/3.7.1/kafka_2.13-3.7.1.tgz
tar xzf kafka_2.13-3.7.1.tgz
ln -s /opt/kafka_2.13-3.7.1 /opt/kafka
```

## Get Help

> [card] [Contact GermainUX Support](https://docs.germainux.com/main/support.md)
>
> The GermainUX team helps you get up and running faster, empower your team, turn data into actionable insights, and optimize performance to deliver measurable business outcomes.

> [card] Can't find what you need?
>
> Search all GermainUX docs with **Ctrl + K**, or use the feedback buttons at the bottom of this page to tell us what is missing.

The content of this document is proprietary to GERMAIN SOFTWARE.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.