Apache ActiveMQ Troubleshooting
Legacy deployments only
Apache ActiveMQ applies only to GermainUX releases earlier than . Beginning with GermainUX , Kafka replaces ActiveMQ, which is no longer supported. For current deployments, see Apache Kafka Setup.
Use this guide to diagnose ActiveMQ startup failures, connection errors, queue backlogs, and storage problems in legacy GermainUX environments.
🔍 Initial checks
Before changing the configuration or deleting any data:
|
Check |
|---|
|
Confirm that ActiveMQ is running. |
|
Verify that it started before Germain Enterprise and the Germain nodes. |
|
Check the ActiveMQ and GermainUX logs for the first relevant error. |
|
Confirm that the broker host and port are reachable. |
|
Check available disk space and memory. |
|
Review queue depth, active consumers, and the dead-letter queue. 
|
Restarting the broker may temporarily restore service, but it can also hide the original cause. Preserve the relevant logs before restarting it.
❌ ActiveMQ does not start
Check the ActiveMQ log for configuration, port, permission, Java, or storage errors.
Common causes include:
|
Cause |
|---|
|
Another process is already using the broker port. |
|
The XML configuration is invalid. |
|
The ActiveMQ service account cannot access its installation or data directories. |
|
The persistent store has no available disk space. |
|
The configured Java version is unavailable or incompatible. |
|
ActiveMQ was not configured using the GermainUX-provided configuration. |
Check the broker status:
bin/activemq status
Start the broker manually to expose startup errors:
bin/activemq start xbean:conf/activemq.conf
Verify that the GermainUX ActiveMQ configuration supplied under the following directory was installed correctly:
ServiceDistribution/install/activemq
See Apache ActiveMQ Setup for the complete legacy setup procedure.
🔗 GermainUX cannot connect to ActiveMQ
Verify the broker URL configured in GermainUX:
/config/germain/application/germain.messaging.brokerUrl
Example:
failover:(tcp://localhost:61616?daemon=true&keepAlive=true)
Confirm that:
|
Item |
|---|
|
The hostname resolves from every Germain Enterprise and node host. |
|
The configured port is listening. |
|
Firewalls and network policies allow the connection. |
|
The broker URL uses the correct protocol, hostname, and port. |
|
Credentials are valid when authentication is enabled. |
|
The ActiveMQ service is fully started before GermainUX services start. |
|
TLS certificates are trusted and have not expired when SSL is enabled. |
Test basic network connectivity from the affected GermainUX host:
nc -vz <broker-host> 61616
If nc is unavailable, use another approved TCP connectivity test.
💾 Persistent store is full
A full persistent store can stop ActiveMQ from accepting messages and may prevent the broker from starting correctly.
Check:
|
Check |
|---|
|
Free space on the filesystem containing the ActiveMQ data directory |
|
Queue depth and message growth |
|
Offline or stalled consumers |
|
Messages accumulating in the dead-letter queue |
|
ActiveMQ store and temporary-store limits |
|
Filesystem permissions and quotas |
Do not manually remove KahaDB journal or index files. Doing so can corrupt the message store.
Instead:
-
Preserve the logs and back up the data directory.
-
Identify the queues consuming the available storage.
-
Restore or restart the affected consumers.
-
Increase available storage when appropriate.
-
Purge messages only after confirming that they are no longer required.
-
Restart ActiveMQ and verify that message processing resumes.
📈 Queue depth continues to increase
A growing queue usually means that messages are arriving faster than they are being consumed, or that no active consumer is available.
For the affected queue, check:
|
Metric |
|---|
|
Number of pending messages |
|
Enqueue and dequeue rates |
|
Number of active consumers |
|
Consumer errors in the GermainUX logs |
|
Database or downstream-service availability |
|
JVM memory and garbage collection |
|
Network latency between GermainUX and ActiveMQ |
Restarting ActiveMQ alone will not resolve a persistent backlog. Identify and correct the stalled or undersized consumer first.
✉️ Messages appear in the dead-letter queue
Messages are normally moved to the dead-letter queue after repeated delivery failures.
Investigate:
|
Possible cause |
|---|
|
Deserialization or message-format errors |
|
Application processing exceptions |
|
Database failures |
|
Missing configuration or dependencies |
|
Redelivery-policy settings |
|
Poison messages that always fail processing |
Do not purge the dead-letter queue before capturing representative errors and determining whether the messages must be recovered or replayed.
🌡️ High memory usage or producer flow control
ActiveMQ may slow or block producers when its configured memory or storage limits are reached.
Review:
|
Metric |
|---|
|
Broker JVM heap usage |
|
ActiveMQ memory-limit settings |
|
Queue sizes and message rates |
|
Large individual messages |
|
Slow or unavailable consumers |
|
Persistent-store utilization |
Increase memory limits only after confirming that the host has sufficient capacity. Otherwise, the operating system may terminate the broker or the broker may spend excessive time in garbage collection.
🔒 Authentication or TLS errors
For authentication failures, verify:
|
Item |
|---|
|
The configured username and password |
|
User and role definitions |
|
Queue authorization rules |
|
Whether credentials were changed without restarting dependent services |
For TLS failures, verify:
|
Item |
|---|
|
Certificate validity dates |
|
Hostname and certificate-name matching |
|
Truststore and keystore paths |
|
File permissions |
|
Trustst |
|
Certificate validity dates |
|
Hostname and certificate-name matching |
|
Truststore and keystore paths |
|
File permissions |
|
Truststore passwords |
|
Supported TLS versions and cipher suites |
Review both sides of the connection because the ActiveMQ and GermainUX logs often contain different parts of the failure.
⚡ Intermittent disconnections
Intermittent broker connections can result from:
|
Cause |
|---|
|
Network interruptions |
|
Load balancer or firewall idle timeouts |
|
Broker restarts |
|
Resource exhaustion |
|
DNS instability |
|
Incorrect failover settings |
|
Long JVM pauses |
Compare timestamps across the ActiveMQ, Germain Enterprise, and node logs. Check whether multiple nodes disconnected simultaneously; this usually indicates a broker or network problem rather than an individual node failure.
✅ Validation
After correcting the issue, confirm that:
|
Validation item |
|---|
|
ActiveMQ starts without errors. |
|
Germain Enterprise and all nodes connect successfully. |
|
Each required queue has an active consumer. |
|
Queue depth is stable or decreasing. |
|
Enqueue and dequeue activity is normal. |
|
The dead-letter queue is not growing unexpectedly. |
|
Disk and memory usage remain within safe limits. |
|
No new connection or processing errors appear in the logs. |
📧 Information to collect for Support
If the problem persists, provide:
|
Item |
|---|
|
GermainUX and ActiveMQ versions |
|
Operating system and Java version |
|
ActiveMQ configuration with credentials removed |
|
Broker URL with credentials removed |
|
ActiveMQ startup and runtime logs |
|
Relevant Germain Enterprise and node logs |
|
Queue statistics and consumer counts |
|
Disk, memory, and JVM-heap usage |
|
Exact timestamps and steps needed to reproduce the problem |
Service: Enterprise
Feature Availability: 2024.1