Clean up state store of an Engine

🔧 Clean Up an Engine State Store

Each GermainUX Engine maintains a local state-store database. It records processing progress, such as which sections of monitored log files have already been parsed.

If this state store becomes corrupted, you can remove and recreate it.

Use only when necessary

Recreating the state store removes the Engine’s local processing history. The Engine may reread files or reprocess data after restarting. Perform this procedure only when the state store is corrupted, a clean reprocessing cycle is intentionally required, or GermainUX Support recommends it.

📋 Before you begin

Confirm that:

Check

The issue is related to the local state store.

You know which Engine uses the affected database.

All Engines using the state store can be stopped.

Sufficient disk space exists for a backup.

You understand the potential effect of data reprocessing.

ENGINE_HOME identifies the correct Engine installation.

Preserve the Engine Manager and Engine logs before proceeding.

🛑 1. Stop the Engine

Stop all Engine processes that use the affected state store.

🏳️ Windows

Use the Windows Services console:

services.msc

Stop the affected GermainUX Engine or Engine Manager service, as appropriate.

🐧 Linux

For a service-managed installation:

sudo systemctl stop <germain-engine-service>

Confirm that no affected Engine process remains:

ps -ef | grep apm-engine

Do not modify the state-store files while an Engine is using them.

📁 2. Open the database directory

Bash
cd "$ENGINE_HOME/database"

Verify the directory before continuing:

pwd
ls -la

You should see the local database scripts and the hsqldb directory.

💾 3. Back up the state store

Create a timestamped backup:

cp -a hsqldb "hsqldb.backup.$(date +%Y%m%d%H%M%S)"

Verify that the backup was created:

ls -ld hsqldb*

Do not continue until you have confirmed that the backup exists.

For a Windows deployment, copy the following directory to a safe backup location:

ENGINE_HOME\database\hsqldb

🧹 4. Clear the existing state

From:

ENGINE_HOME/database

remove the contents of the hsqldb directory:

rm -rf "$ENGINE_HOME/database/hsqldb/"*

Verify that ENGINE_HOME is correctly defined before running this command. Do not replace it with an unresolved or broad path.

A safer alternative is to preserve the original directory and create a new one:

mv hsqldb "hsqldb.previous.$(date +%Y%m%d%H%M%S)"
mkdir hsqldb

This approach is recommended because the original files remain recoverable until validation is complete.

🚀 5. Start the standalone state-store database

Bash
cd "$ENGINE_HOME/database"
nohup ./db-standalone.sh &

Confirm that the database starts without errors.

⚙️ 6. Recreate the state-store schema

Run:

./createStatestore.sh

Review the output and confirm that the schema is created successfully.

Do not restart the Engine if schema creation fails.

⏏️ 7. Shut down the database cleanly

Open the database command utility:

./db-exec.sh

At its prompt, run:

shutdown;
\q

Confirm that the standalone database process has stopped.

⏯️ 8. Restart the Engine

Restart the Engine or Engine Manager using its normal service-management method.

Linux:

sudo systemctl start <germain-engine-service>

Windows:

Start the service from services.msc

✅ Validate the rebuilt state store

Confirm that:

Validation

The Engine Manager starts successfully.

The node heartbeat appears in Germain > State.

The affected Engine is enabled and running.

The state-store files are recreated.

No state-store errors appear in the logs.

Monitoring components resume execution.

New telemetry reaches Germain Enterprise.

Log monitoring does not produce unexpected duplicate data.

CPU, memory, and disk utilization remain normal.

Keep the backup until the Engine has operated successfully for an agreed validation period.

❗ Troubleshooting

⛔ State-store database does not start

Check:

Possible Cause

Java availability

Directory permissions

Available disk space

Whether another state-store process is running

Script execution permissions

Database log output

🔧 Schema creation fails

Verify that:

Check

The standalone database is running.

The hsqldb directory is writable.

The cleanup completed.

The package contains the required schema scripts.

The Engine distribution is complete and version-compatible.

🔍 Engine still does not start

The state store may not be the original cause. Review:

Item

EngineManager.log

Java configuration

JVM options

Germain Enterprise connectivity

Node authorization

Proxy and TLS settings

Component initialization errors

🔁 Unexpected data is reprocessed

Stop the affected Engine and determine whether the original state-store backup must be restored. Do not remove the backup until the resulting data behavior has been validated.

Service: Enterprise

Feature Availability: 2024.1