> ## Documentation Index
> Fetch the complete documentation index at: https://docs.germainux.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Hazelcast Monitoring

> Watch heap pressure and map growth on every member of your Hazelcast cluster, and get alerted before a map runs a member out of memory.

Components
[`Engine`](https://docs.germainux.com/deploy/germain-ux-engine)

Availability
2024.1 or later

> [card] [What Is Monitored](#what-is-monitored)
>
> Heap, map size, map memory

> [card] [How It Works](#how-it-works)
>
> JMX monitor per member

> [card] [Deploy](#deploy)
>
> Enable JMX, run the wizard

> [card] [KPIs](#kpis)
>
> Three KPIs with SLAs

## Overview

GermainUX monitors the Hazelcast clusters your applications use for caching, sessions and shared data. For each member, you see how full its heap is, how many entries each map holds, and how much memory each map takes on that member. Three KPIs, default SLAs and a **Hazelcast Overview** dashboard come ready to use. Your team spots a map that grows without limit, or a member running short of heap, before it slows down or fails the applications that depend on it.

> [card] Heap per Member
>
> Heap usage as a percentage of each member's maximum heap, with an SLA at 75%.

> [card] Growth of Every Map
>
> The number of entries in each of your maps, so a map without a time-to-live or a size limit stands out.

> [card] Memory per Map
>
> The memory each map's entries take on each member, to find which map is filling the heap.

> [card] Ready-Made Dashboard
>
> A **Hazelcast Overview** dashboard with heap usage, peak heap, map size and map memory charts.

### Questions It Answers

* Which Hazelcast member is closest to running out of heap?
* Which map is growing fastest, and since when?
* Which map takes the most memory on each member?
* Did a release change how much data a map holds?
* Is the data spread evenly across the members?
* When did heap usage last stay above 75%?

## What Is Monitored

| Area | What GermainUX monitors |
| - | - |
| **Heap** | Used heap of each member as a percentage of its maximum heap, or of its committed heap when no maximum is set |
| **Map size** | The number of entries in each map. Hazelcast's internal maps, whose names start with `__`, are skipped. |
| **Map memory** | The memory, in bytes, that each map's entries take on each member |
| **Availability** | Up/down status and response time of member endpoints you can reach over HTTP, such as the health check endpoint, with [synthetic HTTP checks](https://docs.germainux.com/automation/synthetic-http-execution.md) |
| **Host** | CPU, memory, disk and network of the servers the members run on, and whether the member process is running. See [Linux monitoring](https://docs.germainux.com/integrations/linux-redhat-suse-monitoring.md) or [Windows monitoring](https://docs.germainux.com/integrations/windows-monitoring.md). |
| **Logs** | Errors and warnings from the member logs, with [File and Log Monitoring](https://docs.germainux.com/integrations/file-log-monitoring.md) |

## Analytics, Alerts and Automation

### AI Analysis

Ask [AI Explore](https://docs.germainux.com/analytics/ai-driven-insights.md) questions such as "Which Hazelcast maps grew the most this week?" [Outlier detection](https://docs.germainux.com/analytics/outlier-detection.md) flags a map or a member whose memory jumps outside its normal range.

### Dashboards and Reports

In the left menu, select **Dashboards > All**, and then open the **Hazelcast Overview** dashboard. It has four charts: **Heap Usage by Member (%)**, **Peak Heap Usage (%)**, **Map Size by Map** and **Map Heap Size by Map**. Each chart compares the selected period with the previous one. Click a chart to open its KPI and pivot by member or map. You can build your own [dashboards and reports](https://docs.germainux.com/dashboards/dashboard-and-report.md) on the same KPIs.

![Hazelcast Overview dashboard with heap usage by member, peak heap usage, map size by map and map heap size by map charts](https://mintcdn.com/germain-ux/eoDC7Ue6Rn5SPFuW/images/main/hazelcast-monitoring/hazelcast-dashboard.png?fit=max\&auto=format\&n=eoDC7Ue6Rn5SPFuW\&q=85\&s=0b8a92e31dbf48a48e74f2e4a18f5158)

Hazelcast Overview Dashboard - GermainUX

The analysis of a chart compares the period with the previous one and breaks the value down by member or map. For **Heap Usage by Member (%)**, you see the average heap usage of each member and how much it changed.

![Heap usage analysis with the average heap usage, the change against the previous period, and a breakdown by member hz-01, hz-02 and hz-03](https://mintcdn.com/germain-ux/eoDC7Ue6Rn5SPFuW/images/main/hazelcast-monitoring/hazelcast-heap-usage-analysis.png?fit=max\&auto=format\&n=eoDC7Ue6Rn5SPFuW\&q=85\&s=f400c7c56fc7c13bf7794c76ff67f361)

Heap Usage Analysis by Member - GermainUX

For **Map Size by Map**, the analysis lists the maps that changed the most, so a map that suddenly grows stands out.

![Map size analysis with the leading factors: the sessions map marked as a spike, product-cache as degraded and rate-limits as better](https://mintcdn.com/germain-ux/eoDC7Ue6Rn5SPFuW/images/main/hazelcast-monitoring/hazelcast-map-size-analysis.png?fit=max\&auto=format\&n=eoDC7Ue6Rn5SPFuW\&q=85\&s=8080816431e3f28b639f69057f3c7f97)

Map Size Analysis by Map - GermainUX

### SLAs and Alerts

Each KPI comes with a default [SLA](https://docs.germainux.com/analytics/slas.md), listed under [KPIs](#kpis). Get an [alert](https://docs.germainux.com/automation/notifications.md) by email, SMS, Slack or Teams when a member's heap or a map stays over its limit.

### Automation

When an SLA is breached, an [automated action](https://docs.germainux.com/automation/automated-actions.md) can run a diagnostic script, call an HTTP endpoint or open an incident.

## How It Works

Hazelcast member
JMX
Hazelcast JMX Monitor
KPIs
Alerts

| Component | What it does |
| - | - |
| **Hazelcast JMX Monitor** | Runs on a GermainUX Engine and connects to one member over JMX on a schedule. It reads `HeapMemoryUsage` from `java.lang:type=Memory`, and `size` and `localOwnedEntryMemoryCost` from each `com.hazelcast:type=IMap` MBean. |
| **Hazelcast rules** | Turn each reading into a fact: heap usage as a percentage, and map size and map memory named after the map |

Each member has its own JMX connection, so you deploy one monitor for each member. The member's host becomes the system name, which is how the KPIs and SLAs tell the members apart.

## Before You Start

* A [GermainUX Engine](https://docs.germainux.com/deploy/germain-ux-engine.md) can reach the JMX port of every Hazelcast member.
* You can change the configuration and JVM options of the members, and restart them.
* You have a JMX user name and password for the members, if JMX authentication is on.
* Firewalls allow the Engine to reach the JMX and RMI ports.

## Deploy

**1. Turn on JMX in Hazelcast**

Set the `hazelcast.jmx` property on every member, in the member configuration:

```xml hazelcast.xml theme={null}
<hazelcast>
    <properties>
        <property name="hazelcast.jmx">true</property>
    </properties>
</hazelcast>
```

You can also start the member with `-Dhazelcast.jmx=true`. See the [Hazelcast monitoring documentation](https://docs.hazelcast.com/hazelcast/latest/maintain-cluster/monitoring) for details.

**2. Open the JVM to remote JMX**

Add the standard Java remote JMX options to each member's JVM, and then restart the member:

```bash theme={null}
-Dcom.sun.management.jmxremote.port=JMX_PORT
-Dcom.sun.management.jmxremote.rmi.port=JMX_PORT
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.password.file=/opt/hazelcast/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=/opt/hazelcast/jmxremote.access
-Dcom.sun.management.jmxremote.ssl=false
-Djava.rmi.server.hostname=MEMBER_HOST
```

Replace `JMX_PORT` with a free port, for example `9999`, and `MEMBER_HOST` with the host name the Engine uses to reach the member. Turn on SSL when your security policy requires it.

**3. Open the Java Monitor Deployment wizard**

In the left menu, go to **Settings > Wizards**. On the **Java Monitor Deployment** card, click **Add**.

**4. Choose the Hazelcast monitor**

On the **Monitoring** step, fill in the fields, and then click **Next**.

| Field | Value |
| - | - |
| **Component Type** | **Hazelcast JMX Monitor** |
| **Monitored Application** | The application to file the data under, for example one named `Hazelcast` |
| **Credentials** | The JMX user name and password. Leave it empty when JMX authentication is off. |

![Java Monitor Deployment wizard on the Monitoring step, with Hazelcast JMX Monitor selected as the Component Type](https://mintcdn.com/germain-ux/eoDC7Ue6Rn5SPFuW/images/main/hazelcast-monitoring/jmx-monitor-deployment-monitoring-step.png?fit=max\&auto=format\&n=eoDC7Ue6Rn5SPFuW\&q=85\&s=e1e0cca8b9970f325b17b6d4df1794c5)

Java Monitor Deployment Wizard, Monitoring Step - GermainUX

**5. Connect to the member**

On the **Connection** step, select or create the **URI** of the member's JMX endpoint, and then click **Next**:

```text theme={null}
service:jmx:rmi:///jndi/rmi://MEMBER_HOST:JMX_PORT/jmxrmi
```

When the Engine runs on the same server as the member, you can enter an **Application Pattern** that matches the member's Java process name instead, and GermainUX attaches to the local JVM.

**6. Deploy it to the Engine**

On the **Deployment** step, enter a **Monitor Name**, such as `Hazelcast hz-01`, select the **Monitoring Node** and Engine, and set the schedule. The default, `0 0 * * * *`, runs once an hour. An SLA trips after 5 violations in a row, so use `0 */5 * * * *` to read the member every 5 minutes. Finish the wizard.

**7. Repeat for the other members**

Run the wizard once for each member of the cluster.

## Validate

**1. Wait for the first run**

The monitor reads the member at its next scheduled time, for example within 5 minutes with `0 */5 * * * *`.

**2. Open the Hazelcast dashboard**

In the left menu, select **Dashboards > All**, and then open **Hazelcast Overview**. **Heap Usage by Member (%)** shows one line for each member you deployed.

**3. Check the maps**

**Map Size by Map** shows one line for each of your maps, with its entry count.

## Troubleshooting

> [accordion] No data appears for a member
>
> Check that the Engine can open a connection to the member's JMX port, and that the port in the URI matches `com.sun.management.jmxremote.port`. When the member sits behind NAT or has several network interfaces, set `java.rmi.server.hostname` to the address the Engine uses, and set `com.sun.management.jmxremote.rmi.port` to the same port.

> [accordion] Heap usage appears, but no maps
>
> The `hazelcast.jmx` property is off, so the member doesn't publish its map MBeans. Set it to `true` and restart the member. Maps also appear only after the application has created them.

> [accordion] The connection fails with an authentication error
>
> The credentials don't match the JMX password file, or the user has no entry in the access file. Check both files on the member, and the **Credentials** selected in the wizard.

> [accordion] Map memory is missing for some maps
>
> Hazelcast reports no memory cost for some map configurations. GermainUX skips readings without a memory value, so those maps show a size but no memory.

## Security and Privacy

Hazelcast monitoring data includes:

* **System data**: the host names of the members
* **Application data**: the names of your maps

GermainUX reads sizes and memory figures only, not the entries stored in the maps.

To mask, anonymize or exclude this data, see [Data Privacy](https://docs.germainux.com/management/compliance-privacy-and-security/data-privacy.md).

## KPIs

| KPI | What it holds | Default SLA |
| - | - | - |
| **Hazelcast Heap Usage** | Heap usage of each member, in percent | OK below `75`, per member |
| **Hazelcast Map Size** | Number of entries in each map | OK below `1000000`, per member and map |
| **Hazelcast Map Heap Size** | Memory taken by each map's entries on a member, in bytes | OK below `104857600` (100 MB), per member and map |

Each SLA is breached after 5 violations in a row. Change the thresholds to fit your cluster. See [KPIs](https://docs.germainux.com/analytics/kpis.md) and [SLAs](https://docs.germainux.com/analytics/slas.md).

## Related Documentation

> [card] [Java Monitoring](https://docs.germainux.com/integrations/java-monitoring.md)
>
> JVM metrics over JMX

> [card] [Analytics](https://docs.germainux.com/analytics.md)
>
> More Analytics Features

> [card] [Automation](https://docs.germainux.com/automation.md)
>
> More Automation Features

## Get Help

The GermainUX team can help you add other Hazelcast MBeans, such as queues, topics or partition data, to the monitor.

> [card] [Contact GermainUX Support](https://docs.germainux.com/getting-started/support.md)
>
> The GermainUX team helps you get up and running faster, empower your team, turn data into actionable insights, and optimize performance to deliver measurable business outcomes.

> [card] Can't Find What You Need?
>
> Search all GermainUX docs with **Ctrl + K**, or use the feedback buttons at the bottom of this page to tell us what is missing.

The content of this document is proprietary to GERMAIN SOFTWARE.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.