> ## Documentation Index
> Fetch the complete documentation index at: https://docs.germainux.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Apache HTTP Server Monitoring

> See when your Apache web servers run out of workers, which URLs fail, and how traffic moves, across every server in one place.

Components
[`Engine`](https://docs.germainux.com/deploy/germain-ux-engine)
[`JS Profiler`](https://docs.germainux.com/deploy/germain-ux-js-profiler)
[`RPA Bot Recorder`](https://docs.germainux.com/deploy/germain-ux-rpa-bot-recorder)
[`RUM JS`](https://docs.germainux.com/deploy/germain-ux-rum-js)
[`RUM Extension`](https://docs.germainux.com/deploy/germain-ux-rum-extension)

Availability
8.6.9 or later

> [card] [What Is Monitored](#what-is-monitored)
>
> Workers, traffic, requests

> [card] [How It Works](#how-it-works)
>
> Status monitor and log parser

> [card] [Deploy](#deploy)
>
> One wizard per server

> [card] [KPIs](#kpis)
>
> Workers, rate, throughput

## Overview

GermainUX shows you how busy each Apache HTTP Server is and what it serves. You see busy and idle workers, request rate, throughput and Apache process memory next to every request from the access log, with its URL, status and client. When a server runs out of workers or a backend starts returning errors, the charts show it within a minute. Your team finds the saturated server without logging in to read `server-status` pages and log files one by one.

> [card] Worker Saturation at a Glance
>
> Busy and idle workers per server, so you see a server reach its worker limit before users time out.

> [card] Every Request, Searchable
>
> Method, URL, status, response size, client, user, referrer and browser for each access-log entry.

> [card] Failures Stand Out
>
> Requests with status `400` or higher count as failed, so error spikes on a URL or a server show up right away.

> [card] Ready-Made Dashboard
>
> The **Apache HTTP** dashboard charts workers, traffic, process memory and TCP retransmissions out of the box.

### Questions It Answers

* Is any Apache server close to running out of workers?
* Which URLs return 5xx errors right now, and since when?
* Is one server handling more traffic or failing more requests than the others?
* Are clients or bots scanning for pages that don't exist?
* Is the network dropping packets between clients and the web server?
* Did traffic really change, or did Apache restart?
* Is every Apache server still sending data to GermainUX?

## What Is Monitored

| Area | What GermainUX monitors |
| - | - |
| **Workers** | Busy and idle workers, from the Apache `server-status` page |
| **Traffic** | Request rate, throughput and average request size, as `mod_status` reports them since the server started |
| **Processes** | The number of Apache processes and the memory they use |
| **Network** | TCP retransmissions on the Apache host |
| **Requests** | Every access-log entry: HTTP method, URL with query string, status code, response size, client address, user, referrer and user agent. A request with status `400` or higher counts as failed. |
| **Availability** | Up/down status and response time of the URLs Apache serves, checked on a schedule with [synthetic HTTP checks](https://docs.germainux.com/automation/synthetic-http-execution.md) |
| **Host** | CPU, memory, disk and network of the Apache host. See [Linux monitoring](https://docs.germainux.com/integrations/linux-redhat-suse-monitoring.md). |
| **User experience** | Real-user page performance, JavaScript errors and Session Replay for the applications Apache serves, with [RUM JS](https://docs.germainux.com/deploy/germain-ux-rum-js.md) or the [RUM Extension](https://docs.germainux.com/deploy/germain-ux-rum-extension.md) |

## Analytics, Alerts and Automation

### AI Analysis

Ask [AI Explore](https://docs.germainux.com/analytics/ai-driven-insights.md) questions such as "Which Apache URLs failed most this week?" and get the answer without building a report. [Outlier detection](https://docs.germainux.com/analytics/outlier-detection.md) flags unusual spikes or drops in busy workers, request rate and failures.

### Dashboards and Reports

GermainUX ships the **Apache HTTP** dashboard. In the left menu, go to **Dashboards > All**, and then open **Apache HTTP**. It has one trend chart per KPI: **Apache Busy Workers**, **Apache Idle Workers**, **Apache Process Count**, **Apache Request Rate**, **Apache Request Size**, **Apache Shared Memory**, **Apache Throughput** and **TCP Retransmissions**.

Busy workers that climb while idle workers fall to zero mean the server has reached its worker limit. New requests then wait in the queue or time out.

![Apache HTTP dashboard over four hours, with busy workers, process count and shared memory rising at 10:00 and 11:00 while idle workers fall, next to request rate, request size, throughput and TCP retransmissions](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apache-http-dashboard.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=bffac5a515924ceb21f3c0aab16c8f68)

Apache HTTP Dashboard - GermainUX

Build your own [dashboards and reports](https://docs.germainux.com/dashboards/dashboard-and-report.md) on the access-log requests: volume and failure rate by server or URL, status-code distribution, and the busiest clients.

### Analysis

Click a chart on the dashboard to open its KPI in the [Analysis](https://docs.germainux.com/dashboards/analytics-dashboard/4-analysis-dashboard.md) screen. The segmentation breakdown lists the servers and applications that changed most, and labels each one as a spike, an outlier or degraded.

![Analysis screen for the Apache Busy Workers KPI, with average busy workers up 361.5% on the previous days, a peak on 16 November in the relationships chart, and request rate and TCP retransmissions as related metrics](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apache-busy-workers-analysis.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=2a3ac6ba3f7e5555f3d9c51b48fbf344)

Analysis of Apache Busy Workers - GermainUX

The relationships chart adds the KPIs related to the one you opened: **Apache Idle Workers** next to **Apache Busy Workers**, **Apache Throughput** next to **Apache Request Rate** and **Apache Request Size**, **Apache Busy Workers** next to **Apache Process Count** and **TCP Retransmissions**, and **Inbound HTTP Request** next to **Apache Busy Workers** and **Apache Request Rate**. The segmentation breakdown compares the servers, so you see which one moved.

### Investigate Backend Errors Behind `mod_proxy`

When an application server behind Apache stops, `mod_proxy` answers `503` to every request it forwards there, and each of these requests counts as failed. Open the **Inbound HTTP Request** KPI in the [Analysis](https://docs.germainux.com/dashboards/analytics-dashboard/4-analysis-dashboard.md) screen, filtered to failed Apache requests: **Success** is `false` and **Member** is `Apache HTTP`. The top row counts the failed requests, compares them with the same hours on the previous days, and shows the hourly trend. The segmentation breakdown shows which server they come from.

In the example below, app-02 stops for 45 minutes on the afternoon of 15 November. `mod_proxy` on **web-02**, which forwards `/app/` and `/api/` requests to app-02, answers `503` to each of them. The screen counts 46 failed requests between 11:00 and 16:00 UTC, up 411.1% on the previous days, all in one hour. The breakdown marks **web-02** as a spike with 44 of them, so you know which backend to check. The failures stop when app-02 is back. An [SLA](https://docs.germainux.com/analytics/slas.md) on the failure rate per server, with an [alert](https://docs.germainux.com/automation/notifications.md), tells you about the next outage as it starts.

![Analysis screen for failed Apache requests from 11:00 to 16:00 UTC on 15 November, with 46 failed requests against 9 on the previous days, one tall bar in the hourly trend, and web-02 marked as a spike with 44 failed requests in the segmentation breakdown](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apache-failed-requests-503-analysis.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=328399bb7b113166642a368f2949c590)

Aggregate Analysis of Failed Apache Requests During a Backend Outage - GermainUX

### Investigate a 404 Scan

A vulnerability scanner or a bot that probes for files that don't exist causes a burst of `404` responses. Open the failed Apache requests in the Analysis screen, as for backend errors. The segmentation breakdown lists the servers, user agents and URLs that the failed requests have in common.

In the example below, a scanner probes both servers for 50 minutes in the night of 16 November, asking for paths such as `/.env`, `/.aws/credentials`, `/xmlrpc.php` and `/backup.tar.gz`. The screen counts 843 failed requests between 23:00 and 04:00 UTC, against 2 at the same hours on the previous days. The breakdown splits them between **web-01** (440) and **web-02** (403), marks the **Robot** user agent device as new, and lists each probed path. None of these paths exists on the site, so the requests don't come from real users.

![Analysis screen for failed Apache requests from 23:00 on 15 November to 04:00 UTC on 16 November, with 843 failed requests against 2 on the previous days, web-01 and web-02 marked as spikes, the Robot user agent device marked as new, and the probed paths /backup.tar.gz, /.aws/credentials, /xmlrpc.php, /.env and /vendor/phpunit in the segmentation breakdown](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apache-404-scan-analysis.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=d43081152d8303af1ebe6240477e03f4)

Aggregate Analysis of Failed Apache Requests During a 404 Scan - GermainUX

### Investigate Packet Loss on a Web Server

**TCP Retransmissions** holds the total number of TCP segments the Apache host has retransmitted since it started, so its line always climbs. Packet loss shows as a steeper climb, not as a spike. Click the **TCP Retransmissions** chart on the **Apache HTTP** dashboard to open it in the Analysis screen, and compare the slope of each server in the segmentation breakdown.

In the example below, a faulty switch port drops packets on **web-02** for 45 minutes in the morning of 15 November. The breakdown marks **web-02** as degraded: its total climbs steeply during the fault, then levels off, and ends up 28.7% higher than on the previous days. The total of **web-01** climbs slowly and evenly, 7.1% above the previous days.

![Analysis screen for the TCP Retransmissions KPI from 06:00 to 12:00 UTC on 15 November, with a step in the relationships chart, and web-02 marked as degraded with a steep climb next to web-01 climbing slowly in the segmentation breakdown](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apache-tcp-retransmissions-analysis.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=7e50a5f833e2e4d0c3f701f97cf490b3)

Aggregate Analysis of TCP Retransmissions - GermainUX

### Investigate a Request Rate Change After a Restart

**Apache Request Rate**, **Apache Throughput** and **Apache Request Size** are averages that `mod_status` computes over the time since Apache started. After days of uptime they barely move, so their lines look flat. When Apache restarts, `mod_status` starts again from zero. The first values cover only the few requests since the restart, and the line then follows the traffic of the hours since. Busy workers, idle workers and process count are current values, so a restart doesn't move them this way. Before you read a jump in these three KPIs as a traffic change, check whether Apache restarted at that time.

In the example below, Apache restarts on **web-01** and **web-02** at 05:00 UTC on 16 November to load a renewed certificate. Before the restart, the request rate is a flat line at about 0.046 requests per second. Right after it, the line dips, because few requests arrive at that hour, and then climbs with the day's traffic to close to 0.1 requests per second in the afternoon. Both servers show the same shape in the segmentation breakdown. Traffic didn't change; only the period that the average covers did.

![Analysis screen for the Apache Request Rate KPI from 03:00 to 08:00 UTC on 16 November, with an average of 0.046 requests per second, a flat relationships chart that dips after the restart and then rises in the afternoon, and the same dip-and-climb line for web-01 and web-02 in the segmentation breakdown](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apache-request-rate-restart-analysis.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=7f342273cd6eaaf47f2cf38b2918fbae)

Aggregate Analysis of Apache Request Rate After a Restart - GermainUX

### SLAs and Alerts

Set [SLAs](https://docs.germainux.com/analytics/slas.md) on busy workers, request rate or the failure rate of requests, and get an [alert](https://docs.germainux.com/automation/notifications.md) by email, SMS, Slack or Teams when one is breached. An SLA on idle workers tells you when a server is about to run out of capacity.

### Correlation with Other Technologies

[Correlate](https://docs.germainux.com/analytics/correlation-and-tracing.md) Apache requests with browser sessions and with the application servers behind Apache, such as [Apache Tomcat](https://docs.germainux.com/integrations/apache-tomcat-monitoring.md), to follow a failed request to its cause.

## How It Works

server-status and access.log
Engine
KPIs
Dashboard
Alerts

The **Apache HTTP Server** wizard applies a server profile that deploys three components to the GermainUX Engine on the Apache host:

| Component | What it does |
| - | - |
| **Apache Monitor - Local - Linux** | Every 60 seconds, reads the Apache `server-status` page, lists the Apache processes, and reads the host's TCP statistics. Stores busy and idle workers, request rate, throughput, request size, process count, process memory and TCP retransmissions. |
| **Apache Log Directory Monitor** | Watches the access log directory and sends new lines of the file named `access.log` to the parser |
| **Apache Access Log Parser** | Reads each access-log line and stores it as an **Inbound HTTP Request** transaction, with the request, client, status, size, referrer and user agent |

The Engine runs the status commands and reads the log files locally, so it runs on the Apache host. See [File and Log Monitoring](https://docs.germainux.com/integrations/file-log-monitoring.md) for how GermainUX reads log files.

## Before You Start

* A GermainUX Engine is deployed on the Apache host, which runs Linux. See [GermainUX Engine](https://docs.germainux.com/deploy/germain-ux-engine.md).
* The Apache host is set up as a server in GermainUX, and the Engine on it is registered for that server.
* `mod_status` is enabled, and `http://APACHE_HOST/server-status` answers on port `80` from the Apache host itself.
* `ExtendedStatus` is `On`, so `server-status` reports request rate, throughput and request size.
* The access log uses the `combined` format and is written to a file named `access.log`.
* The Engine's account can read the access log directory, for example `/var/log/apache2`.
* `/usr/bin/wget` and `/bin/netstat` are installed on the Apache host.

## Deploy

**1. Check the Apache configuration**

In the Apache configuration, check that `server-status` is enabled and that the access log uses the `combined` format:

```apache apache2.conf theme={null}
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog ${APACHE_LOG_DIR}/access.log combined

ExtendedStatus On
<Location "/server-status">
    SetHandler server-status
    Require local
</Location>
```

If you changed these lines, reload Apache.

**2. Open the Apache HTTP Server wizard**

In the left menu, go to **Settings > Wizards**. On the **Apache HTTP Server** card, click **Add**.

**3. Select the Apache servers**

In the **Apply Apache HTTP Server** dialog, select the Apache hosts in **Servers**, and then click **Apply**.

![Apply Apache HTTP Server dialog over the Wizards screen, with one server selected in Servers and the Apply button](https://mintcdn.com/germain-ux/cSCoMMl7KW61OHIM/images/main/apache-http-server-monitoring/apply-apache-http-server-dialog.png?fit=max\&auto=format\&n=cSCoMMl7KW61OHIM\&q=85\&s=3308d5c0cc6d925a0afdfcf39230c93b)

Apply Apache HTTP Server Dialog - GermainUX

**4. Fill in the profile settings**

Fill in the fields:

| Field | Value |
| - | - |
| **Application** | The application the Apache data belongs to. Default: `Apache HTTP`. |
| **Access Log Directory** | The directory that holds `access.log`, for example `/var/log/apache2`. Select a saved directory, or create one. |

Finish the wizard. GermainUX deploys the three components to the Engine on each selected server, and starts reading `server-status` and the access log.

## Validate

**1. Check server-status**

On the Apache host, request the status page the Engine reads:

```bash theme={null}
wget -qO- http://APACHE_HOST/server-status | grep -i "requests currently being processed"
```

Replace `APACHE_HOST` with the host name of the Apache server. The output shows the number of busy and idle workers.

**2. Send test requests**

From any machine that can reach the Apache server, request a page and a path that doesn't exist:

```bash theme={null}
curl -s -o /dev/null -w "%{http_code}\n" http://APACHE_HOST/
curl -s -o /dev/null -w "%{http_code}\n" http://APACHE_HOST/germainux-test-missing
```

**3. Find the data in GermainUX**

After a few minutes, open the **Apache HTTP** dashboard and check that each chart shows the server. Then open the **Inbound HTTP Request** KPI and look for the requests to `/` and `/germainux-test-missing`. Check that the `404` request counts as failed.

## Troubleshooting

> [accordion] The Apache HTTP dashboard stays empty
>
> Check that `wget -qO- http://APACHE_HOST/server-status` returns the status page on the Apache host, and that `mod_status` allows requests from the host itself. Check that the components run on the Engine of the Apache host: in the left menu, go to **Settings > Wizards**, and click **Manage** on the **Apache HTTP Server** card.

> [accordion] Request rate, throughput and request size stay at zero
>
> `server-status` reports these values only when `ExtendedStatus` is `On`. Turn it on and reload Apache.

> [accordion] No Apache requests appear
>
> Check that the access log file is named `access.log`, that it's in the **Access Log Directory** you selected, and that the Engine's account can read it. A log written to another file name, such as `access_log` or `ssl_access.log`, isn't picked up.

> [accordion] Some access-log lines are missing
>
> GermainUX skips lines that don't match the `combined` format. This happens when a `CustomLog` line uses another format, such as `common` or a custom one. Use the `combined` format for `access.log`.

> [accordion] TCP retransmissions are missing
>
> The monitor reads TCP statistics with `/bin/netstat`. Install the package that provides it, such as `net-tools`.

## Security and Privacy

Apache HTTP Server monitoring data includes personal data:

* **Identity data**: the client address and the authenticated user name of each request
* **URLs**: the request path and query string, and the referrer URL. Query strings can contain email addresses, IDs or tokens, so treat them as possibly containing personal or customer data.
* **Device data**: the user agent of each request, with the browser and operating system

To mask, anonymize or exclude this data, see [Data Privacy](https://docs.germainux.com/management/compliance-privacy-and-security/data-privacy.md).

## KPIs

| KPI | What it holds | Data model |
| - | - | - |
| **Apache Busy Workers** | Workers serving a request | `GenericMetric` |
| **Apache Idle Workers** | Workers waiting for a request | `GenericMetric` |
| **Apache Request Rate** | Requests per second, averaged since Apache started | `GenericMetric` |
| **Apache Throughput** | Kilobytes served per second, averaged since Apache started | `GenericMetric` |
| **Apache Request Size** | Average kilobytes per request since Apache started | `GenericMetric` |
| **Apache Process Count** | Number of Apache processes | `GenericMetric` |
| **Apache Shared Memory** | Resident memory (RSS) of all Apache processes | `GenericMetric` |
| **TCP Retransmissions** | Total TCP segments retransmitted on the Apache host since it started. Watch how fast it climbs. | `GenericMetric` |
| **Inbound HTTP Request** | One transaction per access-log line, with the request, client, status and size | `InboundHttpRequest` |

The Apache KPIs have no default SLA. Add the thresholds you need. **Inbound HTTP Request** is a shared KPI that GermainUX uses for other web servers too. See [KPIs](https://docs.germainux.com/analytics/kpis.md) and [SLAs](https://docs.germainux.com/analytics/slas.md).

## Related Documentation

> [card] [Apache Tomcat](https://docs.germainux.com/integrations/apache-tomcat-monitoring.md)
>
> Application servers behind Apache

> [card] [File and Log Monitoring](https://docs.germainux.com/integrations/file-log-monitoring.md)
>
> How GermainUX reads log files

> [card] [Automation](https://docs.germainux.com/automation.md)
>
> More Automation Features

## Get Help

The GermainUX team can help you monitor Apache servers that use a custom log format, another log file name or a non-default status URL.

> [card] [Contact GermainUX Support](https://docs.germainux.com/getting-started/support.md)
>
> The GermainUX team helps you get up and running faster, empower your team, turn data into actionable insights, and optimize performance to deliver measurable business outcomes.

> [card] Can't Find What You Need?
>
> Search all GermainUX docs with **Ctrl + K**, or use the feedback buttons at the bottom of this page to tell us what is missing.

The content of this document is proprietary to GERMAIN SOFTWARE.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.