GSM - Germain Self-Monitoring Script
Feature
Germain’s Self-Monitoring (GSM) monitors and ensures high availability and performance of Germain. That script performs the following:
Check Availability of Germain Services (ActiveMQ, ApacheSolr, Action, Aggregation, Analytics, Config, Engines, Ingestion Services, Query Services, Engine Manager, Session Tracking and Storage Services)
Check Updates and Errors in Germain’s Logs
Check Availability of Germain’s REST Api
Check Availability & Updates of Germain Database (e.g. SQL Tables having no updates "recently", etc)
Generate and email report (example attached)
RED: software feature that is broken/failing/unavailable
ORANGE: software feature slow or erroring out
GREEN: available and fast
Example of a Report sent by GSM script:
Status | Germain Service | Check | Info |
GermainEngineManager-apsep03050 | LogActivity |
CODE
| |
ActiveMQ | AvailabilityCheck | Status: Running, PID(1296 9708) | |
ActiveMQ | BrokerStats | localhost | Temp Percent: 0 | MemoryPercent: 0 | StorePercent: 0 | |
ActiveMQ | QueueStats | apm.action | QueueSize: 0 | ConsumerCount: 1 | EnqueueCount: 28656 | DequeueCount: 28656 | |
ActiveMQ | QueueStats | apm.analytics | QueueSize: 0 | ConsumerCount: 1 | EnqueueCount: 15729162 | DequeueCount: 15729162 | |
ActiveMQ | QueueStats | apm.session | QueueSize: 0 | ConsumerCount: 1 | EnqueueCount: 0 | DequeueCount: 0 | |
ActiveMQ | QueueStats | apm.storage | QueueSize: 0 | ConsumerCount: 2 | EnqueueCount: 7480961 | DequeueCount: 7480961 | |
ActiveMQ | QueueStats | apm.storage.analytics | QueueSize: 0 | ConsumerCount: 2 | EnqueueCount: 189809 | DequeueCount: 189809 | |
ApacheSolr | AvailabilityCheck | Status: Running, PID(1304 1412 2312) | |
ApacheSolr | EndpointAvailability | Rest Endpoint Response Code: 200 | |
GermainActionServices | AvailabilityCheck | Status: Running, PID(1636) | |
GermainActionServices | LogActivity |
CODE
| |
GermainAggregatorServices | AvailabilityCheck | Status: Running, PID(6720) | |
GermainAggregatorServices | LogActivity |
CODE
| |
GermainAnalyticsServices | AvailabilityCheck | Status: Running, PID(1624) | |
GermainAnalyticsServices | LogActivity |
CODE
| |
GermainAPMConfigServices | EndpointAvailability | Rest Endpoint Response Code: 200 | |
GermainAPMConfigServices | EndpointAvailability | Rest Endpoint Response Code: 200 | |
GermainAPMConfigServices-apsep02522 | LogActivity |
CODE
| |
GermainAPMConfigServices-apsep02523 | LogActivity |
CODE
| |
GermainAPMEnginesProd | AvailabilityCheck | Status: Running, PID(16580 4704) | |
GermainAPMIngestionServices-apsep02522 | LogActivity |
CODE
| |
GermainAPMIngestionServices-apsep02523 | LogActivity |
CODE
| |
GermainAPMQueryServices | EndpointAvailability | Rest Endpoint Response Code: 200 | |
GermainAPMQueryServices | EndpointAvailability | Rest Endpoint Response Code: 200 | |
GermainAPMQueryServices-apsep02522 | LogActivity |
CODE
| |
GermainAPMQueryServices-apsep02523 | LogActivity |
CODE
| |
GermainEngineManager-apsep03069 | LogActivity |
CODE
| |
GermainEngineManagerProd | AvailabilityCheck | Status: Running, PID(121228) | |
GermainSessionTrackingServices | AvailabilityCheck | Status: Running, PID(1060) | |
GermainSessionTrackingServices | LogActivity |
CODE
| |
GermainStorageServices | AvailabilityCheck | Status: Running, PID(1648) | |
GermainStorageServices | LogActivity |
CODE
|
Download GSM
That script is available for download here