Germain UX Self-Monitoring (GSM)

Feature

Self-Monitoring (GSM) is a monitoring system that ensures high availability and performance of Germain UX. The GSM script performs the following tasks:

Checks the availability of Germain Services:
- ActiveMQ
- ApacheSolr
- Action
- Aggregation
- Analytics
- Config
- Engines
- Ingestion Services
- Query Services
- Engine Manager
- Session Tracking
- Storage Services
Checks for updates and errors in Germain's logs.
Checks the availability of Germain's REST API.
Checks the availability and updates of Germain Database, such as checking SQL tables for recent updates.
Generates a report and sends it via email. An example of the report is attached.

The report categorizes the status of each software feature as follows:

RED: Indicates a software feature that is broken, failing, or unavailable.
ORANGE: Indicates a software feature that is slow or experiencing errors.
GREEN: Indicates a software feature that is available and fast.

The attached example provides a sample report sent by the GSM script.

Please note that the specific details and format of the report may vary based on the implementation and configuration of the GSM system.

Example of a Report sent by GSM script:

Status	Germain Service	Check	Info
	GermainEngineManager-apsep03050	LogActivity	CODE `Path: '\\serverABC\e$\Software\Germain\GermainEngineProd\logs\EngineManager.log' Modified: 2/20/2020, 9:59:45AM (0.52 minutes ago) Errors: 7 Warn: Errors present in recent log.`
	ActiveMQ	AvailabilityCheck	Status: Running, PID(1296 9708)
	ActiveMQ	BrokerStats \| localhost	Temp Percent: 0 \| MemoryPercent: 0 \| StorePercent: 0
	ActiveMQ	QueueStats \| apm.action	QueueSize: 0 \| ConsumerCount: 1 \| EnqueueCount: 28656 \| DequeueCount: 28656
	ActiveMQ	QueueStats \| apm.analytics	QueueSize: 0 \| ConsumerCount: 1 \| EnqueueCount: 15729162 \| DequeueCount: 15729162
	ActiveMQ	QueueStats \| apm.session	QueueSize: 0 \| ConsumerCount: 1 \| EnqueueCount: 0 \| DequeueCount: 0
	ActiveMQ	QueueStats \| apm.storage	QueueSize: 0 \| ConsumerCount: 2 \| EnqueueCount: 7480961 \| DequeueCount: 7480961
	ActiveMQ	QueueStats \| apm.storage.analytics	QueueSize: 0 \| ConsumerCount: 2 \| EnqueueCount: 189809 \| DequeueCount: 189809
	GermainActionServices	AvailabilityCheck	Status: Running, PID(1636)
	GermainActionServices	LogActivity	CODE `Path: 'E:\Software\GermainService\var\logs\action-services.log' Modified: 2/20/2020, 10:00:04AM (0.19 minutes ago) Errors: 0`
	GermainAggregatorServices	AvailabilityCheck	Status: Running, PID(6720)
	GermainAggregatorServices	LogActivity	CODE `Path: 'E:\Software\GermainAPMService\var\logs\aggregator-services.log' Modified: 2/20/2020, 10:00:15AM (0.01 minutes ago) Errors: 0`
	GermainAnalyticsServices	AvailabilityCheck	Status: Running, PID(1624)
	GermainAnalyticsServices	LogActivity	CODE `Path: '\\serverABC\e$\Software\GermainAPMService\var\logs\analytics-services.log' Modified: 2/20/2020, 10:00:09AM (0.11 minutes ago) Errors: 0`
	GermainAPMConfigServices	EndpointAvailability	Rest Endpoint Response Code: 200
	GermainAPMConfigServices	EndpointAvailability	Rest Endpoint Response Code: 200
	GermainAPMConfigServices-apsep02522	LogActivity	CODE `Path: 'E:\Software\apache-tomcat-8\logs\config-services.log' Modified: 2/20/2020, 10:00:07AM (0.09 minutes ago) Errors: 0`
	GermainAPMConfigServices-apsep02523	LogActivity	CODE `Path: '\\serverABC\e$\Software\apache-tomcat-8\logs\config-services.log' Modified: 2/20/2020, 9:59:45AM (0.44 minutes ago) Errors: 0`
	GermainAPMEnginesProd	AvailabilityCheck	Status: Running, PID(16580 4704)
	GermainAPMIngestionServices-apsep02522	LogActivity	CODE `Path: 'E:\Software\apache-tomcat-8\logs\ingestion-services.log' Modified: 2/20/2020, 10:00:11AM (0.02 minutes ago) Errors: 0`
	GermainAPMIngestionServices-apsep02523	LogActivity	CODE `Path: '\\serverABC\e$\Software\apache-tomcat-8\logs\ingestion-services.log' Modified: 2/20/2020, 10:00:09AM (0.04 minutes ago) Errors: 0`
	GermainAPMQueryServices	EndpointAvailability	Rest Endpoint Response Code: 200
	GermainAPMQueryServices	EndpointAvailability	Rest Endpoint Response Code: 200
	GermainAPMQueryServices-apsep02522	LogActivity	CODE `Path: 'E:\Software\apache-tomcat-8\logs\query-services.log' Modified: 2/20/2020, 9:59:54AM (0.29 minutes ago) Errors: 0`
	GermainAPMQueryServices-apsep02523	LogActivity	CODE `Path: '\\apsep02523\e$\Software\apache-tomcat-8\logs\query-services.log' Modified: 2/20/2020, 9:59:52AM (0.32 minutes ago) Errors: 0`
	GermainEngineManager-apsep03069	LogActivity	CODE `Path: '\\apsep03069\e$\Software\Germain\GermainAPMEngineProd\logs\EngineManager.log' Modified: 2/20/2020, 9:59:58AM (0.32 minutes ago) Errors: 0`
	GermainEngineManagerProd	AvailabilityCheck	Status: Running, PID(121228)
	GermainSessionTrackingServices	AvailabilityCheck	Status: Running, PID(1060)
	GermainSessionTrackingServices	LogActivity	CODE `Path: '\\serverABC\e$\Software\GermainAPMService\var\logs\session-tracking.log' Modified: 2/20/2020, 10:00:01AM (0.26 minutes ago) Errors: 0`
	GermainStorageServices	AvailabilityCheck	Status: Running, PID(1648)
	GermainStorageServices	LogActivity	CODE `Path: 'E:\Software\GermainAPMService\var\logs\storage-services.log' Modified: 2/20/2020, 10:00:11AM (0.08 minutes ago) Errors: 0`