Categorization
Feature
Germain has the ability of categorizing data/facts into groups, in order to identify new problems or issues among a large volume of issue instances. This capability is particularly useful to find new kind of Code Exception or Application Crash among thousands.
System administrators can easily spot Crash or Exception that have not been encountered before, indicating potential issues that require immediate attention. This proactive approach ensures that emerging problems are addressed promptly, preventing any negative impact on system performance or user experience.
In summary, Germain's ability to categorize data/facts into groups provides system administrators with efficient error management. It helps in analyzing error frequency, reducing effort in error review, prioritizing issue resolution, and detecting new or unique errors. These capabilities streamline the troubleshooting process, leading to faster issue resolution and improved overall system performance.
Video overview: https://youtu.be/NaQ_IQRjbU4 .
Configuration
Germain Workspace > Left Menu > Analytics > Categorization
For generic application built with standard tech (e.g. java, goLang, php, c++, etc) this categorization feature is already configured out-of-the-box.
However for more complex application, you will want to define a Categorization to identify more complex Crash or Exception if it involves the analysis of several data sources. This functionality can be easily customized by writing a Javascript processor. Here are several examples
Java Exception Analysis Categorization Example
For java application, Germain analyzes the “path” of an exception/failure and finds the new ones vs the ones that have been reoccurring. Most other APMs only tell you that there is another “nullpointerexception” but don’t help you distinguish between “UserService.getCurrent” and “BusinessLogicService.performLogic” making it very hard to deal with as they often are millions.
Java Exception 1:
Java Exception 2:
Salesforce Apex Exception Categorization Example
Siebel CRM Object Manager Crash Categorization Example
For Siebel CRM for instance, Germain now distinguishes the Siebel Object Manager crashes that are “new” from the ones that are already “known” and reoccurring (in addition, Germain also quantifies the count of users impacted by a crash).
Example of dashboards that help analyze Siebel OM Crashes
How to configure Siebel OM Crash Categorization
This above business impact analysis and identification of a new OM crash comes as a result of Germain analyzing 1 to 3 Siebel files.
Siebel Error:
Siebel FDR file:
Siebel Core file:
Configuration Screen
CODEimportClass(com.germainsoftware.data.Checksum); importClass(com.germainsoftware.apm.storage.categorization.siebel.SiebelCoreCrashParserUtils); var errorCodesWithoutCoreCrash = { 'SBL-SMI-00062': true, // Internal: No more process (multithreaded server) slots available 'SBL-SVR-09127': true, // Internal: Fail to initialize the shared memory resource for the process }; try { var supportingFactIdentifiers = fact.details ? fact.details.split('|') : []; var coreCrash; var knownErrorCode; for (var i=0; i<supportingFactIdentifiers.length; i++) { var identifier = supportingFactIdentifiers[i].split(';'); var factClassName = identifier[0]; var factId = identifier[1]; var factType = identifier[2]; if (factType === 'Siebel:Enterprise Crash') { var errorCode = identifier[3]; if (errorCode && (errorCodesWithoutCoreCrash[errorCode] || errorCode.indexOf('SBL-GEN-') === 0)) { knownErrorCode = errorCode; } } else if (factType === 'Siebel:Core Crash') { coreCrash = queryService.findById(factClassName, factId); } } if (knownErrorCode) { // If this crash is caused by an error that is known to not generate a Core Crash, // perform matching based only on error code, since this is all we have log.debug('FactID: {} - Categorizing based on Error code', fact.id, knownErrorCode); result.exactMatch = Checksum.calculate(knownErrorCode); result.fuzzyMatch = ''; } else if (coreCrash) { // If we have a Core Crash, use that to determine similarity of crashes log.debug('FactID: {} - Categorizing based on Core Crash', fact.id); var uninformativeFunctions = [ { fileName: '/app/siebel/siebsrvr/lib/libsslcosd.so', method: '+0x7185' }, { fileName: '/app/siebel/siebsrvr/lib/libsslcosd.so', method: '+0x797e' }, { fileName: '/app/siebel/siebsrvr/lib/libsscdo90.so', method: '_ZN15CSSOraSqlCursorD1Ev+0x3f' }, { fileName: '/app/siebel/siebsrvr/lib/libsscdo90.so', method: '_ZN15CSSOraSqlCursorD0Ev+0x22' }, { fileName: '/app/siebel/siebsrvr/lib/libsscfdm.so', method: '_ZN16CSSLockSqlCursorD2Ev+0x77' }, { fileName: '/app/siebel/siebsrvr/lib/libsscfdm.so', method: '_ZN16CSSLockSqlCursorD0Ev+0x22' }, ]; var ignore = {}; uninformativeFunctions.forEach(function(f) { ignore[stackElementId(f, false)] = true; }); // Remove recursion in the stack trace, and any generic functions that are not distinctive var stacktrace = SiebelCoreCrashParserUtils.parse(coreCrash.details); stacktrace = SiebelCoreCrashParserUtils.removeRecursion(stacktrace); stacktrace = stacktrace.filter(function(s) { return s.fileName && s.method && s.fileName.indexOf('/app/siebel/') === 0 && !ignore[stackElementId(s, false)]; }); // Categorize stacktraces based on: // 1. Exact match of top 5 function calls // 2. Fuzzy matching of top 10 function calls (including location) var stacktraceId = stacktrace .map(function(s) { return stackElementId(s, false); }) .slice(0, 5) .join('|'); result.exactMatch = Checksum.calculate(stacktraceId); result.fuzzyMatch = stacktrace .map(function(s) { return stackElementId(s, true); }) .slice(0, 10) .join('|'); } } catch (error) { log.error('Error during categorization: {}', error.message); } function stackElementId(element, includeLocation) { if (includeLocation) { return element.fileName + ';' + element.method + ';' + element.location; } return element.fileName + ';' + element.method; }
In the above configuration, the categorization will consider facts from the 'Siebel Component Crash' KPI for categorization (other facts will be ignored). Facts are categorized by passing them through the Javascript script configured, this script performs some analysis of the fact (in this case, analyzes the crash stack trace and error code) to produce two values; an exact match and a fuzzy match. The exact match must match for facts to be considered in the same category, the fuzzy match must match within the configured match threshold (based on a string distance function) to be considered the same category.
Web App Error Categorization Example
For web application, built in any javascript (angular, react, etc), Germain analyzes the types of errors and finds the new ones and their business impact. Most other APMs only tell you that there is another “Uncaught TypeError” but don’t distinguish between “…reading ‘toString’…” and “…reading 'b'….”
Javascript Message/Error 1:
Javascript Message/Error 2:
Service: Analytics
Feature Availability: 2022.1 or later