Data Categorization

Feature

Germain has the feature of categorizing data/facts into groups, which proves to be highly beneficial in identifying new problems or issues among a large volume of data. This capability is particularly useful in error detection and categorization.

By grouping similar errors into a single category, Germain allows system administrators to efficiently analyze and investigate errors. This mechanism provides several advantages in error management:

Frequency Analysis

System administrators can determine the frequency of similar errors occurring. This information is valuable when dealing with a large number of errors. For example, if there are 500 errors to review, knowing that 490 of them are the same helps focus attention on the root cause rather than analyzing each error individually.

Reduced Effort

Categorization significantly reduces the effort required in reviewing errors. Instead of examining each error separately, system administrators can focus on reviewing unique error categories. This streamlines the analysis process and allows for more efficient troubleshooting.

Priority Identification

By categorizing errors, system administrators can identify which errors occur most frequently or affect the largest number of users. This helps prioritize the resolution of critical issues that have a significant impact on the system or user experience.

Detection of -New- Crash or Exception

The categorization feature also enables the identification of new Crash or Exception. System administrators can easily spot Crash or Exception that have not been encountered before, indicating potential issues that require immediate attention. This proactive approach ensures that emerging problems are addressed promptly, preventing any negative impact on system performance or user experience.

In summary, Germain's ability to categorize data/facts into groups provides system administrators with efficient error management. It helps in analyzing error frequency, reducing effort in error review, prioritizing issue resolution, and detecting new or unique errors. These capabilities streamline the troubleshooting process, leading to faster issue resolution and improved overall system performance.

Configuration

For generic application built with standard tech (e.g. java, goLang, php, c++, etc), you won’t need to customize this “categorization”.

However for more complex application, you will want to define a Categorization to identify more complex Crash or Exception if it involves the analysis of several data sources. This functionality can be easily customized by writing a Javascript processor.

Germain Workspace > Left Menu > Analytics > Categorization

Examples

This new error categorization mechanism is currently preconfigured for generic technologies like Angular, Java, .Net, React, http://Salesforce.com , Siebel CRM, etc. Here are a few examples.

• Java app
  For java application, Germain analyzes the “path” of an exception/failure and finds the new ones vs the ones that have been reoccurring. Most other APMs only tell you that there is another “nullpointerexception” but don’t help you distinguish between “UserService.getCurrent” and “BusinessLogicService.performLogic” making it very hard to deal with as they often are millions.

  Java Exception 1:

  

  Java Exception 2:

• Salesforce Apex Code and Exception
  

  

• Siebel CRM
  For Siebel CRM for instance, Germain now distinguishes the Siebel Object Manager crashes that are “new” from the ones that are already “known” and reoccurring (in addition, Germain also quantifies the count of users impacted by a crash).

  • Example - Dashboards that help analyze Siebel OM Crashes

    

    

    

  • How to configure Siebel OM Crash Categorization

    • Siebel Data Sources
      This above business impact analysis and identification of a new OM crash comes as a result of Germain analyzing 1 to 3 Siebel files.

      • Siebel Error:

        

      • Siebel FDR file:

        

      • Siebel Core file:

        

    • Configuration Screen
      

      

      CODE

      importClass(com.germainsoftware.data.Checksum);
      importClass(com.germainsoftware.apm.storage.categorization.siebel.SiebelCoreCrashParserUtils);
      
      var errorCodesWithoutCoreCrash = {
          'SBL-SMI-00062': true,  // Internal: No more process (multithreaded server) slots available
          'SBL-SVR-09127': true,  // Internal: Fail to initialize the shared memory resource for the process
      };
      
      try {
          var supportingFactIdentifiers = fact.details ? fact.details.split('|') : [];
          var coreCrash;
          var knownErrorCode;
          for (var i=0; i<supportingFactIdentifiers.length; i++) {
              var identifier = supportingFactIdentifiers[i].split(';');
              var factClassName = identifier[0];
              var factId = identifier[1];
              var factType = identifier[2];
      
              if (factType === 'Siebel:Enterprise Crash') {
                  var errorCode = identifier[3];
                  if (errorCode && (errorCodesWithoutCoreCrash[errorCode] || errorCode.indexOf('SBL-GEN-') === 0)) {
                      knownErrorCode = errorCode;
                  }
              } else if (factType === 'Siebel:Core Crash') {
                  coreCrash = queryService.findById(factClassName, factId);
              }
          }
      
          if (knownErrorCode) {
              // If this crash is caused by an error that is known to not generate a Core Crash,
              // perform matching based only on error code, since this is all we have
              log.debug('FactID: {} - Categorizing based on Error code', fact.id, knownErrorCode);
              result.exactMatch = Checksum.calculate(knownErrorCode);
              result.fuzzyMatch = '';
          } else if (coreCrash) { 
              // If we have a Core Crash, use that to determine similarity of crashes
              log.debug('FactID: {} - Categorizing based on Core Crash', fact.id);
              var uninformativeFunctions = [
                  { fileName: '/app/siebel/siebsrvr/lib/libsslcosd.so', method: '+0x7185' },
                  { fileName: '/app/siebel/siebsrvr/lib/libsslcosd.so', method: '+0x797e' },
                  { fileName: '/app/siebel/siebsrvr/lib/libsscdo90.so', method: '_ZN15CSSOraSqlCursorD1Ev+0x3f' },
                  { fileName: '/app/siebel/siebsrvr/lib/libsscdo90.so', method: '_ZN15CSSOraSqlCursorD0Ev+0x22' },
                  { fileName: '/app/siebel/siebsrvr/lib/libsscfdm.so', method: '_ZN16CSSLockSqlCursorD2Ev+0x77' },
                  { fileName: '/app/siebel/siebsrvr/lib/libsscfdm.so', method: '_ZN16CSSLockSqlCursorD0Ev+0x22' },
              ];
              var ignore = {};
              uninformativeFunctions.forEach(function(f) { ignore[stackElementId(f, false)] = true; });
      
              // Remove recursion in the stack trace, and any generic functions that are not distinctive
              var stacktrace = SiebelCoreCrashParserUtils.parse(coreCrash.details);
              stacktrace = SiebelCoreCrashParserUtils.removeRecursion(stacktrace);
              stacktrace = stacktrace.filter(function(s) {
                  return s.fileName && s.method
                      && s.fileName.indexOf('/app/siebel/') === 0
                      && !ignore[stackElementId(s, false)];
              });
      
              // Categorize stacktraces based on:
              // 1. Exact match of top 5 function calls
              // 2. Fuzzy matching of top 10 function calls (including location)
              var stacktraceId = stacktrace
                  .map(function(s) { return stackElementId(s, false); })
                  .slice(0, 5)
                  .join('|');
              result.exactMatch = Checksum.calculate(stacktraceId);
              result.fuzzyMatch = stacktrace
                  .map(function(s) { return stackElementId(s, true); })
                  .slice(0, 10)
                  .join('|');
          }
      } catch (error) {
          log.error('Error during categorization: {}', error.message);
      }
      
      function stackElementId(element, includeLocation) {
          if (includeLocation) {
              return element.fileName + ';' + element.method + ';' + element.location;
          }
          return element.fileName + ';' + element.method;
      }

      In the above configuration, the categorization will consider facts from the 'Siebel Component Crash' KPI for categorization (other facts will be ignored). Facts are categorized by passing them through the Javascript script configured, this script performs some analysis of the fact (in this case, analyzes the crash stack trace and error code) to produce two values; an exact match and a fuzzy match. The exact match must match for facts to be considered in the same category, the fuzzy match must match within the configured match threshold (based on a string distance function) to be considered the same category.

• Web App error
  For web application, built in any javascript (angular, react, etc), Germain analyzes the types of errors and finds the new ones and their business impact.

  Most other APMs only tell you that there is another “Uncaught TypeError” but don’t distinguish between “…reading ‘toString’…” and “…reading 'b'….”

  • Javascript Message/Error 1:

    

  • Javascript Message/Error 2:

    

Service: Analytics

Feature Availability: 2022.1 or later