Stream Correlation

⚙️ Configure Stream Correlation

Stream Correlation links related data points from two sources when a unique correlation ID cannot be generated, propagated, or retrieved.

💡 Examples

For example, it can associate:

Instead of requiring an exact identifier, GermainUX uses a customizable cost function to evaluate potential matches. The function can compare timestamps, durations, URLs, users, servers, transaction names, or other shared attributes.

The lower the calculated cost, the more likely the two data points represent the same transaction. A cost of 0 indicates an exact match according to the configured criteria.

🔗 Use with ID-Based Correlation

Stream Correlation can complement ID-Based Correlation by filling gaps where correlation headers or IDs cannot pass through part of the transaction path.

For example, GermainUX may use ID-Based Correlation across most application services while using Stream Correlation to connect a browser request with a backend transaction when the web server does not expose or preserve the correlation header.

Cost and pairing functions are fully customizable for each use case.

🔧 Configuration Example

The following example correlates two streams from a generic web application:

📃 Create a Stream Correlation Scenario

1️⃣ Open Correlation Configuration

Go to:

Germain Workspace -> Left Menu -> Settings → Analytics -> Correlation

Under Stream Correlation Scenarios, select the Add icon.

2️⃣ Name the Scenario

Enter a unique name for the Stream Correlation scenario, then select Next.

3️⃣ Configure the Left Stream

The left stream represents the first source of data to correlate.

Configure:

Select Next when the stream is configured.

4️⃣ Configure the Right Stream

Repeat the same configuration for the second data source.

For this example, select the KPI containing the web-server access-log transactions. Use a compatible partition key and ordering field so records from both streams can be evaluated together.

🧮 Define the Cost Function

The cost function receives one data point from each stream and returns a numeric value representing their similarity:

The following example compares timestamps and durations:

if (left == null || right == null) {
    // Fixed cost for leaving a record unmatched.
    return 20 * 60.0;
}

double cost = 0.0;

long timestampLeft =
    left.timestamp.toEpochSecond(java.time.ZoneOffset.UTC);
long timestampRight =
    right.timestamp.toEpochSecond(java.time.ZoneOffset.UTC);

// Increase the cost according to the timestamp difference.
cost += Math.abs(timestampLeft - timestampRight);

// Increase the cost according to the duration difference.
cost += Math.abs(left.duration - right.duration);

return cost;

This example favors records that occurred at approximately the same time and had similar durations.

The cost function can include additional criteria when needed, such as:

🤝 Define the Pair Function

The pair function determines what GermainUX should do after matching two data points.

In this example, GermainUX copies the backend access-log transaction sequence onto the UX request. This creates a persistent link that can be displayed in the Analysis view.

if (left == null || right == null) {
    return; // No match was found.
}

// Copy the backend transaction identifier to the UX request.
left.backendSequence = right.sequence;

// Save the updated UX request.
processor.updateFact(left);

The pair function can be customized to:

🚀 Activate the Scenario

Save the completed Stream Correlation scenario.

The GermainUX Analytics service will begin processing incoming data from the selected streams using the configured partitioning, ordering, cost, and pair functions.

📋 Configuration Considerations

For reliable results: