OpenTelemetry

NServiceBus version 8 and above supports OpenTelemetry through traces, metrics, and logging.

Enable OpenTelemetry instrumentation in NServiceBus:

endpointConfiguration.EnableOpenTelemetry();

With OpenTelemetry instrumentation enabled, tracing, metrics, and logging can be individually configured via the OpenTelemetry API itself.

Traces

NServiceBus endpoints generate OpenTelemetry traces for incoming and outgoing messages. To capture trace information, add the NServiceBus.Core activity source to the OpenTelemetry configuration:

var tracingProviderBuilder = Sdk.CreateTracerProviderBuilder()
    .AddSource("NServiceBus.Core")
    // ... Add other trace sources
    // ... Add exporters
    .Build();

See the OpenTelemetry samples for instructions on how to send trace information to different tools.

Metrics

NServiceBus endpoints can be configured to expose metrics related to message processing. To capture meter information, add the NServiceBus.Core meter source to the OpenTelemetry configuration:

var meterProviderProvider = Sdk.CreateMeterProviderBuilder()
    .AddMeter("NServiceBus.Core")
    // ... Add other meters
    // ... Add exporters
    .Build();

Emitted meters

• nservicebus.messaging.successes - Total number of messages processed successfully by the endpoint
• nservicebus.messaging.fetches - Total number of messages fetched from the queue by the endpoint
• nservicebus.messaging.failures - Total number of messages processed unsuccessfully by the endpoint

See the OpenTelemetry samples for instructions on how to send metric information to different tools.

Additional metrics

The NServiceBus.Metrics package provides additional metrics that are not yet exposed via OpenTelemetry.

These metrics include:

• Critical time
• Processing time
• Handler time
• Retries

To expose these metrics via OpenTelemetry, a shim can be used. The OpenTelemetry shim sample demonstrates how to set up a shim that exposes NServiceBus.Metrics in an OpenTelemetry format and exports them to Azure Application Insights.

Logging

NServiceBus supports logging out of the box. To collect OpenTelemetry-compatible logging in NServiceBus endpoints, it's possible to configure the endpoint to connect traces and logging when using Microsoft.Extensions.Logging package. See the Connecting OpenTelemetry traces and logs sample for more details.

Alignment of host identifier

It is recommended to align the instance identifier between NServiceBus and OpenTelemetry so all logs, metrics, traces and audit messages can be correlated by a host (instance) if needed.

NServiceBus adds a host identifier to all audit messages and this instance identifier is also used to show performance metrics for each running instance in ServicePulse. The instance identifier used for ServicePulse value can be overriden.

OpenTelemetry also allows to customize the instance id used for service.instance.id in various ways.

Consider aligning the instance ID used by OpenTelemetry and ServiceControl metrics API.

Example

// Generate instance ID shared by all components

// Generate deterministic uuid v4 via
// https://github.com/Faithlife/FaithlifeUtility/blob/master/src/Faithlife.Utility/GuidUtility.cs
var deterministicValue = "MyEndpoint@" + Dns.GetHostName();
Guid serviceInstanceId = GuidUtility.Create(deterministicValue); // or Guid.NewGuid()

// OpenTelemetry
services.AddOpenTelemetry()
    .ConfigureResource(builder =>
        builder.AddService("MyService", serviceInstanceId: serviceInstanceId.ToString()));

// NServiceBus
endpointConfiguration.UniquelyIdentifyRunningInstance()
    .UsingCustomDisplayName("original-instance")
    .UsingCustomIdentifier(serviceInstanceId);

// ServiceControl Metrics
endpointConfiguration
    .EnableMetrics()
    // Not required when already set via UsingCustomIdentifier
    .SendMetricDataToServiceControl("particular.monitoring",
        TimeSpan.FromMinutes(1), serviceInstanceId.ToString());