.NET Core 3.1 introduced support for distributed tracing. Types in the System.Diagnostics namespace may be used to expose and correlate execution data from a distributed system. This sample shows how to extend the NServiceBus pipeline with custom behaviors that publish trace data and how to send that data to Azure Monitor Application Insights for analysis and visualization.
Running the sample
Running the sample requires an Application Insights resource. The Instrumentation Key must be provided by setting the ApplicationInsightsKey
environment variable.
After starting the sample, pressing Enter sends an InitialCommand
. When that command is handled, a FollowUpEvent
is published. Trace data for both actions is pushed to Application Insights and available on the transaction diagnostics pane:
Code overview
Pipeline behaviors
Trace data is exposed by custom pipeline behaviors. Each behavior defines a DiagnosticSource.
static DiagnosticSource diagnosticSource = new DiagnosticListener(MessageTracing.SendMessage);
When a behavior is executed, if there is any listener subscribed to the source, an instance of Activity is created and started with the diagnostic source.
if (diagnosticSource.IsEnabled(MessageTracing.SendMessage))
{
activity = new Activity(MessageTracing.SendMessage);
foreach (var item in context.Headers)
{
activity.AddTag(item.Key, item.Value);
}
diagnosticSource.StartActivity(activity, context.Headers);
// This must happen after StartActivity to ensure the Id is initialized
context.Headers.Add(MessageTracing.ParentActivityIdHeaderName, activity.Id);
}
The new Activity
inherits any existing tracing context (stored in Activity.
), which initializes the ParentId
property and links the new Activity
to the parent Activity
. Message headers are stored in the Tags
collection to provide information about the specific message.
Finally, the activity ID is stored in the outgoing message header to allow the recipient to link its activities to those of the sender.
Application Insights integration
Trace data produced by the behaviors is consumed by tracing listeners registered in NServiceBusInsightsModule
:
if (value.Name == MessageTracing.ReceiveMessage)
{
var listener = new NServiceBusTracingListener(client);
var subscription = value.Subscribe(listener);
subscriptions.Add(subscription);
listeners.Add(listener);
}
When an activity is started by a behavior, a listener is notified by its OnNext
method. The activity is pushed to Application Insights using the TelemetryClient
API:
var key = value.Key;
var activity = Activity.Current;
if (key.EndsWith("Start"))
{
var operation = client.StartOperation<DependencyTelemetry>(activity);
operation.Telemetry.Type = "NServiceBus";
inProgressOperations.AddOrUpdate(activity.Id, _ => operation, (_, __) => operation);
}
else if (key.EndsWith("Stop"))
{
if (inProgressOperations.TryRemove(activity.Id, out var operation))
{
operation.Telemetry.Success = (bool) value.Value;
client.StopOperation(operation);
operation.Dispose();
}
}