Customizing OpenTelemetry tracing

This sample shows how to extend the OpenTelemetry activities in different ways.

Running the project

The code consists of a single endpoint project that sends messages to itself.

Press O to send a CreateOrder message with a randomized OrderId. When the message is handled, two more messages are created: BillOrder and ShipOrder.

As the messages are sent and processed, trace data is exported to the console. Some of the trace data originates from NServiceBus and some from custom code in the sample.

Code walk through

Global configuration

NServiceBus OpenTelemetry instrumentation is not enabled by default. It must be enabled on the endpoint configuration.

Copy code|Copy usings|Edit

var endpointConfiguration = new EndpointConfiguration("MyEndpoint");
endpointConfiguration.EnableOpenTelemetry();

OpenTelemetry is configured to export all traces to the command line. It includes the NServiceBus.* sources built into NServiceBus and a custom activity source defined in the sample (see below).

Copy code|Copy usings|Edit

var resourceBuilder = ResourceBuilder.CreateDefault()
    .AddService(serviceName: "MyEndpoint", serviceInstanceId: Environment.MachineName);

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .SetResourceBuilder(resourceBuilder)
    .AddSource("NServiceBus.*")
    .AddSource(CustomActivitySources.Name)
    .AddConsoleExporter()
    .Build();

Custom activities

The sample includes a custom activity source.

Copy code|Copy usings|Edit

static class CustomActivitySources
{
    public const string Name = "Sample.ActivitySource";
    public static readonly ActivitySource Main = new(Name);
}

The handler for CreateOrder includes a custom activity that wraps around the billing section.

Copy code|Copy usings|Edit

public async Task Handle(CreateOrder message, IMessageHandlerContext context)
{
    using var activity = CustomActivitySources.Main.StartActivity("Billing Order");

    if (message.SimulateFailure)
    {
        throw new MyBusinessException{ ReasonCode = 1};
    }

    Console.WriteLine($"Billing order {message.OrderId}");
    activity?.AddTag("sample.billing.system", "paypal");
    // Calculate order cost
    await context.SendLocal(new BillOrder { OrderId = message.OrderId });

    Console.WriteLine($"Shipping order {message.OrderId}");
    await context.SendLocal(new ShipOrder { OrderId = message.OrderId });
}

This will automatically be created as a child activity of the invoke handler activity created by NServiceBus. The NServiceBus send message activity will treat this custom activity as its parent.

Copy code|Edit
Send CreateOrder
  Process CreateOrder
    Invoke CreateOrderHandler
      Billing Order <-- Custom activity
        Send BillOrder
      Send ShipOrder

Adding tags

Activity.Current may be null if there are no configured trace listeners. Always check if the value is null before calling methods on an Activity instance, or use the null-conditional operator (?.).

The handler for ShipOrder adds tags to the ambient behavior.

Copy code|Copy usings|Edit

class ShipOrderHandler : IHandleMessages<ShipOrder>
{
    public Task Handle(ShipOrder message, IMessageHandlerContext context)
    {
        Console.WriteLine($"Order shipped {message.OrderId}");
        // Figure out what state we are shipping to
        Activity.Current?.AddTag("sample.shipping.state", "STATE");
        return Task.CompletedTask;
    }
}

In the sample, these tags will be added to the NServiceBus invoke handler activity.

Copy code|Edit
Send ShipOrder
  Process ShipOrder
    Invoke ShipOrderHandler <-- Custom tag gets added here

A behavior in the outgoing pipeline adds the size of the message as a tag for all outgoing message activities.

Copy code|Copy usings|Edit

class TraceOutgoingMessageSizeBehavior : Behavior<IOutgoingPhysicalMessageContext>
{
    public override Task Invoke(IOutgoingPhysicalMessageContext context, Func<Task> next)
    {
        Activity.Current?.AddTag("sample.messaging.body.size", context.Body.Length);
        return next();
    }
}

A behavior in the handler pipeline captures a custom business exception and adds its "reason code" as a tag on the handler span created by NServiceBus.

Copy code|Copy usings|Edit

class TraceCustomExceptionInHandlerBehavior : Behavior<IInvokeHandlerContext>
{
    public override async Task Invoke(IInvokeHandlerContext context, Func<Task> next)
    {
        try
        {
            await next();
        }
        catch (MyBusinessException e)
        {
            Activity.Current?.AddTag("sample.business-exception.reason", e.ReasonCode);
            throw;
        }
    }
}

public class MyBusinessException : Exception
{
    public int ReasonCode { get; init; }
}

Running the project

Code walk through

Global configuration

Custom activities

Adding tags

Related Articles

In this article