Using ServiceControl events

Custom notification and alerting using ServiceControl events

ServiceControl events enable the construction of custom notifications and integration that will alert of something going wrong in the system.

ServiceControl Endpoint Plugins collect information from monitored NServiceBus endpoints.

Alerting on FailedMessages Event

Once a message ends up in the error queue ServiceControl will publish a MessageFailed event. The message contains:

• The processing of the message (which endpoint sent and received the message).
• The failure cause (the exception type and message).
• The message itself (the headers and, if using non-binary serialization, also the body).

Subscribing to ServiceControl Events

ServiceControl publishes MessageFailed event when a message gets to the error queue, it is possible to subscribe to these events and act on them (send an email, pager duty and so on).

To subscribe to the MessageFailed event:

• Create an NServiceBus endpoint.
• Install the ServiceControl.Contracts NuGet package.
• Add the message mapping in the UnicastBusConfig section of the endpoint's app.config so that this endpoint will subscribe to the events from ServiceControl as shown:

<UnicastBusConfig>
  <MessageEndpointMappings >
    <add Assembly="ServiceControl.Contracts"
         Endpoint="Particular.ServiceControl" />
  </MessageEndpointMappings>
</UnicastBusConfig>

• Customize the endpoint configuration to use JsonSerializer as the message published by ServiceControl uses Json serialization.
• Also customize the endpoint configuration such that the following conventions are used, as the MessageFailed event that is published by ServiceControl does not derive from IEvent.

This code sample illustrates how to do this customization:

endpointConfiguration.UseSerialization<JsonSerializer>();
var conventions = endpointConfiguration.Conventions();
conventions.DefiningEventsAs(
    type =>
    {
        return typeof(IEvent).IsAssignableFrom(type) ||
               // include ServiceControl events
               type.Namespace != null &&
               type.Namespace.StartsWith("ServiceControl.Contracts");
    });

• The endpoint will also need a message handler, that handles the MessageFailed event. In the following example, there is also a simple HTTP call to HipChat's API to show how to possibly integrate with a 3rd party system to provide notification of the error.

class MessageFailedHandler :
    IHandleMessages<MessageFailed>
{
    public Task Handle(MessageFailed message, IMessageHandlerContext context)
    {
        var failedMessageId = message.FailedMessageId;
        var exceptionMessage = message.FailureDetails.Exception.Message;

        var chatMessage = $@"Message with id: {failedMessageId} failed.
Reason: '{exceptionMessage}'.
Open in ServiceInsight: {GetServiceInsightUri(failedMessageId)}";

        using (var client = new HipchatClient())
        {
            client.PostChatMessage(chatMessage);
        }
        return Task.CompletedTask;
    }

Monitoring events

ServiceControl will also publish events based on collected monitoring data.

See Heartbeat notification events and Custom check notification events for a description of these events.

Decommissioning alert subscribers

ServiceControl uses Event Publishing to expose information to subscribers. When using a persistence based transport an internal reference will be kept to each subscriber. If a subscriber, for an event, cannot be contacted then a log entry will be written with the following error:

Failed dispatching external integration event

An event will also be published and displayed in ServicePulse dashboard with the following text:

'EVENTTYPE' failed to be published to other integration points. Reason for failure: REASON.

To avoid this situation it is important to properly decommission an endpoint that subscribes to ServiceControl events. To do this disable auto-subscription and then unsubscribe from all events.