The Particular Service Platform

Messaging is a great way to build loosely coupled, scalable, and reliable systems. But it has its challenges. The most common difficulty is seeing what's happening in a system. This is one of the problems solved by the Particular Service Platform.

The Platform's goal is to provide a set of tools that make it easier to build and maintain messaging systems. The tools are tailored to the common needs of messaging systems and 'just work', out of the box. They enable developers to focus on more important challenges, such as gaining a better understanding of their business domains.

The Particular Service Platform consists of NServiceBus, ServiceControl, ServicePulse, and ServiceInsight.

The details of each component are discussed below. A Particular Service Platform-based system consists of several NServiceBus endpoints. Endpoints are logical entities that perform business operations. They communicate with each other using messages (via queues) and forward messages to ServiceControl for auditing. ServiceControl stores this audit trail and provides integration points for ServicePulse and ServiceInsight. ServicePulse provides monitoring and recoverability for production systems. ServiceInsight provides debugging and visualization of how the system works.

NServiceBus - where it all begins

NServiceBus is the heart of a distributed system and the Particular Service Platform. It helps create systems that are scalable, reliable, and flexible. Adding NServiceBus to a project can be done via .NET CLI:

dotnet add package NServiceBus

At its core, NServiceBus works by routing messages between endpoints. Messages are plain C# classes that contain meaningful data for the business process being modeled.

public class ProcessOrder
{
    public int OrderId { get; set; }
}

Endpoints are logical entities that send and/or receive messages.

// Sending endpoint
await endpoint.Send(new ProcessOrder { OrderId = 15 });

// Receiving endpoint
public class ProcessOrderHandler : IHandleMessages<ProcessOrder>
{
    public async Task Handle(ProcessOrder messsage, IMessageHandlerContext context)
    {
        // Do something with ProcessOrder message
    }
}

Endpoints can be running in different processes, on different machines, even at different times. NServiceBus ensures that each message reaches its intended destination and is processed.

NServiceBus accomplishes this by providing an abstraction over existing queuing technologies. While it's possible to work directly with queuing systems, NServiceBus provides extra features to make applications more reliable and scalable.

Reliable

NServiceBus offers different ways of ensuring information is not lost due to system failures. First, NServiceBus provides native transaction support for the underlying queuing technologies that support it, as well as its own transaction guarantees through the implementation of the Outbox pattern.

In other cases, NServiceBus has built-in recoverability that can automatically adapt to common failures in a system. For intermittent failures, such as network outages, messages can be retried at regular intervals. For more serious errors, messages are set aside in a separate error queue so that they can be investigated at a later time without impacting the overall performance of the system.

Watch this video for a better picture of how failures can lead to lost data, and how using NServiceBus keeps data safe.

Scalable

NServiceBus is designed to handle a large number of messages. Endpoints are configured for high performance by default, handling multiple messages in parallel. Depending on the workload, the number of messages handled concurrently can be increased to improve message throughput.

In high-volume scenarios, where more messages are produced than a single physical endpoint can handle, the logical endpoint can be scaled out across multiple physical instances running on different machines, sharing the load.

Each endpoint tracks key performance metrics that can be exposed as Windows Performance Counters or collected into a central dashboard. The monitoring demo demonstrates how to find performance bottlenecks and identify endpoints that are ready to scale out.

Simple and testable

NServiceBus is designed with simplicity in mind. Message handlers don't need additional code to manage logging, exception handling, serialization, transactions, or the specifics of a queueing technology. This allows message handler code to focus on business logic.

Long-running business workflows can be modeled in NServiceBus using sagas. A saga is a C# class that handles a number of different messages over time, persisting its state between each step in the workflow. NServiceBus makes sure that each saga instance only processes a single message at a time, keeping its internal state consistent. The NServiceBus saga tutorials provide more details.

Message handlers and sagas can be tested in isolation. Simulating an incoming message is as simple as creating a new C# message object and passing it to the appropriate handler or saga. The framework includes a suite of testing tools that capture the behavior of message handlers and sagas under test, allowing assertions to be made.

Flexible

NServiceBus endpoints can be hosted anywhere code can be executed, such as in a Windows Service, a Docker container, or in the cloud with Azure or AWS. Endpoints can run on a variety of platforms.

NServiceBus works with many different technology stacks, offering choices for transport and persistence. Out of the box, defaults are provided for serialization, dependency injection, and logging. These defaults can be overridden if a specific technology is required.

The NServiceBus message processing and dispatching pipeline is modular and extensible. Message mutators inject code into the pipeline to modify messages as they are being sent or received. More complex pipeline manipulation are possible with behaviors. NServiceBus extensions can be packaged up as features, which can add behaviors to the pipeline and create tasks that get run when an endpoint starts and stops. Many of the existing capabilities of NServiceBus are implemented as behaviors and features.

Particular Service Platform

NServiceBus is designed to work with the rest of the Particular Service Platform. All messages are instrumented with additional headers detailing key information about the message and how it was processed. As each message is processed, it is forwarded to an audit queue, where it is picked up by ServiceControl. ServiceInsight connects to a ServiceControl instance to provide powerful visualizations of a running NServiceBus system, making it easy to understand message flows and timing.

When a message fails to be processed, even after a number of retry strategies have been attempted, NServiceBus will forward the message to an error queue for manual investigation. Messages sent to the error queue are instrumented with headers containing details about the failure including a full exception stack trace. ServiceControl picks up messages from the error queue and makes them available in ServicePulse. Once the root cause of the failure has been found and corrected, all messages caused by the same problem can be retried at once.

Additionally, each endpoint can send heartbeat, health check, and performance metrics through the platform for visualization in ServicePulse, making it easy to see which endpoints are offline, which are ready to scale out, and which require manual intervention.

The real-time monitoring demo provides the ability to experience the Service Platform in action. The Platform Sample package provides the ability to demonstrate the Service Platform from within any Visual Studio solution, without the need to install anything.

See it in action

• Quickstart tutorial - Get a tour through an NServiceBus solution covering all the elements of one-way messaging, publish-subscribe, and automatic recovery from exceptions.
• (Video) Live coding an NServiceBus system - For those who prefer to sit back and watch an experienced developer build a messaging system with NServiceBus.

ServiceControl - data collection

ServiceControl is the backend for ServicePulse and ServiceInsight. It is a background process that collects useful information about an NServiceBus system. This includes messages that cannot be processed, their exceptions, and the ability to collect every message flowing through the system. It can also collect saga state changes, endpoint heartbeats, and perform custom checks. This information is exposed to ServicePulse and ServiceInsight via an HTTP API.

ServiceControl can also be configured to collect detailed performance metrics for display in ServicePulse.

ServiceInsight - visualization

ServiceInsight is a desktop application designed for developers. It provides advanced debugging, visualizations of messages flowing through the system, saga state changes, and more.

It's quicker and easier to spot anomalies and incorrect behavior in a system using message flow diagrams, sequence diagrams, and other visualizations provided by ServiceInsight. More detailed information, such as message headers and message metadata, is also shown.

ServicePulse - monitoring

ServicePulse is a web application designed for administrators. It provides a clear, near real-time, high-level overview of how a system is currently functioning. The UI also provides common failure recovery operations, such as retrying failed messages.

ServicePulse provides a rich, graphical view of detailed performance metrics. These are shown for logical endpoints, physical instances, and even individual message types.

Working with the platform

ServiceControl and ServicePulse are server applications. They should be deployed in each environment, for example: test, QA, and production. ServiceInsight is a client application. It should be deployed on a workstation and connected to ServiceControl in the appropriate environment.

When investigating problems or developing custom checks, it can be useful to have the Platform installed on a development machine.