Connecting multiple SQL Server instances with a backplane

The sample demonstrates how to use NServiceBus.Router to connect endpoints running SQL Server transport that use different instances of SQL Server. This is an alternative to the multi-instance mode of SQL Server transport which has been removed in Version 4.

The RabbitMQ broker is used as a backplane in this sample.

Switch vs Backplane

Both Switch and Backplane approaches can be used replace the deprecated multi-instance mode in connecting endpoints that use different SQL Server databases. The following table contains a side-by-side comparison of both approaches

The Backplane approach, while more complex in terms of deployment, provides more flexibility e.g. some databases might be on-premise while others might be in the cloud.

Throughput

Both approaches can be used to increase the throughput of the entire system when performance of a single SQL Server instance becomes a bottle neck. The key to thing when using the Switch or Backplane for performance reasons is partitioning. When done wrong, it can have the opposite effect and decrease the overall throughput.

To correctly partition the system when using Switch or Backplane first cluster the endpoints based on the volume of messages exchanged. The more messages endpoint exchange, the closer they are. If all endpoints form a single cluster Switch or Backplane won't help. In a healthy system, however, there will be several clusters of endpoints of highly coupled endpoints. Assign each cluster its own instance of SQL Server. Use Switch or Backplane to connect the clusters.

Prerequisites

Ensure an instance of SQL Server (Version 2016 or above for custom saga finders sample, or Version 2012 or above for other samples) is installed and accessible on localhost and port 1433. A Docker image can be used to accomplish this by running docker run -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=yourStrong(!)Password' -p 1433:1433 -d mcr.microsoft.com/mssql/server:latest in a terminal.

Alternatively, change the connection string to point to different SQL Server instance.

At startup each endpoint will create its required SQL assets including databases, tables, and schemas.

This sample automatically creates three databases: backplane_blue, backplane_red and backplane_green

Running the project

1. Start the solution.
2. The text Press <enter> to send a message should be displayed in the Client's console window.
3. Hit enter several times to send some messages.

Verifying that the sample works correctly

1. The Sales console display information about accepted orders in round-robin fashion.
2. The Shipping endpoint displays information that orders were shipped.
3. The Billing endpoint displays information that orders were billed.
4. The Client endpoint displays information that orders were placed.

Code walk-through

This sample contains four endpoints, Client, Sales, Shipping and Billing. The Client endpoint sends a PlaceOrder command to Sales. When PlaceOrder is processed, Sales publishes the OrderAccepted event which is subscribed by Shipping and Billing.

In addition to the four business endpoints, the sample contains three Router endpoints that connect the three databases, Blue, Red and Green, to the RabbitMQ backplane.

Client

The Client endpoint is configured to use its own, Blue, database to harden the security of the solution. This database does not contain sensitive data.

In order to route messages to Sales, Client needs to configure router connection

var routing = transport.Routing();
var routerConfig = routing.ConnectToRouter("Blue");
routerConfig.RouteToEndpoint(typeof(PlaceOrder), "Red.Sales");

Sales and Shipping

The Sales and Shipping endpoints are configured to use the Red database for the transport. As Sales only publishes events and sends replies, it does not need any router configuration.

Shipping subscribes for events published by Sales and it uses the same transport database so router is not involved.

Billing

The Billing endpoint requires even more enhanced security. It uses its own database, Green. In order to subscribe to Sales event it need to register the publisher in the router configuration

var routing = transport.Routing();
var routerConfig = routing.ConnectToRouter("Green");
routerConfig.RegisterPublisher(typeof(OrderAccepted), "Red.Sales");

Routers

Each database is connected to the backplane via a separate router. All three routers share the same configuration

var routerConfig = new RouterConfiguration(routerName);
var sqlInterface = routerConfig.AddInterface<SqlServerTransport>("SQL", t =>
{
    t.ConnectionString(sqlConnectionString);
    t.Transactions(TransportTransactionMode.SendsAtomicWithReceive);
});

var backplaneInterface = routerConfig.AddInterface<RabbitMQTransport>("Backplane", t =>
{
    t.ConnectionString("host=localhost");
    t.UseConventionalRoutingTopology();
});

backplaneInterface.EnableMessageDrivenPublishSubscribe(backplaneSubscriptionStorage);
backplaneInterface.DisableNativePubSub();

routerConfig.AutoCreateQueues();
#pragma warning disable 618
routerConfig.ConfigureDeduplication().EnableInstaller(true);
#pragma warning restore 618

The forwarding of messages between the databases is governed by an endpoint naming convention: the first part of the endpoint name is used as the destination interface name.

var staticRouting = routerConfig.UseStaticRoutingProtocol();

//Forward messages coming from local SQL based on the endpoint name prefix
foreach (var router in otherRouters)
{
    staticRouting.AddRoute(
        destinationFilter: (@interface, dest) =>
        {
            return @interface == "SQL" 
                   && dest.Endpoint != null 
                   && dest.Endpoint.StartsWith(router);
        }, 
        destinationFilterDescription: $"To {router}", 
        gateway: router, 
        iface: "Backplane");
}

//Forward messages coming from backplane to local SQL
staticRouting.AddRoute((@interface, dest) => @interface == "Backplane", "To local", null, "SQL");

Each router has routes to all other SQL databases going through the backplane interface. Each of these routes has a designated gateway set so that messages are not routed directly to the destination, but to the next router.

In addition to that, each router has a route that instructs it to forward all messages coming via the backplane interface to the local SQL interface.

Consistency

The backplane transport (RabbitMQ) offers lower consistency guarantees than the endpoints' transport (SQL Server). The messages can get duplicated while travelling between the databases. This is simulated by the RandomDuplicator router extension which creates duplicates with 50% chance. Each time a duplicate is created, the router logs a warning.

In order to preserve exactly-once message processing guarantees that SQL Server transport offers, messages forwarded from the backplane to the database need to be de-duplicated. The SQL Server de-duplication Router extension addresses this problem.

foreach (var router in otherRouters)
{
    sqlInterface.EnableDeduplication("Backplane", router, 
        () => new SqlConnection(sqlConnectionString), 10);
}