Upgrading a Distributor endpoint from Version 5 to 6

Initial state

This sample uses the same solution as the Version 5 distributor sample:

• Sender sends commands to the scaled out endpoint.
• Worker1 is a master (runs both distributor and a worker node).
• Worker2 is a slave (runs only a worker node).

Upgrade

Adding stand-alone distributor

Versions 6 and above no longer support running an embedded distributor so a separate project has to be added to the solution.

This new project is going to reference NServiceBus Version 5.X because distributor does not have a Version 6 compatible release. It also needs to reference the NServiceBus.Distributor.MSMQ package.

Writing the distributor code

Following code is required to set up the distributor

var busConfiguration = new BusConfiguration();
busConfiguration.EndpointName("Samples.Scaleout.Distributor");
busConfiguration.RunMSMQDistributor(withWorker: false);
busConfiguration.UsePersistence<InMemoryPersistence>();
busConfiguration.EnableInstallers();
using (var bus = Bus.Create(busConfiguration).Start())
{
    Console.WriteLine("Press any key to exit");
    Console.ReadKey();
}

Notice that false is passed as a value for withWorker argument of the RunMSMQDistributor API call.

Reconfiguring sender's routing

In the original sample the sender was sending messages to the master (Worker1). Now it needs to send messages to the new stand-alone distributor

bus.Send("Samples.Scaleout.Distributor", placeOrder);

Upgrading the workers

As both the workers are now only processing messages, they no longer need the NServiceBus.Distributor.MSMQ package. The NServiceBus package needs to be upgraded to 6.x.

There are minor changes required to make the workers compile against the Version 6:

• Changing BusConfiguration to EndpointConfiguration.
• Using the new async endpoint create/start APIs.

Giving workers new identity

In Versions 6 and above each endpoint instance is identified by name of the endpoint and an ID of the instance. Both workers are going to be named Samples.Scaleout.Worker and the instance ID is going to be loaded from the app.config file. If the workers are deployed to separate machines the instance ID can be omitted.

var endpointConfiguration = new EndpointConfiguration("Samples.Scaleout.Worker");
endpointConfiguration.MakeInstanceUniquelyAddressable(
    discriminator: ConfigurationManager.AppSettings["InstanceId"]);

Enlisting with the distributor

In Versions 6 and above there is a new API for enlisting with the distributor. The API requires passing addresses of distributor data and control queues which were previously read from the configuration section.

endpointConfiguration.EnlistWithLegacyMSMQDistributor(
    masterNodeAddress: ConfigurationManager.AppSettings["DistributorAddress"],
    masterNodeControlAddress: ConfigurationManager.AppSettings["DistributorControlAddress"],
    capacity: PushRuntimeSettings.Default.MaxConcurrency);

The last parameter, capacity is the maximum number of messages in-flight between the distributor and the worker. The worker sends an acknowledgement (ACK) message (also called ready message) for each message processed. The distributor keeps track of these ACKs to ensure no worker is flooded with work that it can't process

In Versions 5 and below the capacity was always set to the same value as the maximum concurrency setting, which means there were never more messages in-flight than threads ready to process them on the worker side. If the worker was up and running messages never waited in the queue. This approach limits the potential consequences of worker failure at the expense of throughput.

A new setting was introduced in Version 6, which allows to explicitly control the capacity value. The snippet above shows how to set it to the default concurrency value of Version 6. If the endpoint configuration overrides this parameter with a custom value, an equal or larger value should be provided as capacity when enlisting with distributor.

Upgrading the handlers

Handlers need to be upgraded to conform to new Version 6 API.

Removing the shared code

Because the sender still uses Version 5 and the workers are on Version 6, the shared elements need to be moved to their target projects:

• ConfigErrorQueue class copied to Sender and Distributor projects.
• Sender and Worker projects configured to use unobtrusive mode.

Final state

After the upgrade is done both worker projects are identical (apart from the configuration file). The sender needed only a minor routing correction.