Endpoints using the MSMQ transport are unable to use the competing consumers pattern to scale out by adding additional worker instances. Sender-side distribution is a method of scaling out an endpoint using the MSMQ transport, without relying on a centralized distributor assigning messages to available workers.
When using sender-side distribution:
- Multiple endpoint instances (deployed to different servers) are capable of processing a message that requires scaled-out processing.
- A client sending a message is aware of all the endpoint instances that can process the message.
- The client sends the message to a worker endpoint instance based on round-robin distribution, or a custom distribution strategy.
Using sender-side distribution requires two parts. The first part maps message types to logical endpoints, and occurs in code. The second part maps logical endpoints to physical endpoint instances running on a specific machine.
To map message types to logical endpoints, use the following config:
var transport = endpointConfiguration.UseTransport<MsmqTransport>(); var routing = transport.Routing(); routing.RouteToEndpoint( messageType: typeof(AcceptOrder), destination: "Sales"); routing.RouteToEndpoint( messageType: typeof(SendOrder), destination: "Shipping");
This creates mappings specifying that the
AcceptOrder command is handled by the Sales endpoint, while the
SendOrder command is handled by the Shipping endpoint.
Meanwhile, the logical-to-physical mappings will be configured in the
instance-mapping.xml file, as this information is an operational concern that must be changed for deployment to multiple machines.
UnicastBusConfig/MessageEndpointMappingsconfiguration section, then that message cannot participate in sender-side distribution. The endpoint address specified by a message endpoint mapping is a physical address (
QueueName@MachineName, where machine name is assumed to be
localhostif omitted) which combines the message-to-owner-endpoint and endpoint-to-physical-address concerns in a way that can't be separated.
The routing configuration file specifies how logical endpoint names are mapped to physical queues on specific machines:
<endpoints> <endpoint name="Sales"> <instance machine="VM-S-1"/> <instance machine="VM-S-2"/> <instance machine="VM-S-3"/> </endpoint> </endpoints>
To read more about the instance mapping, refer to the MSMQ routing page.
Sender-side distribution does not use message processing confirmations (the Distributor approach). Therefore the sender has no feedback on the availability of workers and, by default, sends the messages in a round-robin behavior. Should one of the nodes stop processing, the messages will start piling up in its input queue. As such nodes running in sender-side distribution mode require more careful monitoring compared to distributor workers.
For the reasons outlined above, when scaling down (removing a "target" endpoint instance from service), it is important to properly decommission the instance:
- Change the instance mapping file to remove the target endpoint instance.
- Ensure that the updated instance mapping information is distributed to all endpoint instances that might send a message to the target endpoint.
- Allow time (30 seconds by default) for all endpoints to reread the instance mapping file, and ensure no new messages are arriving in the target instance's queue.
- Allow the target endpoint instance to complete processing all messages in its queue.
- Disable the target endpoint instance.
- Check the input queue of the decommissioned instance for leftover messages and move them to other instances if necessary.