This article covers various levels of consistency guarantees with regard to:
- receiving messages
- updating user data
- sending messages
It does not discuss the transaction isolation aspect. Transaction isolation applies only to the process of updating the user data, it does not affect the overall coordination and failure handling.
Transactions
Four levels of guarantees with regard to message processing are offered. A level's availability depends on the selected transport.
Transaction levels supported by NServiceBus transports
The implementation details for each transport are discussed in the dedicated documentation sections. They can be accessed by clicking the links with the transport name in the following table:
When combining SQL Server transport and SQL persistence with the Microsoft SQL Server dialect, be aware of the different connection behaviors.
Transaction scope (Distributed transaction)
In this mode the transport receive operation is wrapped in a TransactionScope
. Other operations inside this scope, both sending messages and manipulating data, are guaranteed to be executed (eventually) as a whole or rolled back as a whole.
If required, the transaction is escalated to a distributed transaction (following a two-phase commit protocol coordinated by MSDTC) if both the transport and the persistence support it. A fully distributed transaction is not always required, for example when using SQL Server transport with SQL persistence, both using the same database connection string. In this case, the ADO.NET driver guarantees that everything happens inside a single database transaction and ACID guarantees are held for the whole processing.
MSMQ will escalate to a distributed transaction right away since it doesn't support promotable transaction enlistments.
Transaction scope mode is enabled by default for the transports that support it (i.e. MSMQ and SQL Server transport). It can be enabled explicitly with the following code:
var transport = endpointConfiguration.UseTransport(new TransportDefinition
{
TransportTransactionMode = TransportTransactionMode.TransactionScope
});
Microsoft SQL Server does not support DTC transactions in all deployment models (such as database mirroring or Always On configurations) and support differs between versions of SQL Server. See Transactions - Always On availability groups and Database Mirroring for more information.
Atomicity and consistency guarantees
When the TransportTransactionMode
is set to TransactionScope
, handlers will execute inside a System.
created by the transport. This means that all the data updates and queue operations are committed or rolled back.
A distributed transaction between the queueing system and the persistent storage guarantees atomic commits but guarantees only eventual consistency.
Consider a system using MSMQ transport and RavenDB persistence implementing the following message exchange scenario with a saga that models a simple order lifecycle:
OrderSaga
receives aStartOrder
message- A new
OrderSagaData
instance is created and stored in RavenDB. OrderSaga
sendsVerifyPayment
message toPaymentService
.- NServiceBus completes the distributed transaction and the DTC instructs MSMQ and RavenDB resource managers to commit their local transactions.
- The
StartOrder
message is removed from the input queue andVerifyPayment
is immediately sent toPaymentService
. - RavenDB acknowledges the transaction commit and begins writing
OrderSagaData
to disk. PaymentService
receives aVerifyPayment
message and immediately responds with aCompleteOrder
message to the originatingOrderSaga
.OrderSaga
receives theCompleteOrder
message and attempts to complete the saga.OrderSaga
queries RavenDB to find theOrderSagaData
instance to complete.- RavenDB has not finished writing
OrderSagaData
to disk and returns an empty result set. OrderSaga
fails to complete.
In the example above the TransactionScope
guarantees atomicity for the OrderSaga
: consuming the incoming StartOrder
message, storing OrderSagaData
in RavenDB and sending the outgoing VerifyPayment
message are committed as one atomic operation. The saga data may not be immediately available for reading even though the incoming message has already been processed. OrderSaga
is thus only eventually consistent. The CompleteOrder
message needs to be retried until RavenDB successfully returns an OrderSagaData
instance.
This mode requires the selected storage to support participating in distributed transactions.
Transport transaction - Sends atomic with Receive
Some transports support enlisting outgoing operations in the current receive transaction. This prevents messages from being sent to downstream endpoints during retries.
Use the following code to use this mode:
var transport = endpointConfiguration.UseTransport(new TransportDefinition
{
TransportTransactionMode = TransportTransactionMode.SendsAtomicWithReceive
});
Consistency guarantees
This mode has the same consistency guarantees as the Receive Only mode, but additionally, it prevents the occurrence of ghost messages since all outgoing operations are atomic with the ongoing receive operation.
Transport transaction - Receive Only
In this mode, the receive operation is wrapped in a transport's native transaction. This mode guarantees that the message is not permanently deleted from the incoming queue until at least one processing attempt (including storing user data and sending out messages) is finished successfully. See also recoverability for more details on retries.
Sends and Publishes are batched and only transmitted until all handlers are successfully invoked. Messages that are required to be sent immediately should use the immediate dispatch option which bypasses batching.
Use the following code to use this mode:
var transport = endpointConfiguration.UseTransport(new TransportDefinition
{
TransportTransactionMode = TransportTransactionMode.ReceiveOnly
});
Consistency guarantees
In this mode some (or all) handlers might get invoked multiple times and partial results might be visible:
- partial updates - where one handler succeeded in updating its data but the other didn't
- partial sends - where some of the messages have been sent but others not
When using this mode all handlers must be idempotent, i.e. the result needs to be consistent from a business perspective even when the message is processed more than once.
See the Outbox
section below for details on how NServiceBus can handle idempotency at the infrastructure level.
Unreliable (Transactions Disabled)
Disabling transactions is generally not recommended, because it might lead to message loss. It might be considered if losing some messages is not problematic and if the messages get outdated quickly, e.g. when sending readings from sensors at regular intervals.
In this mode, when encountering a critical failure such as a system or endpoint crash, the message is permanently lost.
var transport = endpointConfiguration.UseTransport(new TransportDefinition
{
TransportTransactionMode = TransportTransactionMode.None
});
In this mode the transport doesn't wrap the receive operation in any kind of transaction. Should the message fail to process it will be moved straight to the error queue.
Outbox
The Outbox feature provides idempotency at the infrastructure level and allows running in transport transaction mode while still getting the same semantics as Transaction scope mode.
Outbox data needs to be stored in the same database as business data to achieve the exactly-once delivery
.
When using the Outbox, any messages resulting from processing a given received message are not sent immediately but rather stored in the persistence database and pushed out after the handling logic is done. This mechanism ensures that the handling logic can only succeed once so there is no need to design for idempotency.
Avoiding partial updates
In transaction modes lower than TransactionScope there is a risk of partial updates because one handler might succeed in updating business data while another handler fails. To avoid this configure NServiceBus to wrap all handlers in a TransactionScope
that will act as a unit of work and make sure that there are no partial updates. Use the following code to enable a wrapping scope:
var unitOfWorkSettings = endpointConfiguration.UnitOfWork();
unitOfWorkSettings.WrapHandlersInATransactionScope();
This requires that all the data stores used by the handler support enlisting in a distributed transaction (e.g. SQL Server), including the saga store when using sagas.
This might escalate to a distributed transaction if data in different databases are updated.
This API must not be used in combination with transports running in a transaction scope mode.
Controlling transaction scope options
The isolation level and timeout for transaction scopes are configured at the transport level.