The Azure Cosmos DB persister uses the Azure Cosmos DB NoSQL database service for storage.
It is important to read and understand partitioning in Azure Cosmos DB before using NServiceBus..
Persistence at a glance
For a description of each feature, see the persistence at a glance legend.
| Feature | |
|---|---|
| Supported storage types | Sagas, Outbox |
| Transactions | Using TransactionalBatch, with caveats |
| Concurrency control | Optimistic concurrency, optional pessimistic concurrency |
| Scripted deployment | Not supported |
| Installers | Container is created by installers. |
The Outbox feature requires partition planning.
Usage
Add a NuGet package reference to NServiceBus.. Configure the endpoint to use the persister through the following configuration API:
endpointConfiguration.UsePersistence<CosmosPersistence>()
.CosmosClient(new CosmosClient("ConnectionString"));
Token credentials
Using a TokenCredential enables the usage of Microsoft Entra ID authentication such as managed identities for Azure resources instead of the requiring a shared secret in the connection string.
A TokenCredential can be provided by using the corresponding CosmosClient constructor overload when creating the client passed to the persister.
Customizing the database used
By default, the persister will store records in a database named NServiceBus. This can be overwritten by using the following configuration API:
endpointConfiguration.UsePersistence<CosmosPersistence>()
.CosmosClient(new CosmosClient("ConnectionString"))
.DatabaseName("DatabaseName");
Customizing the container used
The container that is used by default for all incoming messages is specified via the DefaultContainer(. configuration API:
endpointConfiguration.UsePersistence<CosmosPersistence>()
.CosmosClient(new CosmosClient("ConnectionString"))
.DefaultContainer(
containerName: "ContainerName",
partitionKeyPath: "/partition/key/path");
Added in version 3.2.0: If the container information is being extracted during runtime from a message instance (header or message body), the default container specified will be overwritten by the last extractor in the pipeline. For example, if a Header Extractor (physical stage) and a Message Extractor (logical stage) are both configured, then the container information within the Message Extractor would be used.
Added in version 3.2.0: The exception to the above is if the extractor fails to pull container information from the message (logical), then the container information in the header (physical) will be used. If this also fails, the configured default container will be used as a fallback. If there is no default container configured, an exception will be thrown.
When installers are enabled, this (default) container will be created if it doesn't exist. To opt-out of creating the default container, either disable the installers or use DisableContainerCreation():
endpointConfiguration.UsePersistence<CosmosPersistence>()
.CosmosClient(new CosmosClient("ConnectionString"))
.DefaultContainer(
containerName: "ContainerName",
partitionKeyPath: "/partition/key/path")
.DisableContainerCreation();
Any other containers that are resolved by extracting partition information from incoming messages need to be manually created in Azure.
The transactions documentation details additional options on how to configure NServiceBus to specify the container using the incoming message headers or contents.
Customizing the CosmosClient provider
When the CosmosClient is configured and used via dependency injection, a custom provider can be implemented:
class CustomCosmosClientProvider
: IProvideCosmosClient
{
// get fully configured via DI
public CustomCosmosClientProvider(CosmosClient cosmosClient)
{
Client = cosmosClient;
}
public CosmosClient Client { get; }
}
and registered on the container:
endpointConfiguration.RegisterComponents(c => c.AddTransient<IProvideCosmosClient, CustomCosmosClientProvider>());
Provisioned throughput rate-limiting
When using provisioned throughput, it is possible for the CosmosDB service to rate-limit usage, resulting in "request rate too large" exceptions indicated by a 429 status code.
When using the Cosmos DB persister with the outbox enabled, "request rate too large" errors may result in handler re-execution and/or duplicate message dispatches depending on which operation is throttled.
Microsoft provides guidance on how to diagnose and troubleshoot request rate too large exceptions.
The Cosmos DB SDK provides a mechanism to automatically retry collection operations when rate-limiting occurs. Besides changing the provisioned RUs or switching to the serverless tier, those settings can be adjusted to help prevent messages from failing during spikes in message volume.
These settings may be set when initializing the CosmosClient via the CosmosClientOptions MaxRetryAttemptsOnRateLimitedRequests and MaxRetryWaitTimeOnRateLimitedRequests properties:
endpointConfiguration.UsePersistence<CosmosPersistence>()
.CosmosClient(new CosmosClient("ConnectionString", new CosmosClientOptions
{
MaxRetryWaitTimeOnRateLimitedRequests = TimeSpan.FromSeconds(30),
MaxRetryAttemptsOnRateLimitedRequests = 9
}));
They may also be set when using a CosmosClientBuilder via the WithThrottlingRetryOptions method:
var cosmosClientBuilder = new CosmosClientBuilder("ConnectionString")
.WithThrottlingRetryOptions(
maxRetryWaitTimeOnThrottledRequests: TimeSpan.FromSeconds(30),
maxRetryAttemptsOnThrottledRequests: 9
);
endpointConfiguration.UsePersistence<CosmosPersistence>()
.CosmosClient(cosmosClientBuilder.Build());
Transactions
The Cosmos DB persister supports using the Cosmos DB transactional batch API. However, Cosmos DB only allows operations to be batched if all operations are performed within the same logical partition key. This is due to the distributed nature of the Cosmos DB service, which does not support distributed transactions.
The transactions documentation provides additional details on how to configure NServiceBus to resolve the incoming message to a specific partition key to take advantage of this Cosmos DB feature.
Outbox
Storage format
Version 3.2.0 and over
A default synthetic partition key will be used for all incoming messages, in the format {endpointName}-{messageId}, if not explicitly overwritten at runtime.
The default partition key should be overwritten whenever the message handler creates or updates business records in CosmosDB. To guarantee atomicity, explicitly set the Outbox partition key to match the partition key of your business record. This is a requirement for including both the business record and the Outbox record in the same Cosmos DB transactional batch. Conversely, for simplicity, you can use the default partition key when a handler's logic does not involve persisting business data.
To support backward compatibility of control messages during migration, the persistence includes a fallback mechanism. When enabled (default), and if a record is not found using the synthetic key format, the system falls back to the legacy {messageId} format. Since the fallback mechanism involves an additional read operation on the Outbox container, it is recommended to turn it off once all legacy records have expired.
endpointConfiguration
.EnableOutbox()
.DisableReadFallback();
Since message identities are not unique across endpoints from a processing perspective, when overwriting the default synthetic key, either separate different endpoints into different containers or override the default synthetic partition key in a way that ensures message identities are unique to each processing endpoint.
Version 3.1.0 and under
The below limitation has been resolved in NServiceBus.. Its recommended that customers update to 3.2 if affected.
For control messages, a default partition key in the format {messageId} will be used, however these Outbox records are not separated by endpoint name. As a result, multiple logical endpoints cannot share the same database and container since message identities are not unique across endpoints from a processing perspective. To avoid conflicts, either separate different endpoints into different containers, override the partition key, or update to NServiceBus.Persistence.CosmosDB 3.2.
Outbox cleanup
When the outbox is enabled, the deduplication data is kept for seven days by default. To customize this time frame, use the following API:
var outbox = endpointConfiguration.EnableOutbox();
outbox.TimeToKeepOutboxDeduplicationData(TimeSpan.FromDays(7));
Outbox cleanup depends on the Cosmos DB time-to-live feature. Failure to remove the expired outbox records is caused by a misconfigured collection that has time-to-live disabled. Refer to the Cosmos DB documentation to configure the collection correctly.