Getting Started
Architecture
NServiceBus
Persistence
ServiceInsight
ServicePulse
ServiceControl
Monitoring
Samples

Azure Storage Queues Transport Configuration

Configuration parameters

The Azure Storage Queues Transport can be configured using the following parameters:

ConnectionString

Defaults: none

PeekInterval

The amount of time that the transport waits before polling the input queue, in milliseconds.

Defaults: 125 ms

MaximumWaitTimeWhenIdle

In order to save money on the transaction operations, the transport optimizes wait times according to the expected load. The transport will back off when no messages can be found on the queue. The wait time will be increased linearly, but it will never exceed the value specified here, in milliseconds.

Defaults: 30000 ms (30 seconds)

PurgeOnStartup

Instructs the transport to remove any existing messages from the input queue on startup.

Defaults: false, i.e. messages are not removed when endpoint starts.

MessageInvisibleTime

The visibilitytimeout mechanism, supported by Azure Storage Queues, causes the message to become invisible after read for a specified period of time. If the processing unit fails to delete the message in the specified time, the message will reappear on the queue. Then another process can retry the message.

Defaults: 30,000 ms (i.e. 30 seconds)

BatchSize

The number of messages that the transport tries to pull at once from the storage queue. Depending on the expected load, the value should vary between 1 and 32 (the maximum).

Starting from version 8.1.3 of the transport (and from version 7.5.6 for the 7.x version of the transport), the batch size is dynamically calculated based on the endpoints message processing concurrency limit unless explicitly specified. The batch size is calculated based on the following formula

MaxConcurrencyBatch Size
11
22
33
......
3232 [max]

If the message processing concurrency limit is higher than the maximum batch size the degree of parallelism is dynamically increased, unless explicitly specified, to fulfill the concurrency needs of the endpoint while avoiding over fetching of messages. This is done to decrease the likelihood of message visibility timeouts.

Versions 7.x to 7.5.5 and 8.x to 8.1.2 default to a batch size of 32.

DegreeOfReceiveParallelism

The number of parallel receive operations that the transport is issuing against the storage queue to pull messages out of it.

Versions 8.1.3 and higher of the transport (or versions 7.5.6 and higher for the 7.x version of the transport) calculate the value at start-up based on the endpoints message processing concurrency limit, using the following formula:

MaxConcurrencyDegreeOfReceiveParallelismBatch Size
111
212
313
...1...
10110
...1...
20120
...1...
32132
50232 / 18
100432 / 32 / 32 / 4
200732 / 32 / 32 / 32 / 32 / 32 / 32 / 8
10003131 x 32 / 8

When the DegreeOfReceiveParallelism is explicitly set, the batch size is dynamically adjusted to fulfill the endpoints message processing limits, if possible, up to the allowed maximum of 32. For example, with a maximum concurrency set to 100 and a DegreeOfReceiveParallelism fixed to 3 an underfetching of 4 messages might occur. Therefore it is advised to tweak both DegreeOfReceiveParallelism to be in alignment with the BatchSize if required. In most cases, the dynamic calculation should be sufficient and no adjustment is required.

Versions 7.x to 7.5.5, and 8.x to 8.1.2, dynamically calculate the value based on the endpoints message processing concurrency limit, using the following equation:

Degree of parallelism = square root of MaxConcurrency
MaxConcurrencyDegreeOfReceiveParallelism
11
103
204
507
10010
20014
100032 [max]

This means that DegreeOfReceiveParallelism message processing loops will receive up to the configured BatchSize number of messages in parallel. For example, using a BatchSize of 32 (the default) and parallelism set to 10 will allow the transport to receive 320 messages from the storage queue at the same time.

SerializeMessageWrapperWith

Messages are wrapped in a transport specific structure containing message metadata. By default, Azure Storage Queues Transport uses the same serializer for the message wrapper as configured for the contained message. It is possible to configure a different serializer for the wrapper using the SerializeMessageWrapperWith option

// serialize the messages using the XML serializer:
endpointConfiguration.UseSerialization<XmlSerializer>();
var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>();
// wrap messages in JSON
transport.SerializeMessageWrapperWith<NewtonsoftJsonSerializer>();

Setting configuration parameters

Settings can be overridden only using configuration API:

var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>();
transport.ConnectionString("DefaultEndpointsProtocol=https;AccountName=[ACCOUNT];AccountKey=[KEY];");
transport.BatchSize(20);
transport.MaximumWaitTimeWhenIdle(TimeSpan.FromSeconds(1));
transport.DegreeOfReceiveParallelism(16);
transport.PeekInterval(TimeSpan.FromMilliseconds(100));
transport.MessageInvisibleTime(TimeSpan.FromSeconds(30));

Connection strings

Note that multiple connection string formats apply when working with Azure storage services. When running against the emulated environment the format is UseDevelopmentStorage=true, but when running against a cloud hosted storage account the format is DefaultEndpointsProtocol=https;AccountName=myAccount;AccountKey=myKey;

For more details refer to Configuring Azure Connection Strings document.

Using aliases for connection strings to storage accounts

It is possible to accidentally leak sensitive information in the connection string for a storage account if it is not properly secured. For example, the information can be leaked if an error occurs when communicating across untrusted boundaries, or if the error information is logged to an unsecured log file.

In order to prevent this, only creating an alias for each connection string is allowed. The alias is mapped to the physical connection string, and connection strings are always referred to by their alias. In the event an alias is leaked, the connection strings are not exposed.

This feature can be enabled when configuring the AzureStorageQueueTransport:

var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>();
transport.UseAccountAliasesInsteadOfConnectionStrings();

See also Using aliases instead of connection strings for multi-account support.

Token-credentials

Enables usage of Microsoft Entra Id authentication such as managed identities for Azure resources instead of the shared secret in the connection string.

Use the corresponding QueueServiceClient, BlobServiceClient or TableServiceClient constructor overload when creating the clients passed to the transport.

Sanitization

If a queue name is longer than 63 characters, the Azure Storage Queues Transport can be configured to sanitize queue names using a custom algorithm. By default, the transport does not sanitize queue names.

For more details refer to Sanitization document.

Serialization

Azure Storage Queues Transport changes the default serializer to JSON. The serializer can be changed using the serialization API.

Related Articles

  • Performance Tuning
    Tips on how to get the best performance from the Azure Storage Queues persistence.