Azure Blob Storage Data Bus will remove the Azure storage blobs used for physical attachments after the message is processed if the
TimeToBeReceived value is specified. When this value isn't provided, the physical attachments will not be removed.
Discarding old Azure Data Bus attachments can be done in one of the following ways:
- Using the built-in method (disabled by default)
- Using an Azure Durable Function
- Using the Blob Lifecycle Management policy
Specify a value for the
TimeToBeReceived property. For more details on how to specify this, see Discarding Old Messages.
0. In versions 3 and above built-in cleanup is disabled by default.
Review the sample to see how to use a durable function to clean up attachments.
Attachment blobs can be cleaned up using the Blob Storage Lifecycle feature. This method allows configuring a single policy for all data bus-related blobs. Those blobs can be either deleted or archived. The policy does not require custom code and is deployed directly to the storage account. This feature can only be used on GPv2 and Blob storage accounts, not on GPv1 accounts.
The following extension methods are available for changing the behavior of
var dataBus = endpointConfiguration.UseDataBus<AzureDataBus>(); dataBus.ConnectionString(azureStorageConnectionString); dataBus.Container(containerName); dataBus.BasePath(basePathWithinContainer); dataBus.BlockSize(blockSize); dataBus.DefaultTTL(timeToLiveInSeconds); dataBus.MaxRetries(maxNumberOfRetryAttempts); dataBus.NumberOfIOThreads(numberOfIoThreads); dataBus.BackOffInterval(backOffIntervalBetweenRetriesInSecs); dataBus.CleanupInterval(cleanupIntervalInMilSecs); dataBus.AuthenticateWithManagedIdentity(storageAccountName, renewFiveMinutesBeforeTokenExpires);
ConnectionString(): The connection string to the storage account for storing databus properties; defaults to
Container(): Container name; defaults to
BasePath(): The blobs' base path in the container; defaults to an empty string.
DefaultTTL: Time in seconds to keep a blob in storage before it is removed; defaults to Int64.MaxValue seconds.
MaxRetries: Number of upload/download retries; defaults to 5 retries.
NumberOfIOThreads: Number of blocks that will be simultaneously uploaded; defaults to 1 thread.
BackOffInterval: The back-off time between retries; defaults to 30 seconds.
BlockSize: The size of a single block for upload when the number of I/O threads is more than 1; defaults to 4MB.
CleanupInterval: The default time interval to perform periodic clean-up of blobs for expired messages with specific TTL; defaults to 5 minutes.
AuthenticateWithManagedIdentity(storageAccountName, renewalTimeBeforeTokenExpires, endpointSuffix): Authenticate with Azure Managed Identity instead of connection string.
storageAccountName: The storage account name used for the data bus.
renewalTimeBeforeTokenExpires: How long before the current token expires a token renewal request should be be issued.
endpointSuffix: Endpoint suffix used for the storage account. The default is set to public Azure cloud (
0 will disable blob cleanup.
var dataBus = endpointConfiguration.UseDataBus<AzureDataBus>(); dataBus.CleanupInterval(0);