Azure Blob Storage Data Bus

Usage

endpointConfiguration.UseDataBus<AzureDataBus, SystemJsonDataBusSerializer>();

Cleanup strategies

Discarding old Azure Data Bus attachments can be done in one of the following ways:

1. Using an Azure Durable Function
2. Using the Blob Lifecycle Management policy

Using an Azure Durable Function

Review the sample to see how to use a durable function to clean up attachments.

Using the Blob Lifecycle Management policy

Attachment blobs can be cleaned up using the Blob Storage Lifecycle feature. This method allows configuring a single policy for all data bus-related blobs. Those blobs can be either deleted or archived. The policy does not require custom code and is deployed directly to the storage account. This feature can only be used on GPv2 and Blob storage accounts, not on GPv1 accounts.

Configuration

Configuring the BlobServiceClient

There are several ways to configure the BlobServiceClient.

Using a preconfigured BlobServiceClient

A fully configured BlobServiceClient can be set through the settings:

var serviceClient = new BlobServiceClient("connectionString");
endpointConfiguration.UseDataBus<AzureDataBus, SystemJsonDataBusSerializer>()
                     .UseBlobServiceClient(serviceClient);

Using a custom provider

A custom provider can be declared that provides a fully configured BlobServiceClient:

public class CustomProvider : IProvideBlobServiceClient
{
    // Leverage dependency injection to use a custom-configured BlobServiceClient
    public CustomProvider(BlobServiceClient serviceClient)
    {
        Client = serviceClient;
    }

    public BlobServiceClient Client { get; }
}

The provider is then registered in the dependency injection container:

endpointConfiguration.UseDataBus<AzureDataBus, SystemJsonDataBusSerializer>();
endpointConfiguration.RegisterComponents(services => services.AddSingleton<IProvideBlobServiceClient, CustomProvider>());

Providing a connection string and container name

endpointConfiguration.UseDataBus<AzureDataBus, SystemJsonDataBusSerializer>()
    .ConnectionString("connectionString")
    .Container("containerName");

Behavior

The following extension methods are available for changing the behavior of AzureDataBus defaults:

var dataBus = endpointConfiguration.UseDataBus<AzureDataBus, SystemJsonDataBusSerializer>();
dataBus.ConnectionString(azureStorageConnectionString);
dataBus.Container(containerName);
dataBus.BasePath(basePathWithinContainer);
dataBus.MaxRetries(maxNumberOfRetryAttempts);
dataBus.NumberOfIOThreads(numberOfIoThreads);
dataBus.BackOffInterval(backOffIntervalBetweenRetriesInSecs);

• ConnectionString(): The connection string to the storage account for storing databus properties; defaults to UseDevelopmentStorage=true.
• Container(): Container name; defaults to databus.
• BasePath(): The blobs' base path in the container; defaults to an empty string.
• MaxRetries: Number of upload/download retries; defaults to 5 retries.
• NumberOfIOThreads: Number of blocks that will be simultaneously uploaded; defaults to 1 thread.
• BackOffInterval: The back-off time between retries; defaults to 30 seconds.