Azure Functions with Azure Storage Queues

Source
NuGet Package NServiceBus.AzureFunctions.StorageQueues (0.x) | License
This is a Preview project.
Target NServiceBus Version: 7.x

Host NServiceBus endpoints with Azure Functions.

Basic usage

Endpoint configuration

static readonly FunctionEndpoint endpoint = new FunctionEndpoint(executionContext =>
{
    var storageQueueTriggeredEndpointConfiguration = StorageQueueTriggeredEndpointConfiguration.FromAttributes();

    return storageQueueTriggeredEndpointConfiguration;
});

The endpoint is automatically configured with the endpoint name, the transport connection string, and the logger passed into the function using a static factory method provided by the ServiceBusTriggeredEndpointConfiguration.FromAttributes method.

Alternatively, the endpoint name can be passed in manually:

static readonly FunctionEndpoint endpoint = new FunctionEndpoint(executionContext =>
{
    var storageQueueTriggeredEndpointConfiguration = new StorageQueueTriggeredEndpointConfiguration("ASQTriggerQueue");

    return storageQueueTriggeredEndpointConfiguration;
});

Azure Function definition

[FunctionName("ASQTriggerQueue")]
public static async Task Run(
    [QueueTrigger(queueName: "ASQTriggerQueue")]
    CloudQueueMessage message,
    ILogger logger,
    ExecutionContext executionContext)
{
    await endpoint.Process(message, executionContext, logger);
}

Configuration

License

The license is provided via the NSERVICEBUS_LICENSE environment variable, which is set via the Function settings in the Azure Portal. Use a local.settings.json file for local development. In Azure, specify a Function setting using the environment variable as the key.

{
  "IsEncrypted": false,
  "Values": {
    ...
    "NSERVICEBUS_LICENSE": "<?xml version=\"1.0\" encoding=\"utf-8\"?><license id=\"1222e1d1-2222-4a46-b1c6-943c442ca710\" expiration=\"2113-11-30T00:00:00.0000000\" type=\"Standard\" LicenseType=\"Standard\" LicenseVersion=\"4.0\" MaxMessageThroughputPerSecond=\"Max\" WorkerThreads=\"Max\" AllowedNumberOfWorkerNodes=\"Max\">. . .</license>"
  }
}

Custom diagnostics

NServiceBus startup diagnostics are disabled by default when using Azure Functions. Diagnostics can be written to the logs with the following snippet:

storageQueueTriggeredEndpointConfiguration.LogDiagnostics();

Persistence

The Azure Storage Queues transport requires a persistence for pub/sub and sagas to work.

var persistence = endpointConfiguration.AdvancedConfiguration.UsePersistence<AzureStoragePersistence>();
persistence.ConnectionString("<connection-string>");

Endpoints that do not have sagas and do not require pub/sub can omit persistence registration using the following transport option:

endpointConfiguration.Transport.DisablePublishing();

Error queue

For recoverability to move the continuously failing messages to the error queue rather than the Functions poison queue, the error queue must be created in advance and configured using the following API:

endpointConfiguration.AdvancedConfiguration.SendFailedMessagesTo("error");

Known constraints and limitations

When using Azure Functions with Azure Storage Queues, the following points must be taken into consideration:

  • Endpoints cannot create their own queues or other infrastructure using installers; the infrastructure required by the endpoint to run must be created in advance. For example:
    • Queues for commands
    • Subscription records in storage for events
  • The Configuration API exposes NServiceBus transport configuration options via the configuration.Transport method to allow customization; however, not all of the options will be applicable to execution within Azure Functions.
  • When using the default recoverability or specifying custom number of immediate retries, the number of delivery attempts specified on the underlying queue or Azure Functions host must be greater than the number of the immediate retries. The Azure Functions default is 5 (DequeueCount) for the Azure Storage Queues trigger.

Features dependent upon delayed delivery

The delayed delivery feature of the Azure Storage Queues transport polls for the delayed messages information and must run continuously in the background. With the Azure Functions Consumption plan, this time is limited to the function execution duration with some additional non-deterministic cool off time. Past that time, delayed delivery will not work as expected until another message to process or the Function is kept "warm".

For features that require timely execution of the delayed delivery related features, use one of the following options:

  • Keep the Function warm in the Consumption plan
  • Use an App Service Plan for Functions hosting
  • Use a Premium plan

The following features are supported but are not guaranteed to execute timely on the Consumption plan:

The following features require an explicit opt-in:

var recoverability = endpointConfiguration.AdvancedConfiguration.Recoverability();
recoverability.Delayed(settings =>
{
    settings.NumberOfRetries(numberOfDelayedRetries);
    settings.TimeIncrease(timeIncreaseBetweenDelayedRetries);
});

Preparing the Azure Storage account

Queues must be provisioned manually.

Subscriptions to events are created when the endpoint executes at least once. To ensure the endpoint processes all the events, subscriptions should be created manually.

Samples


Last modified