Azure Storage Queues Sanitization

Component: Azure Storage Queues Transport
NuGet Package NServiceBus.Azure.Transports.WindowsAzureStorageQueues (8-pre)
Target NServiceBus Version: 7.x
This page targets a pre-release version and is subject to change prior to the final release.

In Versions 8.0 and above, the Azure Storage Queues transport no longer sanitizes queues by default.

Azure Storage Queues naming rules

  1. A queue name must start with a letter or number, and can only contain letters, numbers, and the dash (-) character.
  2. The first and last letters in the queue name must be alphanumeric.
  3. The dash (-) character cannot be the first or last character.
  4. Consecutive dash characters are not permitted in the queue name.
  5. All letters in a queue name must be lowercase.
  6. A queue name must be between 3 and 63 characters long.

Custom sanitization

To sanitize queue names, a sanitization function containing the required logic can be registered.

var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>();
transport.SanitizeQueueNamesWith(queueName => queueName.Replace('.', '-'));

When an endpoint is started, the sanitizer function will be invoked for each queue the transport creates.

Backward compatibility with versions 7 and below

To remain backward compatible with endpoints versions 7 and below, endpoints version 8 and above should be configured to perform sanitization based on version 7 and below rules. The following custom code will ensure queues are sanitized in a backwards compatible manner.

var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>();
transport.SanitizeQueueNamesWith(BackwardsCompatibleQueueNameSanitizer.WithMd5Shortener);

Sanitization code for MD5 and SHA1:

public static class BackwardsCompatibleQueueNameSanitizer
{
    public static string WithMd5Shortener(string queueName)
    {
        return Sanitize(queueName, useMd5Hashing: true);
    }

    public static string WithSha1Shortener(string queueName)
    {
        return Sanitize(queueName, useMd5Hashing: false);
    }

    static string Sanitize(string queueName, bool useMd5Hashing = true)
    {
        var queueNameInLowerCase = queueName.ToLowerInvariant();
        return ShortenQueueNameIfNecessary(SanitizeQueueName(queueNameInLowerCase), useMd5Hashing);
    }

    static string ShortenQueueNameIfNecessary(string sanitizedQueueName, bool useMd5Hashing)
    {
        if (sanitizedQueueName.Length <= 63)
        {
            return sanitizedQueueName;
        }

        var shortenedName = useMd5Hashing ? ShortenWithMd5(sanitizedQueueName) : ShortenWithSha1(sanitizedQueueName);

        return $"{sanitizedQueueName.Substring(0, 63 - shortenedName.Length - 1).Trim('-')}-{shortenedName}";
    }

    static string SanitizeQueueName(string queueName)
    {
        // this can lead to multiple '-' occurrences in a row
        var sanitized = invalidCharacters.Replace(queueName, "-");
        return multipleDashes.Replace(sanitized, "-");
    }

    static string ShortenWithMd5(string test)
    {
        //use MD5 hash to get a 16-byte hash of the string
        using (var provider = MD5.Create())
        {
            var inputBytes = Encoding.Default.GetBytes(test);
            var hashBytes = provider.ComputeHash(inputBytes);
            //generate a GUID from the hash:
            return new Guid(hashBytes).ToString();
        }
    }

    static string ShortenWithSha1(string queueName)
    {
        using (var provider = SHA1.Create())
        {
            var inputBytes = Encoding.Default.GetBytes(queueName);
            var hashBytes = provider.ComputeHash(inputBytes);

            return ToChars(hashBytes);
        }
    }

    static string ToChars(byte[] hashBytes)
    {
        var chars = new char[hashBytes.Length * 2];
        for (var i = 0; i < chars.Length; i += 2)
        {
            var byteIndex = i / 2;
            chars[i] = HexToChar((byte)(hashBytes[byteIndex] >> 4));
            chars[i + 1] = HexToChar(hashBytes[byteIndex]);
        }

        return new string(chars);
    }

    static char HexToChar(byte a)
    {
        a &= 15;
        return a > 9 ? (char)(a - 10 + 97) : (char)(a + 48);
    }

    static Regex invalidCharacters = new Regex(@"[^a-z0-9\-]", RegexOptions.Compiled);
    static Regex multipleDashes = new Regex(@"\-+", RegexOptions.Compiled);
}

Future consideration prior to using sanitization

When implementing custom sanitization, consider factors such as readability and discover-ability. Things to consider:

  • Truncated long queue names could conflict.
  • Hashed queue names could lead to difficult names to use during production troubleshooting or debugging.
  • Sanitized queue names stay in the system and cannot be replaced until no longer used.

Possible way to avoid sanitization is to define endpoint name short and meaningful.

Samples

Related Articles


Last modified