Amazon SQS Transport

Simple Queue Service (SQS) is a message queue service provided by Amazon Web Services.

Transport at a glance

Advantages

• Fully managed turn-key messaging infrastructure. SQS queues requires little effort to set up, maintain, and manage over time.
• Integrates seamlessly with other services provided by AWS, such as IAM, CloudWatch, and Lambda. For organizations already committed to AWS, SQS is a natural choice.
• Can be used as a gateway between endpoints that may not have direct connectivity to each other.
• Can send and receive large messages that exceed the queue limitations by storing large payloads in S3. For more information review the documentation for the transport topology and configuration options.

Disadvantages

• Like other message brokers, there is no local store-and-forward mechanism available. If an endpoint cannot reach SQS, either due to network problems or if SQS is unavailable, the endpoint will not be able to send nor receive messages.
• Can be expensive with large volumes of messages.

Prerequisites

An AWS IAM account with a pair of Access Keys is required.

The IAM account requires the following permissions to provision infrastructure and run the transport:

SQS permissions

• CreateQueue
• DeleteMessage
• DeleteMessageBatch
• GetQueueUrl
• ReceiveMessage
• SendMessage
• SendMessageBatch
• SetQueueAttributes
• GetQueueAttributes
• ChangeMessageVisibility
• ChangeMessageVisibilityBatch
• PurgeQueue

SNS permissions

• CreateTopic
• ListTopics
• GetTopicAttributes
• SetTopicAttributes
• SetEndpointAttributes
• Publish
• Subscribe
• Unsubscribe
• ListSubscriptions
• ListSubscriptionsByTopic
• GetSubscriptionAttributes
• SetSubscriptionAttributes

In addition to the above permissions the queue subscribing to a topic needs sqs:SendMessage permission to enable the topics delivering messages to the subscribing queue.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SomeSid",
      "Effect": "Allow",
      "Principal": {
        "AWS": "yourPrincipal"
      },
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:yourQueueArn",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:sns:yourTopicArn"
        }
      }
    },
  ]
}

S3 permissions

• CreateBucket
• DeleteObject
• GetObject
• PutObject
• PutLifecycleConfiguration
• GetLifecycleConfiguration
• ListAllMyBuckets

Other permissions

• If using server-side encryption of SQS queues, all NServiceBus endpoints (as well as ServiceControl) will require the kms:GenerateDataKey permission in order to support key management.

Configuration

By default, AWS Access Key ID, AWS Secret Access Key and AWS Region Key are discovered from environment variables of the machine that is running the endpoint:

• Access Key ID goes in AWS_ACCESS_KEY_ID
• Secret Access Key goes in AWS_SECRET_ACCESS_KEY
• Region Key goes in AWS_REGION

// S3 bucket only required for messages larger than 256KB
var transport = new SqsTransport
{
    S3 = new S3Settings("myBucketName", "my/key/prefix")
};

endpointConfiguration.UseTransport(transport);

For more configuration options consult the configuration options page.

Retries and timeouts

The SQS transport uses the default retry and timeout values implemented by the AWS SDK for .NET:

Batching

Messages sent from within a handler are batched with up to ten messages per batch depending on the size of the message. Messages sent outside a handler are not batched.