Amazon SQS Transport

Source
NuGet Package NServiceBus.AmazonSQS (6-pre)
This page targets a pre-release version and is subject to change prior to the final release.

Simple Queue Service (SQS) is a message queue service provided by Amazon Web Services.

Transport at a glance

Feature
TransactionsNone, ReceiveOnly (Message visibility timeout)
Pub/SubNative (Requires SNS, supports hybrid-mode for migration purposes)
TimeoutsNative (Requires FIFO Queues)
Large message bodiesNative (Requires S3)
Scale-outCompeting consumer
Scripted DeploymentBuilt-in CLI, C#
InstallersOptional

Advantages

  • Fully managed turn-key messaging infrastructure. SQS queues requires very little effort to set up, maintain and manage over time.
  • Integrates seamlessly with other services provided by AWS, such as IAM, CloudWatch, and Lambda. For organizations already committed to AWS, SQS is a natural choice.
  • Can be used as a gateway between endpoints that may not have direct connectivity to each-other.
  • Can send and receive large messages that exceed the queue limitations by storing large payloads in S3. For more information review the documentation for the transport topology and configuration options.

Disadvantages

  • Like other message brokers, there is no local store-and-forward mechanism available. If an endpoint cannot reach SQS, either due to network problems or if SQS is unavailable, the endpoint will not be able to send nor receive messages.
  • Can be relatively expensive when using larger volumes of messages.

Prerequisites

An AWS IAM account with a pair of Access Keys is required.

The IAM account requires the following permissions:

SQS permissions

  • CreateQueue
  • DeleteMessage
  • DeleteMessageBatch
  • GetQueueUrl
  • ReceiveMessage
  • SendMessage
  • SendMessageBatch
  • SetQueueAttributes
  • GetQueueAttributes
  • ChangeMessageVisibility
  • ChangeMessageVisibilityBatch
  • PurgeQueue

SNS permissions

  • ListTopics
  • Unsubscribe
  • SetEndpointAttributes
  • ListSubscriptions
  • GetSubscriptionAttributes
  • SetSubscriptionAttributes

In addition to the above permissions the queue subscribing to a topic needs sqs:SendMessage permission to enable the topics delivering messages to the subscribing queue.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "SomeSid",
      "Effect": "Allow",
      "Principal": {
        "AWS": "yourPrincipal"
      },
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:yourQueueArn",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:sns:yourTopicArn"
        }
      }
    },
  ]
}

S3 permissions

  • CreateBucket
  • DeleteObject
  • GetObject
  • PutObject
  • PutLifecycleConfiguration
  • GetLifecycleConfiguration
  • ListAllMyBuckets

By default, AWS Access Key ID, AWS Secret Access Key and AWS Region Key are discovered from environment variables of the machine that is running the endpoint:

  • Access Key ID goes in AWS_ACCESS_KEY_ID
  • Secret Access Key goes in AWS_SECRET_ACCESS_KEY
  • Region Key goes in AWS_REGION
// S3 bucket only required for messages larger than 256KB
var transport = new SqsTransport
{
    S3 = new S3Settings("myBucketName", "my/key/prefix")
};

endpointConfiguration.UseTransport(transport);

For more configuration options consult the configuration options page.

Retries and timeouts

The SQS transport uses the default retry and timeout values implemented by the AWS SDK for .NET:

ParameterDefault value
MaxErrorRetries4
RequestTimeout100s
ReadWriteTimeout300s
NServiceBus will perform immediate and delayed retries in addition to retries performed internally by the SQS client.

Batching

Messages sent from within a handler are batched with up to ten messages per batch depending on the size of the message. Messages sent outside a handler are not batched.

Samples

Related Articles

  • Amazon SQS Delayed Delivery
    Describes the native delayed delivery implementation in the SQS transport.
  • Configuration Options
    Configuration options for the SQS transport.
  • Native integration
    Considerations when integrating NServiceBus endpoints with native Amazon SQS publishers and consumers.
  • Performance Tuning
    Guidance to tweak the performance of the SQS transport.
  • SQS Transport Scripting
    Example code and scripts to facilitate deployment and operational actions against the SQS Transport.
  • Topology
    Identify the physical components used by the Amazon SQSL transport and how they interact.
  • Transaction Support
    Explore the transaction modes the transport supports.
  • Troubleshooting
    Tips on what to do when the SQS transport is not behaving as expected.
  • Why Use SQS?
    Explore when it makes sense to use SQS as a transport.

Last modified