- Fully managed turn-key messaging infrastructure. SQS queues requires very little effort to set up, maintain and manage over time.
- Integrates seamlessly with other services provided by AWS, for example, IAM, CloudWatch, Lambda, etc. For organizations already committed to AWS, SQS is a natural choice.
- Can be used as a gateway between endpoints that may not have direct connectivity to each-other.
- Like other message brokers, there is no local store-and-forward mechanism available. If an endpoint cannot reach SQS, either due to network problems or if SQS is unavailable, the endpoint will not be able to send nor receive messages.
- Can be relatively expensive when using larger volumes of messages.
The IAM account requires the following:
By default, AWS Access Key ID and AWS Secret Access Key are discovered from environment variables of the machine that is running the endpoint:
- Access Key ID goes in
- Secret Access Key goes in
var transport = endpointConfiguration.UseTransport<SqsTransport>(); transport.Region("ap-southeast-2"); // S3 bucket only required for messages larger than 256KB transport.S3BucketForLargeMessages("myBucketName", "my/key/prefix");
Amazon SQS can handle large, continuous throughput but if there are sudden spikes, the service may apply throttling.
When throttling happens, the following exception is logged:
2017-11-14 23:10:24,314|ERROR|18|NServiceBus.Transports.SQS.MessageDispatcher|Exception from Send. Amazon.SQS.AmazonSQSException: Request is throttled. ---> Amazon.Runtime.Internal.HttpErrorResponseException: The remote server returned an error: (403) Forbidden. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
Throttling is more likely to happen when sending a large number messages concurrently. For example, using a list of tasks when using async/await.
To avoid Amazon throttling errors, limit the maximum number of concurrent sends. For example, allow only a small amount of messages to be sent concurrently as outlined in the sending large amount of messages guidelines or send messages sequentially.
Throttling can happen during any send or receive operation and can happen during the following scenarios:
- Incoming message (receiving)
- Sending from within a handler
- Sending outside of a handler
For incoming messages throttling errors can be safely ignored as the message pump will try to fetch the next available message again.
Failing message sends raise an exception when throttled. The exception will be handled by the recoverability feature mechanism. An incoming message that continuously fails due to throttling errors will be moved to the error queue.
A throttling error could result in partial message delivery while the incoming message is not processed successfully and can occur regardless of using the default batched message dispatch or when using immediate dispatch.
Throttling errors are similar to any other technical error that can occur.
As message sending does not happen within a handler context any failures during sending will not rely or be covered by the recoverability feature mechanism. Any retry logic must be manually implemented.
When throttling occurs with no custom error logic implemented, one or more messages might not have been transmitted to Amazon SQS. The custom retry logic could either retry all messages to be sent again, including already succeeded messages or only retry individual messages that failed.