AWS re:Invent SVC 105: AWS Messaging

What people said

Recommends John Rotenstein · 2017-01 · Stack Overflow · on “AWS SQS queue improve performance”

Having messages stored in a queue is good - in fact, that's the purpose of using a queue. If your backend systems cannot consume messages at the rate that they are produced, the queue will act as a buffer to retain the messages until they can be processed. A good example is this AWS re:Invent presentation where a queue is shown with more than 200 million messages: Building Elastic, High-Performance Systems with Amazon SQS and Amazon SNS [this video] If it is important to process the messages quickly, then scale your consumers to match the rate of message production (or faster, so you can consume backlog). You mention that your process "picks up the JSON message from the SQS queue, deletes the mes

Recommends Dimos · 2017-02 · Stack Overflow · on “Point to point messaging in scalabale application?”

``Probably once message is read from receiver it can put it in separate queue and batch job can save it to DB from there``, yes that's right, one more queue will be needed. I would advise not to use queues that store messages only in-memory. Most of the queues now persist messages in disk as well. The superiority to databases comes thanks to the access patterns (not memory vs disk). Better scalability is mainly because queues can be distributed much easier than databases and they are much more elastic. Check [this video] to see what I'm talking about.

Recommends John Rotenstein · 2018-06 · Stack Overflow · on “Best way to copy messages within 2 SQS queues across AWS accounts”

Permissions are a secondary matter. The primary question is how to "copy messages from one SQS queue to the other across AWS accounts". Amazon SQS very recently introduced the ability to trigger AWS Lambda functions when a message arrives. You could create a Lambda function that then creates the message in another queue (which could be in a different account, region, etc). However, the original message will be deleted once it is processed , so it isn't really "copying" the message. The better method would be: Instead of sending a message to an Amazon SQS message, send it to an Amazon SNS topic You can then subscribe the Amazon SQS queues to the topic -- yes, you can subscribe multiple queues

Recommends John Rotenstein · 2019-07 · Stack Overflow · on “How can I know if SQS has a new message?”

The answer you linked is accurate. You must write a program that polls SQS for a message (or up to 10 messages). It is more efficient to use long polling because you require less calls. If you wish to know about a message very quickly, then you will need to poll continually. That is, as soon as it comes back and says "nothing to receive", you should call it again. To reduce the frequency of these calls, you can set long polling , up to a maximum of 20 seconds. This means that, if there are no messages in the queue, the ReceiveMessages() option will take 20 seconds before it returns a response of "no messages". If, however, a message arrives in the meantime, it will respond immediately. The l

Cites Hitul Mistry · 2013-01 · Stack Overflow · on “Amazon SQS(Simple Queuing Service) Speed?”

Recently at AWS:reInvent the Messaging Team did a demo of SQS throughput using EC2 and SQS. And SQS was able to achieve tens of thousands of messages a second on a single EC2 machine. The video is available here: [this video] If you are using EC2 micro or small instances and a single threaded application, you won't be able to get those kinds of numbers. We have customers who are regularly making more requests than the numbers show in the demo app. The speed of SQS mostly depends on how scalable you write your senders and receivers. Hope this helps.