I have been working on Amazon Simple Queue Service (SQS) since last week. This is the first time that I play with SQS and it disappoints me in several ways. Maybe I’m just not familiar enough with this product, but if someone could point out my misunderstandings of SQS, I would be absolutely appreciative instead of being ashamed for my ignorance.
Before last week, I naively assumed that, as the name of Message Queue indicates, SQS must have some kind of asynchronous API to notify client that new message has come. Unfortunately it doesn’t, which leaves me no choice but to start a thread to poll the queue, which is really clumsy since a short interval will cause unnecessary bandwidth and CPU waste whereas a long interval makes my code less responsive.
I guess this behaviour is due to SQS uses HTTP as its fundamental protocol, which doesn’t have a way to push data from server to client.
What really surprised me is that I found SQS is not as real-time as I thought. I wrote a test case for Sending and Receiving SQS message. Basically, it sends a message and then calls the receiving API to check if the message is successfully sent. Sometimes the green bulb of nunit shined happily. But most of the time, the receive operation got nothing and the assertion miserably failed, which means the message doesn’t appear in the queue immediately after sending is done and I need to poll the queue for a while to check if it’s really there.
The high latency also makes cleaning up the queue awkward. As SQS doesn’t have any API for emptying the queue, for each test case, a new queue with a random name is created in setup and destroyed in teardown, which prevents messages from the last test interfere future tests. This works, but creating a queue in SQS is really a time consuming operation. It makes running the test suite too slow to be practical.
Now what I do is polling the queue for new messages and deleting them in teardown. It seems to solve the problem and is way faster than creating and deleting queues but I’m not quite sure if the messages are really deleted, as occasionally my test suite still fails due to messages sent by the last execution of test.