Case Study: How Industry are getting benefits from AWS SQS

In this article, we are going to discuss a cool new technology of AWS that is Simple Queue Service(SQS). Also, get a glimpse of how industries are getting benefits from this service.

Before jumping into the service let’s talk about AWS.

What is AWS?

AWS is an Amazon Public cloud providing service that provides servers, storage, networking, remote computing, email, mobile development, and security like tonnes of services.

It is a highly reliable, scalable, and secured platform. To know more you can follow this blog.

What is Amazon SQS?

Amazon Simple Queue Service is a fully managed message queuing service. It is used where for a simple product there are distributed applications exchanging messages between one another.

Messaging queue enables the user to separate and scale microservices, distributed systems, and serverless applications.

Using SQS, you can send, store, and receive messages between software components at any volume. It guarantees to not use any other service available and no loss of any of the messages as well.

AWS SQS can start with the help of the tools such as Amazon Console, command-line interface, and SDK. It supports HTTP over SSL (HTTPS) and Transport Layer Security (TLS) protocols for security.

Amazon SQS uses two types of Queues:

• Standard Queue: It doesn’t guarantee the order but offers at least one delivery and maximum throughput.
• FIFO(First In First Out): This takes only one delivery at a time on the basis of the time it arrives i. e. the element that comes first will always be processed first. It’s a very good service of AWS but costlier.

How AWS SQS works

Distributed Queues:

There are mainly three parts in a distributed messaging system: the component of the distributed system(distributed on Amazon SQS server), your queue, and the message in the queue.

Message lifecycle:

The following scenario describes the lifecycle of an Amazon SQS message in a queue, from creation to deletion:

There are some more following techniques are used to complete the whole SQS service:

• Visibility timeout: The time for which one message becomes invisible to all the resources, this can be used when a process is already started and no other can again start that.
• Dead letter queues: The queues that can target messages that can’t be processed by the source queue.
• Delay Queue: Delay queues let you postpone the delivery of new messages to a queue for a number of seconds, this works when a consumer application requires additional time to process a message.
• Short and long polling: Amazon SQS provides short polling and long polling to receive messages from a queue. By default, queues use short polling.
• Temporary Queues: Temporary queues help you save development time and deployment costs when using common message patterns such as request-response.

Benefits of Amazon SQS

• Security: You can control who can send messages to and receive messages from an Amazon SQS queue.
• Durability — For the safety of your messages, Amazon SQS stores them on multiple servers. Standard queues support at-least-once message delivery, and FIFO queues support exactly-once message processing.
• Availability — Amazon SQS uses redundant infrastructure to provide highly-concurrent access to messages and high availability for producing and consuming messages.

• Scalability — Amazon SQS can process each buffered request independently, scaling transparently to handle any load increases or spikes without any provisioning instructions.
• Reliability — Amazon SQS locks your messages during processing so that multiple producers can send and multiple consumers can receive messages at the same time.
• Customization — Your queues don’t have to be exactly alike — for example, you can set a default delay on a queue. You can store the contents of messages larger than 256 KB using Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB, with Amazon SQS holding a pointer to the Amazon S3 object, or you can split a large message into smaller messages.

Oyster Case Study

New York-based Oyster.com shares unvarnished reviews of hotels in nearly 200 destinations worldwide. The company’s own investigators visit each location to assess cleanliness, amenities, service, and overall quality. What sets Oyster apart from similar sites is its extensive collection of photographs. Oyster takes hundreds of photos for each property, and every review includes dozens of untouched images (submitted by guests as well as investigators) that allow potential visitors to compare a hotel’s marketing material with reality.

The Challenge

Since its 2009 launch, Oyster has published more than one million high-quality digital images. When this massive volume of images became too hard to handle in-house, the company decided to offload the content to a central repository on Amazon Simple Storage Service (Amazon S3).

Oyster reprocesses its entire collection of photographic images a few times each year to update the copyright year and, if necessary, to change the watermarks. Using their previous solution, reprocessing the entire collection of photographs required about 800 hours to complete. In addition, Oyster often recreated existing images in new formats and sizes for mobile and tablet devices. Resizing existing images and adding new ones was slowing down the rate at which the company was able to process the collection.

Moreover, there were numerous software bugs in the multiprocessing solution that the company used, but since the solution didn’t scale, Oyster didn’t bother to fix them.

Why Amazon Web Services

While the company is still running one local server, the bulk of the processing work now takes place on the AWS Cloud. Oyster is using a customized Amazon Linux AMI within Amazon EC2. Within this new environment, the company connects to Amazon S3 and Amazon Simple Queue Service (Amazon SQS) using boto, a Python interface to AWS. The images themselves are processed with the ImageMagick software available in the AMI package.

Oyster uses Amazon EC2 instances and Amazon SQS in an integrated workflow to generate the sizes they need for each photo. The team processes a few thousand photos each night, using Amazon EC2 Spot Instances. When Oyster processes the entire collection, it can use up to 100 Amazon EC2 instances. The team uses Amazon SQS to communicate the photos that need to be processed and the status of the jobs.

The Benefits

Oyster’s old system needed approximately 400 hours to process one million photos. By using AWS, the company can process the same number of photos in about 20 hours — a 95 percent improvement.

Oyster has also been able to reduce in-house hardware expenses by repurposing two of its old servers, which were sitting idle more than 80 percent of the time.

AWS Services used

• Amazon EC2
• Amazon SQS
• Amazon Linux AMI

Conclusion

We come to know about the Amazon SQS, how it works and how industries are getting benefits from this.

If you like this don’t forget to clap below. Your small effort gives us a lot of motivation.

I have a list of related articles, in case you like to follow this list — Cloud Computing.

Thanks for reading!!!