Application Services

SQS – Simple Queue Service

• SQS is a distributed web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.

• SQS helps decouple the components of an application so they can run independently.

• Messages can be retrieved via SQS API

• The producer and consumer can run at their own independent throughput.

• The queue acts as a buffer between consumer and producer. Ensures delivery of messages at least once. Ensure your application isn’t affected by processing the same message multiple times.

• Allows multiple readers and writers. Single queue can be used simultaneously by various applications – helps scale out applications

• SQS Message size up to 256KB of text in any format. May consist of 1-10 messages.

• Does not guarantee FIFO messages. If order is important, add sequencing information in each message.

• For SQS, you have to pull messages. It doesn’t push messages – unlike SNS. You are billed at 64KB Chunks

Pricing

• First 1 million SQS Requests per month are free.

• $0.50 per 1 million SQS requests per month thereafter.

• 64KB chunk = 1 request. So a message of 256KB = 4 requests.

• Each messages has a visibility timeout – 12 hours by default. Visibility timeout period only starts when a worker node has picked up the message for processing. During this interval, the message is invisible to other processor workers.

• SQS can do auto-scaling. If queue grows beyond a threshold, instantiate new web/app servers. Use Auto scaling + SQS to achieve this.

Exam Tip - De-couple ➔ SQS

SWF – Simple Workflow Service

• SWS is a web service that makes it easy to coordinate work across distributed application components. Co-ordinate tasks & workflows.

• Amazon uses SWS to process orders on its website.

• No EC2 components involved.

• It can also involve human actors.

Trick Question – when to use SQS or SWS

Attribute	SQS	SWS
Retention	14 days	1 year
API	Message Oriented	Task Oriented
Assignment	Might be assigned multiple times	Only once
State	Write code to implement tracking	Keeps Track of State & Events

SWF Actors

1. WF Starters – e-commerce application

2. WF Deciders – Control flow of activity tasks.

3. WF Activity workers – Carry out actual task

SNS – Simple Notification Service

• Makes it easy to setup, operate and send notifications from the cloud.

• Immediate delivery to subscribers or other applications

• SNS consists of Topics and you can publish messages to topics.

• You can send emails, text and other alerts. Apple Push, Android etc.

• Publish messages to SQS queues, trigger Lambda functions. Lambda function can then manipulate information and then send to other SNS Topics

• SNS is Push based messaging.

• You can group multiple recipients using topics. Recipients can subscribe to topics to receive notifications.

• Flexible message delivery over multiple protocols.

• Is used in conjunction with CloudWatch and AutoScaling.

EC2 instances pull SQS messages from a standard SQS queue on a FIFO (First In First out) basis. – False

Elastic Transcoder

• Allows to convert media files from source to different media formats.

• You pay the minutes you transcode and the resolution

• S3 → Lambda Function → E. Transcoder → S3

API Gateway

• Managed web service which enables developers to publish, monitor and secure APIs at any scale.

• Create an API that acts as front door for applications to access data, business logic or any functionality from your backend services

• API Caching – Cache your endpoint’s responses. Reduces load on endpoints based on duration of TTLs

• Low cost & Efficient. Scales

• Throttle requests as required to prevent attacks.

• Log requests to CloudWatch.

• For application built on top of multiple domains, you need to enable CORS on API Gateway.

Amazon Kinesis

• Streaming data is something which is generated by thousands of data sources – stock prices, game information, social network data, geo-spatial data, purchases from online stores, IoT sensor data.

• Kinesis is an AWS platform to analyze streaming data.

• Kinesis Streams

  • Stores data for 24 hours to 7 days.
  • Data stored in shards.
  • Data consumers (EC2 instances) analyze the stream and then derive results/take next actions.
  • Data capacity of stream is a function of the number of shards you specify for the stream.

• Kinesis Firehose

  • Don’t have to worry about shards, streams – completely automated.
  • No automatic data retention window. Data is either immediately analyzed or sent to S3 and then to Redshift, elastic search cluster
  • Data is immediately analyzed via Lambda.

• Kinesis Analytics –

  • Run SQL type queries on top of data contained in Streams or Firehose and store the results in S3 / Redshift and Elastic Search cluster.