← All notes

AWS - Cloud Integrations

March 30, 2025

Introduction

  • When we start deploying multiple applications, they will inevitably need to communicate with one another

  • There are two patterns of applications communication

  • Synchronous between applications can be problematic if there are sudden spikes of traffic

  • What if you need to suddenly encode 1000 videos but usually it's 10?

  • In that case, it's better to decouple your applications:

    • Using SQS: queue model
    • Using SNS: pub/sub model
    • Using Kinesis: real-time data streaming model

  • These services can scale independently from our application

  • SQS: Queue service in AWS

  • SNS: Notification service in AWS

Amazon SQS - Standard Queue

  • Oldest AWS offering (over 10 years old)
  • Fully managed service (~serverless), use to decouple applications
  • Scales from 1 message per second to 10.000s per second
  • Default retention of messages: 4 days, maximum of 14 days
  • No limit to how many messages can be un the queue
  • Messages are deleted after they're read by consumers
  • Low latency (<10 ms on publish and receive)
  • Consumers share the work to read messages & scale horizontally

Amazon SQS - FIFO Queue

  • FIFO -> First In First Out (ordering of messages in the queue)
  • Messages are processed in order by the consumer

Amazon Kinesis Data Streams

  • For the exam: Kinesis = real-time big data streaming
  • Managed service to collect, process, and analyze real-time streaming data at any scale
  • Too detailed for the Cloud Practitioner exam but good to know:
    • Amazon kinesis Data Streams: Low latency streaming to ingest data at scale from hundreds of thousands of sources
    • Amazon Data Firehose: Load kinesis Data Streams into Amazon S3, Redshift, OpenSearch, etc...

Amazon SNS

  • What if you want to send one message to many receivers?
  • The "event publishers" only sends message to one SNS topic
  • As many "event subscribers" as we want to listen to the SNS topic notifications
  • Each subscriber to the topic will get all the messages
  • Up to 12.500.000 subscriptions per topic, 100.000 topics limit

Amazon MQ

  • SQS, SNS are "cloud-native" services: proprietary protocols from AWS
  • Traditional applications running from on-repmises may use open protocols such as: MQTT, AMQP, STOMP, Openwire, WSS
  • When migrating to the cloud, instead of re-engineering the application to use SQS, and SNS, we can use Amazon MQ
  • Amazon MQ is a managed Message broker service for:
    • Rabbit MQ
    • ActiveMQ
  • Amazon MQ doesn't "scale" as much as SQS / SNS
  • Amazon MQ runs on servers, can run in Multi-AZ with failover
  • Amazon MQ has both queue feature (~SQS) and topic features (~SNS)