Learning Message Queues & Pub Sub

Introduction

• Simple project I created with references and examples to learn about message queues and publisher/subscriber pattern with different technologies.

To Do

• RabbitMQ Message Queue Example
• RabbitMQ PubSub Example
• Apache Kafka Message Queue Example
• Apache Kafka PubSub Example
• Observer Pattern (GoF) Example
• BullMQ Message Queue Example
• AWS SQS Example using LocalStack
• AWS SNS Example using LocalStack
• AWS S3 Example using LocalStack
• AWS Dynamodb Example using LocalStack

Prerequisites

• Install Bun: https://bun.sh/docs/installation
• Install LocalStack: https://docs.localstack.cloud/getting-started/installation/
• Install Docker & Docker-Compose: https://docs.docker.com/engine/install/

a. Install dependencies & up docker-compose

sudo chmod +x setup.sh && ./setup.sh

b. To see RabbitMQ Management UI, go to: http://localhost:15672

• Username: guest
• Password: guest

c. To see LocalStack UI, go to: https://app.localstack.cloud/inst/default/status

Observer Pattern Example

• Credits: https://youtu.be/ioYkXh8NhKc

1. Run script

bun run observer-pattern

RabbitMQ Message Queue Example

1. Run producer (send message)

bun run rabbitmq-queue-producer

2. Run consumer (consume message)

bun run rabbitmq-queue-consumer

RabbitMQ PubSub Example

1. Run subscriber to stay listening... (receive notifications in subscriber)

bun run rabbitmq-pubsub-subscriber

2. Run publish (publish message to subscribers consume)

bun run rabbitmq-pubsub-publisher

Apache Kafka Message Queue Example

1. Run producer (send messages)

bun run apache-kafka-queue-producer

2. Run consumer (consume messages)

bun run apache-kafka-queue-consumer

Apache Kafka PubSub Example

1. Run publisher (send messages to topic x)

bun run apache-kafka-pubsub-publisher

2. Run subscriber (subscribers consume messages from topic x)

bun run apache-kafka-pubsub-subscriber

BullMQ Message Queue Example

1. Run job

bun run bullmq-mq-job

2. Run worker

bun run bullmq-mq-worker

AWS SQS Example

CLI

• Prerequisites
  • Install AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
  • Verify your AWS Credentials:

   aws --version
   export AWS_ACCESS_KEY_ID="test"
   export AWS_SECRET_ACCESS_KEY="test"
   export AWS_DEFAULT_REGION="us-east-1"
   aws configure --profile localstack
   aws configure list

1. Creating QUEUE

aws sqs create-queue --endpoint-url http://localhost:4566 --queue-name cli-test-queue --profile localstack

2. Listing QUEUEs

aws sqs list-queues --endpoint-url http://localhost:4566 --profile localstack

3. Sending Messages

aws sqs send-message --endpoint-url http://localhost:4566 --queue-url http://localhost:4566/000000000000/cli-test-queue --message-body "Testing Message" --message-attributes file://./src/aws-sqs/message.json --profile localstack

4. Receiving Messages

aws sqs receive-message --endpoint-url http://localhost:4566 --queue-url http://localhost:4566/000000000000/cli-test-queue --attribute-names All --message-attribute-names All  --profile localstack

Code

1. Run example

bun run aws-sqs-example

AWS SNS Example

1. Run example

bun run aws-sns-example

AWS S3 Example

1. Run example

bun run aws-s3-example

AWS Dynamodb Example

1. Run example

bun run aws-dynamodb-example

Images Example

License

MIT

Copyright (c) June 2024-present, Alex Galhardo