Debug AWS S3/SQS Event Notifications using AWS CLI

Requirement:

  1. S3 bucket “my-s3”
  2. SQS queue “my-sqs”
  3. SQS Event to be triggered if file with suffix _SUCCESS is created under /foo/ folder (or subfolders under /foo/)

Things to watch out for in the Terraform S3 settings:

  1. events is set to ["s3:ObjectCreated:*"]
  2. filter_prefix should be set to foo/ and not /foo/ (There is no leading / for foo)

Firing Test Event

  1. Get the SQS URL
  2. Setup AWS CLI
  3. Create a dummy file named _SUCCESS in the local folder
  4. Start SQS listener using AWS CLI
  5. Update S3 bucket using AWS CLI
$ aws s3 cp _SUCCESS s3://<my-s3-bucket>/foo/bar/_SUCCESS && \
  aws --region <aws-region> sqs receive-message --queue-url <sqs-queue-url> --attribute-names All --message-attribute-names All --max-number-of-messages 1 --wait-time-seconds 20

Published
Categorized as AWS

By Shanker Balan

Shanker Balan is a devops and infrastructure freelancer with over 14 years of industry experience in large scale Internet systems. He is available for both short term and long term projects on contract. Please use the Contact Form for any enquiry.

Leave a comment

Your email address will not be published. Required fields are marked *