Use SNS instead of SQS for communication between Producer and Consumer

[blauerberg]

Issue #, if available:

Description of changes:

The Lambda handler was short-polling SQS, which resulted in a very large number of API requests to SQS.

This activity was consuming a significant portion of my free monthly tier of SQS. I fixed this issue by using SNS instead of SQS.

This change may result in reduced message ordering accuracy (unless we use FIFO queues instead of standard ones).
Since the events are not triggered frequently, this may not pose any significant issues. The solution has been working well in my environment.

This repository solution has been very helpful in reducing costs. Thanks!

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[vsr2158]

Thanks for brining this up. I can see this can be an issue in account where there are lot of SQS polling going on likely not in management account. Have you encountered this issue or this is based on an optimisation you wanted to suggest?

Dont want to make this switch without doing a through check hence asking.

[blauerberg]

@vsr2158

Thank you for checking.

Yes, this issue actually occurred on my sandbox account under the AWS Control Tower.

As you pointed out, I believe it's the consumer account performing the polling, not the management account.

[vsr2158]

The consumer Lambda runs in the Management account. I meant to say SQS polling issue should NOT be encountered in Management account because there is likely not much SQS actiovity going on. Please let me know if otherwise and share (cleaned) logs so I can look deeper.

[blauerberg]

@vsr2158

Apologies for the confusion.

As you mentioned, the Consumer Lambda runs in the management account.

In my case, the issue seemed to occur while other accounts has no activity.

However, my assumption was based solely on the SQS Free Tier metrics visible in the management account; I have not yet isolated via logs which account performed the polling.

I will revert code and investigate the logs further when I get a chance.

Thanks.