Skip to content

Commit 8307feb

Browse files
author
Ben Smith
authored
Merge pull request #1420 from raffaeu/raffaeu-feature-sns-sqs-message-router-cdk
Raffaeu feature sns sqs message router cdk
2 parents 9273614 + 2e380f1 commit 8307feb

File tree

12 files changed

+434
-0
lines changed

12 files changed

+434
-0
lines changed

‎sns-sqs-message-content-router-cdk/README.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Amazon SNS to Amazon SQS by filtering content (Message-content router pattern)
2+
3+
This pattern creates an SNS topic and various SQS destination queues. Messages received by the SNS topic are routed to various SQS queues according to routing logic. The pattern is an implementation of the Integration pattern: "Content-Based Message route" available here: [https://www.enterpriseintegrationpatterns.com/patterns/messaging/ContentBasedRouter.html](https://www.enterpriseintegrationpatterns.com/patterns/messaging/ContentBasedRouter.html)
4+
5+
Learn more about this pattern at Serverless Land Patterns: [https://serverlessland.com/patterns/sns-sqs-message-content-router-cdk](https://serverlessland.com/patterns/sns-sqs-message-content-router-cdk)
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html) (AWS SAMCDK) installed
15+
16+
## Deployment Instructions
17+
18+
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
19+
```
20+
git clone https://github.com/aws-samples/serverless-patterns
21+
```
22+
1. Change directory to the pattern directory:
23+
```
24+
cd sns-sqs-message-content-router-cdk/src
25+
```
26+
1. From the command line, use AWS CDK to deploy the Stack:
27+
```
28+
cdk deploy
29+
```
30+
1. Note the outputs from the CDK deployment process. These contain the resource names and/or ARNs which are used for testing.
31+
32+
## How it works
33+
34+
The CDK Stack pick-up a configuration from `stackconfiguration.json` stored inside the `src` folder. This files contains a JSON structure that describe the routing logic. The stack will also create a DeadQueue letter for each Queue.
35+
36+
The parameter `IsAttribute` specifies to CDK if the filter is of type attribute or content.
37+
38+
```json
39+
{
40+
"Sns": {
41+
"TopicName": "content-router"
42+
},
43+
"Sqs": [
44+
{
45+
"QueueName": "country-usa",
46+
"IsAttribute": true,
47+
"Filter": {
48+
"FilterName": "country",
49+
"FilterValues": [
50+
"united states of america",
51+
"usa"
52+
]
53+
}
54+
},
55+
{
56+
"QueueName": "country-germany",
57+
"IsAttribute": true,
58+
"Filter": {
59+
"FilterName": "country",
60+
"FilterValues": [
61+
"germany",
62+
"de"
63+
]
64+
}
65+
},
66+
{
67+
"QueueName": "language-english",
68+
"IsAttribute": false,
69+
"Filter": {
70+
"FilterName": "language",
71+
"FilterValues": [
72+
"english",
73+
"en"
74+
]
75+
}
76+
}
77+
]
78+
}
79+
```
80+
81+
## Testing
82+
83+
In order to test the solution, you need to login into your AWS Console and generate a couple of Amazon SNS messages that mimics the filters provided in `stackconfiguration.js`.
84+
85+
**Scenario-1 Attribute based**
86+
87+
Create an SNS message with the following information:
88+
89+
```
90+
Message Body:
91+
{
92+
"Message":"Test Message to be received by Queue3 only"
93+
}
94+
Message Attribute:
95+
Type - String; Name - country; Value - usa
96+
```
97+
98+
**Scenario-2 Content based**
99+
100+
Create an SNS message with the following information:
101+
102+
```
103+
Message body based filter. Copy Json to the message body and add attibutes given below. This message should be received by Queue1 and Queue3
104+
Message Body:
105+
{
106+
"Message":"Test Message to be received by Queue2 only",
107+
"Filter":{
108+
"Language": ["en"]
109+
}
110+
}
111+
```
112+
113+
Then head to the Amazon SQS queues, pull the messages and verify that the messages have been routed to the correct queue.
114+
115+
## Cleanup
116+
117+
1. Delete the stack
118+
```bash
119+
cdk destroy
120+
```
121+
----
122+
Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
123+
124+
SPDX-License-Identifier: MIT-0

‎sns-sqs-message-content-router-cdk/example-pattern.json‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
{
2+
"title": "Content based router with SNS and SQS filters",
3+
"description": "Create an SNS topic and multiple SQS queues with content filtered based on configurable criteria.",
4+
"language": "TypeScript",
5+
"level": "200",
6+
"framework": "CDK",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This CDK project demonstrates how to create a Message Content-based routing pattern using Amazon SNS and Amazon SQS.",
11+
"The number of Queues and the Filters are dynamically defined inside the stackconfiguration.json file. The solution can be adapted for any specific requirement.",
12+
"This architecture is an implementation of the Integration pattern: Content-Based Message Router explained in the Integration Patterns book.",
13+
"The content can be filtered by attributes or by message body which grants full flexibility."
14+
]
15+
},
16+
"gitHub": {
17+
"template": {
18+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/sns-sqs-message-content-router-cdk",
19+
"templateURL": "serverless-patterns/sns-sqs-message-content-router-cdk",
20+
"projectFolder": "sns-sqs-message-content-router-cdk",
21+
"templateFile": "sns-sqs-message-content-router-cdk/src/lib/src-stack.ts"
22+
}
23+
},
24+
"resources": {
25+
"bullets": [
26+
{
27+
"text": "Amazon SNS subscription filter policies",
28+
"link": "https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html"
29+
}
30+
]
31+
},
32+
"deploy": {
33+
"text": [
34+
"cdk deploy"
35+
]
36+
},
37+
"testing": {
38+
"text": [
39+
"See the Github repo for detailed testing instructions."
40+
]
41+
},
42+
"cleanup": {
43+
"text": [
44+
"Delete the stack: <code>cdk destroy</code>."
45+
]
46+
},
47+
"authors": [
48+
{
49+
"name": "Raffaele Garofalo (Raf)",
50+
"image": "https://avatars.githubusercontent.com/raffaeu",
51+
"bio": "Raffaele Garofalo (Raf) is a Senior Solutions Architect at AWS and member of the AWS Serverless TFC (Technical Field Communities).",
52+
"linkedin": "raffaeu",
53+
"twitter": "raffaeu"
54+
}
55+
]
56+
}

‎sns-sqs-message-content-router-cdk/src/.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
*.js
2+
!jest.config.js
3+
*.d.ts
4+
node_modules
5+
6+
# CDK asset staging directory
7+
.cdk.staging
8+
cdk.out

‎sns-sqs-message-content-router-cdk/src/.npmignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
*.ts
2+
!*.d.ts
3+
4+
# CDK asset staging directory
5+
.cdk.staging
6+
cdk.out

‎sns-sqs-message-content-router-cdk/src/bin/src.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env node
2+
import 'source-map-support/register';
3+
import * as cdk from 'aws-cdk-lib';
4+
import { SrcStack } from '../lib/src-stack';
5+
6+
const app = new cdk.App();
7+
new SrcStack(app, 'cdk-stack-message-router', {
8+
9+
});

‎sns-sqs-message-content-router-cdk/src/cdk.json‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"app": "npx ts-node --prefer-ts-exts bin/src.ts",
3+
"watch": {
4+
"include": [
5+
"**"
6+
],
7+
"exclude": [
8+
"README.md",
9+
"cdk*.json",
10+
"**/*.d.ts",
11+
"**/*.js",
12+
"tsconfig.json",
13+
"package*.json",
14+
"yarn.lock",
15+
"node_modules",
16+
"test"
17+
]
18+
},
19+
"context": {
20+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
21+
"@aws-cdk/core:checkSecretUsage": true,
22+
"@aws-cdk/core:target-partitions": [
23+
"aws",
24+
"aws-cn"
25+
],
26+
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
27+
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
28+
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
29+
"@aws-cdk/aws-iam:minimizePolicies": true,
30+
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
31+
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
32+
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
33+
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
34+
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
35+
"@aws-cdk/core:enablePartitionLiterals": true,
36+
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
37+
"@aws-cdk/aws-iam:standardizedServicePrincipals": true,
38+
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
39+
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
40+
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
41+
"@aws-cdk/aws-route53-patters:useCertificate": true,
42+
"@aws-cdk/customresources:installLatestAwsSdkDefault": false
43+
}
44+
}

‎sns-sqs-message-content-router-cdk/src/jest.config.js‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
module.exports = {
2+
testEnvironment: 'node',
3+
roots: ['<rootDir>/test'],
4+
testMatch: ['**/*.test.ts'],
5+
transform: {
6+
'^.+\\.tsx?$': 'ts-jest'
7+
}
8+
};

‎sns-sqs-message-content-router-cdk/src/lib/src-stack.ts‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
import * as cdk from 'aws-cdk-lib';
2+
import { Construct } from 'constructs';
3+
import * as configuration from '../stackconfiguration.json';
4+
import { AllowListReceiptFilter } from 'aws-cdk-lib/aws-ses';
5+
6+
export class SrcStack extends cdk.Stack {
7+
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
8+
super(scope, id, props);
9+
10+
// parameters
11+
const account = cdk.Stack.of(this).account;
12+
const stackName = cdk.Stack.of(this).stackName;
13+
const snsName = `${account}-${stackName}-topic-${configuration.Sns.TopicName}`;
14+
15+
// sns topic
16+
const sns = new cdk.aws_sns.Topic(this, 'cdk-sns-topic', {
17+
displayName: snsName,
18+
topicName: snsName
19+
});
20+
21+
// create all required sqs
22+
for (let q = 0; q < configuration.Sqs.length; q++) {
23+
const queueConfig = configuration.Sqs[q];
24+
// dead queue
25+
const sqsDead = new cdk.aws_sqs.Queue(this, `cdk-sqs-dead-${q + 1}`, {
26+
queueName: `${account}-${stackName}-dead-${queueConfig.QueueName}`,
27+
removalPolicy: cdk.RemovalPolicy.DESTROY
28+
});
29+
// queue
30+
const sqs = new cdk.aws_sqs.Queue(this, `cdk-sqs-queue-${q + 1}`, {
31+
queueName: `${account}-${stackName}-queue-${queueConfig.QueueName}`,
32+
removalPolicy: cdk.RemovalPolicy.DESTROY,
33+
deadLetterQueue: {
34+
maxReceiveCount: 3,
35+
queue: sqsDead
36+
}
37+
});
38+
if (queueConfig.IsAttribute) {
39+
// subscription attribute
40+
sns.addSubscription(new cdk.aws_sns_subscriptions.SqsSubscription(sqs, {
41+
filterPolicy: {
42+
[queueConfig.Filter.FilterName]: cdk.aws_sns.SubscriptionFilter.stringFilter({
43+
allowlist: queueConfig.Filter.FilterValues
44+
})
45+
},
46+
rawMessageDelivery: true
47+
}));
48+
} else {
49+
// subscription content
50+
sns.addSubscription(new cdk.aws_sns_subscriptions.SqsSubscription(sqs, {
51+
filterPolicyWithMessageBody: {
52+
filter: cdk.aws_sns.FilterOrPolicy.policy({
53+
[queueConfig.Filter.FilterName]: cdk.aws_sns.FilterOrPolicy.filter(cdk.aws_sns.SubscriptionFilter.stringFilter({
54+
allowlist: queueConfig.Filter.FilterValues
55+
}))
56+
})
57+
58+
},
59+
rawMessageDelivery: true
60+
}));
61+
}
62+
}
63+
}
64+
}

‎sns-sqs-message-content-router-cdk/src/package.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"name": "src",
3+
"version": "0.1.0",
4+
"bin": {
5+
"src": "bin/src.js"
6+
},
7+
"scripts": {
8+
"build": "tsc",
9+
"watch": "tsc -w",
10+
"test": "jest",
11+
"cdk": "cdk"
12+
},
13+
"devDependencies": {
14+
"@types/jest": "^29.2.5",
15+
"@types/node": "18.11.18",
16+
"aws-cdk": "2.61.0",
17+
"jest": "^29.3.1",
18+
"ts-jest": "^29.0.3",
19+
"ts-node": "^10.9.1",
20+
"typescript": "~4.9.4"
21+
},
22+
"dependencies": {
23+
"aws-cdk-lib": "^2.84.0",
24+
"constructs": "^10.0.0",
25+
"source-map-support": "^0.5.21"
26+
}
27+
}

‎sns-sqs-message-content-router-cdk/src/stackconfiguration.json‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{
2+
"Sns": {
3+
"TopicName": "content-router"
4+
},
5+
"Sqs": [
6+
{
7+
"QueueName": "country-usa",
8+
"IsAttribute": true,
9+
"Filter": {
10+
"FilterName": "country",
11+
"FilterValues": [
12+
"united states of america",
13+
"usa"
14+
]
15+
}
16+
},
17+
{
18+
"QueueName": "country-germany",
19+
"IsAttribute": true,
20+
"Filter": {
21+
"FilterName": "country",
22+
"FilterValues": [
23+
"germany",
24+
"de"
25+
]
26+
}
27+
},
28+
{
29+
"QueueName": "language-english",
30+
"IsAttribute": false,
31+
"Filter": {
32+
"FilterName": "language",
33+
"FilterValues": [
34+
"english",
35+
"en"
36+
]
37+
}
38+
}
39+
]
40+
}

0 commit comments

Comments
 (0)