Stats

Logs are stored in CloudWatch Insights. There are various ways we can slice and dice this. Some useful canned queries are:

API Requests by token (GET param)

fields @timestamp, @message, queryStringParameters.auth_token
| sort @timestamp desc
| filter @message like /(?i)(api\/v1)./ and httpMethod = 'GET'
| stats count() by  queryStringParameters.auth_token

API Requests by token (header)

fields @timestamp, @message, headers.Authorization
| sort @timestamp desc
| filter @message like /(?i)(api\/v1)./ and httpMethod = 'GET'
| stats count() by headers.Authorization

Widget Requests by embed page

fields @timestamp, @message, queryStringParameters.utm_source
| sort @timestamp desc
| filter @message like /(?i)(api\/v1)./ and httpMethod = 'GET'
| filter queryStringParameters.utm_medium="widget"
| stats count() by queryStringParameters.utm_source

Note this is raw number of requests.

Exporting logs to s3

To do this you'll need to have access to s3 and cloudwatch in the production-aggregator-api-dc account.

• Create a bucket - I made aggregator-api-prod-logs
• Add permissions to the bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.eu-west-2.amazonaws.com"
            },
            "Action": "s3:GetBucketAcl",
            "Resource": "arn:aws:s3:::aggregator-api-prod-logs"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.eu-west-2.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::aggregator-api-prod-logs/20210506/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }
    ]
}

Replace 20210506 with a unique string - I used the date that I was exporting logs for.

• Go to the 'Export logs to s3' action on the relevant log group.
• Pick your time range, select the s3 bucket, and add the unique string from above to the S3 bucket prefix field.
• Export