Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ See each command's linked README for more details, or click on [📚](https://do
#### `stepfunctions`
- `instrument`: Instrument [AWS Step Function](src/commands/stepfunctions) with Datadog to get logs and traces. [📚](https://docs.datadoghq.com/serverless/step_functions/installation/?tab=datadogcli)
- `uninstrument`: Uninstrument [AWS Step Function](src/commands/stepfunctions). [📚](https://docs.datadoghq.com/serverless/step_functions/installation/?tab=datadogcli)
- `flare`: Gather [AWS Step Function](src/commands/stepfunctions) configuration, execution history, and logs for Datadog support. [📚](src/commands/stepfunctions/README.md#flare)

#### `synthetics`
- `run-tests`: Run [Continuous Testing tests](src/commands/synthetics) from the CI. [📚](https://docs.datadoghq.com/continuous_testing/)
Expand Down
51 changes: 50 additions & 1 deletion src/commands/stepfunctions/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# stepfunctions commands

You can use the `stepfunctions instrument` command to instrument your Step Functions with Datadog. This command enables instrumentation by subscribing Step Function logs to a [Datadog Forwarder](https://docs.datadoghq.com/logs/guide/forwarder/).
The Step Functions commands allow you to manage Datadog instrumentation and troubleshooting for your AWS Step Functions:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The Step Functions commands allow you to manage Datadog instrumentation and troubleshooting for your AWS Step Functions:
The Step Function commands allow you to manage Datadog instrumentation and troubleshooting for your AWS Step Functions:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion for consistency -- as in line 5, Step Function is used when it's an attributive noun


- Use the `stepfunctions instrument` command to instrument your Step Functions with Datadog. This command enables instrumentation by subscribing Step Function logs to a [Datadog Forwarder](https://docs.datadoghq.com/logs/guide/forwarder/).
- Use the `stepfunctions uninstrument` command to remove Datadog instrumentation from your Step Functions.
- Use the `stepfunctions flare` command to collect diagnostic information for troubleshooting with Datadog support.

You can also add the `stepfunctions instrument` command to your CI/CD pipelines to enable Datadog instrumentation for all of your Step Functions. Run the command after your normal serverless application deployment, so that changes made by this command do not get overridden by changes in the CI/CD pipeline.

Expand All @@ -23,6 +27,38 @@ Run the `uninstrument` command to unsubscribe a Step Function log group from the
datadog-ci stepfunctions uninstrument --step-function <step-function-arn> --forwarder <forwarder-arn> [--dry-run]
```

### `flare`
Run the `flare` command to gather state machine configuration, execution history, logs, and project files for Datadog support troubleshooting. This command collects diagnostic information about your Step Functions and creates a flare file that can be shared with Datadog support.

```bash
datadog-ci stepfunctions flare --state-machine <state-machine-arn> --case-id <case-id> --email <email> [--with-logs] [--start] [--end] [--max-executions] [--dry-run]
```

Example:
```bash
# Basic flare collection
datadog-ci stepfunctions flare \
--state-machine arn:aws:states:us-east-1:123456789012:stateMachine:MyStateMachine \
--case-id 12345 \
--email support@example.com

# Include logs from the last 7 days
datadog-ci stepfunctions flare \
--state-machine arn:aws:states:us-east-1:123456789012:stateMachine:MyStateMachine \
--case-id 12345 \
--email support@example.com \
--with-logs \
--start "2025-06-11" \
--end "2025-06-18"

# Dry run to preview without sending
datadog-ci stepfunctions flare \
--state-machine arn:aws:states:us-east-1:123456789012:stateMachine:MyStateMachine \
--case-id 12345 \
--email support@example.com \
--dry-run
```

## Arguments

### instrument
Expand All @@ -44,6 +80,19 @@ datadog-ci stepfunctions uninstrument --step-function <step-function-arn> --forw
| `--forwarder` | | :white_check_mark: | The ARN of the [Datadog Forwarder](https://docs.datadoghq.com/logs/guide/forwarder/) to subscribe Step Function log groups. | |
| `--dry-run` | `-d` | | Preview changes without applying them. | `false` |

### flare

| Argument | Shorthand | Required | Description | Default |
| ----------------- | --------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `--state-machine` | `-s` | :white_check_mark: | The ARN of the Step Functions state machine to collect diagnostic information from. | |
| `--case-id` | `-c` | :white_check_mark: | The Datadog support case ID to associate with this flare. | |
| `--email` | `-e` | :white_check_mark: | The email address associated with the support case. | |
| `--with-logs` | | | Include CloudWatch logs from the state machine's log group in the flare. | `false` |
| `--start` | | | Start time for log collection (ISO 8601 format). Only used with `--with-logs`. | |
| `--end` | | | End time for log collection (ISO 8601 format). Only used with `--with-logs`. | |
| `--max-executions`| | | Maximum number of recent executions to include in the flare. | `10` |
| `--dry-run` | `-d` | | Preview the flare collection without creating or sending files. | `false` |

## Installation

1. Install the Datadog CLI
Expand Down
227 changes: 227 additions & 0 deletions src/commands/stepfunctions/__tests__/fixtures/stepfunctions-flare.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
import {SubscriptionFilter, OutputLogEvent} from '@aws-sdk/client-cloudwatch-logs'
import {DescribeStateMachineCommandOutput, ExecutionListItem, HistoryEvent, Tag} from '@aws-sdk/client-sfn'

export const stateMachineConfigFixture = (
props: Partial<DescribeStateMachineCommandOutput> = {}
): DescribeStateMachineCommandOutput => {
const defaults: DescribeStateMachineCommandOutput = {
$metadata: {},
stateMachineArn: 'arn:aws:states:us-east-1:123456789012:stateMachine:MyWorkflow',
name: 'MyWorkflow',
status: 'ACTIVE',
definition: JSON.stringify({
StartAt: 'HelloWorld',
States: {
HelloWorld: {
Type: 'Pass',
Result: 'Hello World!',
End: true,
},
},
}),
roleArn: 'arn:aws:iam::123456789012:role/MyRole',
type: 'STANDARD',
creationDate: new Date('2024-01-01'),
loggingConfiguration: {
level: 'ALL',
includeExecutionData: true,
destinations: [
{
cloudWatchLogsLogGroup: {
logGroupArn: 'arn:aws:logs:us-east-1:123456789012:log-group:/aws/vendedlogs/states/MyWorkflow-Logs',
},
},
],
},
}

return {...defaults, ...props}
}

export const sensitiveStateMachineConfigFixture = (): DescribeStateMachineCommandOutput => {
return stateMachineConfigFixture({
definition: JSON.stringify({
StartAt: 'ProcessPayment',
States: {
ProcessPayment: {
Type: 'Task',
Resource: 'arn:aws:lambda:us-east-1:123456789012:function:ProcessPayment',
Parameters: {
'ApiKey.$': '$.credentials.apiKey',
SecretToken: 'secret-12345-token',
DatabasePassword: 'super-secret-password',
},
End: true,
},
},
}),
description: 'Payment processing workflow with sensitive data',
})
}

export const executionsFixture = (): ExecutionListItem[] => {
return [
{
executionArn: 'arn:aws:states:us-east-1:123456789012:execution:MyWorkflow:execution1',
stateMachineArn: 'arn:aws:states:us-east-1:123456789012:stateMachine:MyWorkflow',
name: 'execution1',
status: 'SUCCEEDED',
startDate: new Date('2024-01-01T10:00:00Z'),
stopDate: new Date('2024-01-01T10:01:00Z'),
},
{
executionArn: 'arn:aws:states:us-east-1:123456789012:execution:MyWorkflow:execution2',
stateMachineArn: 'arn:aws:states:us-east-1:123456789012:stateMachine:MyWorkflow',
name: 'execution2',
status: 'FAILED',
startDate: new Date('2024-01-01T09:00:00Z'),
stopDate: new Date('2024-01-01T09:01:00Z'),
},
]
}

interface SensitiveExecution {
executionArn: string
stateMachineArn: string
name: string
status: string
startDate: Date
stopDate: Date
input: string
output: string
}

export const sensitiveExecutionFixture = (): SensitiveExecution => {
return {
executionArn: 'arn:aws:states:us-east-1:123456789012:execution:MyWorkflow:execution1',
stateMachineArn: 'arn:aws:states:us-east-1:123456789012:stateMachine:MyWorkflow',
name: 'execution1',
status: 'SUCCEEDED',
startDate: new Date('2024-01-01T10:00:00Z'),
stopDate: new Date('2024-01-01T10:01:00Z'),
input: '{"creditCard": "4111-1111-1111-1111", "cvv": "123", "amount": 100}',
output: '{"transactionId": "secret-transaction-id", "authToken": "Bearer secret-token"}',
}
}

export const executionHistoryFixture = (): HistoryEvent[] => {
return [
{
timestamp: new Date('2024-01-01T10:00:00Z'),
type: 'ExecutionStarted',
id: 1,
previousEventId: 0,
executionStartedEventDetails: {
input: '{"orderId": "12345"}',
roleArn: 'arn:aws:iam::123456789012:role/MyRole',
},
},
{
timestamp: new Date('2024-01-01T10:00:01Z'),
type: 'TaskStateEntered',
id: 2,
previousEventId: 1,
stateEnteredEventDetails: {
name: 'ProcessPayment',
input: '{"orderId": "12345", "amount": 100}',
},
},
{
timestamp: new Date('2024-01-01T10:00:59Z'),
type: 'TaskStateExited',
id: 3,
previousEventId: 2,
stateExitedEventDetails: {
name: 'ProcessPayment',
output: '{"success": true, "transactionId": "tx-12345"}',
},
},
{
timestamp: new Date('2024-01-01T10:01:00Z'),
type: 'ExecutionSucceeded',
id: 4,
previousEventId: 3,
executionSucceededEventDetails: {
output: '{"result": "completed"}',
},
},
]
}

export const stepFunctionTagsFixture = (): Tag[] => {
return [
{key: 'Environment', value: 'test'},
{key: 'Service', value: 'payment-processor'},
{key: 'Team', value: 'platform'},
]
}

export const logSubscriptionFiltersFixture = (): SubscriptionFilter[] => {
return [
{
filterName: 'datadog-forwarder',
destinationArn: 'arn:aws:lambda:us-east-1:123456789012:function:DatadogForwarder',
filterPattern: '',
logGroupName: '/aws/vendedlogs/states/MyWorkflow-Logs',
},
]
}

export const cloudWatchLogsFixture = (): OutputLogEvent[] => {
return [
{
timestamp: 1704106800000,
message: 'Execution started',
ingestionTime: 1704106801000,
},
{
timestamp: 1704106801000,
message: 'Processing payment for order 12345',
ingestionTime: 1704106802000,
},
{
timestamp: 1704106859000,
message: 'Payment processed successfully',
ingestionTime: 1704106860000,
},
{
timestamp: 1704106860000,
message: 'Execution completed',
ingestionTime: 1704106861000,
},
]
}

export const MOCK_STATE_MACHINE_ARN = 'arn:aws:states:us-east-1:123456789012:stateMachine:MyWorkflow'
export const MOCK_REGION = 'us-east-1'
export const MOCK_CASE_ID = 'case-123456'
export const MOCK_EMAIL = 'test@example.com'
export const MOCK_API_KEY = 'test-api-key-1234'

export const MOCK_AWS_CREDENTIALS = {
accessKeyId: 'AKIAIOSFODNN7EXAMPLE',
secretAccessKey: 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
sessionToken: undefined,
}

export const MOCK_FRAMEWORK = 'Serverless Framework'

export const MOCK_OUTPUT_DIR = '.datadog-ci/flare/stepfunctions-MyWorkflow-1704106800000'

export const MOCK_INSIGHTS_CONTENT = `# Step Functions Flare Insights

Generated: 2024-01-01T10:00:00.000Z

## State Machine Configuration
- Name: MyWorkflow
- ARN: arn:aws:states:us-east-1:123456789012:stateMachine:MyWorkflow
- Type: STANDARD
- Status: ACTIVE

## Framework
Serverless Framework

## Environment
- Region: us-east-1
- CLI Version: 1.0.0
`
Loading
Loading