Add rate limiting and retry configuration to Contentstack provider #44
Conversation
- Add rate_limit, rate_burst, and max_retries provider attributes - Update provider schema to support new rate limiting options - Add automatic retry logic integration with contentstack-go-sdk - Update provider documentation with new attributes - Add simplified examples for provider and rate-limiting configurations - Update go.mod to use enhanced contentstack-go-sdk with rate limiting This enables reliable Terraform operations by automatically handling Contentstack API rate limits (10 req/sec) with configurable throttling and exponential backoff retry logic for 429 responses. Related: contentstack-go-sdk rate limiting implementation
- Add retryableRoundTripper for seamless SDK integration - Implement retry policy for 429 responses only - Add max_retries, retry_wait_min, retry_wait_max provider attributes - Update examples and documentation - Use exponential backoff with configurable wait times - Maintain compatibility with existing rate limiting
|
Hi @demeyerthom , |
There was a problem hiding this comment.
Overall looks good! I left 2 comments to fix.
Can you also add a changie file for the changelog? It triggers our deployment processes:
- install changie (https://changie.dev/guide/installation/)
- run
changie new - fill in the questions
- add generated file to the commit
| BaseURL: config.BaseURL.Value, | ||
| AuthToken: config.AuthToken.Value, | ||
| HTTPClient: httpClient, | ||
| RateLimit: config.RateLimit.Value, |
There was a problem hiding this comment.
I think those fields are not part of the config are they?
There was a problem hiding this comment.
Hi @demeyerthom,
Good catch! Those RateLimit and RateBurst fields are part of the SDK changes I've proposed in contentstack-go-sdk PR labd/contentstack-go-sdk#17.
The fields aren't in the published SDK (v0.1.0) yet, so this terraform provider PR depends on those SDK changes being merged first.
Would you be open to reviewing and merging the SDK PR first? Once that's released, this provider PR will work with the new SDK version. Happy to make any adjustments needed on either PR!
Thanks for your time reviewing this!
| @@ -0,0 +1,33 @@ | |||
| terraform { | |||
There was a problem hiding this comment.
This bit will not show in the docs. You want to add it to examples/provider/main.tf, and then run go generate to generate the correct docs for it to show
2 participants
This PR implements rate limiting and automatic retry configuration for the Contentstack Terraform provider to address API rate limit issues that cause terraform operations to fail with 429 errors.
Changes implemented:
Provider Rate Limiting Configuration: Added three new optional provider attributes:
rate_limit(float64): Maximum requests per second (defaults to 10.0 to comply with Contentstack API limits)rate_burst(int): Maximum burst size for short bursts above the rate limit (defaults to 10)max_retries(int): Maximum retry attempts for 429 responses with exponential backoff (defaults to 3)Integration with Enhanced SDK: Updated go.mod to use the enhanced contentstack-go-sdk that includes:
golang.org/x/time/rateRetry-AfterheadersDocumentation Updates: Auto-generated provider documentation now includes all new rate limiting attributes with detailed descriptions.
Example Configurations: Added simplified examples demonstrating:
examples/provider/main.tf)examples/rate-limiting/main.tf)Rationale:
Contentstack's Management API has a 10 requests/second rate limit. When Terraform makes concurrent requests (during plan/apply operations with multiple resources), it frequently hits 429 errors causing operations to fail. This is particularly problematic for users managing large numbers of content types and global fields (~35+ resources as mentioned in the related issue).
The solution provides configurable rate limiting that prevents hitting API limits, plus automatic retry logic that gracefully handles any 429 responses that do occur, making Terraform operations reliable and robust.
Fixes #3
Related: contentstack-go-sdk#16
NEW FEATURES | UPGRADE NOTES | ENHANCEMENTS | BUG FIXES | EXPERIMENTS
rate_limitprovider attribute to configure API request throttlingrate_burstprovider attribute to allow controlled request burstsmax_retriesprovider attribute to configure automatic retry attempts for 429 responses