Skip to content

Handle Throttling Limit in Fabric Item Jobs #15

Open
billybillysss wants to merge 1 commit intomicrosoft:mainfrom
billybillysss:feature/add-throttling-limit-handling
Open

Handle Throttling Limit in Fabric Item Jobs #15
billybillysss wants to merge 1 commit intomicrosoft:mainfrom
billybillysss:feature/add-throttling-limit-handling

Conversation

@billybillysss
Copy link

This PR addresses an issue encountered when calling Fabric item jobs concurrently with deferrable=True and wait-for-terminate=True. The issue arises due to throttling limits in the Fabric API, which returns a 429 status code with the error code 'RequestBlocked'. https://learn.microsoft.com/en-us/rest/api/fabric/articles/throttling

When this occurs, the response lacks a "status" field, causing the job to fail while attempting to parse the JSON response.
Changes Introduced:

New FabricRunItemStatus State:
    Added a new status, Throttled, to handle scenarios where the API throttles requests.
    Included this new status in the THROTTLED_STATES group.

Modified _async_send_request function in the Hook:
    Updated the method to include status_code, status, and retry_after (extracted from the response header) in the returned dictionary format.

Modified run function in the Trigger:
    When the item status is in FabricRunItemStatus.THROTTLED_STATES, the job will pause execution and sleep for the number of seconds specified in the retry-after header.

I have tested successfully to run 20 Fabric item jobs concurrently without encountering failures

@billybillysss
Copy link
Author

Hey there, this is my first time raising a PR, so if there’s any additional information I need to provide or anything I can improve, please feel free to let me know. Thanks! 😊

@jpeak5
Copy link

jpeak5 commented Jun 27, 2025

Thanks @billybillysss for this patch, it is working well for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants