-
Notifications
You must be signed in to change notification settings - Fork 289
Clarify message idempotency section #4144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -109,19 +109,15 @@ If you don’t need to ensure that your handlers complete, you may specify your | |
|
|
||
| See the links below for how to ensure handlers are finished in your SDK. | ||
|
|
||
| #### Ensuring your messages are processed exactly once {#exactly-once-message-processing} | ||
| #### Message IDs and handling Continue-As-New {#exactly-once-message-processing} | ||
|
|
||
| Many developers want their message handlers to run exactly once--to be idempotent--in cases where the same Signal or Update is delivered twice or sent by two different call sites. Temporal deduplicates messages for you on the server, but there is one important case when you need to think about this yourself when authoring a Workflow, and one when sending Signals and Updates. | ||
| Usually, you'll want your message handlers to run exactly once--to be idempotent--in cases where the same Signal or Update is delivered twice. For Updates, Temporal handles this for you on the server, by deduplicating according to the Update ID. For Signals, you should use a custom idempotency key that you send as part of your own signal inputs, implementing the deduplication in your Workflow code. The Update ID is set automatically to a UUID, but you can set it yourself. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd move the last sentence about UUID next to the Update sentence. The Signal sentence breaks the flow there in the middle.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It may be simpler to just ask people to check idempotency. And then explain the reasoning that it's necessary for signals and update plus CaN . |
||
|
|
||
| When your workflow Continues-As-New, you should handle deduplication yourself in your message handler. This is because Temporal's built-in deduplication doesn't work across [Continue-As-New](/workflow-execution/continue-as-new) boundaries, meaning you would risk processing messages twice for such Workflows if you don't check for duplicate messages yourself. | ||
| However, if you are using Updates with [Continue-As-New](/workflow-execution/continue-as-new) you should implement the deduplication in your Workflow code, since Update ID deduplication by the server is per Workflow run. | ||
|
|
||
| To deduplicate in your message handler, you can use an idempotency key. | ||
| (In addition to these application-level identifiers, both Signals and Updates automatically use request IDs to deduplicate retried client calls. You do not need to do anything to enable this.) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't mind dropping the parentheses.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On second thought, if the user can't change/do anything about it, do we need to explain this here at all?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have the same question.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, the two topics are deduplication of framework-level retries, and deduplication of messages that are duplicates according to an application-domain definition of equivalence. We could get rid of it. The reason I put it there is because I have personally confused them in the past, which leads me to suspect others will. Put differently, whenever request IDs are discussed, idempotency is also. If we tell people that they are on the hook for signal idempotency then I think many people will confuse that with a statement that Temporal doesn't use request IDs for signal. |
||
|
|
||
| Clients can provide an idempotency key. This can be important because Temporal's SDKs provide a randomized key by default, which means Temporal only deduplicates retries from the same call. For Updates, if you craft an Update ID, Temporal will deduplicate any calls that use that key. This is useful when you have two different callsites that may send the same Update, or when your client itself may get retried. For Signals, you can provide a key as part of your Signal arguments. | ||
|
|
||
| Inside your message handler, you can check your idempotency key--the Update ID or the one you provided to the Signal--to check whether the Workflow has already handled the update. | ||
|
|
||
| See the links below for examples of solving this in your SDK. | ||
| See the links below for examples of handling idempotency and Continue-As-New in your SDK. | ||
|
|
||
| #### Authoring message handler patterns | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like that this is focused on CaN now. The part about the same Update/Signal coming in from different clients is now only implied. That might be fine.