-
Notifications
You must be signed in to change notification settings - Fork 782
workflow history propagation docs #5153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: v1.18
Are you sure you want to change the base?
Changes from 9 commits
a8c2450
5e033dd
bbe7f12
e8848cf
4fea981
b1fe587
45a128a
4d9e8ea
14d79c1
2b6bd66
5f99352
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,258 @@ | ||
| --- | ||
| type: docs | ||
| title: "Workflow history propagation" | ||
| linkTitle: "History propagation" | ||
| weight: 2500 | ||
| description: "Share a parent workflow's execution history with child workflows and activities for chain-of-custody, audit, and AI-agent context" | ||
| --- | ||
|
|
||
| By default, a child workflow or activity only sees the input it was scheduled with. The parent's execution history — what activities ran, what child workflows it spawned, what events it observed — is invisible. Workflow history propagation lets a parent opt-in to share that history with the work it schedules. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add a some scenarios into this article as well, in case most likely, that some comes straight to this article from a search (like I suggest in the overview)
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add a some scenarios into this article as well, in case most likely, that some comes straight to this article from a search (like I suggest in the overview)
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| This is useful for: | ||
|
|
||
| - **Chain-of-custody and audit** — a fraud check, compliance gate, or settlement step can verify what already happened upstream rather than trusting the caller's claims. | ||
| - **Cross-app workflows** — a downstream app, often owned by a different team, can confirm an upstream activity ran without an out-of-band query. | ||
|
cicoyle marked this conversation as resolved.
|
||
| - **AI agents and long-running orchestrations** — agents that loop with `ContinueAsNew` or fan out to sub-agents can carry context (tool calls, model outputs, intermediate decisions) forward without re-prompting from scratch. | ||
|
|
||
| ## Propagation scopes | ||
|
|
||
| Two scopes are exposed. The default is no propagation. | ||
|
|
||
| | Option helper | Scope value on `propagatedHistory.Scope()` | What gets sent | When to use it | | ||
| | ------------- | --------------------------- | -------------- | -------------- | | ||
| | `workflow.PropagateLineage()` | `LINEAGE` (`HISTORY_PROPAGATION_SCOPE_LINEAGE`) | Caller's own events plus the full ancestor chain inherited from the caller's own parent | Full chain-of-custody, downstream wants to see everything that happened before, all the way to the root | | ||
| | `workflow.PropagateOwnHistory()` | `OWN_HISTORY` (`HISTORY_PROPAGATION_SCOPE_OWN_HISTORY`) | Caller's own events only, ancestral lineage is dropped | Trust boundary — the caller is willing to vouch for what *it* did but the receiver shouldn't see further upstream | | ||
|
|
||
| The helpers (`PropagateLineage` / `PropagateOwnHistory`) are what you pass when scheduling. The scope values (`LINEAGE` / `OWN_HISTORY`) are what `propagatedHistory.Scope()` returns on the receiver — they're the same thing. | ||
|
|
||
| `PropagateOwnHistory` is the trust boundary: choosing it tells the runtime to stop forwarding any history the caller itself received. This is the right choice when the receiver is a less-trusted app, a third party, or operates under different compliance rules. | ||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any more guidance we can provide here? For example PropagateOwnHistory only has to be used if you have no child workflows, but this is very unlikely typically. |
||
| ## Setting it up | ||
|
|
||
| A parent opts a single child workflow or activity into propagation via a per-call option. Other calls in the same workflow are unaffected. The default is no propagation. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. opts?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, I can change it if preferred: |
||
|
|
||
| {{< tabpane text=true >}} | ||
|
|
||
| {{% tab header="Python" %}} | ||
|
|
||
| ```python | ||
| import dapr.ext.workflow as wf | ||
|
|
||
| @wfr.workflow(name='MerchantCheckout') | ||
| def merchant_checkout(ctx: wf.DaprWorkflowContext, order_json: str): | ||
| # Activity does NOT receive propagated history (default). | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| yield ctx.call_activity(validate_merchant, input=order_json) | ||
|
|
||
| # Child workflow DOES receive parent's history (LINEAGE). | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| result = yield ctx.call_child_workflow( | ||
| process_payment, | ||
| input=order_json, | ||
| propagation=wf.PropagationScope.LINEAGE, | ||
| ) | ||
| return result | ||
| ``` | ||
|
|
||
| {{% /tab %}} | ||
|
|
||
| {{% tab header=".NET" %}} | ||
|
|
||
| ```csharp | ||
| using Dapr.Workflow; | ||
|
|
||
| public sealed class MerchantCheckoutWorkflow : Workflow<Order, string> | ||
| { | ||
| public override async Task<string> RunAsync(WorkflowContext ctx, Order order) | ||
| { | ||
| // Activity does NOT receive propagated history (default). | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| await ctx.CallActivityAsync<bool>(nameof(ValidateMerchantActivity), order); | ||
|
|
||
| // Child workflow DOES receive parent's history (Lineage). | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| return await ctx.CallChildWorkflowAsync<string>( | ||
| nameof(ProcessPaymentWorkflow), | ||
| order, | ||
| new ChildWorkflowTaskOptions(PropagationScope: HistoryPropagationScope.Lineage)); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| {{% /tab %}} | ||
|
|
||
| {{% tab header="Go" %}} | ||
|
|
||
| ```go | ||
|
cicoyle marked this conversation as resolved.
|
||
| import ( | ||
| "github.com/dapr/durabletask-go/workflow" | ||
| ) | ||
|
|
||
| func MerchantCheckout(ctx *workflow.WorkflowContext) (any, error) { | ||
| // Activity does NOT receive propagated history (default). | ||
| if err := ctx.CallActivity("ValidateMerchant").Await(nil); err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| // Child workflow DOES receive parent's history (LINEAGE). | ||
| var result string | ||
| if err := ctx.CallChildWorkflow("ProcessPayment", | ||
| workflow.WithHistoryPropagation(workflow.PropagateLineage()), | ||
| ).Await(&result); err != nil { | ||
| return nil, err | ||
| } | ||
| return result, nil | ||
| } | ||
| ``` | ||
|
|
||
| {{% /tab %}} | ||
|
|
||
| {{< /tabpane >}} | ||
|
|
||
| ## Receiving propagated history | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| Inside a child workflow or activity, call `GetPropagatedHistory()` (Go / .NET) or `get_propagated_history()` (Python). It returns the propagated history if the caller opted in, or `None` / `nil` if it didn't. | ||
|
|
||
| The example below assumes the parent shown above registered as `MerchantCheckout` with an activity `ValidateMerchant` and a child workflow `FraudDetection` (called with `Lineage`). | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| {{< tabpane text=true >}} | ||
|
|
||
| {{% tab header="Python" %}} | ||
|
|
||
| ```python | ||
| import dapr.ext.workflow as wf | ||
|
|
||
| @wfr.workflow(name='FraudDetection') | ||
| def fraud_detection(ctx: wf.DaprWorkflowContext, order_json: str): | ||
| history = ctx.get_propagated_history() | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Presumably this line will only return history for ValidateMerchant and not ProcessPayment child workflow. Maybe worth saying that |
||
| if history is None: | ||
| return 'no upstream history' | ||
|
|
||
| if not ctx.is_replaying: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does this is_replaying do? It would be nice to have a comment here and why this matters. |
||
| print(f'scope={history.scope}, workflows={[w.name for w in history.get_workflows()]}') | ||
|
|
||
| try: | ||
| merchant_wf = history.get_workflow_by_name('MerchantCheckout') | ||
| validation = merchant_wf.get_activity_by_name('ValidateMerchant') | ||
| except wf.PropagationNotFoundError as exc: | ||
| return f'missing required upstream step: {exc}' | ||
|
|
||
| if not validation.completed: | ||
| return 'merchant validation did not complete, rejecting' | ||
| return 'approved' | ||
| ``` | ||
|
|
||
| {{% /tab %}} | ||
|
|
||
| {{% tab header=".NET" %}} | ||
|
|
||
| ```csharp | ||
| using Dapr.Workflow; | ||
|
|
||
| public sealed class FraudDetectionWorkflow : Workflow<Order, string> | ||
| { | ||
| public override Task<string> RunAsync(WorkflowContext ctx, Order order) | ||
| { | ||
| var history = ctx.GetPropagatedHistory(); | ||
| if (history is null) | ||
| return Task.FromResult("no upstream history"); | ||
|
|
||
| if (!ctx.IsReplaying) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same comment here as above |
||
| Console.WriteLine($"received {history.Entries.Count} workflow segment(s)"); | ||
|
|
||
| // Verify MerchantCheckout is present in the ancestor chain. | ||
| var merchant = history.FilterByWorkflowName(nameof(MerchantCheckoutWorkflow)); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any reason this is not history.GetByWorkflowName for API consistency? Why Filter and not Get? Can you ask Whit?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah not sure, that is not consistent with the other sdks. Will ask in discord. |
||
| if (merchant.Entries.Count == 0) | ||
| return Task.FromResult("MerchantCheckout missing from propagated history, rejecting"); | ||
|
|
||
| return Task.FromResult("approved"); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| {{% /tab %}} | ||
|
|
||
| {{% tab header="Go" %}} | ||
|
|
||
| ```go | ||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/dapr/durabletask-go/workflow" | ||
| ) | ||
|
|
||
| func FraudDetection(ctx *workflow.WorkflowContext) (any, error) { | ||
| propagatedHistory := ctx.GetPropagatedHistory() | ||
| if propagatedHistory == nil { | ||
| return "no upstream history", nil | ||
| } | ||
|
|
||
| // The history exposes both raw events and per-app/per-instance chunks. | ||
| fmt.Printf("scope: %s, %d events from apps %v\n", | ||
| propagatedHistory.Scope(), len(propagatedHistory.Events()), propagatedHistory.GetAppIDs()) | ||
|
|
||
| // Drill into a specific upstream workflow's activities. | ||
| merchantWf, err := propagatedHistory.GetWorkflowByName("MerchantCheckout") | ||
| if err != nil { | ||
| return nil, fmt.Errorf("expected MerchantCheckout in propagated history: %w", err) | ||
| } | ||
|
cicoyle marked this conversation as resolved.
|
||
| validation, err := merchantWf.GetActivityByName("ValidateMerchant") | ||
| if err != nil { | ||
| return nil, fmt.Errorf("expected ValidateMerchant in propagated history: %w", err) | ||
| } | ||
| if !validation.Completed { | ||
| return "merchant validation didn't complete, rejecting", nil | ||
| } | ||
| return "approved", nil | ||
| } | ||
| ``` | ||
|
|
||
| {{% /tab %}} | ||
|
|
||
| {{< /tabpane >}} | ||
|
|
||
| Whatever the language, the returned propagated-history object exposes the same conceptual shape: | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| - **Events** — the flat list of upstream history events in order | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| - **Scope** — which scope the parent chose (`OWN_HISTORY` or `LINEAGE`) | ||
|
cicoyle marked this conversation as resolved.
|
||
| - **Per-workflow entries** — one entry per ancestor workflow, each tagged with that workflow's app ID, name, instance ID, and the index range of events it covers | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what is the actual property name here and is this also a structured type? |
||
| - **App IDs** — deduplicated list of every app that contributed events to the chain | ||
| - **Filters** — convenience accessors to find an entry by workflow name, or events by app / instance / workflow name. Drilling into an entry lets you look up the specific activities or child workflows the upstream workflow ran | ||
|
|
||
| Exact method names differ by SDK (`GetWorkflowByName` in Go and .NET, `get_workflow_by_name` in Python; `WorkflowResult` vs `Entries`; etc.) — see the per-SDK docs for the precise surface. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. tell me this is minimal though and only language semantic differences. The entries one confuses me given that this is the first time that word has been used. What are entries and why is this different? |
||
|
|
||
| ## Cross-app workflows | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| When a parent workflow in App A calls a child workflow in App B, the propagated chunks travel between sidecars. Two security knobs apply: | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| - **mTLS** — when Dapr is deployed with mTLS (the default for Helm / `dapr init -k`), inter-sidecar traffic is encrypted and authenticated. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does mTLS have to do with propagation? The statement below just say communication is encrypted. |
||
| - **`WorkflowHistorySigning`** — an opt-in `Configuration` feature that signs each propagated chunk with the producing app's SPIFFE identity. Receivers can verify chunks weren't tampered with after they left the producer. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Make this its own heading and then reference the article in this PR Workflow history signing.With history signing, the propagated context can be cryptographically verified. For more information see [workflow history signing]({{% ref "workflow-history-signing.md" %}} |
||
|
|
||
| If `WorkflowHistorySigning` is not enabled, daprd logs a warning per dispatch: | ||
|
|
||
| > `propagating unsigned workflow history to child workflow '...' (signing is not configured; chunks cannot be cryptographically verified by the receiver)` | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| {{% alert title="Treat unsigned chunks as untrusted" color="warning" %}} | ||
| Without signing, propagated chunks are functional but not cryptographically verifiable. Don't treat unsigned propagated history as authoritative for high-value decisions (payments, approvals). Enable `WorkflowHistorySigning` for production deployments that depend on chain-of-custody. | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| {{% /alert %}} | ||
|
|
||
| ## ContinueAsNew and rerun | ||
|
|
||
| Propagated history flows correctly through both `ContinueAsNew` and rerun: | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| - A workflow that received propagated history and calls `ContinueAsNew` passes the same incoming chunks to its next generation. | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
| - Rerunning a workflow re-issues activity / child-workflow calls with the same propagation scope they were originally scheduled with — `LINEAGE` stays `LINEAGE`, `OWN_HISTORY` stays `OWN_HISTORY`. | ||
|
cicoyle marked this conversation as resolved.
Outdated
|
||
|
|
||
| This makes long-running agents and crash-recovery scenarios behave the way you'd expect: the receiving generation/rerun sees the same history the original run did. | ||
|
|
||
| ## Next steps | ||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Discover [how to apply workflow signing]({{% ref "workflow-history-signing.md" %}} to attest the validity of a previous workflow or activity step. |
||
| {{< button text="Workflow patterns >>" page="workflow-patterns.md" >}} | ||
|
|
||
| ## Related links | ||
|
|
||
| - [Workflow overview]({{< ref workflow-overview.md >}}) | ||
| - [Workflow features and concepts]({{< ref workflow-features-concepts.md >}}) | ||
| - [Workflow architecture]({{< ref workflow-architecture.md >}}) | ||
| - [How to author a workflow]({{< ref howto-author-workflow.md >}}) | ||
| - [Workflow API reference]({{< ref workflow_api.md >}}) | ||
| - Try out the example code in each SDK: | ||
| - [Python example](https://github.com/dapr/python-sdk/blob/main/examples/workflow/history_propagation.py) | ||
| - [.NET quickstart](https://github.com/dapr/quickstarts/tree/master/workflows/csharp/sdk-context-propagation) | ||
| - [Go example](https://github.com/dapr/go-sdk/tree/main/examples/workflow-history-propagation) | ||
Uh oh!
There was an error while loading. Please reload this page.