-
Notifications
You must be signed in to change notification settings - Fork 782
workflow history propagation docs #5153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: v1.18
Are you sure you want to change the base?
Changes from 5 commits
a8c2450
5e033dd
bbe7f12
e8848cf
4fea981
b1fe587
45a128a
4d9e8ea
14d79c1
2b6bd66
5f99352
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -203,6 +203,17 @@ Workflows can also wait for multiple external event signals of the same name, in | |||||
|
|
||||||
| Learn more about [external system interaction.]({{% ref "workflow-patterns.md#external-system-interaction" %}}) | ||||||
|
|
||||||
| ## Workflow history propagation | ||||||
|
|
||||||
| A parent workflow can opt to share its execution history with downstream child workflows and activities. Two scopes are available: | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| - **Lineage** — caller's events plus the full ancestor chain (chain-of-custody) | ||||||
| - **Own history** — caller's events only (a trust boundary, ancestral lineage dropped) | ||||||
|
|
||||||
| The receiving workflow/activity reads its inherited history via `ctx.GetPropagatedHistory()` and can verify what happened upstream — useful for fraud checks, compliance gates, and long-running AI agents that need context across hops. | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| For details, the scope comparison, and code examples, see [History propagation]({{< ref workflow-history-propagation.md >}}). | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| ## Purging | ||||||
|
|
||||||
| Workflow state can be purged from a state store, purging all its history and removing all metadata related to a specific workflow instance. The purge capability is used for workflows that have run to a `COMPLETED`, `FAILED`, or `TERMINATED` state. | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,147 @@ | ||||||
| --- | ||||||
| type: docs | ||||||
| title: "Workflow history propagation" | ||||||
| linkTitle: "History propagation" | ||||||
| weight: 2500 | ||||||
| description: "Share a parent workflow's execution history with child workflows and activities for chain-of-custody, audit, and AI-agent context" | ||||||
| --- | ||||||
|
|
||||||
| By default, a child workflow or activity only sees the input it was scheduled with. The parent's execution history — what activities ran, what child workflows it spawned, what events it observed — is invisible. Workflow history propagation lets a parent opt-in to share that history with the work it schedules. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add a some scenarios into this article as well, in case most likely, that some comes straight to this article from a search (like I suggest in the overview)
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add a some scenarios into this article as well, in case most likely, that some comes straight to this article from a search (like I suggest in the overview)
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| This is useful for: | ||||||
|
|
||||||
| - **Chain-of-custody and audit** — a fraud check, compliance gate, or settlement step can verify what already happened upstream rather than trusting the caller's claims. | ||||||
| - **Cross-app workflows** — a downstream app, often owned by a different team, can confirm an upstream activity ran without an out-of-band query. | ||||||
|
cicoyle marked this conversation as resolved.
|
||||||
| - **AI agents and long-running orchestrations** — agents that loop with `ContinueAsNew` or fan out to sub-agents can carry context (tool calls, model outputs, intermediate decisions) forward without re-prompting from scratch. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| ## Propagation scopes | ||||||
|
|
||||||
| Two scopes are exposed. The default is no propagation. | ||||||
|
|
||||||
| | Option helper | Scope value on `propagatedHistory.Scope()` | What gets sent | When to use it | | ||||||
| | ------------- | --------------------------- | -------------- | -------------- | | ||||||
| | `workflow.PropagateLineage()` | `LINEAGE` (`HISTORY_PROPAGATION_SCOPE_LINEAGE`) | Caller's own events plus the full ancestor chain inherited from the caller's own parent | Full chain-of-custody, downstream wants to see everything that happened before, all the way to the root | | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| | `workflow.PropagateOwnHistory()` | `OWN_HISTORY` (`HISTORY_PROPAGATION_SCOPE_OWN_HISTORY`) | Caller's own events only, ancestral lineage is dropped | Trust boundary — the caller is willing to vouch for what *it* did but the receiver shouldn't see further upstream | | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| The helpers (`PropagateLineage` / `PropagateOwnHistory`) are what you pass when scheduling. The scope values (`LINEAGE` / `OWN_HISTORY`) are what `propagatedHistory.Scope()` returns on the receiver — they're the same thing. | ||||||
|
|
||||||
| `PropagateOwnHistory` is the trust boundary: choosing it tells the runtime to stop forwarding any history the caller itself received. This is the right choice when the receiver is a less-trusted app, a third party, or operates under different compliance rules. | ||||||
|
|
||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any more guidance we can provide here? For example PropagateOwnHistory only has to be used if you have no child workflows, but this is very unlikely typically. |
||||||
| ## Setting it up | ||||||
|
|
||||||
| A parent opts a single `CallActivity` or `CallChildWorkflow` into propagation via the `WithHistoryPropagation` option. Other activity / child calls in the same workflow are unaffected. | ||||||
|
|
||||||
| ```go | ||||||
|
cicoyle marked this conversation as resolved.
|
||||||
| import ( | ||||||
| "github.com/dapr/durabletask-go/workflow" | ||||||
| ) | ||||||
|
|
||||||
| func MerchantCheckout(ctx *workflow.WorkflowContext) (any, error) { | ||||||
| // Activity does NOT receive propagated history (default). | ||||||
| if err := ctx.CallActivity("ValidateMerchant").Await(nil); err != nil { | ||||||
| return nil, err | ||||||
| } | ||||||
|
|
||||||
| // Child workflow DOES receive parent's history (LINEAGE). | ||||||
| var result string | ||||||
| if err := ctx.CallChildWorkflow("ProcessPayment", | ||||||
| workflow.WithHistoryPropagation(workflow.PropagateLineage()), | ||||||
| ).Await(&result); err != nil { | ||||||
| return nil, err | ||||||
| } | ||||||
| return result, nil | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ## Receiving propagated history | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| Inside a child workflow or activity, call `ctx.GetPropagatedHistory()`. It returns the propagated history if the caller opted in, or `nil` if it didn't. | ||||||
|
|
||||||
| ```go | ||||||
| import ( | ||||||
| "fmt" | ||||||
|
|
||||||
| "github.com/dapr/durabletask-go/workflow" | ||||||
| ) | ||||||
|
|
||||||
| func FraudDetection(ctx *workflow.WorkflowContext) (any, error) { | ||||||
| propagatedHistory := ctx.GetPropagatedHistory() | ||||||
| if propagatedHistory == nil { | ||||||
| return "no upstream history", nil | ||||||
| } | ||||||
|
|
||||||
| // The history exposes both raw events and per-app/per-instance chunks. | ||||||
| fmt.Printf("scope: %s, %d events from apps %v\n", | ||||||
| propagatedHistory.Scope(), len(propagatedHistory.Events()), propagatedHistory.GetAppIDs()) | ||||||
|
|
||||||
| // Drill into a specific upstream workflow's activities. The names here | ||||||
| // match the parent workflow and activity shown above (registered as | ||||||
| // "MerchantCheckout" and "ValidateMerchant"). | ||||||
| merchantWf, err := propagatedHistory.GetWorkflowByName("MerchantCheckout") | ||||||
| if err != nil { | ||||||
| return nil, fmt.Errorf("expected MerchantCheckout in propagated history: %w", err) | ||||||
| } | ||||||
|
cicoyle marked this conversation as resolved.
|
||||||
| validation, err := merchantWf.GetActivityByName("ValidateMerchant") | ||||||
| if err != nil { | ||||||
| return nil, fmt.Errorf("expected ValidateMerchant in propagated history: %w", err) | ||||||
| } | ||||||
| if !validation.Completed { | ||||||
| return "merchant validation didn't complete, rejecting", nil | ||||||
| } | ||||||
|
|
||||||
| return "approved", nil | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| The returned `PropagatedHistory` carries: | ||||||
|
|
||||||
| - `Events()` — the flat list of upstream `HistoryEvent`s in order | ||||||
| - `Scope()` — which scope the parent chose (`OWN_HISTORY` or `LINEAGE`) | ||||||
| - `GetWorkflows()` — per-workflow chunks (each is a `WorkflowResult` tagged with `appId`, `workflowName`, `instanceId`, `startEventIndex`, `eventCount`) | ||||||
| - `GetAppIDs()` — deduplicated list of every app that contributed events | ||||||
| - Top-level filters: `GetWorkflowByName(name)`, `GetWorkflowsByName(name)`, `GetEventsByAppID(appID)`, `GetEventsByInstanceID(id)`, `GetEventsByWorkflowName(name)` | ||||||
|
|
||||||
| Each `WorkflowResult` further exposes: | ||||||
|
|
||||||
| - `GetActivityByName(name)` / `GetActivitiesByName(name)` — find activities the upstream workflow ran. Returns an `ActivityResult` with `Completed bool` and the result payload. | ||||||
| - `GetChildWorkflowByName(name)` / `GetChildWorkflowsByName(name)` — find child workflows the upstream workflow scheduled. | ||||||
|
|
||||||
| The singular `*ByName` methods return `ErrPropagationNotFound` when no match exists, the plural variants return an empty slice. | ||||||
|
|
||||||
| ## Cross-app workflows | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| When a parent workflow in App A calls a child workflow in App B, the propagated chunks travel between sidecars. Two security knobs apply: | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| - **mTLS** — when Dapr is deployed with mTLS (the default for Helm / `dapr init -k`), inter-sidecar traffic is encrypted and authenticated. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does mTLS have to do with propagation? The statement below just say communication is encrypted.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. would remove this |
||||||
| - **`WorkflowHistorySigning`** — an opt-in `Configuration` feature that signs each propagated chunk with the producing app's SPIFFE identity. Receivers can verify chunks weren't tampered with after they left the producer. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Make this its own heading and then reference the article in this PR Workflow history signing.With history signing, the propagated context can be cryptographically verified. For more information see [workflow history signing]({{% ref "workflow-history-signing.md" %}} |
||||||
|
|
||||||
| If `WorkflowHistorySigning` is not enabled, daprd logs a warning per dispatch: | ||||||
|
|
||||||
| > `propagating unsigned workflow history to child workflow '...' (signing is not configured; chunks cannot be cryptographically verified by the receiver)` | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| {{% alert title="Treat unsigned chunks as untrusted" color="warning" %}} | ||||||
| Without signing, propagated chunks are functional but not cryptographically verifiable. Don't treat unsigned propagated history as authoritative for high-value decisions (payments, approvals). Enable `WorkflowHistorySigning` for production deployments that depend on chain-of-custody. | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
| {{% /alert %}} | ||||||
|
|
||||||
| ## ContinueAsNew and rerun | ||||||
|
|
||||||
| Propagated history flows correctly through both `ContinueAsNew` and rerun: | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| - A workflow that received propagated history and calls `ContinueAsNew` passes the same incoming chunks to its next generation. | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
| - Rerunning a workflow re-issues activity / child-workflow calls with the same propagation scope they were originally scheduled with — `LINEAGE` stays `LINEAGE`, `OWN_HISTORY` stays `OWN_HISTORY`. | ||||||
|
cicoyle marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| This makes long-running agents and crash-recovery scenarios behave the way you'd expect: the receiving generation/rerun sees the same history the original run did. | ||||||
|
|
||||||
| ## Next steps | ||||||
|
|
||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Discover [how to apply workflow signing]({{% ref "workflow-history-signing.md" %}} to attest the validity of a previous workflow or activity step. |
||||||
| {{< button text="Workflow patterns >>" page="workflow-patterns.md" >}} | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we add a note on payload size and limits here? I imagine that when workflow history prop is enabled the payloads increase a bit? Maybe call that out and that the default size is 4mb etc... |
||||||
|
|
||||||
| ## Related links | ||||||
|
|
||||||
| - [Workflow overview]({{< ref workflow-overview.md >}}) | ||||||
| - [Workflow features and concepts]({{< ref workflow-features-concepts.md >}}) | ||||||
| - [Workflow architecture]({{< ref workflow-architecture.md >}}) | ||||||
| - [How to author a workflow]({{< ref howto-author-workflow.md >}}) | ||||||
| - [Workflow API reference]({{< ref workflow_api.md >}}) | ||||||
| - Try out the full SDK example: | ||||||
| - [Go example](https://github.com/dapr/go-sdk/tree/main/examples/workflow-history-propagation) | ||||||
Uh oh!
There was an error while loading. Please reload this page.