Repository navigation
docs(csharp): document the .NET FileSystem serializer (#2540) - #280
GarrettBeatty wants to merge 4 commits into
Conversation
The .NET SDK now supports an optional per-operation `Serializer` (an ILambdaSerializer) on StepConfig, CallbackConfig, InvokeConfig, WaitForConditionConfig<TState>, and ChildContextConfig; when null the globally-registered serializer is used. Update the C# tabs on the serialization page (Default serialization, SerDes interface, StepConfig, CallbackConfig) and the StepConfigExample/CallbackConfigExample examples. Map/Parallel C# tabs left as-is pending two-level (item + whole-result) support.
MapConfig<TItem> and ParallelConfig now expose ItemSerializer for each item/branch result; the aggregated batch envelope is SDK-internal and not user-serialized (no whole-result serializer). Update the Map/Parallel C# tab and MapConfigExample.
Fill in the C# tabs of the FileSystem serdes section: walkthrough, constructor signature, inner-serializer/compression, storage modes (Always/Overflow), and path encoding (Uri/Hash), with matching csharp example snippets. Note preview/masking and an execution-wide default serializer as planned follow-ups. Documents aws/aws-lambda-dotnet#2558.
GitHub Actions
Inactive
GitHub Actions
Failure
GitHub Actions
Failure
Document the new FileSystemSerializer(basePath, storageMode?, pathEncoding?) constructor that reuses the host-boundary ILambdaSerializer as the inner (bound by the durable runtime when used through a per-operation slot), and note that an explicit inner wins and that using it with no bound inner throws.
GitHub Actions
Inactive
GitHub Actions
Failure
GitHub Actions
Failure
GitHub Actions
Inactive
GitHub Actions
Inactive
GitHub Actions
Failure
GitHub Actions
Failure
Document missing operation config signatures, incremental parallel branches, custom subtype rules, terminal serializer failures, and local checkpoint-token omission for Core 2.2.0 and Testing 1.1.0. Leave the serialization guide and its examples to PRs #279 and #280. Keep shared prose and other language updates in PR #300. All remaining Markdown changes are inside C# tabs; no file overlaps #279 or #280. Validate formatting, spelling, snippets, vendored assets, rendered links, and a clean Zensical build. Retained examples match the previously compiled and runtime-checked versions.
zhongkechen
left a comment
zhongkechen
left a comment
There was a problem hiding this comment.
Carrying over 11 outstanding .NET serialization review comments from #300 to #280. Each inline thread links to the original feedback and the follow-up confirming it remains unaddressed here. These corrections remain open in #280; resolving their source threads in #300 records the transfer, not a completed fix.
| A `FileSystemSerializer` built without an inner serializer only resolves the global | ||
| serializer when it is used through a per-operation slot (for example | ||
| `StepConfig.Serializer`). Using it as a plain `ILambdaSerializer` with no bound inner — | ||
| for instance as the host-boundary serializer — throws `InvalidOperationException`. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by SilanHe, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
C# failure mode is half right.
NotSupportedExceptionis thrown when used as a plainILambdaSerializer.InvalidOperationExceptionis thrown only when the inner serializer is missing.
| // The inner serializer controls the on-the-wire format. Wrap one to compress results before | ||
| // they are written to the filesystem — FileSystemSerializer just handles storage. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by SilanHe, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
Four em-dashes, plus long comments that repeat the page prose. Both break the authoring guide.
The current version still has an em dash and a prose-style explanatory comment at lines 6–7; the count in the original feedback describes the version reviewed in #300.
| The default is the single `ILambdaSerializer` registered at the host boundary — the SDK | ||
| uses it for every durable operation result and for the handler's return value. An | ||
| optional per-operation override is available (see | ||
| [Custom SerDes on durable operations](#custom-serdes-on-durable-operations)): set | ||
| `Serializer` on `StepConfig`, `CallbackConfig`, `InvokeConfig`, | ||
| `WaitForConditionConfig<TState>`, or `ChildContextConfig` to use a different | ||
| `ILambdaSerializer` for that one operation. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
The C# tabs on this page state one fact five times: the registered serializer is the default, and a per-operation serializer overrides it. The statements are at lines 91-94, 132-138, 231-234, 247-248, and 331-334. Could the page state it once, here, and keep the later tabs to what each section adds? This list also leaves out
ItemSerializeronMapConfig<TItem>andParallelConfig, and the per-branch serializer onIDurableParallel.Branch.
| .NET has no separate per-operation SerDes interface — it reuses the standard | ||
| `ILambdaSerializer` contract. Serialization defaults to the single `ILambdaSerializer` | ||
| registered on `ILambdaContext.Serializer`, and an individual operation can override it | ||
| by setting `Serializer` (an `ILambdaSerializer`) on that operation's config. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
Core 2.0.0 added
IDurableResultSerializer, an optional interface for per-operation serializers. ItsSerialize<T>andDeserialize<T>overloads receive aDurableSerializationContext, which hasEntityIdandDurableExecutionArnproperties. The runtime calls these overloads when the configured serializer implements the interface.FileSystemSerializerimplements it. So "no separate per-operation SerDes interface" is not accurate for 2.x. Could this tab documentIDurableResultSerializer, the way the Java tab now documentsSerDesContext?
| [Custom SerDes on durable operations](#custom-serdes-on-durable-operations)): set | ||
| `Serializer` on `StepConfig`, `CallbackConfig`, `InvokeConfig`, | ||
| `WaitForConditionConfig<TState>`, or `ChildContextConfig` to use a different | ||
| `ILambdaSerializer` for that one operation. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
This tab now says a per-operation override affects only the selected operation. The C# tab of "Custom SerDes example" (lines 284-285) still says "The custom serializer applies to every durable operation result, not a single operation." With
StepConfig.Serializerand the other per-operation properties, that sentence is no longer true. Could that tab change in this PR too, so the page gives one answer?
In this PR, the conflicting "Custom SerDes example" wording is at lines 215–216 of this file; this thread is anchored to the changed per-operation guidance.
| Set `CallbackConfig.Serializer` to deserialize the callback payload with a specific | ||
| `ILambdaSerializer`. When it is `null` (default), the payload the external system | ||
| delivers is deserialized with the `ILambdaSerializer` registered at the host boundary. | ||
| Only the deserialize path is used for callbacks. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
This also uses passive voice. How about this?
Set
CallbackConfig.Serializerto deserialize the callback payload with a specificILambdaSerializer. When it isnull, the SDK deserializes the payload with theILambdaSerializerregistered at the host boundary. The SDK never serializes a callback result, so it calls onlyDeserialize.
| Set `MapConfig<TItem>.ItemSerializer` (and `ParallelConfig.ItemSerializer`) to serialize | ||
| each item / branch **result** with a specific `ILambdaSerializer`. When `null` (default), | ||
| item results use the `ILambdaSerializer` registered at the host boundary. There is no | ||
| separate whole-result serializer: the aggregated batch envelope (per-item statuses and | ||
| completion reason) is an SDK-internal, source-generated structure and is not | ||
| user-serialized — only the per-item results are. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
The last sentence here chains two colons, uses passive voice, and names an implementation detail ("source-generated"). How about this?
Set
MapConfig<TItem>.ItemSerializerorParallelConfig.ItemSerializerto serialize each item or branch result with a specificILambdaSerializer. When it isnull, item results use the serializer registered at the host boundary. The SDK writes the batch envelope (item statuses and the completion reason) itself, so there is no whole-result serializer.
| `FileSystemStorageMode.Always` (default) writes every result to a file; the checkpoint | ||
| holds only the pointer. `FileSystemStorageMode.Overflow` keeps the result inline until it | ||
| would exceed the durable execution checkpoint size limit, then spills to a file. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
In Overflow mode, the .NET serializer base64-encodes the serialized bytes into a
{"data": ...}envelope. It then compares the envelope's size with 255 KiB. Base64 adds a third to the size. I tested 2.2.0: a 190 KiB string stayed inline, and a 192 KiB string went to a file. So inline storage ends near 191 KiB of serialized bytes, not at the checkpoint limit. The comment inFileSystemSerdesOverflow.cs(lines 12-13) also says "~256 KB". Could both state the 255 KiB envelope rule, the way the Java tab states its own?
| Not yet supported in the .NET SDK. Set `FileSystemSerializer` per operation via | ||
| `StepConfig.Serializer` (or `ItemSerializer` for Map/Parallel). An execution-wide default | ||
| serializer hook is planned as a follow-up; until then, the single `ILambdaSerializer` you | ||
| register at the host boundary applies to any operation that does not set its own. |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
"Not yet supported" and "is not supported" say the same thing, and "yet" promises future work. The Java tab gives the reason, and the same reason applies here:
FileSystemSerializerthrowsNotSupportedExceptionat the host boundary. How about this?The FileSystem serializer cannot be the handler-wide default, because the host-boundary serializer also handles Lambda input and output, which have no operation identity. Set it per operation through
StepConfig.SerializerorItemSerializer.
| var config = new CallbackConfig | ||
| { | ||
| Timeout = TimeSpan.FromHours(1), | ||
| Serializer = new DefaultLambdaJsonSerializer(), |
There was a problem hiding this comment.
Carried over from the original review comment on #300 by yaythomas, with the follow-up confirming it remains unaddressed in #280. Please track the correction here.
Original feedback (line references refer to the version reviewed in #300):
DefaultLambdaJsonSerializeris also the usual host-boundary serializer, so this override looks the same as the default. The Step and Map examples useCamelCaseLambdaJsonSerializer. Could this one use it too?
Summary
Fills in the C# tabs of the existing FileSystem serdes section of the serialization reference, documenting the new .NET
FileSystemSerializer. Companion to the SDK PR aws/aws-lambda-dotnet#2558.What changed
C# tabs (previously "Not available") now covered in
docs/sdk-reference/state/serialization.md:FileSystemSerializerwrapping an innerILambdaSerializer, passed viaStepConfig.Serializer(and Child/WaitForCondition/Map/Parallel).inner,basePath,storageMode,pathEncoding.AlwaysvsOverflow.UrivsHash.New C# example snippets under
examples/csharp/sdk-reference/serialization/(FileSystemSerdesWalkthrough / Signature / Compression / Overflow / PathEncoding).Noted as follow-ups (not yet in the .NET SDK)