Skip to content

Commit b5233d2

Browse files
committed
docs(telemetry): describe number, line size and message limits
Document the Decimal rules, including negative zero, the DynamoDB string fallback, the line limit of LambdaLogExporter, the message cap, the cause truncation step, and that Python records have no cause. State that WARN filtering applies only with the JSON log format, and which envelope-shaped results are decoded or passed through.
1 parent a0d1c85 commit b5233d2

1 file changed

Lines changed: 42 additions & 15 deletions

File tree

  • packages/aws-durable-execution-sdk-python-telemetry-exporter

‎packages/aws-durable-execution-sdk-python-telemetry-exporter/README.md‎

Lines changed: 42 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -91,12 +91,17 @@ double-encode output that is already JSON (see
9191
[Log and monitor Python Lambda functions](https://docs.aws.amazon.com/lambda/latest/dg/python-logging.html)
9292
and [Lambda log formats](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs-logformat.html)).
9393

94-
`max_record_size_bytes` (default 256 KB) limits the record in `message`. The
95-
JSON escaping of `message` can make the line up to about 2x the record; a
96-
CloudWatch Logs event can be up to 1 MB
94+
`max_record_size_bytes` (default 256,000 bytes) limits the printed line: the
95+
envelope, including the JSON escaping of `message`, in UTF-8 bytes without the
96+
trailing newline. Truncation measures that line in one pass. A CloudWatch Logs
97+
event can be up to 1 MB
9798
([PutLogEvents](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html)).
98-
The lines have `level` `INFO`, so Lambda drops them when the function's
99-
application log level is `WARN` or above.
99+
U+2028 and U+2029 are written as `\u2028` and `\u2029`, so a tool that treats
100+
them as line breaks still sees one line per record.
101+
102+
The lines have `level` `INFO`. With the JSON log format and an application log
103+
level of `WARN` or above, Lambda drops them. With the Text log format, Lambda
104+
does not filter them.
100105

101106
### CloudWatchLogsExporter
102107

@@ -113,6 +118,9 @@ CloudWatchLogsExporter(log_group_name="/custom/durable-execution-telemetry")
113118
`PutItem` keyed by `partition_key` (default `pk`) = `executionArn`. With the
114119
default `sort_key="sk"` (= `emittedAt`) every export adds an item; pass
115120
`sort_key=None` for a key-only table that upserts. IAM: `dynamodb:PutItem`.
121+
A number DynamoDB cannot store (more than 38 significant digits, a magnitude
122+
outside 1E-130 to 9.99...E+125, NaN or Infinity) is written as a string
123+
attribute holding its JSON text, so it never drops the item.
116124

117125
```python
118126
DynamoDBExporter(table_name="durable-execution-telemetry")
@@ -257,9 +265,20 @@ has the same shape, recursively, and is present only when the SDK provides a
257265
cause.
258266

259267
In Python, the plugin hooks carry only an `ErrorObject` (type, message, data,
260-
stack trace) with no cause, so Python records have `name` and `message` and
261-
never have `cause`. The exporter does not build a cause from other operations
262-
or from `ErrorData`. `include_errors=False` removes operation errors.
268+
stack trace) with no cause, so no error in a Python record has a `cause`: not
269+
the execution `error`, not an operation error. The exporter does not build a
270+
cause from other operations or from `ErrorData`. `include_errors=False`
271+
removes operation errors.
272+
273+
Every `message` is capped at 8,192 characters. A longer message keeps its first
274+
8,192 characters followed by `…`.
275+
276+
When a record is over its exporter's size limit, truncation first removes every
277+
`cause` and sets `droppedCauses: true`, then runs its usual steps (operation
278+
results, then whole operations, then input, then output). Python records have
279+
no cause, so the first step never changes them. With the message cap, a record
280+
whose operations, input and output are dropped fits any limit of 64 KB or
281+
more.
263282

264283
Names are each language's own and differ across SDKs. For example, a step that
265284
throws `PaymentError` can be reported as `StepError` in JS,
@@ -275,21 +294,29 @@ default serializer's (`ExtendedTypeSerDes`) envelope, a JSON object whose keys
275294
are exactly `t` and `v` with a string `t`, it is decoded with that serializer
276295
and converted to plain JSON:
277296

278-
- `Decimal`: an integer when it is integral and `adjusted() < 300`; otherwise a
279-
number when the float round-trips exactly; otherwise `str(value)` (for
280-
example `"12345678901234567.89"` or `"1E+5000"`). NaN and Infinity become
281-
`null`.
297+
- `Decimal`: an integer when it is integral and its absolute value is at most
298+
`2**53 - 1` (9007199254740991), the largest integer a JavaScript reader parses
299+
exactly. Otherwise a float when the float round-trips exactly and is within
300+
DynamoDB's number range (at most 38 significant digits, magnitude 1E-130 to
301+
9.99...E+125). Otherwise `str(value)`, for example `"9007199254740993"`,
302+
`"12345678901234567.89"` or `"1E+299"`. NaN and Infinity become `null`.
303+
Negative zero (`Decimal("-0")`, `Decimal("-0.0")`) is integral, so it becomes
304+
`0` and loses its sign.
282305
- `datetime` and `date`: `isoformat()`. `UUID`: string. `bytes`: base64.
283306
`tuple`: list. `BatchResult`: `to_dict()`.
284307

285308
Any other result is parsed as JSON, or kept as the raw string when it is not
286-
JSON. A result that fails to decode keeps its parsed or raw value, and the rest
287-
of the record is still emitted.
309+
JSON. In parsed JSON, NaN and Infinity become `null`; nothing else changes. A
310+
result that fails to decode keeps its parsed or raw value, and the rest of the
311+
record is still emitted.
288312

289313
The hooks do not say which serializer wrote a result, so the envelope is
290314
recognized by shape alone. A result written by `JsonSerDes` or a custom
291315
`SerDes` that has this exact shape is decoded too: `{"t": "s", "v": "x"}` is
292-
reported as `"x"`.
316+
reported as `"x"`. Only the core serializer's known tags are decoded, and the
317+
core may coerce `v` while decoding: `{"t": "b", "v": "yes"}` is reported as
318+
`true`. An unknown tag, such as `{"t": "temperature", "v": 21.5}`, fails to
319+
decode and passes through unchanged.
293320

294321
## Requirements
295322

0 commit comments

Comments
 (0)