Idea, not a plan. Recording a design discussion so it does not have to be re-derived. Nothing here is scheduled; no PR should close this. Came out of #96 (candidate plot types), where it was noted that several "can you plot X" requests are really "plot a computed signal".
The idea
Let a database_options entry produce a new, named Signal from an existing one — smoothing, low-pass, derivative — instead of a new plot type. A derived Signal composes with time_series, loop, psd, spectrogram and the planned histogram all at once, where a plot type buys one behaviour.
Three shapes considered
- (a) In-place transform, as a per-signal display option — a
smoothing key next to period_resampling in signals.<name>. Discarded. It silently changes the numbers under every plot type while the trace still claims to be the measurement.
- (b) A new derived Signal — the transform produces an additional Signal with a new name, groupable alongside its parent. Preferred. The clinician cannot apply it by accident: they had to name the thing.
Metadata.is_derived and Metadata.parent_signal_name already exist for this and are currently set nowhere.
- (c) Smoothing as a
trace_options rendering concern — data untouched, only the drawing smoothed. Cheap and honest about the underlying data, but risks the same misreading as (a): what is on screen is not what was measured.
Prior art already in the tree
period_resampling is exactly shape (a), shipped: configured per signal (with a numerics global default) and applied at Signal construction in signal_container.time_series_from_dataframe. It also carries the warning: spectral._framed_power refuses to compute a PSD or spectrogram on a decimated signal, because step-decimation with no anti-alias filter shows aliased energy that reads as a real rhythm. One in-place transform already forced a hand-written guard into every spectral type.
Caveats found
The one that undercuts the premise:
- (b) relocates the trap rather than removing it. A derived Signal is first-class, so it can be fed to
psd/spectrogram like a measured one — and a smoothed signal's PSD is still a lie about the roll-off, which the existing guard will not catch because it is not decimation. The transform descriptor on the Signal, and the spectral types reading it, are therefore part of this feature, not a follow-up.
- Metadata inheritance is a correctness rule, not a convenience. A derived signal built from a decimated parent that does not inherit
period_resampling silently bypasses the existing refusal. "Inherit everything" is wrong too — smoothing preserves units, a derivative does not.
Two forks to settle before building:
- Display or analysis?
extract stops at format and returns DataFrames; Signals are built afterwards — so a derived signal would appear in plots and be absent from extract_patient output. Pushing derivation into _format does not rescue it: _format is per-datasource, and a parent may live in another one. Display-only means the Signal layer and is cheap; analysis means a DataFrame-level step both pipelines call, and is a much larger build.
- Where derivation runs argues with ADR-0013. It must run after the datasource loop (a parent may be elsewhere) and before
assemble_plot_groups (so groups and derived plots can reference the new signal) — but ADR-0013 puts reference qualification inside assembly's first step, and a derived entry's own references need qualifying before that. Either qualification is hoisted into a phase both call, or the ADR is amended.
Manageable, but they will surface:
- Every derived signal gets its own subplot by default —
assemble_plot_groups gives a plot to any signal no configured group took. "Smoothed ABP over raw ABP" is therefore two config edits for one intent; wants a shorthand.
raw_name stops meaning what CONTEXT.md says it means ("its identifier in the source data"). A derived signal's raw_name is in no file — a glossary amendment, not just code.
- The xlsx half is not free. Plot types get their sheet via
SHEET_NAME + CellReader on the definition; a derived-signal section is not a plot type and inherits none of it, so it needs its own reader in database_options_xlsx.py plus the .json parity tests/unit/test_example_assets.py enforces.
- Chaining and missing parents. Smooth-then-differentiate needs dependency ordering and cycle detection; and since
wrapper.main skips a datasource that fails to load, a parent can legitimately be absent at runtime — derivation must degrade with a logged refusal like the derived plot types do, not crash.
- Annotations key to
plot_name plus {datasource_name, raw_name, display_name}, so renaming a derived signal orphans annotations on it. True of groups today, but database_options is shared and edited while annotations.json is per-patient and persistent, so churn is likelier here.
Scope discipline if it is ever built
One parent, one operation, no arithmetic between signals. Two-parent arithmetic needs a common time grid across sources with different sample rates, and resampling onto that grid reintroduces exactly the aliasing problem the spectral guard already refuses. That deserves its own decision, not a config key.
Idea, not a plan. Recording a design discussion so it does not have to be re-derived. Nothing here is scheduled; no PR should close this. Came out of #96 (candidate plot types), where it was noted that several "can you plot X" requests are really "plot a computed signal".
The idea
Let a
database_optionsentry produce a new, named Signal from an existing one — smoothing, low-pass, derivative — instead of a new plot type. A derived Signal composes withtime_series,loop,psd,spectrogramand the planned histogram all at once, where a plot type buys one behaviour.Three shapes considered
smoothingkey next toperiod_resamplinginsignals.<name>. Discarded. It silently changes the numbers under every plot type while the trace still claims to be the measurement.Metadata.is_derivedandMetadata.parent_signal_namealready exist for this and are currently set nowhere.trace_optionsrendering concern — data untouched, only the drawing smoothed. Cheap and honest about the underlying data, but risks the same misreading as (a): what is on screen is not what was measured.Prior art already in the tree
period_resamplingis exactly shape (a), shipped: configured per signal (with anumericsglobal default) and applied at Signal construction insignal_container.time_series_from_dataframe. It also carries the warning:spectral._framed_powerrefuses to compute a PSD or spectrogram on a decimated signal, because step-decimation with no anti-alias filter shows aliased energy that reads as a real rhythm. One in-place transform already forced a hand-written guard into every spectral type.Caveats found
The one that undercuts the premise:
psd/spectrogramlike a measured one — and a smoothed signal's PSD is still a lie about the roll-off, which the existing guard will not catch because it is not decimation. The transform descriptor on the Signal, and the spectral types reading it, are therefore part of this feature, not a follow-up.period_resamplingsilently bypasses the existing refusal. "Inherit everything" is wrong too — smoothing preserves units, a derivative does not.Two forks to settle before building:
extractstops atformatand returns DataFrames; Signals are built afterwards — so a derived signal would appear in plots and be absent fromextract_patientoutput. Pushing derivation into_formatdoes not rescue it:_formatis per-datasource, and a parent may live in another one. Display-only means the Signal layer and is cheap; analysis means a DataFrame-level step both pipelines call, and is a much larger build.assemble_plot_groups(so groups and derived plots can reference the new signal) — but ADR-0013 puts reference qualification inside assembly's first step, and a derived entry's own references need qualifying before that. Either qualification is hoisted into a phase both call, or the ADR is amended.Manageable, but they will surface:
assemble_plot_groupsgives a plot to any signal no configured group took. "Smoothed ABP over raw ABP" is therefore two config edits for one intent; wants a shorthand.raw_namestops meaning what CONTEXT.md says it means ("its identifier in the source data"). A derived signal's raw_name is in no file — a glossary amendment, not just code.SHEET_NAME+CellReaderon the definition; a derived-signal section is not a plot type and inherits none of it, so it needs its own reader indatabase_options_xlsx.pyplus the.jsonparitytests/unit/test_example_assets.pyenforces.wrapper.mainskips a datasource that fails to load, a parent can legitimately be absent at runtime — derivation must degrade with a logged refusal like the derived plot types do, not crash.plot_nameplus{datasource_name, raw_name, display_name}, so renaming a derived signal orphans annotations on it. True of groups today, butdatabase_optionsis shared and edited whileannotations.jsonis per-patient and persistent, so churn is likelier here.Scope discipline if it is ever built
One parent, one operation, no arithmetic between signals. Two-parent arithmetic needs a common time grid across sources with different sample rates, and resampling onto that grid reintroduces exactly the aliasing problem the spectral guard already refuses. That deserves its own decision, not a config key.