feat(sdk)!: PlotMarkers builtin + unify markers/transforms into pj.data_processors.v1 (kind discriminator)#135
Open
Alvvalencia wants to merge 14 commits into
Open
feat(sdk)!: PlotMarkers builtin + unify markers/transforms into pj.data_processors.v1 (kind discriminator)#135Alvvalencia wants to merge 14 commits into
Alvvalencia wants to merge 14 commits into
Conversation
New SDK builtin type for time-series plot markers (findings), the foundation of the anomaly-detection pipeline feature. - PlotMarkers (kPlotMarkers = 18): homogeneous, id-less marker records (Region / Event / ValueBand / Label) with shared semantic fields (status, severity, category, label, description, color, metadata) + a PlotMarkers list container. No id/source/scope by design — identity is owned by the host marker store; location carries provenance/reach. - Hand-written PJ.PlotMarkers wire codec following the image_annotations_codec idiom; PlotMarkers.proto contract. - Registered in C++ + C-ABI builtin enums, name()/parse()/typeOf(), and the ABI layout sentinel (static_assert id == 18). - Round-trip codec tests; full pj_base suite green (32/32) under ASAN. - Design (use-cases, architecture) + wire-format docs. ColorRGBA reused via image_annotations.hpp per the existing house convention (mesh3d / scene_entities do the same). abi/baseline.abi refresh + SDK MINOR version bump deferred to the release-cut step (additions-only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plugin-facing contract for plot markers, mirroring the object
read/write host pattern.
- C vtable PJ_marker_store_host_vtable_t + fat pointer + an owning
PJ_marker_query_handle_t (plugin_data_api.h): add / remove / query /
query_bytes / release_query. Markers cross as serialized PJ.PlotMarkers
bytes; ids cross as uint64.
- C++ MarkerStoreHostView (plugin_data_api.hpp) wrapping the codec so
callers work in typed sdk::PlotMarker; add(single|batch) -> ids,
remove(id), query -> [{id, marker}]. + MarkerEntry.
- MarkerStoreService trait (service_traits.hpp), "pj.marker_store.v1".
ABI is additive (new service); abi/baseline.abi refresh deferred to the
release-cut step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ox slot Markers become a builtin object published to the ObjectStore via the generic object-write surface; the producer republishes the whole PlotMarkers set (last-writer-publish). The dedicated pj.marker_store.v1 service (vtable, MarkerStoreHostView, service trait) is removed. Add a tail-appended, idempotent toolbox-host slot register_object_topic_on_dataset(DatasetId, ...) so a toolbox can annotate an existing dataset (the object-write path was previously source-scoped). Add marker object-topic naming helpers (markerObjectTopicName / kGlobalMarkerTopic) to plot_markers.hpp. Update the ABI layout sentinel (88->96) and the two mock vtables. The PlotMarkers type + codec are unchanged. Docs (plot_markers_*) updated to the ObjectStore + republish model. MINOR bump + abi/baseline.abi refresh deferred to release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…retention toolbox ABI slots Dialog protocol: add the MarkerTimeline custom widget (TimelineMark setters/event/parser/callback) for an editable multi-marker strip, with a single shared TimelineMark<->JSON codec used by all four sites. Toolbox host ABI (tail-appended, struct_size-gated): lookup_topic_on_dataset on the object-read vtable (dataset-scoped topic resolution; the name-only lookup is ambiguous across datasets) and set_object_topic_retention on the write vtable (keep-latest-N so a republishing producer's snapshots don't accumulate). Toolbox-host vtable 96->104; sentinels + mock vtables updated. Add sdk::markerSeriesKey (single-slash join tolerating a leading-'/' field) so producer and overlay build the per-series key identically. Doc refresh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Additive dialog-protocol surface (ABI-safe, no version bump): - setSaveFilePicker / isSaveFilePicker — a native "Save As" picker (getSaveFileName) so a not-yet-existing file can be created; reported via the existing onFileSelected. - ChartMarker + setChartMarkers / chartMarkers — overlay markers (events/regions/ value-bands) on a chart preview, alongside setChartSeries. - Fix a stale comment that referenced QLineSeries (the preview is Qwt, not Qt Charts). Cleanup: decodeMarker now drives the shared builtin_wire::parseFields loop like the sibling codecs, replacing the hand-rolled continue/break/skip control flow. Behavior and wire format are unchanged (codec round-trip tests pass). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RcygzPWUrHf9jHGGwzEzZo
Reconcile builtin-object ABI: VoxelGrid keeps slot 18 (released in 0.10.0), PlotMarkers moves to slot 19. SDK 0.8.0 -> 0.10.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeoYQHmF6ixvYv9KXXgyVh
Adding the PlotMarkers canonical builtin (slot 19) is a MINOR addition over main's 0.11.0, so the merged SDK is a distinct 0.12.0 — not the upstream 0.11.0 (which lacks PlotMarkers). Keeps version identity clean for downstream pins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeoYQHmF6ixvYv9KXXgyVh
… ephemeral preview)
The whole-series analog of pj.data_processors.v1: a plugin submits a marker
generator by data (script + input series + an output marker-topic + params) and
the HOST runs it over the whole series and publishes the resulting PlotMarkers to
the ObjectStore. Nothing executable crosses the boundary.
- PJ_markers_host_vtable_t (plugin_data_api.h): create/remove/list/config +
tail-appended set_marker_preview / clear_marker_preview for a live, ephemeral
(non-persisted) preview generator read back through the object surface.
- MarkersHostView (sdk/plugin_data_api.hpp) + MarkersHostService trait.
- params_json {"scope":"all"} makes a global generator publish across every
dataset; absent = the active dataset only.
- markers_api_test.cpp: fake-host ABI test (forwarding, count-then-fill,
borrowed-string lifetime, binary-safe payload, preview slots).
Single output_marker_topic (not an outputs[] array): a generator writes exactly
one marker topic (global or per-series).
NOTE: MINOR version bump (conanfile.py / PJ_PACKAGE_VERSION / recipe.yaml) and the
abidiff baseline refresh are pending pre-merge — this commit adds API on a feature
branch (WIP) and does not bump the release version yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QsY1SE2yBsX2qXvkgEnX8U
Bring v0.11.0 SDK work (voxel grid, data-processors API, RangeSlider markers) into the markers feature branch. Resolved widget_data.hpp additively: kept the feature's ChartMarker/TimelineMark + MarkerTimeline setters AND main's RangeSliderMarker + setRangeSliderMarkers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rh6GtzBNRVXXBUhFWhXEam
Unify the whole-series host-driven service under one SDK contract,
PJ_generators_host_vtable_t / GeneratorsHostService ("pj.generators.v1"),
with a string `kind` discriminator:
- kind="markers" (objects -> ObjectStore) is implemented; shared `language`
param, compile-only validate_script, ephemeral-preview flag, and out_topics
return apply across kinds.
- kind="transform" (per-sample timeseries -> DerivedEngine) is RESERVED, not
implemented; its end-state home is decided when the transform-editor work merges.
Removes PJ_markers_host_vtable_t / MarkersHostService (the old pj.markers.v1) and
markers_api_test; adds generators_api_test. A speculative kind="field" that briefly
existed on this branch was dropped (zero consumers, violated the object/timeseries
engine split); it changed no ABI struct since `kind` is a runtime string.
Versioning: API removal, normally MAJOR. Ships as 0.13.0 because no public tag ever
carried pj.markers.v1 (no released plugin breaks). The first public release carrying
pj.generators.v1 MUST be tagged 1.0.0 -- recorded in CHANGELOG.md + a comment by
`version` in conanfile.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013N1nzqco4Y9AQhnQgXo5tQ
…ta_processors.v1 Brings origin/main (#134 data-processors script API) into feature/plot-markers and collapses the two parallel whole-series host services into ONE: Pablo's `pj.data_processors.v1`, now carrying a string `kind` discriminator so a plugin chooses `markers`/`transform` (and future engines) from the SAME `create` call. SDK changes: - Generalized `PJ_data_processors_host_vtable_t.create_data_processor` with kind/language/flags/out_topics; ephemeral preview is now the PJ_DATA_PROCESSOR_FLAG_EPHEMERAL flag (removed the separate ephemeral slot); validate_data_processor_script gains `kind`. - Removed the interim `pj.generators.v1` entirely (vtable, host_t, GeneratorsHostService, GeneratorsHostView). - DataProcessorsHostView = unified `create(kind,…)` + thin shims createTransform/createEphemeralTransform (Status, signatures unchanged) / createMarkers. - Folded generators_api_test.cpp into data_processors_api_test.cpp (transform+markers). - CHANGELOG/conanfile reworded; stays 0.13.0 (no public tag carried the service yet; first public unified release MUST be 1.0.0). Builds green (RelWithDebInfo + debug/ASAN); 48/48 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHPK8VduUcoenjL8AtoSVd
Alvvalencia
changed the title
Applies the pre-commit clang-format pass that the merge commit (bf7eb0f) skipped via --no-verify. Formatting only — no semantic change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHPK8VduUcoenjL8AtoSVd
Completes the formatting pass over the full PR diff (PlotMarkers.proto column alignment + dialog_plugin_typed.hpp signature wrap) that the narrowly-scoped previous pass missed. Formatting only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHPK8VduUcoenjL8AtoSVd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Unifies the two parallel whole-series host services into one —
pj.data_processors.v1—with a string
kinddiscriminator, so a plugin choosesmarkers/transform(and futureengines) from the same
createcall. Also lands the canonical PlotMarkers builtinobject + codec.
This supersedes the interim
pj.generators.v1that earlier revisions of this branch added:that service is removed; its markers backend now lives under
kind="markers"on Pablo'sexisting
pj.data_processors.v1.ABI / API changes
PJ_data_processors_host_vtable_t.create_data_processorgeneralized withkind/language/flags/out_topics(the resolved sink names).PJ_DATA_PROCESSOR_FLAG_EPHEMERALflag oncreate_data_processor(the separate
create_data_processor_ephemeralslot is removed).validate_data_processor_scriptgains a leadingkindargument.pj.generators.v1entirely (vtable, host_t,GeneratorsHostService,GeneratorsHostView).DataProcessorsHostView= unifiedcreate(kind, …)+ thin convenience shimscreateTransform/createEphemeralTransform(Status return, signatures unchanged) /createMarkers.generators_api_test.cppfolded intodata_processors_api_test.cpp(transform + markers).Versioning
Stays
0.13.0: no public tag has carriedpj.data_processors.v1yet, so no released pluginbreaks. The first public release that carries the unified
pj.data_processors.v1must betagged
1.0.0(pre-1.0 break rule).This changes the
create_data_processor/validate_data_processor_scriptABI signatures, sothe host that IMPLEMENTS the service (
DataProcessorsRuntimeHost, PJ4 #234) must adopt thenew vtable fill. Callers keep working via the C++ shims; only
validateScriptcall-sitesgain a leading
kind("transform"). Merge this SDK first (it is the bottom layer); PJ4 / pluginsthen bump to it. The kind="transform" engine routing is owned by the transform-editor PRs.
Verification
./build.sh(RelWithDebInfo) +./build.sh --debug(ASAN) green;./test.sh48/48 pass.unified service (host + plugin both built against this ABI).
🤖 Generated with Claude Code