Skip to content

Add Labcyte Echo Support#1001

Draft
alexjamesgodfrey wants to merge 33 commits into
PyLabRobot:v1b1from
alexjamesgodfrey:labcyte-echo-plate-access
Draft

Add Labcyte Echo Support#1001
alexjamesgodfrey wants to merge 33 commits into
PyLabRobot:v1b1from
alexjamesgodfrey:labcyte-echo-plate-access

Conversation

@alexjamesgodfrey

@alexjamesgodfrey alexjamesgodfrey commented Apr 20, 2026

Copy link
Copy Markdown

Summary

Support for Labcyte (acq. Beckman) Echo acoustic liquid handlers under PyLabRobot's v1b1 architecture.

This started as Echo 650 only, but the branch now covers both the Echo 650 and the Echo 525 through a single Echo frontend, and has adopted the driver-architecture refactor @c-reiter and I discussed with @rickwierenga below. I've folded the two stacked follow-ups (#5 for the 525, and the driver refactor on top of it) directly into this branch so it's one coherent PR to review rather than a three-deep stack — credit to @c-reiter for the 525 work, the EchoMockServer, and the refactor commits (their authorship is preserved in the history).

What's here:

  • One Echo frontend, model as a parameterEcho(host, model="Echo 650" | "Echo 525"), backed by an ECHO_MODELS registry (the two differ only in droplet increment, 2.5 nL vs 25 nL, and the version strings the firmware advertises). Echo525 remains as a deprecation shim.
  • EchoDriver is an ABC with MedmanEchoDriver (real Medman SOAP-over-HTTP) and EchoChatterboxDriver (logs, no I/O) as sibling implementations, injected via Echo(driver=...). This is the shape @rickwierenga signed off on, and keeps the door open for a vendor-SDK driver / the iDOT later.
  • Lock/unlock, instrument info, DIO/access-state polling, and event-stream support.
  • Source/destination plate access operations and safe close-door handling, exposed through the generic plate_access capability (PlateAccess + EchoPlateAccessBackend).
  • Source plate-map upload, plate survey, saved-survey retrieval, dry-plate support, and focus calibration.
  • PLR-native transfer generation: build_echo_transfer_plan() from PLR resources/wells, higher-level transfer() / transfer_wells(), EchoPlatePosition source/destination modeling, and liquid-tracker updates (apply_volumes_to_plate()).
  • Raw DoWellTransfer execution for pre-built Echo protocol XML via do_well_transfer() + EchoTransferPrintOptions, still available underneath.
  • Per-transfer destination offsets: optional dx / dy / tag on EchoPlannedTransfer, emitted into each <wp> element (firmware slots confirmed present from a hardware Medman capture — see the offset discussion in the thread; they're zero on a plate-to-plate reformat). The default path is byte-identical to before, so nothing changes for existing callers.
  • Hardware-free testing: EchoMockServer replays captured Echo responses, so the full survey → dry → transfer path has regression coverage without an instrument.

v1b1 conformance notes

  • P-01 (driver ≠ capability method names): the EchoDriver methods that used to share names with PlateAccess operations (lock, open_source_plate, close_door, …) are renamed to vendor-endpoint verbs (lock_instrument, present_source_gripper, close_instrument_door, …), so EchoPlateAccessBackend now does a real abstract-op → vendor-verb translation instead of pass-through-by-identical-name.
  • Capability bypass: run_picklist routes its lock/unlock through self.plate_access rather than the driver. (close_door is deliberately kept on the driver inside run_picklist — the capability's close_door polls an access-state precondition the picklist context never establishes; there's a comment explaining it.)

Todo

  • Add pylabrobot.labcyte namespace
  • Add EchoDriver, EchoPlateAccessBackend, and Echo frontend
  • Implement Echo Medman SOAP-over-HTTP transport
  • Handle gzip-compressed responses and Echo request quirks
  • Add instrument setup/teardown path
  • Add GetInstrumentInfo
  • Add lock/unlock/session control
  • Add DIO/access-state polling
  • Normalize source/destination access state
  • Add source plate present/retract operations
  • Add destination plate present/retract operations
  • Add safe close_door() behavior
  • Add generic PLR plate_access capability
  • Wire Echo into PlateAccess
  • Add event stream parsing/reading support
  • Add source plate map upload
  • Add PlateSurvey
  • Add GetSurveyData
  • Add DryPlate
  • Add survey_source_plate() helper workflow
  • Add SetSurveyData
  • Add transfer-volume metadata helpers
  • Add raw DoWellTransfer execution
  • Add EchoTransferPrintOptions
  • Parse Echo transfer report XML
  • Add focused unit tests for access, survey, events, and raw transfer execution
  • Add user docs for Echo access, survey, and transfer execution
  • Run focused pytest validation
  • Run focused ruff validation
  • Add PLR-native Echo protocol generation from PLR resources/wells (build_echo_transfer_plan())
  • Add a higher-level transfer(...) API instead of requiring raw Echo protocol XML (transfer() / transfer_wells())
  • Model Echo source/destination load positions as PLR resources (EchoPlatePosition)
  • Decide how Echo transfer results update PLR liquid trackers (apply_volumes_to_plate())
  • Add Echo 525 support (single frontend, model=)
  • Refactor to EchoDriver ABC + Medman/Chatterbox siblings (per @rickwierenga)
  • Add EchoMockServer for hardware-free survey/transfer regression tests
  • Expose optional per-transfer dx / dy / tag destination offsets
  • Close v1b1 P-01 (driver ≠ capability names) + capability-bypass findings
  • Run live 650 validation of the expanded survey/transfer path (currently mock-covered via EchoMockServer, not yet re-run on the instrument)
  • Migrate the per-RPC TCP transport to plr.io
  • Integrate physical plate movement with PLR arms/workcells, end to end
  • Add an example: robot loads plate → Echo surveys/transfers → robot unloads plate
  • Split echo.py into a package (driver.py / plate_access_backend.py / echo.py / chatterbox.py / models.py) — it's over the v1b1 file-size guidance
  • Move the trigger+poll orchestration off the PlateAccess frontend into the backend (touches the shared capability — want to coordinate before changing it)
  • Unify the two <wp> offset generators (picklist.py uses a µm convention; echo.py emits raw device units)
  • Add broader docs explaining what is Echo-native vs PLR-native

Validation

  • uv run --with pytest --with pytest-timeout --with pytest-asyncio python -m pytest pylabrobot/labcyte/echo_tests.py pylabrobot/labcyte/echo525_tests.py pylabrobot/labcyte/picklist_tests.py pylabrobot/capabilities/plate_access/plate_access_tests.py
  • uv run --with ruff ruff check pylabrobot/labcyte pylabrobot/capabilities/plate_access

@alexjamesgodfrey alexjamesgodfrey changed the title Add Labcyte Echo Medman support Add Labcyte Echo Support Apr 20, 2026
@alexjamesgodfrey
alexjamesgodfrey force-pushed the labcyte-echo-plate-access branch from 41d9b6e to 5e7fa00 Compare May 6, 2026 20:38
@xaviersvk

Copy link
Copy Markdown

Hey @alexjamesgodfrey — awesome work on this, thanks for sharing it!

Quick question: in _make_transfer_protocol_xml each <wp> only carries n / dn / v
(source, dest, volume). I'm wondering if <wp> might also take a per-transfer offset /
XY coordinate that just didn't show up in your capture.

The Echo can dispense to arbitrary positions on a destination (MALDI plates, slides,
membranes — feature-based stuff), so if your Wireshark session was a normal plate-to-plate
reformat, an optional offset attribute would never have hit the wire even if the firmware
supports it. Might just be hiding rather than missing.

Could be totally wrong (haven't tested on hardware), maybe offsets only live at the
plate-type level (A1OffsetX/Y etc.). But did you ever see anything on <wp> beyond n/dn/v,
or have a capture from a non-plate destination?

Happy to help test on hardware if useful — thanks again!

c-reiter and others added 2 commits June 24, 2026 02:54
The Echo 525 speaks the identical Medman SOAP-over-HTTP protocol as the
Echo 650 (POST /Medman, gzip-compressed SOAP envelopes, same RPC method
set and DoWellTransfer <wp> layout), verified against a Wireshark capture
of a 525 (Model "Echo 525", software 2.7.3) running a HiFi PCR protocol.

The only functional difference is transfer volume granularity: the 525
dispenses in 25 nL increments (GetTransferVolIncrNl/MinimumNl == 25) vs
the 650's 2.5 nL. Rather than fork the 4.3k-line driver, this:

- parameterizes the volume increment in echo.py (backward-compatible
  defaults preserve all Echo 650 behavior) and adds a driver-factory hook
  to Echo so subclasses can inject model-specific drivers;
- adds echo525.py with Echo525 / Echo525Driver overriding only the 525
  defaults (25 nL increment, observed 2.6/2.7.3 protocol/client versions);
- adds echo525_tests.py (10 tests) pinned to the captured device traffic,
  including a byte-exact reproduction of the captured 384-well transfer.

Existing Echo 650 test suite (78 tests) still passes unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an in-process asyncio mock of the Echo Medman protocol (POST /Medman +
gzip SOAP), replaying real responses captured from the physical Echo 525, so
Echo/Echo525 can be driven end-to-end with no instrument attached. Mirrors the
MicroSpinMockServer pattern from PyLabRobot#1047.

- Replays 33 captured device responses keyed by SOAP method; the 384-well
  DoWellTransfer print-map is trimmed to 3 wells to keep the fixture ~3.5 KB.
- Models the instrument lock so motion/transfer RPCs issued without the lock
  get the Echo's real "Caller does not own the lock" fault.
- 4 new end-to-end tests: captured 525 identity (Model "Echo 525"), 25 nL
  increment read over the wire, full lock->transfer->unlock flow, lock gating.

14 Echo 525 tests pass; Echo 650 suite (78) still green.
@c-reiter

c-reiter commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

@xaviersvk - we can shed some light on this from hardware. We captured the Medman traffic of an Echo 525 running a plate-to-plate HiFi PCR reformat (~13.8k SOAP messages) while building 525 support on top of this PR (alexjamesgodfrey#5 against this branch), so we have the real <wp> elements the Echo software itself emits.

Your intuition is right: <wp> carries more than n/dn/v. Every <wp> the firmware emitted (9,360 of them) had these 11 attributes:

oid  v  n  r  c  dn  dr  dc  dx  dy  tag

So dx / dy — a per-transfer destination XY offset — are part of the protocol. In our capture they were 0 on every element, exactly as you guessed: a normal plate-to-plate reformat emits the offset slots but leaves them zero ("hiding rather than missing").

The transfer report (<w> in the DoWellTransfer response) backs up the geometry model. Each well echoes dx/dy (=0) next to the firmware's resolved absolute acoustic target gx/gy and transducer tz, e.g.:

<w n="A2" r="0" c="1" dn="A1" dr="0" dc="0" vt="150" avt="150"
   dx="0" dy="0" gx="107259" gy="56368.8" tz="18198.704"
   fld="CP-Buffer" sv="1488.689" tof="35.548" .../>

i.e. destination = dn/dr/dc + dx/dy, which the firmware resolves into absolute gx/gy. The offset clearly feeds into absolute positioning.

Caveats (where you'd still be the one to confirm on hardware): our run was plate-to-plate, so dx/dy were never non-zero anywhere in the capture. We can confirm the attributes exist and are accepted/echoed by the firmware, but not that a non-zero offset is honored, nor the exact units (likely the same device-unit space as gx/gy, ~10⁵, but that's inference). A capture from a MALDI/slide/feature-based destination would pin that down — happy to compare notes if you run one.

Practical takeaway for this PR: the minimal <wp n dn v> works because the firmware fills in r/c/dr/dc (derivable from n/dn) and defaults dx/dy/tag. So exposing per-transfer offsets later is a small change — add optional dx/dy (and tag) to _make_transfer_protocol_xml; the firmware already has the slots.

c-reiter added 3 commits June 24, 2026 04:31
- new user-guide page docs/user_guide/labcyte/echo-525.md covering the 25 nL
  increment, the Echo525 frontend, the EchoMockServer hardware-free workflow,
  and live-validation override
- link it from the Labcyte index toctree and add a note on the Echo 650 page
- add Echo525 / Echo525Driver / EchoMockServer to the labcyte API reference

The documented mock-server example runs verbatim against EchoMockServer.
Document host/port/start/stop so the mock server's public surface is fully
covered (matches the response_for docstring already present).
…tured responses

Addresses the open "document live validation for the expanded survey/transfer
path" item from PyLabRobot#1001 in a hardware-free, reproducible way: drives
survey_source_plate (SetPlateMap -> PlateSurvey -> GetSurveyData -> DryPlate)
followed by DoWellTransfer against EchoMockServer, which replays responses
captured from a physical Echo 525 running that exact path.
@c-reiter

c-reiter commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

@alexjamesgodfrey — while building Echo 525 support on top of this (alexjamesgodfrey/pylabrobot#5 against this branch) we went through the Todo checklist, and a bunch of the unchecked boxes actually look already implemented in the current head — might be worth re-ticking so the PR reads as complete:

  • Add PLR-native Echo protocol generation from PLR resources/wells → build_echo_transfer_plan() / _make_transfer_protocol_xml()
  • Add a higher-level transfer(...) API instead of raw protocol XML → transfer() / transfer_wells()
  • Model Echo source/destination load positions as PLR resources/PlateHolders → EchoPlatePosition + source_position / destination_position
  • Decide how Echo transfer results should update PLR liquid trackers → update_volume_trackers + apply_volumes_to_plate() / _apply_transfer_volume_tracking()

Still genuinely open from what we can tell: the live 650 run, deeper arm/workcell plate-movement integration, and the "Echo-native vs PLR-native" explainer doc.

On "run or document live validation for the expanded survey/transfer path": we captured the Medman traffic of a physical Echo 525 running a full survey + transfer + dry-plate sequence and built an in-process mock (EchoMockServer) that replays those real responses. It now drives the whole expanded path hardware-free in a test — survey_source_plate (SetPlateMap → PlateSurvey → GetSurveyData → DryPlate) → DoWellTransfer — and asserts the real survey well + dry mode + RPC order. Since the 525 and 650 share the Medman protocol, that mock is reusable as 650 validation too. Happy to break it out of the 525 PR into a standalone contribution here if that's useful to you.

Thanks again for this — the 525 was a near-trivial add precisely because the driver was structured so well. 🙏

…ator

Adds an open-source, vendor-SDK-free path to run an Echo picklist end-to-end:

- picklist.py: Transfer model, read_picklist() for the Echo cherry-pick CSV
  (a public data format), an EchoProtocolGenerator interface, and
  NaiveEchoProtocolGenerator (group by source plate type, emit <wp> in picklist
  order, pass through DestX/YOffset). Ordering is deliberately NOT optimised -
  the vendor's head-travel/serpentine path-finding is proprietary and is left to
  an out-of-tree generator that implements the same interface.
- Echo.run_picklist(): parse -> generate -> survey -> DoWellTransfer, no GUI.
  Plate loading/access is left to the caller (robotic arm) via PlateAccess.

The generator seam lets SDK owners swap in a vendor-exact generator without any
vendor code entering this repo. 5 new tests; full labcyte suite 98 green.
c-reiter and others added 9 commits July 2, 2026 15:24
- ruff format + import sorting across the labcyte files
- TYPE_CHECKING import so run_picklist's Transfer/EchoProtocolGenerator
  annotations resolve (F821)
- rename a loop var shadowing dataclasses.field (F402); drop a duplicate
  'fluids' annotation (mypy no-redef)
- echo_mock.py mypy: typed _load_captured_responses result, type _server as
  asyncio.Server, int() the port

ruff, mypy (labcyte), typos and the labcyte tests all pass.
Per maintainer guidance (target v1b1 Device/Driver conventions), collapse the
Echo525 frontend/driver subclasses into a single Echo selected by model:

- add an ECHO_MODELS registry (Echo 650 = 2.5 nL/3.1, Echo 525 = 25 nL/2.6);
  EchoDriver(model=...) resolves the increment + version defaults from it.
- Echo(host, model="Echo 525") and Echo(driver=<any EchoDriver>) — the driver
  is now injectable (enabling chatterbox / vendor-SDK siblings), replacing the
  driver_class subclassing hook.
- Echo525 / Echo525Driver become thin deprecation shims (warn -> model="Echo 525").
- migrate 525/picklist tests + docs to the model= API; add deprecation-alias tests.

100 labcyte tests pass. Follow-up will make EchoDriver an ABC with
MedmanEchoDriver + EchoChatterboxDriver siblings.
…blings

Completes the driver-architecture refactor Rick approved on the PR:

- EchoDriver is now an abstract base holding all Medman protocol logic; the
  transport (_send_request / open_event_stream) is abstract.
- MedmanEchoDriver: the concrete SOAP-over-HTTP implementation (real instrument).
- EchoChatterboxDriver: hardware-free sibling that logs each RPC and returns
  SUCCEEDED/OK with no I/O (dry-run command sequences), injectable via
  Echo(driver=EchoChatterboxDriver()).
- Echo builds a MedmanEchoDriver by default; Echo525Driver shim now subclasses it.
- update base/525 tests + docs + API reference; 102 labcyte tests pass.

This is the sibling-driver-under-one-ABC shape (Medman / chatterbox / future
vendor-SDK or iDOT) injected into a single Echo frontend.
Per @rickwierenga's review on the PR:
- EchoDriver stores self.spec (an EchoModel); transfer_volume_increment_nl,
  client_version, protocol_version, model are computed properties off it
  (no longer flattened onto the instance / no override kwargs).
- model is typed as a Literal["Echo 650", "Echo 525"].
- abstract stubs keep only their docstring (no redundant NotImplementedError).
- removed the Echo525 / Echo525Driver deprecation shims and echo525.py
  entirely — nothing is merged yet, so there is nothing to deprecate; the 525
  is just Echo(model="Echo 525"). Updated tests, exports, docs and API ref.

100 labcyte tests pass.
- ruff format + import sorting across the touched labcyte files
- type Echo(model=) and EchoDriver(model=) as the EchoModelName Literal so
  mypy is happy passing model through to the driver
- mypy fixes in echo_mock.py (typed _load_captured_responses result; type
  _server as asyncio.Server so .sockets resolves; int() the port)
- rename a loop var that shadowed the dataclasses.field import (F402) and
  drop a duplicate 'fluids' annotation (no-redef) in echo.py
- remove now-unused EchoDriver import from echo_tests.py

ruff (format/lint/isort), mypy (labcyte), typos and the 100 labcyte tests all pass.
The Echo firmware's <wp> transfer elements carry an optional per-transfer
destination XY offset (dx/dy, in device units ~1e5 scale) and a free-form
tag label, both echoed back in the transfer report next to the resolved
gx/gy. Thread these through the PLR transfer path so a caller can drop off
the destination well centre and/or label individual transfers.

- EchoPlannedTransfer gains dx: float = 0.0, dy: float = 0.0,
  tag: Optional[str] = None. This is the ergonomic, typed, public per-transfer
  spec already accepted by transfer()/transfer_wells()/build_echo_transfer_plan;
  the (source, destination, volume) tuple shorthand keeps the zero-offset default.
- _make_transfer_protocol_xml emits dx/dy (paired, when either is non-zero) and
  tag (when not None). Default transfers stay byte-identical to the minimal
  <wp n dn v> form, guarded by test_transfer_plan_default_wp_is_byte_identical.
- EchoTransferredWell gains dest_x_offset/dest_y_offset parsed from the report's
  echoed dx/dy for a complete round-trip.
- Tests: non-default dx/dy/tag emission, tag-only emission, byte-identical
  default (tuple and EchoPlannedTransfer forms), and report-side offset parsing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r ≠ capability names)

The EchoDriver ABC exposed public methods whose names collided with the
PlateAccess capability API (lock/unlock/get_access_state/open_source_plate/
close_source_plate/open_destination_plate/close_destination_plate/close_door),
and EchoPlateAccessBackend merely pass-through delegated by identical name.

Rename the driver methods to the vendor Medman endpoint verbs they actually
build, so the Driver surface is generic transport/vendor verbs and the backend
performs a real abstract-capability-op -> vendor-verb translation:

  lock                     -> lock_instrument            (LockInstrument)
  unlock                   -> unlock_instrument          (UnlockInstrument)
  get_access_state         -> read_access_state          (GetDIOEx2-derived)
  open_source_plate        -> present_source_gripper     (PresentSrcPlateGripper)
  close_source_plate       -> retract_source_gripper     (RetractSrcPlateGripper)
  open_destination_plate   -> present_destination_gripper (PresentDstPlateGripper)
  close_destination_plate  -> retract_destination_gripper (RetractDstPlateGripper)
  close_door               -> close_instrument_door       (CloseDoor)

All driver-internal callers, the EchoPlateAccessBackend delegation (which keeps
the capability-standard method names), and the affected tests are updated. The
Echo frontend convenience methods and the capability backend method names are
unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ity (v1b1)

run_picklist called PlateAccess operations straight on the driver
(self.driver.lock / unlock). Route lock/unlock through the PlateAccess
capability (self.plate_access.lock / unlock) so plate-access ops go through the
capability layer rather than reaching around it into the driver.

The door close is deliberately left on the driver
(self.driver.close_instrument_door). The capability's close_door confirms
source/destination access is retracted and then polls the access state until the
door reports closed; run_picklist assumes plates are already loaded and never
surveys access state, so that precondition/poll is neither established nor
observable here (e.g. the captured-response mock has no GetDIOEx2 reply and would
raise before any transfer). Firing the vendor CloseDoor directly preserves the
picklist's fire-and-continue sequencing; a code comment records the rationale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants