DuckDB: bulk writes

[fungiboletus]

What

Writing a batch to DuckDB now costs a handful of statements whatever the number of series or strings, like PostgreSQL, TimescaleDB and ClickHouse.

• duckdb_registration.rs: sensors, units, label names and descriptions, strings and labels of a whole batch are looked up with chunked IN lists and inserted through appenders (add_column, so ids still come from the table sequences).
• duckdb_publishers.rs: one appender per value table for the whole batch instead of one per sensor; string samples use the batch dictionary.
• The #[cached] wrappers are gone (with forget_sensor_id and the cache clears). They were filled inside the transaction, so a rollback could leave ids of sensors that were never committed.

Numbers

Release build, DuckDB file, same machine and scripts (tests/perf/scale.sh 3000, tests/perf/strings.sh 3000), one run each, indicative:

write	before	after
3000 new series (30 000 samples)	2.69 s	0.23 s
the same series again	0.37 s	0.11 s
3000 series x 10 strings, 50 distinct	2.51 s	0.20 s
one series of 10 000 distinct strings	1.50 s	0.16 s

Reads are unchanged. The aggregated selector read is still one query per series (about 0.3 s for 100 series); it is noted in done/duckdb-bulk-writes.md as a separate task.

Tests

• Four new unit tests on an in-memory DuckDB: everything written once, same sensor twice in the input, a rollback leaves every table empty and the batch can be written again, lookups across the chunk size, empty input.
• DuckDB suite (236 lib, 274 integration) and default build (239 lib, 287 integration) pass; clippy --tests -D warnings on both. The backend-generic tests (2100 new sensors in a batch, string dictionaries, labels written once, concurrent first writes) cover this path.
• cargo test --doc with the duckdb feature cannot find libduckdb.dylib on macOS locally; CI is not affected.

Docs

Task file moved to done/, the ClickHouse pre-production task closed (PR 44 is merged, CI green), and docs/BACKENDS.md corrected: DuckDB has stored microsecond timestamps since the TIMESTAMP_MS columns were replaced.

🤖 Generated with Claude Code