Plugin refactor

[TheGreatCodeholio]

Summary

This PR is a significant rework of the plugin system, call conclusion pipeline, and runtime configuration. The changes fall into five areas:

• Plugin call_end split into blocking and deferred execution phases
• Shared JSON context passed through the call lifecycle
• New ConfigurationService for plugins to request runtime config changes
• call_concluder correctness and safety fixes
• Decoupling of uploader credentials from the core System / Config structs

---

1. Blocking / Deferred Plugin Execution

plugin_api.h, plugin_manager.cc/.h, global_structs.h

call_end now takes two additional parameters:
virtual int call_end(Call_Data_t& call_info, nlohmann::ordered_json& plugin_ctx);

Plugins are classified in config as "pluginType": "blocking" (default) or "pluginType": "deferred".

• Blocking plugins run serially before files are finalized. They can read and write plugin_ctx, which is merged into call_info.call_json under the plugin's name. Failed blocking plugins are retried with exponential backoff.
• Deferred plugins run concurrently via std::async after all blocking plugins succeed and the updated JSON is written to disk. They receive a read-only snapshot of call_info. Failed deferred plugins are also retried
  independently.

Retry tracking is split into blocking_plugin_retry_list and deferred_plugin_retry_list on Call_Data_t. Files are only finalized (moved from tempDir to captureDir) after both phases complete successfully.

Three bugs fixed in this dispatch logic (see last section).

All existing plugins (broadcastify, openmhz, rdioscanner, simplestream, stat_socket) updated to the new signature.

---

2. Shared JSON Context

Each blocking plugin gets a nlohmann::ordered_json object keyed by plugin name in call_info.call_json. This lets plugins write structured data into the call's JSON file — useful for transcription results, upload receipts, or
any per-plugin metadata that should travel with the call record.

The JSON file is written twice: once before upload script / plugins run (so the upload script sees complete call metadata), and again after all blocking plugins complete (so any plugin-written context is persisted to disk
before deferred plugins receive it).

---

3. RuntimeConfigurationService

New files: config_service.h, config_service.cc, examples/example_config_plugin.cc/.h

Plugins can now request runtime changes to SDR source and system parameters without restarting. The service exposes a thread-safe command queue processed on the main loop:

// From any plugin:
get_config_service()->set_source_gain(0, 45.0, plugin_name, callback);
get_config_service()->set_system_squelch_db(0, -60.0, plugin_name, callback);

Supported parameters:

• Source: gain, gain-by-name, error/PPM, gain mode (AGC), antenna, signal detector threshold
• System: squelch, analog/digital levels, min/max duration, min TX duration, record_unknown, hide_encrypted, hide_unknown, conversation mode, tau, max_dev, filter_width
• Global: call timeout

Changes trigger on_config_change(ConfigChangeInfo&) callbacks to all running plugins, so plugins can react to changes made by other plugins.

Source::set_error now immediately applies the correction to the SDR hardware (osmosdr/USRP) rather than only updating the in-memory value. Source::get_gain_mode accessor added.

Sources and Systems track a config_index (their array position in the JSON) so the config service can look them up for save_config.

save_config() added to persist runtime changes back to the config file.

---

4. Call Concluder: Temp-Dir-First Workflow + Correctness Fixes

call_concluder.cc

Temp-dir-first archival

All call artifacts (.wav, .m4a, .json, raw wav) are written to tempDir during processing. Only after all plugins succeed are they moved to their final destinations under captureDir. Cross-device moves (e.g. tmpfs/ramdisk →
disk) fall back to copy+delete. Call_Data_t gains final_* fields for the archive destinations.

remove_call_files replaced by two focused functions:

• cleanup_tmp_call_files — deletes everything still in tempDir
• finalize_call_files — moves keepers to captureDir, respects audio_archive, transmission_archive, and call_log flags

Bug fixes

• boost::filesystem → std::filesystem throughout
• Thread-safe RNG: rand() (unseeded, not thread-safe) replaced with a seeded std::mt19937 behind a mutex
• localtime → localtime_r: std::localtime is not thread-safe; manage_call_data_workers runs on the main thread but the original upload_call_worker also called it from worker threads
• read() loop handles EINTR: process output capture now correctly retries on signal interruption
• Partial concat list cleanup: if write_concat_list fails, any partial file is removed before returning
• Short transmissions always deleted from tempDir: previously, transmission_archive suppressed deletion of short segments, leaving them to accumulate
• is_invalid_loudnorm_value simplified using trim_whitespace / lowercase_copy helpers already present in the file

---

5. Uploader Decoupling

config.cc, global_structs.h, system_impl.cc/.h

upload_server and bcfy_calls_server removed from Config. api_key, bcfy_api_key, and bcfy_system_id removed from System. These were only ever used by the uploader plugins; keeping them on shared structs forced the core to know
about plugin-specific config.

The three previously hard-coded "internal" plugins (openmhz_uploader, broadcastify_uploader, unit_script) are no longer auto-loaded via add_internal_plugin. They are now regular plugins configured in the plugins array. The
add_internal_plugin function is removed entirely.

The compressWav validation for Broadcastify is moved from config.cc into Broadcastify_Uploader::init(), where it belongs.

---

6. Plugin Manager Lifecycle Fixes

Three bugs fixed in this session:

Deferred plugins silently skipped after blocking retry. When blocking plugins failed on the initial attempt, upload_call_worker returned early before reaching plugman_call_end_deferred. On the retry pass, call_info.status ==
RETRY caused plugman_call_end_deferred to take the retry-list path — but deferred_plugin_retry_list was empty (deferred had never run), so all deferred plugins were skipped and files finalized without them. Fixed by adding
deferred_plugins_ran to Call_Data_t; plugman_call_end_deferred always does a full run on first invocation regardless of status.

Failed plugins promoted to PLUGIN_RUNNING. start_plugins set plugin->state = PLUGIN_RUNNING unconditionally for any plugin that didn't call continue inside the function. Plugins already in PLUGIN_FAILED (from parse_config or
init) skipped the PLUGIN_INITIALIZED block but still hit the assignment. Fixed with an early continue on PLUGIN_FAILED.

Missing retry logging. Per-plugin retry attempt logging was lost in the blocking path rewrite. Restored.

---

Documentation

Per-plugin documentation added under docs/plugins/:

• broadcastify-calls.md
• openmhz.md
• rdio-scanner.md
• simplestream.md
• stat-socket.md
• unit-script.md

docs/Plugins.md and docs/CONFIGURE.md updated to reflect the new plugin configuration format, pluginType, and removal of top-level uploader keys.

simplestream plugin enabled in CMakeLists.txt.

---

Test Plan

• Existing blocking-only config: verify calls conclude, JSON written, files archived in captureDir
• Deferred plugin: verify it runs after blocking completes, including when blocking initially fails and retries
• Blocking plugin failure + retry: verify retry logging appears, deferred still runs after blocking eventually succeeds
• Plugin parse_config / init failure: verify plugin is skipped (not marked RUNNING), no callbacks received
• ConfigurationService: verify gain/squelch changes apply to hardware at runtime
• tempDir on ramdisk (/dev/shm): verify cross-device move fallback works
• save_config: verify runtime changes are persisted to config file
• Uploader plugins (broadcastify, openmhz, rdioscanner) configured as regular plugins: verify uploads succeed