feat(integrations): Splunk data source + anomaly forwarding (#99) - #145
Draft
Jovonni wants to merge 1 commit into
Draft
feat(integrations): Splunk data source + anomaly forwarding (#99)#145Jovonni wants to merge 1 commit into
Jovonni wants to merge 1 commit into
Conversation
Adds a bidirectional Splunk integration, reusing the existing connector / data-source / integration-settings patterns: - SplunkConnector (core/integrations): REST export search (input) + HTTP Event Collector (output), bearer or basic auth, env + integration_settings config - model runner: 'splunk' data_source branch runs an SPL search and returns a DataFrame (self-contained, mirrors the elasticsearch branch) - models train/execute: splunk_search / splunk_index params + dispatch - orchestrator: best-effort forward of each run's anomalies to Splunk HEC when enabled (forward_anomalies), never blocking detection - settings: register 'splunk', mask token/hec_token/password, /test probe - settings UI: Splunk card under Data Engines - docs/SPLUNK.md - dedicated test subsuite (26 unit tests, fully mocked, runs in the CI unit job) Closes #99
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Integrates Splunk in both directions (requested in #99):
How it reuses existing infrastructure
SplunkConnector(core/integrations/splunk.py) sits alongsideElasticsearchConnector/SparkConnector— REST export search for input, HTTP Event Collector for output, bearer or basic auth. Config viaintegration_settings(from_config) orSPLUNK_*env.splunkbranch in_load_data_for_model, mirroring the elasticsearch branch (self-containedrequests, returns a DataFrame with numeric coercion).splunk_search/splunk_indexparams + a dispatch branch, exactly like the existingspark/elasticsearch/source_groupsources.forward_anomaliesis enabled — best-effort, so a Splunk outage never affects detection or persistence.splunkregistered in the whitelist, secrets (token/hec_token/password) masked,/testconnectivity probe, and a Splunk card in the Data Engines settings panel.docs/SPLUNK.md.Tests (dedicated subsuite, CI-wired)
core/tests/test_services/test_splunk_connector.py+test_splunk_wiring.py— 26 unit tests, all mocked (no live Splunk): SPL normalization, search parsing + bearer/basic auth, error handling, HEC output + auth header, batch send counts, connectivity probe, settings whitelist/masking/_test_splunk, and orchestrator forwarding gating (enabled/disabled/flag-off/error-safe). Placed undertest_services/so they run in the CI unit job (the-kfilter excludes anything undertest_integrations/).Local: 26/26 green, runner compiles,
tsc/next lint/next buildall clean.Follow-up (not in this PR)
Splunk-as-a-physical-source inside
source_groups(so a source group can mix Splunk with other sources) is a natural next step.Closes #99