Skip to content

feat(qradar): add IBM QRadar collector (#253)#425

Open
SamuelHassine wants to merge 2 commits into
mainfrom
feature/253-qradar-collector
Open

feat(qradar): add IBM QRadar collector (#253)#425
SamuelHassine wants to merge 2 commits into
mainfrom
feature/253-qradar-collector

Conversation

@SamuelHassine

@SamuelHassine SamuelHassine commented Jun 16, 2026

Copy link
Copy Markdown
Member

Summary

Adds an OpenAEV collector for IBM QRadar (requested in #253). It validates OpenAEV detection expectations by running Ariel (AQL) searches against QRadar for events that match the attack signatures produced during a simulation, then reports a verdict and a trace linking back to the QRadar Log Activity view.

Modeled on the existing splunk-es / elastic SIEM collector pattern (pyoaev CollectorDaemon plus expectation/trace service providers), with the SIEM-specific layer implemented for QRadar.

What it does

  • Builds an AQL query from the attack signatures, creates an Ariel search, polls it to completion, and parses the returned events
  • Authentication via an authorized service token (SEC header, preferred) or HTTP basic
  • Signature matching on sourceip, destinationip, and parent process via the URL property, bounded by the expectation time window
  • Retry mechanism with a configurable offset to handle event ingestion latency
  • Trace generation with links to the QRadar Log Activity view
  • Detection expectations only (QRadar is a detection source)

API endpoints

  • POST /api/ariel/searches?query_expression=<AQL> (create search)
  • GET /api/ariel/searches/{search_id} (poll status)
  • GET /api/ariel/searches/{search_id}/results (fetch results)

Self-review fixes

Copilot could not review this PR (it errored out), so these come from a manual pass over the full collector:

  • Removed the dead QRadarAlert._raw PrivateAttr: it was passed through the constructor (which Pydantic v2 ignores) so it was always None and never read.
  • Corrected the _build_trace_url_from_expectation docstring to match the actual behavior (a Log Activity filter from source/destination IPs only, not the full AQL query) and renamed the misleading kql_* locals to filter_* since QRadar uses AQL.
  • Fixed the README configuration section: the loader selects a single source (first of .env, config.yml, environment variables); it does not merge them.
  • Fixed the documented default collector name (QRadar, not IBM QRadar) to match ConfigLoaderCollector.

Tests

  • 125 unit tests, 86% coverage (codecov/patch threshold is 80%)
  • black / isort / flake8 clean
  • Registered in .circleci/config.yml (docker build + publish)

Notes

  • The collector icon (src/img/qradar-logo.png) is a placeholder and should be replaced with the official IBM QRadar logo before release.

Closes #253

Detection-expectation collector that runs Ariel (AQL) searches against IBM QRadar to validate detections. Modeled on the splunk-es/elastic collector pattern (pyoaev CollectorDaemon, expectation and trace services). Supports authorized service token (SEC header) or basic auth, source/destination IP and parent-process (URL) matching, and retry with configurable offset. Registered in the CircleCI docker build/publish pipeline.
Copilot AI review requested due to automatic review settings June 16, 2026 14:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

- Remove the dead QRadarAlert._raw PrivateAttr: it was assigned through
  the constructor (QRadarAlert(..., _raw=row)), which Pydantic v2 ignores,
  so it always stayed None and was never read anywhere.
- Correct the _build_trace_url_from_expectation docstring to describe the
  actual behavior: it builds a QRadar Log Activity filter from
  source/destination IPs only, not the full AQL query (no parent-process
  URL match, no search time window). Rename the misleading kql_* locals to
  filter_* since QRadar uses AQL, not KQL.
- Fix the README configuration section: the loader selects a single source
  (first of .env, config.yml, environment variables); it does not merge
  env vars over YAML over defaults.
- Fix the documented default collector name (QRadar, not IBM QRadar) to
  match ConfigLoaderCollector.
@SamuelHassine

Copy link
Copy Markdown
Member Author

Review and fix summary

Copilot errored out and could not review this PR, so I did a full independent senior review of the QRadar collector (which shares the framework with splunk-es / elastic). Changes pushed in 90367f2:

  • Removed dead QRadarAlert._raw PrivateAttr (src/services/models.py): it was assigned through the constructor, which Pydantic v2 ignores, so it was always None and never read. Dropped the field, the constructor argument and the unused PrivateAttr import.
  • Corrected _build_trace_url_from_expectation docstring (src/services/trace_service.py): it builds a QRadar Log Activity filter from source/destination IPs only, not the full AQL query (no URL match, no search time window). Also renamed the misleading kql_* locals to filter_* since QRadar uses AQL, not KQL.
  • Fixed README configuration precedence: the loader selects a single source (first of .env, config.yml, environment variables) and does not merge them.
  • Fixed the documented default collector name (QRadar, not IBM QRadar) to match ConfigLoaderCollector.

Reviewed and considered acceptable (no change):

  • AQL is built with interpolated values, but the inputs are constrained to IP-typed signatures and injects/agent UUIDs, so injection surface is limited and this matches the native Ariel query approach.

Status:

  • All CI checks are green (tests including Test qradar, linter, formatting, signed commits, codecov patch/project, docker image build).
  • No review threads to resolve (Copilot review failed to run).
  • mergeable: MERGEABLE. The only thing left is a maintainer approval (REVIEW_REQUIRED); I could not approve since I am the PR author.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(qradar): collector

3 participants