Skip to content

Feature: Added support for BulkEnrichDomains, EnrichDomainRisk, GetDomainProfile actions with widgets, tests, and domain enrichment playbook - #1301

Open
briluza wants to merge 28 commits into
chronicle:mainfrom
DomainTools:domaintools_v14.0
Open

briluza wants to merge 28 commits into
chronicle:mainfrom
DomainTools:domaintools_v14.0

Conversation

@briluza

@briluza briluza commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Added support for BulkEnrichDomains, EnrichDomainRisk, GetDomainProfile actions, tests, and domain enrichment playbook


Description

Please provide a detailed description of your changes. This helps reviewers understand your work and its context.

What problem does this PR solve?

  • Added three new enrichment actions; BulkEnrichDomains, EnrichDomainRisk, and
    GetDomainProfile, this actions give analysts automated domain threat intelligence
    directly on the case wall. These leverage the updated domaintools-api package
    and cover risk scoring, WHOIS identity, hosting, and registration details that
    were missing from the existing action set.

How does this PR solve the problem?

  • Added BulkEnrichDomains, EnrichDomainRisk, and GetDomainProfile actions using
    the Iris Enrich and Iris Investigate APIs
  • Added a DomainToolsParser and datamodels layer to normalize API responses
    into structured EntityResult outputs
  • Updated DomainToolsManager to handle batched Iris Enrich calls and domain
    extraction
  • Added a DomainTools - Domain Enrichment playbook that automatically runs
    EnrichDomainRisk then GetDomainProfile on domain/hostname entities
  • Fixed InvestigateDomain entity join bug
  • Added unit tests and result example JSON files for all new actions

Any other relevant information (e.g., design choices, tradeoffs, known issues):
N/A


Checklist:

Please ensure you have completed the following items before submitting your PR.
This helps us review your contribution faster and more efficiently.

General Checks:

  • I have read and followed the project's contributing.md guide.
  • My code follows the project's coding style guidelines.
  • I have performed a self-review of my own code.
  • My changes do not introduce any new warnings.
  • My changes pass all existing tests.
  • I have added new tests where appropriate to cover my changes. (If applicable)
  • I have updated the documentation where necessary (e.g., README, API docs). (If applicable)

Open-Source Specific Checks:

  • My changes do not introduce any Personally Identifiable Information (PII) or sensitive customer data.
  • My changes do not expose any internal-only code examples, configurations, or URLs.
  • All code examples, comments, and messages are generic and suitable for a public repository.
  • I understand that any internal context or sensitive details related to this work are handled separately in internal systems (Buganizer for Google team members).

For Google Team Members and Reviewers Only:

  • I have included the Buganizer ID in the PR title or description (e.g., "Internal Buganizer ID: 123456789" or "Related Buganizer: go/buganizer/123456789").
  • I have ensured that all internal discussions and PII related to this work remain in Buganizer.
  • I have tagged the PR with one or more labels that reflect the pull request purpose.

Screenshots (If Applicable)

If your changes involve UI or visual elements, please include screenshots or GIFs here.
Ensure any sensitive data is redacted or generalized.

image image image image image

Further Comments / Questions

N/A

secops-parsers-helper-bot and others added 22 commits August 7, 2026 13:56
…inProfile

  actions, widgets, tests, and domain enrichment playbook
…, and PascalCase JsonResult example filenames
[IDEV-2515_2516_2517_2519] feat(domain_tools): Added support for BulkEnrichDomains,  EnrichDomainRisk, GetDomainProfile actions with widgets, tests, and domain   enrichment playbook
[IDEV-2572] domaintools v14.0: Address Bugbash Feedbacks
@briluza
briluza requested review from a team and adarshtwy as code owners September 30, 2026 13:26
@github-actions
github-actions Bot requested a review from a team as a code owner October 5, 2026 12:40
@github-actions github-actions Bot added the Stale label Oct 27, 2026


@output_handler
def main() -> None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, migrating this new action to inherit from TIPCommon.base.action.Action (or decomposing main into focused helpers for parameter extraction, domain collection, enrichment, and output generation) would make it much easier to read and maintain. Right now main() spans 139 lines, whereas we aim to keep functions under 50 lines with a single level of abstraction.

Also, please add the return type annotation -> None to main():

Suggested change
def main() -> None:
def main() -> None:

print_value=True,
)
verify_ssl: str = extract_configuration_param(
siemplify, provider_name=INTEGRATION_NAME, param_name="Verify SSL", is_mandatory=True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can fit this extract_action_param call on a single line since it is 85 characters (well under the 100-character limit):

        manual_domains_str = extract_action_param(siemplify, "Domains", print_value=True)

if manual and merge_with_entities:
domains = list(dict.fromkeys(manual + entity_domains))
elif manual:
domains = manual

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When merge_entities is False (or when case_domains is empty), manual is assigned directly without deduplication, whereas lines 89 and 93 deduplicate with dict.fromkeys(...). If a user passes duplicate domains in the comma-separated Domains parameter, duplicate API lookups and table rows will be produced.

Let's deduplicate manual here as well:

Suggested change
domains = manual
domains = list(dict.fromkeys(manual))

return

if len(domains) > BULK_ENRICH_MAX_DOMAINS:
siemplify.LOGGER.warn(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can collapse this logger.info call onto a single line (84 characters) to keep the control flow compact:

            siemplify.LOGGER.info(f"Truncating {len(domains)} domains to {max_domains}")

BulkEnrichDomains.main()

assert action_output.results.execution_state == ExecutionState.COMPLETED
assert "1 young domains" in action_output.results.output_message

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This make_siemplify_mock call is 88 characters and fits cleanly on a single line:

        s = make_siemplify_mock(entities=[e1, e2], params={"Merge With Case Entities": "true"})

class TestEnrichDomainRisk:

@set_metadata(integration_config_file_path=CONFIG_PATH, entities=[DOMAIN_ENTITY])
def test_enrich_success(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add parameter and return type annotations (monkeypatch: pytest.MonkeyPatch, -> None) across the test methods in TestEnrichDomainRisk:

Suggested change
def test_enrich_success(
def test_high_risk_marks_suspicious(self, monkeypatch: pytest.MonkeyPatch) -> None:

class TestGetDomainProfile:

@set_metadata(integration_config_file_path=CONFIG_PATH, entities=[DOMAIN_ENTITY])
def test_profile_success(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In _make_iris_model, registrant_name="John Doe", create_date="2020-01-15", and ip_country_code="US" are passed as str values, whereas DomainToolsParser.get_identity, get_registration, and get_hosting now pass dict | None ({"value": ..., "count": ...}) into Identity, Registration, and Hosting.

Let's pass the dictionary structure here (and add type annotations to _make_iris_model) so the unit tests exercise the real datamodel shape produced by DomainToolsParser.

assert "No domain profiles could be built" in action_output.results.output_message

@set_metadata(integration_config_file_path=CONFIG_PATH, entities=[DOMAIN_ENTITY])
def test_rdap_not_found_still_succeeds(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_rdap_not_found_still_succeeds (and its docstring "When RDAP returns 404, action still succeeds with Iris data") is left over from before commit 32020681c removed the RDAP call from GetDomainProfile.py — right now it is identical to test_happy_path_all_sources. Let's either remove this duplicate test or repurpose it to test a real edge case (such as when create_date is missing or malformed).

class TestClassifyDomainRisk:
"""Unit tests for classify_domain_risk() — no mocking needed."""

def test_high_risk(self):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add return type annotations (-> None) to the test methods in test_risk_classification.py:

Suggested change
def test_high_risk(self):
def test_malicious_by_high_score(self) -> None:

@github-actions github-actions Bot removed the Stale label Oct 30, 2026
@briluza

briluza commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @adarshtwy for the feedback, will take a look and address them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants