Skip to content

feat(paloalto): create cortex XSOAR collector (#301) #348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mariot wants to merge 27 commits into
base: main
Choose a base branch
from
Open

feat(paloalto): create cortex XSOAR collector (#301) #348

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
53859f0
[palo-alto-cortex-xsoar] feature(core): create collector (#301)
mariot Apr 28, 2026
154c6aa
[palo-alto-cortex-xsoar] add tests
mariot Apr 28, 2026
2d3bc5e
[palo-alto-cortex-xsoar] use HttpUrl type of api_url
mariot Apr 28, 2026
e7e8cb9
[palo-alto-cortex-xsoar] add tests
mariot Apr 28, 2026
346f07e
[palo-alto-cortex-xsoar] withdrawing unnecessary alias in CustomFields
guzmud May 6, 2026
82b873f
[palo-alto-cortex-xsoar] adding timeout value to requests in client API
guzmud May 6, 2026
fef8355
[palo-alto-xsoar] adding automatic retries with increasing backoff
guzmud May 6, 2026
7be995e
[palo-alto-xsoar] chore(utils): deleting leftover debug print
guzmud May 11, 2026
b810702
[palo-alto-xsoar] feat(fetcher): adding alert_process_image_name to m…
guzmud May 11, 2026
869cccb
[palo-alto-xsoar] feat(apiclient): session creation made during init
guzmud May 11, 2026
f72e078
add ub9 image
mariot Jun 9, 2026
87ead61
revert API use
mariot Jun 9, 2026
a7ff4a6
Revert "revert API use"
mariot Jun 17, 2026
16f0511
feat(expectation): add regex to find implant
mariot Jun 18, 2026
eec4604
Potential fix for pull request finding
mariot Jun 23, 2026
2bf7589
Potential fix for pull request finding
mariot Jun 23, 2026
8e62948
Potential fix for pull request finding
mariot Jun 23, 2026
69fcc14
refactor: add Pydantic type hinting to ioc_extractor and alert_fetcher
mariot Jun 23, 2026
06dfa8e
refactor: remove redundant include_paths from IOC extraction
mariot Jun 23, 2026
070bc8d
feat: improve error handling in IOC extraction to prevent batch failure
mariot Jun 23, 2026
3f6add3
test: add tests for ioc_extractor and allow extra fields in CustomFields
mariot Jun 23, 2026
e3d8608
test: fix ioc_extractor tests for JSON escaping and mocking
mariot Jun 23, 2026
d8e6120
fix: resolve timezone drift in alert fetching by correctly handling l…
mariot Jun 23, 2026
fd06eb5
fix: preserve original timezone in alert fetcher queries
mariot Jun 23, 2026
b29fe0e
fix: ensure timezone offset is always present in Palo Alto API queries
mariot Jun 23, 2026
b42fef2
fix: small changes
mariot Jun 23, 2026
4236506
fix: typings
mariot Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,14 @@ jobs:
working_directory: ~/openaev/palo-alto-cortex-xdr
name: Tests for palo-alto-cortex-xdr collector
command: poetry run pytest
- run:
working_directory: ~/openaev/palo-alto-cortex-xsoar
name: Install dependencies for palo-alto-cortex-xsoar
command: poetry run pip install pytest factory-boy pyoaev msticpy

@Kakudou Kakudou Jun 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question:
Any reason to use mticpy, i saw that you are using transform.iocextract but you have to build the ioc_regex.
So i don't really get what's the benefit against re or ioc_finder already used in a connector (cf. https://github.com/OpenCTI-Platform/connectors/blob/master/external-import/crowdstrike/src/crowdstrike_feeds_services/utils/ioc_extractor.py)

@mariot mariot Jun 23, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ioc_finder is great, but it lacks the ability to retreive command line infos. If we were to use ioc_finder, we will need to pass the entire string to it, the query it again with re to get all the infos we need. With mticpy, we can define the regex we need and pass to it once.

- run:
working_directory: ~/openaev/palo-alto-cortex-xsoar
name: Tests for palo-alto-cortex-xsoar collector
command: poetry run pytest
build_docker_images:
working_directory: ~/openaev
docker:
Expand Down Expand Up @@ -305,6 +313,19 @@ jobs:
fi
docker save -o ~/openaev/images/collector-palo-alto-cortex-xdr openaev/collector-palo-alto-cortex-xdr:${CIRCLE_SHA1}
docker save -o ~/openaev/images/collector-palo-alto-cortex-xdr-ubi9 openaev/collector-palo-alto-cortex-xdr:${CIRCLE_SHA1}-ubi9
- run:
working_directory: ~/openaev/palo-alto-cortex-xsoar
name: Build Docker image openaev/collector-palo-alto-cortex-xsoar
command: |
if [[ "${CIRCLE_BRANCH}" == "main" ]]; then
docker build --pull --progress=plain -t openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1} --build-arg PYOAEV_GIT_BRANCH_OVERRIDE="${CIRCLE_BRANCH}" .
docker build --pull --progress=plain -t openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1}-ubi9 -f Dockerfile_ubi9 --build-arg PYOAEV_GIT_BRANCH_OVERRIDE="${CIRCLE_BRANCH}" .
else
docker build --pull --progress=plain -t openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1} .
docker build --pull --progress=plain -t openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1}-ubi9 -f Dockerfile_ubi9 .
fi
docker save -o ~/openaev/images/collector-palo-alto-cortex-xsoar openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1}
docker save -o ~/openaev/images/collector-palo-alto-cortex-xsoar-ubi9 openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1}-ubi9
- persist_to_workspace:
root: ~/openaev
paths:
Expand Down Expand Up @@ -455,7 +476,14 @@ jobs:
docker image load < collector-palo-alto-cortex-xdr-ubi9
docker tag openaev/collector-palo-alto-cortex-xdr:${CIRCLE_SHA1}-ubi9 openaev/collector-palo-alto-cortex-xdr:${IMAGETAG}-ubi9
docker tag openaev/collector-palo-alto-cortex-xdr:${CIRCLE_SHA1}-ubi9 openbas/collector-palo-alto-cortex-xdr:${IMAGETAG}-ubi9


docker image load < collector-palo-alto-cortex-xsoar
docker tag openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1} openaev/collector-palo-alto-cortex-xsoar:${IMAGETAG}
docker tag openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1} openbas/collector-palo-alto-cortex-xsoar:${IMAGETAG}
docker image load < collector-palo-alto-cortex-xsoar-ubi9
docker tag openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1}-ubi9 openaev/collector-palo-alto-cortex-xsoar:${IMAGETAG}-ubi9
docker tag openaev/collector-palo-alto-cortex-xsoar:${CIRCLE_SHA1}-ubi9 openbas/collector-palo-alto-cortex-xsoar:${IMAGETAG}-ubi9

echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
docker push openaev/collector-mitre-attack:${IMAGETAG}
docker push openaev/collector-mitre-attack:${IMAGETAG}-ubi9
Expand Down Expand Up @@ -521,7 +549,10 @@ jobs:
docker push openaev/collector-palo-alto-cortex-xdr:${IMAGETAG}-ubi9
docker push openbas/collector-palo-alto-cortex-xdr:${IMAGETAG}
docker push openbas/collector-palo-alto-cortex-xdr:${IMAGETAG}-ubi9

docker push openaev/collector-palo-alto-cortex-xsoar:${IMAGETAG}
docker push openaev/collector-palo-alto-cortex-xsoar:${IMAGETAG}-ubi9
docker push openbas/collector-palo-alto-cortex-xsoar:${IMAGETAG}
docker push openbas/collector-palo-alto-cortex-xsoar:${IMAGETAG}-ubi9
if [ "${IS_LATEST}" == "true" ]
then
docker tag openaev/collector-mitre-attack:${IMAGETAG} openaev/collector-mitre-attack:latest
Expand Down Expand Up @@ -556,6 +587,8 @@ jobs:
docker tag openaev/collector-google-workspace:${IMAGETAG} openbas/collector-google-workspace:latest
docker tag openaev/collector-palo-alto-cortex-xdr:${IMAGETAG} openaev/collector-palo-alto-cortex-xdr:latest
docker tag openaev/collector-palo-alto-cortex-xdr:${IMAGETAG} openbas/collector-palo-alto-cortex-xdr:latest
docker tag openaev/collector-palo-alto-cortex-xsoar:${IMAGETAG} openaev/collector-palo-alto-cortex-xsoar:latest
docker tag openaev/collector-palo-alto-cortex-xsoar:${IMAGETAG} openbas/collector-palo-alto-cortex-xsoar:latest

docker push openaev/collector-mitre-attack:latest
docker push openbas/collector-mitre-attack:latest
Expand Down Expand Up @@ -589,6 +622,8 @@ jobs:
docker push openbas/collector-google-workspace:latest
docker push openaev/collector-palo-alto-cortex-xdr:latest
docker push openbas/collector-palo-alto-cortex-xdr:latest
docker push openaev/collector-palo-alto-cortex-xsoar:latest
docker push openbas/collector-palo-alto-cortex-xsoar:latest
fi
- slack/notify:
event: fail
Expand Down
4 changes: 4 additions & 0 deletions palo-alto-cortex-xsoar/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.idea
.venv
.run
*.lock
1 change: 1 addition & 0 deletions palo-alto-cortex-xsoar/.python-version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.13
32 changes: 32 additions & 0 deletions palo-alto-cortex-xsoar/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
FROM python:3.13-alpine AS builder

# poetry version available on Ubuntu 24.04
RUN pip3 install poetry==2.1.3

RUN apk update && apk upgrade

ARG installdir=/collector
ADD . ${installdir}
RUN cd ${installdir} && poetry build

FROM python:3.13-alpine AS runner

# Declare the build argument
ARG PYOAEV_GIT_BRANCH_OVERRIDE

ARG installdir=/collector
COPY --from=builder ${installdir} ${installdir}
RUN cd ${installdir}/dist && pip3 install --no-cache-dir "$(ls *.whl)[prod]"

RUN if [[ ${PYOAEV_GIT_BRANCH_OVERRIDE} ]] ; then \
echo "Forcing specific version of client-python" && \
apk add --no-cache git && \
pip install pip3-autoremove && \
pip-autoremove pyoaev -y && \
pip install git+https://github.com/OpenAEV-Platform/client-python@${PYOAEV_GIT_BRANCH_OVERRIDE} ; \
fi
Comment on lines +21 to +27

# necessary for icon location
WORKDIR ${installdir}

CMD ["python3", "-m", "src"]
44 changes: 44 additions & 0 deletions palo-alto-cortex-xsoar/Dockerfile_ubi9
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
FROM registry.access.redhat.com/ubi9/ubi-minimal AS base

RUN set -eux; \
microdnf -y --setopt=install_weak_deps=0 install python3.12; \
microdnf clean all;


FROM base AS builder

RUN set -eux; \
microdnf -y --setopt=install_weak_deps=0 install python3.12-pip; \
pip3.12 install poetry==2.1.3; \
microdnf -y remove python3.12-pip; \
microdnf clean all;

WORKDIR /collector
COPY ./ ./
RUN set -eux; \
poetry build


FROM base AS runner

ARG PYOAEV_GIT_BRANCH_OVERRIDE=""

WORKDIR /collector
COPY --from=builder /collector/ ./

RUN set -eux; \
microdnf -y --setopt=install_weak_deps=0 install python3.12-pip; \
(cd dist && pip3.12 install --no-cache-dir "$(ls *.whl)[prod]"); \
if [ -n "${PYOAEV_GIT_BRANCH_OVERRIDE}" ] ; then \
echo "Forcing specific version of client-python"; \
microdnf -y --setopt=install_weak_deps=0 install git-core; \
pip3.12 install pip3-autoremove; \
pip-autoremove pyoaev -y; \
pip3.12 install git+https://github.com/OpenAEV-Platform/client-python@${PYOAEV_GIT_BRANCH_OVERRIDE}; \
microdnf -y remove git-core; \
fi; \
microdnf -y remove python3.12-pip; \
microdnf clean all;

CMD ["python3.12", "-m", "src"]

Loading
Loading