Implement workaround for PanicException serialization

[doomedraven]

Added a workaround for PanicException serialization failures by dynamically registering a module. This allows for successful pickle/unpickle operations.

@wmetcalf can you review it? without this it just breaks process.py for me

[wmetcalf]

sure I'll have a look

[wmetcalf]

Hey @doomedraven — thanks for chasing this down. I tried to reproduce the failure and ended up somewhere different: I couldn't get the stub approach to work, but I did find something in PebbleEngine that wedges process.py exactly the way you describe. So rather than nitpick this patch: try this instead.

The wedge: a worker that dies before pickup orphans its task forever

pebble's per-task timeout only starts once a task is running. If the worker dies before picking the task up, pebble can't associate the dead worker with any task (find_expired_task → LookupError → return), so the future never resolves. The task never runs, never times out, is never marked failed, and stays in _pending forever — wedging both the saturation check and the drain loop.

It isn't panic-specific, and that's what makes it worth fixing on its own. Any initializer failure does it (real pebble 5.1.0, one worker, one task):

clean        -> ['success']
valueerror   -> NEVER COMPLETED (task orphaned)
segv         -> NEVER COMPLETED (task orphaned)
exit         -> NEVER COMPLETED (task orphaned)

Note the asymmetry: a worker dying during a task is handled correctly — pebble reports ProcessExpired, marks the task failed and respawns. Dying during init is the hole, and init_worker() compiles the whole YARA ruleset, so it's doing real work at exactly the wrong moment.

The fix: reap tasks that were scheduled but never ran

--- a/lib/cuckoo/core/processing_engine/pebble.py
+++ b/lib/cuckoo/core/processing_engine/pebble.py
@@
     def __init__(self, task_fn, worker_init, source, parallel, timeout,
-                max_tasks=0, max_count=0):
+                 max_tasks=0, max_count=0, stall_grace=300):
         super().__init__(task_fn, worker_init, source, parallel, timeout)
         self.max_tasks = max_tasks
         self.max_count = max_count
+        self.stall_grace = stall_grace
         self._pending = {}  # future -> task_id
+        self._scheduled_at = {}  # future -> time.monotonic() when scheduled
 
     def _done(self, future):
         """Pebble done-callback: fires in the pool's internal thread."""
         task_id = self._pending.pop(future, None)
+        self._scheduled_at.pop(future, None)
 
+    def _reap_stalled(self):
+        """Fail tasks that were scheduled but never ran.
+
+        pebble only applies a task's timeout once that task is RUNNING. If the
+        worker dies before picking it up, pebble cannot associate the dead
+        worker with any task, so the future never resolves: the task never
+        runs, never times out, is never marked failed, and stays in
+        ``_pending`` forever. Both the scheduling loop (via the saturation
+        check) and the drain loop then spin indefinitely.
+
+        Anything still pending past ``timeout + stall_grace`` is therefore
+        presumed dead and failed explicitly."""
+        if not self.stall_grace or not self._pending:
+            return
+
+        deadline = self.timeout + self.stall_grace
+        now = time.monotonic()
+        for future in list(self._pending):
+            if now - self._scheduled_at.get(future, now) < deadline:
+                continue
+            task_id = self._pending.pop(future, None)
+            self._scheduled_at.pop(future, None)
+            log.error(
+                "[%s] Task never ran (worker died before pickup?); marking it failed after %ss",
+                task_id, deadline,
+            )
+            with suppress(Exception):
+                future.cancel()
+            if task_id is not None:
+                self.source.mark_failed(task_id)
+
     def run(self):
@@
             while not self.max_count or count < self.max_count:
+                # Fail anything that was scheduled but never picked up, so a
+                # dead worker can't wedge the saturation check below forever.
+                self._reap_stalled()
+
@@
                     self._pending[future] = task.id
+                    self._scheduled_at[future] = time.monotonic()
                     future.add_done_callback(self._done)
@@
             # Drain: wait for all in-flight tasks to finish before returning.
+            # Reap here too: a task orphaned by a dead worker would otherwise
+            # keep this loop spinning forever.
             while self._pending:
+                self._reap_stalled()
                 time.sleep(0.2)

stall_grace=300 is conservative — it fires at timeout + 5min, so it can't race a legitimately slow task. 0 disables it.

This needs one companion change, because future.cancel() raises CancelledError, which is BaseException-derived on 3.8+:

@@ def _done(self, future):
-        except (pebble.ProcessExpired, Exception) as error:
+        except BaseException as error:
+            # BaseException, not Exception: anything escaping this callback
+            # propagates into pebble's message-manager thread and kills it.
             log.exception("[%s] Exception when processing task: %s", task_id, error)

That one's worth having regardless. _done runs as a concurrent.futures done-callback, and Future._invoke_callbacks only guards except Exception — so a BaseException escapes into message_manager_loop (which only catches BrokenProcessPool) and kills the thread: marked_failed: [], _pending never drains, run() spins forever. prefork.py:89 already uses except BaseException:; the pebble engine was the odd one out.

Tests — tests/test_pebble_engine_stall.py, 3 pass with the patch; the end-to-end one hangs out without it

The end-to-end test bounds run() with SIGALRM on the calling thread, because a worker thread would get its own sqlite connection and miss the fixture's tables:

def _worker_init_dies():
    """Stands in for any initializer failure -- exception or hard crash.
    Both kill the worker before it takes a task."""
    raise ValueError("worker init failed")


def test_reaper_fails_a_task_that_was_never_picked_up():
    source = _CountingSource()
    engine = _engine(source, timeout=1, stall_grace=1)
    future = _NeverFinishes()
    engine._pending[future] = 77
    engine._scheduled_at[future] = time.monotonic() - 3600  # long overdue

    engine._reap_stalled()

    assert engine._pending == {}, "overdue task left in _pending"
    assert source.failed == [77], "overdue task was never marked failed"


def test_reaper_leaves_a_task_that_is_still_within_its_deadline():
    ...
    assert engine._pending == {future: 88}, "in-flight task was reaped early"


def test_engine_returns_when_the_worker_dies_during_init(db, temp_pe32, monkeypatch):
    engine = PebbleEngine(task_fn=_task_fn_noop, worker_init=_worker_init_dies,
                          source=TaskSource(db), parallel=1, timeout=3,
                          max_count=1, stall_grace=3)

    def _bail(signum, frame):
        raise TimeoutError("engine.run() never returned -- the dead worker orphaned the task")

    previous = signal.signal(signal.SIGALRM, _bail)
    signal.setitimer(signal.ITIMER_REAL, 90)
    try:
        engine.run()
    finally:
        signal.setitimer(signal.ITIMER_REAL, 0)
        signal.signal(signal.SIGALRM, previous)

    assert engine._pending == {}, "drain loop left a stalled task in _pending"
    with db.session.begin():
        assert db.view_task(tid).status == TASK_FAILED_PROCESSING

With stall_grace=0 (reaper off), the last one fails exactly as advertised:

E   TimeoutError: engine.run() never returned -- the dead worker orphaned the task

On the sys.modules stub itself

I don't think it can work, and it's worth writing down so nobody tries it again. PyO3 builds PanicException via PyErr_NewExceptionWithDoc("pyo3_runtime.PanicException", …), which sets the type's __module__ but never registers a pyo3_runtime module in sys.modules — not on import, and not even after a panic actually fires:

REAL panic raised: <class 'pyo3_runtime.PanicException'> | __module__: pyo3_runtime | base: BaseException
pyo3_runtime in sys.modules AFTER a real panic: False

pickle of real panic:       PicklingError -> ... import of module 'pyo3_runtime' failed
pickle WITH this PR's stub: PicklingError -> ... it's not the same object as pyo3_runtime.PanicException
stub catches real panic?    False

The pickling happens in the child (pebble/pool/process.py:435 send_result → Connection.send), and the pool forks, so the child inherits the stub — then pickle.save_global's identity check (getattr(mod, 'PanicException') is obj) fails against the real type. Same outcome either way, just a different message. The if "pyo3_runtime" not in sys.modules guard also can never be skipped, so the stub permanently owns that name; anyone later writing the natural-looking except pyo3_runtime.PanicException: would get the fake class and match nothing.

If we do want panics to survive the trip, the place to do it is the worker boundary in run_task — convert to a picklable RuntimeError carrying str(exc) and the traceback. That's engine-agnostic and keeps the real message (RuntimeError: PanicException: attempt to calculate the remainder with a divisor of zero) instead of a PicklingError. Happy to add that hunk if you want it.

Two bits of cheap hardening, no known trigger

Being upfront that these are belt-and-braces, not fixes for anything I could reproduce: init_worker's yara pre-compile and get_yara()'s scan both guard with except Exception:, and a PyO3 panic is BaseException-derived, so it would slip past either one. Widening them is a one-word change each (with a type(e).__module__ != "pyo3_runtime": raise guard in get_yara so Ctrl-C still propagates).

I say "no known trigger" deliberately — I installed yara-x 1.19.0 and tried ~20 ways to make it panic. It degrades cleanly every time: syntax errors, unknown modules, undefined identifiers, duplicate rules, bad hex strings, malformed PE/ELF/Mach-O/dotnet input all come back as CompileError / ScanError / TimeoutError, which init_yara's prune-and-retry loop already handles properly. The only hard failure I could induce was a stack overflow from a synthetically absurd rule ('not ' * 100000), and that's a SIGSEGV — no Python handler catches it, which is the reaper's job, not these two.

So: yara-x looks well-behaved, and our handling of it looks right. Happy to drop these two if you'd rather keep the diff tight.

The gap

I still can't tell you which extension is raising the PanicException you're seeing — I couldn't get one out of yara-x at all. Could you paste the traceback? If your workers are dying quietly at startup with nothing marked failed, the reaper is the fix. If the panic is surfacing somewhere else, I'd rather chase that than guess.

Everything above sits on top of your branch: ruff clean, engine + yara suites at 28 passed / 1 skipped (the skip is test_yara_x, which is also why CI wouldn't have caught any of this). Glad to open it as a PR against your branch so you keep the credit for finding it.

[doomedraven]

well that fix worked for me, so at least for now i have my servers working, i will review on monday your suggestion, thank you for checking. the traceback just in case:

2026-08-07 00:09:26,184 [lib.cuckoo.core.processing_engine.pebble] INFO: Processing analysis data for Task #462060
2026-08-07 00:09:26,313 [Task 462060 (3286593)] [dev_utils.mongodb] INFO: Successfully connected to MongoDB at 10.34.3.20:27017
2026-08-07 00:09:26,347 [lib.cuckoo.core.processing_engine.pebble] ERROR: [462060] Exception when processing task: Can't pickle <class 'pyo3_runtime.PanicException'>: import of module 'pyo3_runtime' failed
Traceback (most recent call last):
  File "/opt/CAPEv2/utils/../lib/cuckoo/core/processing_engine/pebble.py", line 52, in _done
    future.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
_pickle.PicklingError: Can't pickle <class 'pyo3_runtime.PanicException'>: import of module 'pyo3_runtime' failed
2026-08-07 00:09:26,348 [lib.cuckoo.core.data.tasking] INFO: setstat task 462060 status failed_processing

[doomedraven]

yes your changes seems to work, i have pushed those + few other fixes to another things, thanks for review @wmetcalf

[doomedraven]

now error with current code, so as is yara_x issue will see with Victor as im on latest version of rust/yara-x/python-yara-x

Traceback (most recent call last):                                                          
  File "/opt/CAPEv2/utils/process.py", line 201, in run_task
    process(                                                                 
  File "/opt/CAPEv2/utils/process.py", line 137, in process
    RunProcessing(task=task_dict, results=results).run()                           
  File "/opt/CAPEv2/utils/../lib/cuckoo/core/plugins.py", line 364, in run
    result = self.process(module)                                                                                                 
  File "/opt/CAPEv2/utils/../lib/cuckoo/core/plugins.py", line 322, in process                           
    data = current.run()                                                                                          
  File "/opt/CAPEv2/utils/../modules/processing/CAPE.py", line 473, in run
    self.process_file(                                       
  File "/opt/CAPEv2/utils/../modules/processing/CAPE.py", line 246, in process_file
    file_info, pefile_object = f.get_all()                                                  
  File "/opt/CAPEv2/utils/../lib/cuckoo/common/objects.py", line 795, in get_all
    "yara": self.get_yara(),                                                 
  File "/opt/CAPEv2/utils/../lib/cuckoo/common/objects.py", line 623, in get_yara
    yara_results = rules.scan_file(self.file_path)                                 
pyo3_runtime.PanicException: assertion `left == right` failed: yara_x::Scanner is unsendable, but sent to another thread
  left: ThreadId(2)                                                                                                               
 right: ThreadId(1) 

[wmetcalf]

Your fix yara-x commit is right, and I can say that with more than an opinion: I'd written thread-affinity tests against my own version of this, and all four pass against e477564 unmodified. Same two lines I'd landed on, so that's independent agreement rather than me agreeing with myself.

Three follow-ups came out of chasing this, so I've put them against this branch rather than a cleanup PR that never happens — #3152. Three separate commits, drop any of them freely:

1. ruff is red on the current head — import sys in pebble.py went unused with the stub, and the new engine tests have whitespace on blank lines. Your pre-commit hook fails on both.
2. Nothing guards the Scanner fix from coming back. test_yara_x skips whenever yara-x isn't installed — the default — so CI never executes that branch at all, which is why this reached production. Four tests, skipped without yara-x so CI behaviour is unchanged.
3. get_yara()'s fallback recompiles per file. Pre-existing, but it's in the function you just touched: a category that can't produce rules forces a full six-category recompile for every scanned file (~3s each on a production ruleset, and CAPE.py/procmemory.py call it per extracted payload). Now a 300s backoff.

One thing worth knowing if you ever want to kill the per-scan yara_x.Scanner(): caching it per-thread doesn't work either. I tried it and backed it out — Scanner is unsendable for drop as well as use, so a cached one gets dropped on the wrong thread on fork (pebble respawning a worker) and at interpreter shutdown for daemon web threads, printing

RuntimeError: yara_x::Scanner is unsendable, but is being dropped on another thread

straight to stderr from PyO3. Not worth trading a panic for that to save ~0.19ms/file. There's a comment at the cache site saying so.

And on the yara-x side generally: I installed 1.19.0 and spent a while trying to make it misbehave — syntax errors, unknown modules, duplicate rules, malformed PE/ELF/Mach-O/dotnet input. It degrades cleanly into CompileError/ScanError/TimeoutError every time, which init_yara's prune-and-retry already handles. So I don't think there's anything to raise with Victor — it was correctly telling us we were misusing Scanner.