Skip to content

Fix cancelled timer payload lifetimes and compact timer queues - #119

Merged
egorsmkv merged 4 commits into
masterfrom
fix/cancelled-timer-lifetimes
Oct 11, 2026
Merged

egorsmkv merged 4 commits into
masterfrom
fix/cancelled-timer-lifetimes

Conversation

@egorsmkv

@egorsmkv egorsmkv commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #118.

Cancelled timers behind an earlier live deadline kept callback, args and context until heap removal, pinning completed timeout tasks and closed asyncpg statement caches. TimerCallback now takes Python ownership under a timer-only mutex and releases it outside the lock. Dispatch claims ownership under the same lock and keeps running callbacks valid through concurrent cancellation. The existing call_soon/Handle layout and invocation path are unchanged.

Both timer queues compact after more than 100 entries and over half the queue are cancelled. Idempotent accounting handles cross-thread cancellation/removal and shutdown; removed shells go to the ready drain. Tests cover immediate ownership release, contexts, repeated cancellation, finalizer reentry, cancellation after dequeue, expired timeouts, running invocation ownership and heap ordering.

TDD: the first commit fails 12 lifetime regressions on 0.1.63; a local Rust regression found 257 queued entries instead of 1. Real baseline PostgreSQL CI retains 500 caches and 1500 prepared statements after 500 closed connections; asyncio/uvloop retain 0. Fixed PostgreSQL 3.14 CI reports 0 for caches, statements and protocols, with 39 lifecycle tests passing. Fixed local 20,000-timer/timeout probes retain 0 payloads/tasks after turns: Python retention drops from 87.2 MB / 101.0 MB to 307 B / 29.9 KB. Timer RSS drops from 125.2 to 33.1 MiB and timeout RSS from 150.4 to 34.1 MiB in fresh-process probes.

Validation: 51 differential tests passed (6 intentional control skips); 19 new free-threaded tests plus 5 existing timer tests passed. Local core: 628 passed; ecosystem: 78 passed. Two core Unix-socket failures reproduce on the baseline; local Rust syscall-restricted cases are documented, not counted as passing. Clippy with all targets/features, fmt, Ruff and targeted Pyright passed. CI is being monitored; a Python 3.10-only benchmark compatibility failure was corrected to use wait_for and skip unavailable asyncio.timeout.

Performance: two reversed-order release-build pairs, 14 samples per build, 1M operations/sample. call_soon is stable (-0.4%, noise); timer-only synchronization costs about 15–30 ns/op (+5.8% retained/cancelled, +11.5% discarded). Scattered cancelled timers measured -2.7%; no general speedup claim.

Measurements and reproduction probes are in benches/timer_cancellation.py and the baseline PostgreSQL CI / passing fixed CI. The report file was removed before merge.

The PostgreSQL/memory checks run against a real CI PostgreSQL service. The author's 300-second Granian production workload was not repeated.

@egorsmkv
egorsmkv marked this pull request as ready for review October 11, 2026 09:07
@egorsmkv
egorsmkv merged commit 8564a04 into master Oct 11, 2026
44 checks passed
@egorsmkv
egorsmkv deleted the fix/cancelled-timer-lifetimes branch October 11, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cancelled TimerHandle keeps its callback, args and context alive until every earlier timer has fired (asyncio.timeout, asyncpg statement cache)

1 participant