Skip to content

Bump PocketIC to 16.0.0 - #9624

Merged
hpeebles merged 4 commits into
masterfrom
claude/bump-pocket-ic-16
Sep 29, 2026
Merged

hpeebles merged 4 commits into
masterfrom
claude/bump-pocket-ic-16

Conversation

@hpeebles

@hpeebles hpeebles commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Bumps PocketIC from 11.0.0 to 16.0.0 (library and server binary) and adapts the integration tests to what changed underneath. This is meant to merge ahead of #9618.

PocketIC 16 doesn't change the library API in any way we use. Its replica, however, is nine months newer, and three of those changes affect the tests.

1. Every canister now pays a base fee, and storage costs 2.5× more

dfinity/ic#10292 charges every canister 10k cycles/s (about 0.86B cycles a day) regardless of what it uses. Storage went from 127,000 to 317,500 cycles per GiB per second (dfinity/ic#9127, #9424, #9752). Both are live on mainnet. Measured idle burn:

Canister PocketIC 11 (cycles/day) PocketIC 16 (cycles/day)
Minimal wat canister 0.69M 866M
Fresh user canister 46M 980M
LocalUserIndex 481M 1.98B

This broke tests in two ways:

  • Clock jumps drained canisters. PocketIC starts at May 2021. Several tests jump to the current time (ensure_time_at_least_day0, the daily puzzle and video call tests), about 5.4 years, which now burns roughly 1.9T from child canisters that start with 0.5T. Those canisters ran out of cycles and were uninstalled. Because envs are pooled, later tests that drew the same env failed too. Fix: envs now start at the current time (with_initial_time), so none of those jumps are needed.
  • Cycle-refund residuals grew. The refunder can send only the liquid balance, which excludes the freezing threshold, now about 26B (30 days of the base fee). The residual went from about 82B to about 108B, over the 100B limits in cycles_refunder_tests and delete_user_tests. Fix: raised both limits to 125B and updated the explanation, including the one in the refunder's README.

2. Cross-subnet messages take 10 rounds instead of 1

Measured with a direct message between users on different subnets: delivered after exactly 10 ticks, deterministically, versus 1 tick on PocketIC 11. Same-subnet delivery is still immediate. The replica's move to asynchronous state hashing (dfinity/ic#8464) is the likely cause, and PocketIC has no setting for it.

New users go to whichever LocalUserIndex has fewest users. In pooled envs, whether a test's users (and so their chats, groups and communities) share a subnet therefore depends on the tests that ran before it. Fix: raised the ticks wherever a test waits on a cross-subnet hop. Where I measured, I used the measured count plus a margin:

  • Group → community conversion: 24 ticks when the community lands on the other subnet, so 30 in place of 20.
  • Verification changes: 13 ticks to reach a community via its LocalUserIndex, so a TICKS_FOR_VERIFICATION_TO_PROPAGATE constant of 20.

The slower delivery also affects bounded-wait calls. Jumping the clock (for example by a day, to expire a p2p swap) while such a call is still crossing subnets makes it pass its deadline ("Call deadline has expired"), so deposit_refunded_if_swap_expires now lets the offer land before jumping.

join_private_group_with_invitation_succeeds and call_push_tests::invariant_1_only_small_private_groups_ring also got more ticks. Both already flake on master for the same reason, which PocketIC 16 made worse.

3. Pay-as-you-go HTTPS outcall pricing is behind a flag

pricing_version: 2 is honoured only where the replica's flexible_http_requests flag is on. It was switched on in dfinity/ic#11399 (2026-09-02). I checked all 42 mainnet subnets: every one runs a replica built after that commit, with the flag enabled. PocketIC 16 predates the switch, so it enables the flag only with beta features, and beta features consist of just that flag. Fix: envs now enable beta features, matching mainnet. This is needed to test pay-as-you-go outcalls, such as the OpenAI moderation migration that follows #9618.

Other

  • One video call test sets the clock to the current real time so that its JWTs verify. A pooled env can now already be ahead of that, and PocketIC 16 rejects SettingTimeIntoPast, so the test uses a fresh env.
  • The stop, act, start retry tests (fire_and_forget_handler_tests, send_direct_message_tests) now wait out a cross-subnet round trip before restarting the canister. Otherwise the first attempt lands after the restart and nothing fails.
  • daily_puzzle_start_submit_and_streak now discards its env. It pushes a made-up light_up puzzle for the current day into the LocalUserIndex, which shadowed the real puzzle in daily_puzzle_end_to_end whenever that test drew the same env on a day whose real puzzle is also light_up. That now happens reliably, because pooled envs reach 2026-11-01.
  • local_canister_creator reads the default effective canister id from the topology JSON itself. pocket-ic 16's Topology requires fields, such as cost_schedule, that the PocketIC server bundled with dfx doesn't return, so deploy-local.sh would otherwise panic.
  • pocket-ic 16.0.0 pins thiserror = "=2.0.18", so the lockfile moves thiserror down one patch version from 2.0.19.

Test plan

  • ./scripts/run-integration-tests.sh build 8 passes on PocketIC 16.0.0

🤖 Generated with Claude Code

PocketIC 16's replica charges every canister a base fee and prices storage 2.5x higher, and
delivers cross-subnet messages after 10 rounds rather than 1. Adapt the integration tests:

- Start envs at the current time, since jumping from PocketIC's 2021 genesis to today now drains
  child canisters of their cycles
- Allow for the freezing threshold, now ~26B, in the cycles refunder's residual balance
- Tick enough for cross-subnet hops to land where tests wait on them
- Enable beta features, ie. the `flexible_http_requests` flag, which is enabled on every mainnet
  subnet and lets HTTPS outcalls use pay-as-you-go pricing
- Use a fresh env where a test sets the clock to the current time, since a pooled env's clock may
  already be past it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

canbench 🏋 (dir: .)

./canbench_results.yml is up to date ✅

---------------------------------------------------

Benchmark: add_reactions
  total:
    instructions: 1.57 B (-0.96%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: import_events
  total:
    instructions: 124.06 M (0.28%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: migrate_imported_user_metrics
  total:
    instructions: 10.78 M (1.05%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: push_simple_text_messages
  total:
    instructions: 640.51 M (0.01%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: search_messages
  total:
    instructions: 128.23 M (0.77%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

…olation and local canister creation

- Tick enough for cross-subnet hops at the remaining sites which assert on the other user's copy
- Let the stop/act/start retry tests wait out a cross-subnet round trip before restarting the
  canister, else the first attempt only lands after the restart and nothing fails
- Discard the env after `daily_puzzle_start_submit_and_streak`, whose made up `light_up` puzzle
  for today otherwise shadows the real one in any later test which draws the env on a day the
  real puzzle is also `light_up`
- Read the default effective canister id from the topology JSON in `local_canister_creator`, since
  pocket-ic 16's `Topology` can't parse the response of the PocketIC server bundled with dfx

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hpeebles and others added 2 commits September 29, 2026 01:56
- `join_private_group_with_invitation_succeeds` and the community broadcast in
  `invariant_1_only_small_private_groups_ring` waited too few rounds for a cross-subnet hop (both
  also flake on master for the same reason)
- `deposit_refunded_if_swap_expires` jumped the clock a day while the bounded wait call delivering
  the offer to user2 could still be in flight across subnets, so it passed its deadline and user2
  never had the swap to update

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hpeebles
hpeebles merged commit e8541d4 into master Sep 29, 2026
13 checks passed
@hpeebles
hpeebles deleted the claude/bump-pocket-ic-16 branch September 29, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant