Skip to content

Commit b9474ef

Browse files
authored
docs: Rust stable structures guide (#73)
## Summary - Explains the `ic-stable-structures` library and when to use it over heap storage - Covers `Storable` trait implementation with CBOR serialization, `MemoryManager` partitioning with named `MemoryId` constants, and complete working canister examples - Includes `StableBTreeMap`, `StableCell`, `StableVec`, and `StableLog` with their two-region pattern - Decision table comparing heap vs stable memory; upgrade verification workflow - Common mistakes section (MemoryId reuse, missing `Bound`, upgrade-erased heap state) ## Sync recommendation informed by dfinity/cdk-rs — ic-stable-structures, icskills stable-memory skill
1 parent cbd5c76 commit b9474ef

1 file changed

Lines changed: 304 additions & 12 deletions

File tree

Lines changed: 304 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,313 @@
11
---
22
title: "Stable Structures"
3-
description: "Use StableBTreeMap, StableVec, and MemoryManager for upgrade-safe Rust data"
3+
description: "Use StableBTreeMap, StableCell, StableLog, StableVec, and MemoryManager for upgrade-safe persistent storage in Rust canisters"
44
sidebar:
55
order: 2
66
---
77

8-
TODO: Write content for this page.
8+
Stable structures are data structures that read and write directly to stable memory, bypassing the heap entirely. Unlike heap data, stable memory survives canister upgrades — no `pre_upgrade`/`post_upgrade` serialization hooks required.
99

10-
<!-- Content Brief -->
11-
Use stable data structures in Rust canisters for upgrade-safe storage. Cover StableBTreeMap, StableVec, StableCell, MemoryManager for multiple data structures, when to use stable vs heap storage, and migration patterns between stable structure versions. Include practical examples for common patterns (key-value store, user records, counters).
10+
The [`ic-stable-structures`](https://docs.rs/ic-stable-structures/latest/ic_stable_structures/) crate provides the building blocks. This page covers how to use them in Rust canisters.
1211

13-
<!-- Source Material -->
14-
- Portal: developer-tools/cdks/rust/stable-structures.mdx, canister-state.mdx
15-
- icskills: stable-memory
16-
- Rust CDK: https://docs.rs/ic-cdk/latest/ic_cdk/
12+
## Why stable structures
1713

18-
<!-- Cross-Links -->
19-
- guides/backends/data-persistence -- general persistence guide
20-
- concepts/orthogonal-persistence -- conceptual background
21-
- guides/canister-management/lifecycle -- upgrades and stable memory
14+
On ICP, heap memory (Wasm linear memory) is preserved across update calls within a session but is **wiped on every canister upgrade**. Any data you store in a plain `HashMap` or `Vec` on the heap will be lost the next time you `icp deploy`.
15+
16+
The two approaches to persistence across upgrades are:
17+
18+
| Approach | When to use |
19+
|----------|-------------|
20+
| **Stable structures** | Recommended for all new canisters. Data lives in stable memory directly — no serialization step, no instruction-limit risk. |
21+
| **Pre/post upgrade hooks** | Simple to add to existing code, but does not scale. Serializing large datasets in `pre_upgrade` can hit the instruction limit and brick the canister. |
22+
23+
Stable structures eliminate the upgrade risk entirely. The `MemoryManager` partitions stable memory (which can grow to hundreds of GB) into independent virtual regions, one per data structure.
24+
25+
## Add the dependency
26+
27+
Add `ic-stable-structures` to your `Cargo.toml`:
28+
29+
```toml
30+
[package]
31+
name = "backend"
32+
version = "0.1.0"
33+
edition = "2021"
34+
35+
[lib]
36+
crate-type = ["cdylib"]
37+
38+
[dependencies]
39+
ic-cdk = "0.19"
40+
ic-stable-structures = "0.7"
41+
candid = "0.10"
42+
serde = { version = "1", features = ["derive"] }
43+
ciborium = "0.2"
44+
```
45+
46+
`ciborium` provides CBOR serialization for custom types stored in stable memory. CBOR is compact and fast — preferred over Candid for this use case.
47+
48+
## Available structures
49+
50+
The crate provides several persistent data structures. See the [full API reference](https://docs.rs/ic-stable-structures/latest/ic_stable_structures/) to look up each type:
51+
52+
| Type | Use case |
53+
|------|----------|
54+
| `StableBTreeMap` | Key-value store. Keys must implement `Storable + Ord`. |
55+
| `StableCell` | Single persistent value — counters, configuration, state flags. |
56+
| `StableLog` | Append-only log. Efficient for event streams and audit trails. |
57+
| `StableVec` | Ordered sequence. Efficient indexed access. |
58+
| `StableBTreeSet` | Set of unique keys. Efficient membership tests and range queries. |
59+
| `StableMinHeap` | Priority queue — smallest element dequeued first. |
60+
61+
## Implement Storable for custom types
62+
63+
Every key and value stored in a stable structure must implement the `Storable` trait, which defines how the value is serialized to and from bytes in stable memory. Primitive types (`u64`, `u32`, `bool`, `f64`), `String`, `Vec<u8>`, and `Principal` implement `Storable` already.
64+
65+
For custom structs, implement `Storable` using CBOR serialization:
66+
67+
```rust
68+
use ic_stable_structures::storable::{Bound, Storable};
69+
use candid::CandidType;
70+
use serde::{Deserialize, Serialize};
71+
use std::borrow::Cow;
72+
73+
#[derive(CandidType, Serialize, Deserialize, Clone)]
74+
struct User {
75+
id: u64,
76+
name: String,
77+
created_at: u64,
78+
}
79+
80+
impl Storable for User {
81+
// Use Unbounded to avoid compatibility issues when adding new fields.
82+
// Bound::Bounded requires a fixed max_size — exceeding it after a
83+
// schema change breaks deserialization of existing data.
84+
const BOUND: Bound = Bound::Unbounded;
85+
86+
fn to_bytes(&self) -> Cow<'_, [u8]> {
87+
let mut buf = vec![];
88+
ciborium::into_writer(self, &mut buf).expect("failed to encode User");
89+
Cow::Owned(buf)
90+
}
91+
92+
// `into_bytes` was added in ic-stable-structures 0.7. If you are upgrading
93+
// from 0.6.x, add this method — it is not required in 0.6.
94+
fn into_bytes(self) -> Vec<u8> {
95+
let mut buf = vec![];
96+
ciborium::into_writer(&self, &mut buf).expect("failed to encode User");
97+
buf
98+
}
99+
100+
fn from_bytes(bytes: Cow<'_, [u8]>) -> Self {
101+
ciborium::from_reader(bytes.as_ref()).expect("failed to decode User")
102+
}
103+
}
104+
```
105+
106+
## MemoryManager and MemoryId
107+
108+
The `MemoryManager` partitions a single stable memory region into virtual regions. Each data structure is allocated its own `MemoryId`. Two structures that share a `MemoryId` corrupt each other's data — always use distinct IDs.
109+
110+
```rust
111+
use ic_stable_structures::{
112+
memory_manager::{MemoryId, MemoryManager, VirtualMemory},
113+
DefaultMemoryImpl,
114+
};
115+
116+
type Memory = VirtualMemory<DefaultMemoryImpl>;
117+
118+
// Declare IDs as constants to prevent accidental reuse.
119+
const USERS_MEM_ID: MemoryId = MemoryId::new(0);
120+
const COUNTER_MEM_ID: MemoryId = MemoryId::new(1);
121+
const LOG_INDEX_MEM_ID: MemoryId = MemoryId::new(2);
122+
const LOG_DATA_MEM_ID: MemoryId = MemoryId::new(3);
123+
```
124+
125+
`StableLog` requires two separate memory regions — one for the index and one for the data.
126+
127+
## Canister wiring example
128+
129+
The following snippet shows how to wire a `StableBTreeMap` and a `StableCell` into a canister using `thread_local!`. The `User` struct and its `Storable` implementation are defined in the previous section.
130+
131+
```rust
132+
use ic_stable_structures::{
133+
memory_manager::{MemoryId, MemoryManager, VirtualMemory},
134+
DefaultMemoryImpl, StableBTreeMap,
135+
};
136+
use ic_cdk::{init, post_upgrade};
137+
use std::cell::RefCell;
138+
139+
type Memory = VirtualMemory<DefaultMemoryImpl>;
140+
141+
const USERS_MEM_ID: MemoryId = MemoryId::new(0);
142+
const COUNTER_MEM_ID: MemoryId = MemoryId::new(1);
143+
144+
thread_local! {
145+
static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
146+
RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));
147+
148+
static USERS: RefCell<StableBTreeMap<u64, User, Memory>> =
149+
RefCell::new(StableBTreeMap::init(
150+
MEMORY_MANAGER.with(|m| m.borrow().get(USERS_MEM_ID))
151+
));
152+
153+
static COUNTER: RefCell<ic_stable_structures::StableCell<u64, Memory>> =
154+
RefCell::new(ic_stable_structures::StableCell::init(
155+
MEMORY_MANAGER.with(|m| m.borrow().get(COUNTER_MEM_ID)),
156+
0u64,
157+
).expect("failed to init COUNTER"));
158+
}
159+
160+
#[init]
161+
fn init() {}
162+
163+
#[post_upgrade]
164+
fn post_upgrade() {
165+
// Stable data auto-restores — re-initialize timers or transient state here.
166+
}
167+
```
168+
169+
Key patterns:
170+
171+
- `thread_local! { RefCell<StableBTreeMap<...>> }` is the standard pattern. The `RefCell` allows interior mutability inside `thread_local!`; the stable structure itself lives in stable memory, not the heap.
172+
- `#[post_upgrade]` is safe to omit for stable data (the data restores automatically), but defining it explicitly is the right place to re-initialize timers or other transient state after an upgrade.
173+
- `ic_cdk::api::time()` returns nanoseconds since the Unix epoch as a `u64`.
174+
175+
For a fully runnable canister with canister methods and `ic_cdk::export_candid!()`, see the [unit_testable_rust_canister](https://github.com/dfinity/examples/tree/master/rust/unit_testable_rust_canister) example in dfinity/examples.
176+
177+
## Multiple data structures
178+
179+
When a canister needs more than one stable structure, allocate a unique `MemoryId` for each. `StableLog` requires two IDs — one for its index and one for its data:
180+
181+
```rust
182+
// Declare all IDs as named constants to prevent accidental reuse.
183+
const USERS_MEM_ID: MemoryId = MemoryId::new(0);
184+
const COUNTER_MEM_ID: MemoryId = MemoryId::new(1);
185+
// StableLog requires two separate memory regions.
186+
const LOG_INDEX_MEM_ID: MemoryId = MemoryId::new(2);
187+
const LOG_DATA_MEM_ID: MemoryId = MemoryId::new(3);
188+
189+
// Initialize StableLog in thread_local! the same way as other structures:
190+
// StableLog::init(index_memory, data_memory).expect("failed to init LOG")
191+
```
192+
193+
IDs are stable across upgrades — never renumber them. Adding a new structure always gets the next available integer.
194+
195+
## StableVec usage
196+
197+
`StableVec` is an ordered, indexed sequence backed by stable memory. Use it when you need positional access by index rather than key-based lookup:
198+
199+
```rust
200+
use ic_stable_structures::{StableVec, memory_manager::{MemoryId, MemoryManager, VirtualMemory}, DefaultMemoryImpl};
201+
use ic_cdk::{query, update};
202+
use std::cell::RefCell;
203+
204+
type Memory = VirtualMemory<DefaultMemoryImpl>;
205+
const VEC_MEM_ID: MemoryId = MemoryId::new(0);
206+
207+
thread_local! {
208+
static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
209+
RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));
210+
211+
static ITEMS: RefCell<StableVec<u64, Memory>> =
212+
RefCell::new(StableVec::init(
213+
MEMORY_MANAGER.with(|m| m.borrow().get(VEC_MEM_ID))
214+
).expect("failed to init ITEMS"));
215+
}
216+
217+
#[update]
218+
fn push_item(value: u64) {
219+
ITEMS.with(|v| v.borrow_mut().push(&value).expect("failed to push"));
220+
}
221+
222+
#[query]
223+
fn get_item(index: u64) -> Option<u64> {
224+
ITEMS.with(|v| v.borrow().get(index))
225+
}
226+
```
227+
228+
## When to use heap vs stable memory
229+
230+
| Scenario | Use |
231+
|----------|-----|
232+
| Data that must survive upgrades (user records, balances, settings) | Stable structures |
233+
| Large datasets that could grow beyond a few MB | Stable structures — stable memory can grow to hundreds of GB |
234+
| Temporary computation state within a single call | Heap (`Vec`, `HashMap`) |
235+
| Caches that can be rebuilt after an upgrade | Heap (`Vec`, `HashMap`) reconstructed in `#[post_upgrade]` |
236+
| Small configuration that changes rarely | `StableCell` |
237+
238+
The pre/post upgrade hook pattern (serializing heap state in `pre_upgrade` and deserializing in `post_upgrade`) becomes dangerous as datasets grow. Both hooks run under a fixed instruction limit. Exceeding the limit traps the upgrade, leaving the canister stuck. Stable structures bypass this risk entirely because no serialization step occurs during upgrade.
239+
240+
## Verify persistence across upgrades
241+
242+
To confirm your canister actually persists data:
243+
244+
```bash
245+
# Start a local network
246+
icp network start -d
247+
248+
# Deploy the canister
249+
icp deploy backend
250+
251+
# Add some data
252+
icp canister call backend add_user '("Alice")'
253+
# Expected: (0 : nat64)
254+
255+
icp canister call backend add_user '("Bob")'
256+
# Expected: (1 : nat64)
257+
258+
# Record the count before upgrade
259+
icp canister call backend get_user_count '()'
260+
# Expected: (2 : nat64)
261+
262+
# Redeploy (simulates a code update + upgrade)
263+
icp deploy backend
264+
265+
# Count must still be 2 — not 0
266+
icp canister call backend get_user_count '()'
267+
# Expected: (2 : nat64)
268+
269+
# Data must still be retrievable
270+
icp canister call backend get_user '(0)'
271+
# Expected: (opt record { id = 0 : nat64; name = "Alice"; created_at = ... })
272+
```
273+
274+
If the count drops to 0 after redeployment, the data is not in stable memory. Check that the structures are initialized inside `thread_local!` using the `MemoryManager`, not as plain heap types.
275+
276+
## Common mistakes
277+
278+
**Reusing a `MemoryId` for two different structures** corrupts both. Declare IDs as named constants to prevent mistakes.
279+
280+
**Using `thread_local! { RefCell<HashMap<...>> }` for persistent state.** This is heap memory and is wiped on every upgrade. Use `StableBTreeMap` instead.
281+
282+
**Using `Bound::Bounded` with a `max_size` that is too small.** If you add a field to a struct later, existing records that fit the old `max_size` may still encode larger than expected, or the new layout may exceed the bound and break deserialization. Prefer `Bound::Unbounded` unless you have a specific reason to bound the size.
283+
284+
**Omitting `#[post_upgrade]` when you have timers or transient state.** Stable data is safe without a `#[post_upgrade]` hook — the structures read from stable memory automatically on first access. The real reason to define the hook is to re-initialize timers and other transient heap state that is lost on upgrade. If your canister uses timers, omitting `#[post_upgrade]` means timers silently stop firing after an upgrade.
285+
286+
**Serializing heap data in `pre_upgrade` as the sole persistence strategy.** This does not scale. For canisters with user-facing data, use stable structures from the start.
287+
288+
## Schema evolution
289+
290+
Stable memory is persistent — once you deploy a canister, existing serialized bytes must remain readable after you add or change fields. `Bound::Unbounded` is the safe default because it allows the encoded size to grow without constraint, so adding a new field to a CBOR-serialized struct does not break reads of old records.
291+
292+
**Adding a field:** Use `Option<T>` for new fields so old records (which have no bytes for the field) deserialize correctly as `None`. CBOR skips unknown fields on deserialization, so a plain new field also works — but `Option<T>` makes the intent explicit.
293+
294+
```rust
295+
// Before upgrade:
296+
struct User { id: u64, name: String }
297+
298+
// After upgrade — old records deserialize with email = None:
299+
struct User { id: u64, name: String, email: Option<String> }
300+
```
301+
302+
**Changing a key type:** Changing the key type of a `StableBTreeMap` (for example, from `u32` to `u64`) is a breaking change — all existing keys are stored as the old type and the new type will not read them. To migrate, allocate a new `MemoryId` for a new map, copy data from the old map in `#[post_upgrade]`, then remove the old `MemoryId` in a subsequent upgrade once migration is complete.
303+
304+
**Never change `Bound::Bounded` max_size for live data.** Lowering it truncates existing records. Raising it is safe but may require a separate migration if old records are smaller than the new bound expects.
305+
306+
## Next steps
307+
308+
- [`ic-stable-structures` API reference](https://docs.rs/ic-stable-structures/latest/ic_stable_structures/) — complete trait and type documentation
309+
- [Data persistence guide](../../guides/backends/data-persistence.md) — cross-language comparison of persistence patterns
310+
- [Orthogonal persistence](../../concepts/orthogonal-persistence.md) — how ICP manages canister state
311+
- [Canister lifecycle](../../guides/canister-management/lifecycle.md) — upgrades, snapshots, and state management
312+
313+
<!-- Upstream: informed by dfinity/icskills skills/stable-memory/SKILL.md; dfinity/portal docs/building-apps/developer-tools/cdks/rust/stable-structures.mdx; dfinity/examples rust/unit_testable_rust_canister -->

0 commit comments

Comments
 (0)