Repository navigation
Commit 6411f1e
[executorch][native-ggml] Size the scheduler hash for graph leafs too
## Ulterior Motive
Running Gemma 4 31B through ExecuTorch Native's GGML engine on CUDA, to find why it decodes slower than llama.cpp on the same GGUF.
## Rationale
**What**: The backend scheduler is created for `2 * graph_size` tensors.
**Why**: `ggml_backend_sched` hashes every node and every leaf of a graph it splits, and asserts that its hash set holds them all. It was sized for `graph_size`, which caps nodes and leafs separately, so a graph whose nodes and leafs together outgrew it failed that assertion, though either count alone fit.
## Details
- **Before**: `ggml_backend_sched_new(..., graph_size, ...)`; ggml's own source notes the hash "needs to be size*2 to account for leafs".
- **After**: `2 * graph_size`, room for a full graph of `graph_size` nodes and `graph_size` leafs.
`test_graph_with_as_many_leafs_as_nodes` adds 2,100 separate constants to an input: 2,100 nodes and as many leafs, within the default graph size of 4,096.
Authored with Claude Code.
Differential Revision: [D130721991](https://our.internmc.facebook.com/intern/diff/D130721991/)
ghstack-source-id: 469829014
Pull-Request: #248471 parent aff7e4c commit 6411f1e
2 files changed
Lines changed: 15 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
492 | 492 | | |
493 | 493 | | |
494 | 494 | | |
| 495 | + | |
| 496 | + | |
495 | 497 | | |
496 | 498 | | |
497 | 499 | | |
498 | 500 | | |
499 | | - | |
| 501 | + | |
500 | 502 | | |
501 | 503 | | |
502 | 504 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1510 | 1510 | | |
1511 | 1511 | | |
1512 | 1512 | | |
| 1513 | + | |
| 1514 | + | |
| 1515 | + | |
| 1516 | + | |
| 1517 | + | |
| 1518 | + | |
| 1519 | + | |
| 1520 | + | |
| 1521 | + | |
| 1522 | + | |
| 1523 | + | |
| 1524 | + | |
1513 | 1525 | | |
1514 | 1526 | | |
1515 | 1527 | | |
| |||
0 commit comments