feat: ssz engine API transport#9382
Open
nazarhussain wants to merge 12 commits into
Open
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request implements EIP-8161, introducing an SSZ-REST transport for the Engine API to improve communication efficiency. It adds a new SszRestClient and specialized encoding/decoding logic, updating the ExecutionEngineHttp class to prioritize SSZ-REST for key methods with a fallback to JSON-RPC. Review feedback identified critical bugs in the SSZ encoding and decoding for getBlobs requests and responses, specifically regarding the incorrect use of offsets for fixed-size lists. Additionally, improvements were suggested to enhance redundancy by supporting multiple engine URLs, refactor duplicated versioning logic into a helper method, and utilize existing utility functions for hex-to-byte conversions.
Comment on lines
+191
to
+201
| const engineUrl = opts?.urls?.[0] ?? "http://localhost:8551"; | ||
| const baseUrl = engineUrl.replace(/\/+$/, ""); | ||
| this.sszRestClient = new SszRestClient({ | ||
| baseUrl, | ||
| jwtSecretHex: opts?.jwtSecretHex, | ||
| jwtId: opts?.jwtId, | ||
| jwtVersion: opts?.jwtVersion, | ||
| timeout: opts?.timeout, | ||
| }); | ||
| this.logger.info("SSZ-REST Engine API transport enabled (EIP-8161)", {url: baseUrl}); | ||
| } |
Contributor
There was a problem hiding this comment.
The SszRestClient is initialized using only the first URL from opts.urls. While the fallback to JSON-RPC (which handles multiple URLs) ensures correctness, this implementation will cause a timeout or connection error on every Engine API call if the primary execution node is offline before falling back. Consider making SszRestClient aware of all configured URLs or implementing a mechanism to share the currently active/healthy URL between the JSON-RPC and SSZ-REST clients to maintain redundancy efficiency.
| if (this.sszRestClient) { | ||
| try { | ||
| const version = | ||
| ForkSeq[fork] >= ForkSeq.fulu ? 5 : ForkSeq[fork] >= ForkSeq.electra ? 4 : ForkSeq[fork] >= ForkSeq.deneb ? 3 : ForkSeq[fork] >= ForkSeq.capella ? 2 : 1; |
Contributor
There was a problem hiding this comment.
The logic for determining the Engine API version based on the fork is repeated in notifyNewPayload, notifyForkchoiceUpdate (line 421), and getPayload (line 555). This duplication increases the risk of inconsistency when new forks are added. It is recommended to refactor this into a private helper method, e.g., getEngineApiVersion(fork: ForkName): number.
Contributor
|
| Benchmark suite | Current: c49c6fe | Previous: 1d0e0b9 | Ratio |
|---|---|---|---|
| Full columns - reconstruct all 20 blobs | 1.7634 ms/op | 574.63 us/op | 3.07 |
| Full columns - reconstruct half of the blobs out of 20 | 715.22 us/op | 155.07 us/op | 4.61 |
Full benchmark results
| Benchmark suite | Current: c49c6fe | Previous: 1d0e0b9 | Ratio |
|---|---|---|---|
| getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 1.0589 ms/op | 838.65 us/op | 1.26 |
| getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 40.136 us/op | 37.597 us/op | 1.07 |
| BLS verify - blst | 666.75 us/op | 714.67 us/op | 0.93 |
| BLS verifyMultipleSignatures 3 - blst | 1.3898 ms/op | 1.2826 ms/op | 1.08 |
| BLS verifyMultipleSignatures 8 - blst | 2.1887 ms/op | 2.0297 ms/op | 1.08 |
| BLS verifyMultipleSignatures 32 - blst | 6.8883 ms/op | 6.2793 ms/op | 1.10 |
| BLS verifyMultipleSignatures 64 - blst | 13.334 ms/op | 12.347 ms/op | 1.08 |
| BLS verifyMultipleSignatures 128 - blst | 26.033 ms/op | 24.319 ms/op | 1.07 |
| BLS deserializing 10000 signatures | 646.54 ms/op | 608.96 ms/op | 1.06 |
| BLS deserializing 100000 signatures | 6.5258 s/op | 6.0955 s/op | 1.07 |
| BLS verifyMultipleSignatures - same message - 3 - blst | 822.79 us/op | 768.48 us/op | 1.07 |
| BLS verifyMultipleSignatures - same message - 8 - blst | 959.15 us/op | 860.64 us/op | 1.11 |
| BLS verifyMultipleSignatures - same message - 32 - blst | 1.5663 ms/op | 1.4950 ms/op | 1.05 |
| BLS verifyMultipleSignatures - same message - 64 - blst | 2.5112 ms/op | 2.2952 ms/op | 1.09 |
| BLS verifyMultipleSignatures - same message - 128 - blst | 4.1202 ms/op | 3.9097 ms/op | 1.05 |
| BLS aggregatePubkeys 32 - blst | 17.672 us/op | 17.009 us/op | 1.04 |
| BLS aggregatePubkeys 128 - blst | 62.992 us/op | 60.816 us/op | 1.04 |
| getSlashingsAndExits - default max | 51.181 us/op | 42.204 us/op | 1.21 |
| getSlashingsAndExits - 2k | 420.09 us/op | 327.15 us/op | 1.28 |
| proposeBlockBody type=full, size=empty | 797.91 us/op | 648.86 us/op | 1.23 |
| isKnown best case - 1 super set check | 180.00 ns/op | 161.00 ns/op | 1.12 |
| isKnown normal case - 2 super set checks | 178.00 ns/op | 154.00 ns/op | 1.16 |
| isKnown worse case - 16 super set checks | 180.00 ns/op | 154.00 ns/op | 1.17 |
| validate api signedAggregateAndProof - struct | 1.5504 ms/op | 1.4433 ms/op | 1.07 |
| validate gossip signedAggregateAndProof - struct | 1.5597 ms/op | 1.4329 ms/op | 1.09 |
| batch validate gossip attestation - vc 640000 - chunk 32 | 112.23 us/op | 101.14 us/op | 1.11 |
| batch validate gossip attestation - vc 640000 - chunk 64 | 97.753 us/op | 89.376 us/op | 1.09 |
| batch validate gossip attestation - vc 640000 - chunk 128 | 91.170 us/op | 83.114 us/op | 1.10 |
| batch validate gossip attestation - vc 640000 - chunk 256 | 91.402 us/op | 79.304 us/op | 1.15 |
| bytes32 toHexString | 293.00 ns/op | 289.00 ns/op | 1.01 |
| bytes32 Buffer.toString(hex) | 177.00 ns/op | 159.00 ns/op | 1.11 |
| bytes32 Buffer.toString(hex) from Uint8Array | 247.00 ns/op | 229.00 ns/op | 1.08 |
| bytes32 Buffer.toString(hex) + 0x | 174.00 ns/op | 162.00 ns/op | 1.07 |
| Return object 10000 times | 0.22110 ns/op | 0.20660 ns/op | 1.07 |
| Throw Error 10000 times | 3.4414 us/op | 3.1635 us/op | 1.09 |
| toHex | 96.004 ns/op | 87.189 ns/op | 1.10 |
| Buffer.from | 90.534 ns/op | 98.504 ns/op | 0.92 |
| shared Buffer | 63.504 ns/op | 53.617 ns/op | 1.18 |
| fastMsgIdFn sha256 / 200 bytes | 1.5220 us/op | 1.4050 us/op | 1.08 |
| fastMsgIdFn h32 xxhash / 200 bytes | 161.00 ns/op | 143.00 ns/op | 1.13 |
| fastMsgIdFn h64 xxhash / 200 bytes | 209.00 ns/op | 194.00 ns/op | 1.08 |
| fastMsgIdFn sha256 / 1000 bytes | 4.8800 us/op | 4.5560 us/op | 1.07 |
| fastMsgIdFn h32 xxhash / 1000 bytes | 251.00 ns/op | 227.00 ns/op | 1.11 |
| fastMsgIdFn h64 xxhash / 1000 bytes | 267.00 ns/op | 243.00 ns/op | 1.10 |
| fastMsgIdFn sha256 / 10000 bytes | 42.884 us/op | 39.868 us/op | 1.08 |
| fastMsgIdFn h32 xxhash / 10000 bytes | 1.3150 us/op | 1.2370 us/op | 1.06 |
| fastMsgIdFn h64 xxhash / 10000 bytes | 857.00 ns/op | 799.00 ns/op | 1.07 |
| send data - 1000 256B messages | 4.6703 ms/op | 3.9784 ms/op | 1.17 |
| send data - 1000 512B messages | 5.0105 ms/op | 4.1560 ms/op | 1.21 |
| send data - 1000 1024B messages | 5.2856 ms/op | 4.2706 ms/op | 1.24 |
| send data - 1000 1200B messages | 5.0649 ms/op | 4.6114 ms/op | 1.10 |
| send data - 1000 2048B messages | 5.8145 ms/op | 4.6796 ms/op | 1.24 |
| send data - 1000 4096B messages | 6.7340 ms/op | 5.3242 ms/op | 1.26 |
| send data - 1000 16384B messages | 16.698 ms/op | 18.163 ms/op | 0.92 |
| send data - 1000 65536B messages | 266.73 ms/op | 156.89 ms/op | 1.70 |
| enrSubnets - fastDeserialize 64 bits | 776.00 ns/op | 722.00 ns/op | 1.07 |
| enrSubnets - ssz BitVector 64 bits | 299.00 ns/op | 249.00 ns/op | 1.20 |
| enrSubnets - fastDeserialize 4 bits | 110.00 ns/op | 102.00 ns/op | 1.08 |
| enrSubnets - ssz BitVector 4 bits | 276.00 ns/op | 247.00 ns/op | 1.12 |
| prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 222.66 us/op | 196.76 us/op | 1.13 |
| prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 253.38 us/op | 222.31 us/op | 1.14 |
| prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 376.40 us/op | 323.68 us/op | 1.16 |
| prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 651.09 us/op | 572.57 us/op | 1.14 |
| prioritizePeers score 0:0 att 64-1 sync 4-1 | 758.13 us/op | 668.78 us/op | 1.13 |
| array of 16000 items push then shift | 1.3324 us/op | 1.2022 us/op | 1.11 |
| LinkedList of 16000 items push then shift | 8.2820 ns/op | 7.1190 ns/op | 1.16 |
| array of 16000 items push then pop | 74.093 ns/op | 62.856 ns/op | 1.18 |
| LinkedList of 16000 items push then pop | 6.2450 ns/op | 5.6880 ns/op | 1.10 |
| array of 24000 items push then shift | 1.8720 us/op | 1.7670 us/op | 1.06 |
| LinkedList of 24000 items push then shift | 7.7810 ns/op | 6.7330 ns/op | 1.16 |
| array of 24000 items push then pop | 101.14 ns/op | 88.165 ns/op | 1.15 |
| LinkedList of 24000 items push then pop | 6.3780 ns/op | 5.8480 ns/op | 1.09 |
| intersect bitArray bitLen 8 | 4.7600 ns/op | 4.5820 ns/op | 1.04 |
| intersect array and set length 8 | 29.857 ns/op | 28.067 ns/op | 1.06 |
| intersect bitArray bitLen 128 | 24.362 ns/op | 23.175 ns/op | 1.05 |
| intersect array and set length 128 | 501.56 ns/op | 478.82 ns/op | 1.05 |
| bitArray.getTrueBitIndexes() bitLen 128 | 1.0850 us/op | 1.0230 us/op | 1.06 |
| bitArray.getTrueBitIndexes() bitLen 248 | 1.8350 us/op | 1.7540 us/op | 1.05 |
| bitArray.getTrueBitIndexes() bitLen 512 | 3.8610 us/op | 3.5150 us/op | 1.10 |
| Full columns - reconstruct all 6 blobs | 131.39 us/op | 229.42 us/op | 0.57 |
| Full columns - reconstruct half of the blobs out of 6 | 69.507 us/op | 123.01 us/op | 0.57 |
| Full columns - reconstruct single blob out of 6 | 34.642 us/op | 31.465 us/op | 1.10 |
| Half columns - reconstruct all 6 blobs | 407.37 ms/op | 368.26 ms/op | 1.11 |
| Half columns - reconstruct half of the blobs out of 6 | 203.62 ms/op | 183.97 ms/op | 1.11 |
| Half columns - reconstruct single blob out of 6 | 73.274 ms/op | 65.882 ms/op | 1.11 |
| Full columns - reconstruct all 10 blobs | 218.42 us/op | 308.58 us/op | 0.71 |
| Full columns - reconstruct half of the blobs out of 10 | 100.69 us/op | 150.97 us/op | 0.67 |
| Full columns - reconstruct single blob out of 10 | 33.743 us/op | 29.720 us/op | 1.14 |
| Half columns - reconstruct all 10 blobs | 669.71 ms/op | 607.35 ms/op | 1.10 |
| Half columns - reconstruct half of the blobs out of 10 | 339.31 ms/op | 306.71 ms/op | 1.11 |
| Half columns - reconstruct single blob out of 10 | 73.269 ms/op | 65.294 ms/op | 1.12 |
| Full columns - reconstruct all 20 blobs | 1.7634 ms/op | 574.63 us/op | 3.07 |
| Full columns - reconstruct half of the blobs out of 20 | 715.22 us/op | 155.07 us/op | 4.61 |
| Full columns - reconstruct single blob out of 20 | 31.825 us/op | 30.019 us/op | 1.06 |
| Half columns - reconstruct all 20 blobs | 1.3473 s/op | 1.2127 s/op | 1.11 |
| Half columns - reconstruct half of the blobs out of 20 | 684.84 ms/op | 602.83 ms/op | 1.14 |
| Half columns - reconstruct single blob out of 20 | 74.435 ms/op | 65.175 ms/op | 1.14 |
| Set add up to 64 items then delete first | 2.7917 us/op | 1.9657 us/op | 1.42 |
| OrderedSet add up to 64 items then delete first | 3.6257 us/op | 3.1124 us/op | 1.16 |
| Set add up to 64 items then delete last | 2.5290 us/op | 1.9689 us/op | 1.28 |
| OrderedSet add up to 64 items then delete last | 3.4966 us/op | 3.2183 us/op | 1.09 |
| Set add up to 64 items then delete middle | 2.2228 us/op | 2.0268 us/op | 1.10 |
| OrderedSet add up to 64 items then delete middle | 4.9971 us/op | 4.6593 us/op | 1.07 |
| Set add up to 128 items then delete first | 4.5267 us/op | 3.8780 us/op | 1.17 |
| OrderedSet add up to 128 items then delete first | 6.9866 us/op | 5.7324 us/op | 1.22 |
| Set add up to 128 items then delete last | 4.1795 us/op | 3.6930 us/op | 1.13 |
| OrderedSet add up to 128 items then delete last | 6.3013 us/op | 5.7317 us/op | 1.10 |
| Set add up to 128 items then delete middle | 4.1089 us/op | 3.6715 us/op | 1.12 |
| OrderedSet add up to 128 items then delete middle | 13.901 us/op | 11.533 us/op | 1.21 |
| Set add up to 256 items then delete first | 10.062 us/op | 7.2146 us/op | 1.39 |
| OrderedSet add up to 256 items then delete first | 17.001 us/op | 10.882 us/op | 1.56 |
| Set add up to 256 items then delete last | 7.6424 us/op | 7.2939 us/op | 1.05 |
| OrderedSet add up to 256 items then delete last | 14.357 us/op | 11.249 us/op | 1.28 |
| Set add up to 256 items then delete middle | 8.7479 us/op | 7.2014 us/op | 1.21 |
| OrderedSet add up to 256 items then delete middle | 41.116 us/op | 33.806 us/op | 1.22 |
| pass gossip attestations to forkchoice per slot | 2.5204 ms/op | 2.4786 ms/op | 1.02 |
| forkChoice updateHead vc 100000 bc 64 eq 0 | 380.40 us/op | 376.63 us/op | 1.01 |
| forkChoice updateHead vc 600000 bc 64 eq 0 | 2.3172 ms/op | 2.2451 ms/op | 1.03 |
| forkChoice updateHead vc 1000000 bc 64 eq 0 | 3.8475 ms/op | 3.7397 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 320 eq 0 | 2.3006 ms/op | 2.2394 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 1200 eq 0 | 2.3498 ms/op | 2.2596 ms/op | 1.04 |
| forkChoice updateHead vc 600000 bc 7200 eq 0 | 2.8099 ms/op | 2.5647 ms/op | 1.10 |
| forkChoice updateHead vc 600000 bc 64 eq 1000 | 2.9033 ms/op | 2.7800 ms/op | 1.04 |
| forkChoice updateHead vc 600000 bc 64 eq 10000 | 2.9463 ms/op | 2.8555 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 64 eq 300000 | 6.6201 ms/op | 6.5558 ms/op | 1.01 |
| computeDeltas 1400000 validators 0% inactive | 12.237 ms/op | 12.047 ms/op | 1.02 |
| computeDeltas 1400000 validators 10% inactive | 11.472 ms/op | 11.309 ms/op | 1.01 |
| computeDeltas 1400000 validators 20% inactive | 10.411 ms/op | 10.297 ms/op | 1.01 |
| computeDeltas 1400000 validators 50% inactive | 8.0577 ms/op | 7.8950 ms/op | 1.02 |
| computeDeltas 2100000 validators 0% inactive | 18.294 ms/op | 17.938 ms/op | 1.02 |
| computeDeltas 2100000 validators 10% inactive | 17.135 ms/op | 16.772 ms/op | 1.02 |
| computeDeltas 2100000 validators 20% inactive | 15.691 ms/op | 15.293 ms/op | 1.03 |
| computeDeltas 2100000 validators 50% inactive | 9.1535 ms/op | 8.9372 ms/op | 1.02 |
| altair processAttestation - 250000 vs - 7PWei normalcase | 1.8371 ms/op | 1.8084 ms/op | 1.02 |
| altair processAttestation - 250000 vs - 7PWei worstcase | 2.6456 ms/op | 2.5994 ms/op | 1.02 |
| altair processAttestation - setStatus - 1/6 committees join | 103.51 us/op | 96.291 us/op | 1.07 |
| altair processAttestation - setStatus - 1/3 committees join | 205.18 us/op | 186.41 us/op | 1.10 |
| altair processAttestation - setStatus - 1/2 committees join | 297.05 us/op | 273.98 us/op | 1.08 |
| altair processAttestation - setStatus - 2/3 committees join | 382.89 us/op | 353.14 us/op | 1.08 |
| altair processAttestation - setStatus - 4/5 committees join | 517.92 us/op | 485.67 us/op | 1.07 |
| altair processAttestation - setStatus - 100% committees join | 605.80 us/op | 580.68 us/op | 1.04 |
| altair processBlock - 250000 vs - 7PWei normalcase | 3.8178 ms/op | 3.9507 ms/op | 0.97 |
| altair processBlock - 250000 vs - 7PWei normalcase hashState | 14.501 ms/op | 16.669 ms/op | 0.87 |
| altair processBlock - 250000 vs - 7PWei worstcase | 19.926 ms/op | 20.563 ms/op | 0.97 |
| altair processBlock - 250000 vs - 7PWei worstcase hashState | 40.310 ms/op | 38.943 ms/op | 1.04 |
| phase0 processBlock - 250000 vs - 7PWei normalcase | 1.3762 ms/op | 1.1894 ms/op | 1.16 |
| phase0 processBlock - 250000 vs - 7PWei worstcase | 16.465 ms/op | 15.828 ms/op | 1.04 |
| altair processEth1Data - 250000 vs - 7PWei normalcase | 272.44 us/op | 269.37 us/op | 1.01 |
| getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:16 | 3.9090 us/op | 2.9790 us/op | 1.31 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:220 | 19.835 us/op | 19.532 us/op | 1.02 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:43 | 6.1300 us/op | 5.5950 us/op | 1.10 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:19 | 3.5500 us/op | 3.5160 us/op | 1.01 |
| getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1021 | 86.319 us/op | 88.239 us/op | 0.98 |
| getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11778 | 1.2581 ms/op | 1.2912 ms/op | 0.97 |
| getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 1.8044 ms/op | 1.7168 ms/op | 1.05 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 1.7661 ms/op | 1.7076 ms/op | 1.03 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 3.4005 ms/op | 3.5063 ms/op | 0.97 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 2.0475 ms/op | 1.9469 ms/op | 1.05 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 3.9388 ms/op | 3.8649 ms/op | 1.02 |
| Tree 40 250000 create | 390.22 ms/op | 295.39 ms/op | 1.32 |
| Tree 40 250000 get(125000) | 104.49 ns/op | 86.760 ns/op | 1.20 |
| Tree 40 250000 set(125000) | 1.0998 us/op | 951.61 ns/op | 1.16 |
| Tree 40 250000 toArray() | 9.3965 ms/op | 10.581 ms/op | 0.89 |
| Tree 40 250000 iterate all - toArray() + loop | 9.3744 ms/op | 11.634 ms/op | 0.81 |
| Tree 40 250000 iterate all - get(i) | 35.355 ms/op | 33.541 ms/op | 1.05 |
| Array 250000 create | 2.0343 ms/op | 2.0266 ms/op | 1.00 |
| Array 250000 clone - spread | 644.97 us/op | 620.43 us/op | 1.04 |
| Array 250000 get(125000) | 0.28300 ns/op | 0.29900 ns/op | 0.95 |
| Array 250000 set(125000) | 0.28000 ns/op | 0.29900 ns/op | 0.94 |
| Array 250000 iterate all - loop | 54.910 us/op | 55.267 us/op | 0.99 |
| phase0 afterProcessEpoch - 250000 vs - 7PWei | 60.551 ms/op | 37.378 ms/op | 1.62 |
| Array.fill - length 1000000 | 2.3159 ms/op | 2.0468 ms/op | 1.13 |
| Array push - length 1000000 | 8.1421 ms/op | 8.4481 ms/op | 0.96 |
| Array.get | 0.20653 ns/op | 0.20008 ns/op | 1.03 |
| Uint8Array.get | 0.23955 ns/op | 0.23596 ns/op | 1.02 |
| phase0 beforeProcessEpoch - 250000 vs - 7PWei | 18.758 ms/op | 12.160 ms/op | 1.54 |
| altair processEpoch - mainnet_e81889 | 347.20 ms/op | 233.51 ms/op | 1.49 |
| mainnet_e81889 - altair beforeProcessEpoch | 38.589 ms/op | 14.140 ms/op | 2.73 |
| mainnet_e81889 - altair processJustificationAndFinalization | 7.2690 us/op | 5.4250 us/op | 1.34 |
| mainnet_e81889 - altair processInactivityUpdates | 3.6384 ms/op | 3.3578 ms/op | 1.08 |
| mainnet_e81889 - altair processRewardsAndPenalties | 21.055 ms/op | 18.261 ms/op | 1.15 |
| mainnet_e81889 - altair processRegistryUpdates | 553.00 ns/op | 509.00 ns/op | 1.09 |
| mainnet_e81889 - altair processSlashings | 145.00 ns/op | 127.00 ns/op | 1.14 |
| mainnet_e81889 - altair processEth1DataReset | 144.00 ns/op | 123.00 ns/op | 1.17 |
| mainnet_e81889 - altair processEffectiveBalanceUpdates | 3.0607 ms/op | 1.1567 ms/op | 2.65 |
| mainnet_e81889 - altair processSlashingsReset | 700.00 ns/op | 668.00 ns/op | 1.05 |
| mainnet_e81889 - altair processRandaoMixesReset | 1.3590 us/op | 1.0180 us/op | 1.33 |
| mainnet_e81889 - altair processHistoricalRootsUpdate | 151.00 ns/op | 124.00 ns/op | 1.22 |
| mainnet_e81889 - altair processParticipationFlagUpdates | 442.00 ns/op | 410.00 ns/op | 1.08 |
| mainnet_e81889 - altair processSyncCommitteeUpdates | 124.00 ns/op | 101.00 ns/op | 1.23 |
| mainnet_e81889 - altair afterProcessEpoch | 41.376 ms/op | 41.403 ms/op | 1.00 |
| capella processEpoch - mainnet_e217614 | 946.14 ms/op | 805.48 ms/op | 1.17 |
| mainnet_e217614 - capella beforeProcessEpoch | 53.293 ms/op | 54.905 ms/op | 0.97 |
| mainnet_e217614 - capella processJustificationAndFinalization | 7.2600 us/op | 6.5970 us/op | 1.10 |
| mainnet_e217614 - capella processInactivityUpdates | 14.868 ms/op | 16.810 ms/op | 0.88 |
| mainnet_e217614 - capella processRewardsAndPenalties | 95.276 ms/op | 90.735 ms/op | 1.05 |
| mainnet_e217614 - capella processRegistryUpdates | 4.5450 us/op | 4.4440 us/op | 1.02 |
| mainnet_e217614 - capella processSlashings | 151.00 ns/op | 133.00 ns/op | 1.14 |
| mainnet_e217614 - capella processEth1DataReset | 145.00 ns/op | 129.00 ns/op | 1.12 |
| mainnet_e217614 - capella processEffectiveBalanceUpdates | 5.6640 ms/op | 19.067 ms/op | 0.30 |
| mainnet_e217614 - capella processSlashingsReset | 704.00 ns/op | 682.00 ns/op | 1.03 |
| mainnet_e217614 - capella processRandaoMixesReset | 1.4540 us/op | 1.4640 us/op | 0.99 |
| mainnet_e217614 - capella processHistoricalRootsUpdate | 147.00 ns/op | 131.00 ns/op | 1.12 |
| mainnet_e217614 - capella processParticipationFlagUpdates | 436.00 ns/op | 439.00 ns/op | 0.99 |
| mainnet_e217614 - capella afterProcessEpoch | 111.04 ms/op | 105.87 ms/op | 1.05 |
| phase0 processEpoch - mainnet_e58758 | 333.72 ms/op | 278.72 ms/op | 1.20 |
| mainnet_e58758 - phase0 beforeProcessEpoch | 64.293 ms/op | 58.051 ms/op | 1.11 |
| mainnet_e58758 - phase0 processJustificationAndFinalization | 6.7140 us/op | 5.7090 us/op | 1.18 |
| mainnet_e58758 - phase0 processRewardsAndPenalties | 15.868 ms/op | 15.406 ms/op | 1.03 |
| mainnet_e58758 - phase0 processRegistryUpdates | 2.2800 us/op | 2.1750 us/op | 1.05 |
| mainnet_e58758 - phase0 processSlashings | 153.00 ns/op | 125.00 ns/op | 1.22 |
| mainnet_e58758 - phase0 processEth1DataReset | 145.00 ns/op | 207.00 ns/op | 0.70 |
| mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 885.58 us/op | 788.58 us/op | 1.12 |
| mainnet_e58758 - phase0 processSlashingsReset | 815.00 ns/op | 842.00 ns/op | 0.97 |
| mainnet_e58758 - phase0 processRandaoMixesReset | 1.3910 us/op | 1.2260 us/op | 1.13 |
| mainnet_e58758 - phase0 processHistoricalRootsUpdate | 147.00 ns/op | 127.00 ns/op | 1.16 |
| mainnet_e58758 - phase0 processParticipationRecordUpdates | 1.2400 us/op | 1.0260 us/op | 1.21 |
| mainnet_e58758 - phase0 afterProcessEpoch | 31.820 ms/op | 32.522 ms/op | 0.98 |
| phase0 processEffectiveBalanceUpdates - 250000 normalcase | 949.68 us/op | 965.40 us/op | 0.98 |
| phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 1.5919 ms/op | 1.7241 ms/op | 0.92 |
| altair processInactivityUpdates - 250000 normalcase | 10.545 ms/op | 11.294 ms/op | 0.93 |
| altair processInactivityUpdates - 250000 worstcase | 10.600 ms/op | 11.907 ms/op | 0.89 |
| phase0 processRegistryUpdates - 250000 normalcase | 2.1270 us/op | 2.3670 us/op | 0.90 |
| phase0 processRegistryUpdates - 250000 badcase_full_deposits | 140.44 us/op | 135.45 us/op | 1.04 |
| phase0 processRegistryUpdates - 250000 worstcase 0.5 | 65.597 ms/op | 55.355 ms/op | 1.19 |
| altair processRewardsAndPenalties - 250000 normalcase | 13.903 ms/op | 15.341 ms/op | 0.91 |
| altair processRewardsAndPenalties - 250000 worstcase | 14.910 ms/op | 14.796 ms/op | 1.01 |
| phase0 getAttestationDeltas - 250000 normalcase | 5.4250 ms/op | 5.1814 ms/op | 1.05 |
| phase0 getAttestationDeltas - 250000 worstcase | 5.5057 ms/op | 5.2889 ms/op | 1.04 |
| phase0 processSlashings - 250000 worstcase | 59.121 us/op | 60.847 us/op | 0.97 |
| altair processSyncCommitteeUpdates - 250000 | 11.761 ms/op | 13.359 ms/op | 0.88 |
| BeaconState.hashTreeRoot - No change | 181.00 ns/op | 165.00 ns/op | 1.10 |
| BeaconState.hashTreeRoot - 1 full validator | 80.576 us/op | 83.651 us/op | 0.96 |
| BeaconState.hashTreeRoot - 32 full validator | 1.0572 ms/op | 871.40 us/op | 1.21 |
| BeaconState.hashTreeRoot - 512 full validator | 8.3775 ms/op | 8.5394 ms/op | 0.98 |
| BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 123.02 us/op | 108.18 us/op | 1.14 |
| BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 1.6351 ms/op | 1.4944 ms/op | 1.09 |
| BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 22.536 ms/op | 19.452 ms/op | 1.16 |
| BeaconState.hashTreeRoot - 1 balances | 102.81 us/op | 83.475 us/op | 1.23 |
| BeaconState.hashTreeRoot - 32 balances | 1.3980 ms/op | 726.56 us/op | 1.92 |
| BeaconState.hashTreeRoot - 512 balances | 5.1574 ms/op | 6.4646 ms/op | 0.80 |
| BeaconState.hashTreeRoot - 250000 balances | 192.70 ms/op | 244.62 ms/op | 0.79 |
| aggregationBits - 2048 els - zipIndexesInBitList | 18.722 us/op | 21.700 us/op | 0.86 |
| regular array get 100000 times | 22.077 us/op | 23.773 us/op | 0.93 |
| wrappedArray get 100000 times | 21.939 us/op | 23.842 us/op | 0.92 |
| arrayWithProxy get 100000 times | 17.066 ms/op | 9.8230 ms/op | 1.74 |
| ssz.Root.equals | 20.489 ns/op | 22.181 ns/op | 0.92 |
| byteArrayEquals | 20.409 ns/op | 21.999 ns/op | 0.93 |
| Buffer.compare | 8.4970 ns/op | 9.0660 ns/op | 0.94 |
| processSlot - 1 slots | 10.184 us/op | 10.091 us/op | 1.01 |
| processSlot - 32 slots | 2.0114 ms/op | 2.3811 ms/op | 0.84 |
| getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 3.8863 ms/op | 5.2961 ms/op | 0.73 |
| getCommitteeAssignments - req 1 vs - 250000 vc | 1.7017 ms/op | 1.5906 ms/op | 1.07 |
| getCommitteeAssignments - req 100 vs - 250000 vc | 3.4448 ms/op | 3.2868 ms/op | 1.05 |
| getCommitteeAssignments - req 1000 vs - 250000 vc | 3.6989 ms/op | 3.5257 ms/op | 1.05 |
| findModifiedValidators - 10000 modified validators | 731.23 ms/op | 765.32 ms/op | 0.96 |
| findModifiedValidators - 1000 modified validators | 477.21 ms/op | 534.32 ms/op | 0.89 |
| findModifiedValidators - 100 modified validators | 285.91 ms/op | 332.47 ms/op | 0.86 |
| findModifiedValidators - 10 modified validators | 234.41 ms/op | 250.42 ms/op | 0.94 |
| findModifiedValidators - 1 modified validators | 160.16 ms/op | 160.74 ms/op | 1.00 |
| findModifiedValidators - no difference | 175.02 ms/op | 195.13 ms/op | 0.90 |
| migrate state 1500000 validators, 3400 modified, 2000 new | 3.7918 s/op | 3.1077 s/op | 1.22 |
| RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 3.5700 ns/op | 3.5400 ns/op | 1.01 |
| state getBlockRootAtSlot - 250000 vs - 7PWei | 442.46 ns/op | 405.61 ns/op | 1.09 |
| computeProposerIndex 100000 validators | 1.2932 ms/op | 1.3680 ms/op | 0.95 |
| getNextSyncCommitteeIndices 1000 validators | 2.7840 ms/op | 2.8164 ms/op | 0.99 |
| getNextSyncCommitteeIndices 10000 validators | 24.975 ms/op | 24.244 ms/op | 1.03 |
| getNextSyncCommitteeIndices 100000 validators | 82.069 ms/op | 83.639 ms/op | 0.98 |
| computeProposers - vc 250000 | 572.69 us/op | 544.43 us/op | 1.05 |
| computeEpochShuffling - vc 250000 | 39.577 ms/op | 38.479 ms/op | 1.03 |
| getNextSyncCommittee - vc 250000 | 9.6895 ms/op | 9.3979 ms/op | 1.03 |
| nodejs block root to RootHex using toHex | 91.573 ns/op | 104.48 ns/op | 0.88 |
| nodejs block root to RootHex using toRootHex | 55.391 ns/op | 54.763 ns/op | 1.01 |
| nodejs fromHex(blob) | 842.58 us/op | 747.91 us/op | 1.13 |
| nodejs fromHexInto(blob) | 606.88 us/op | 601.13 us/op | 1.01 |
| nodejs block root to RootHex using the deprecated toHexString | 377.69 ns/op | 446.07 ns/op | 0.85 |
| nodejs byteArrayEquals 32 bytes (block root) | 25.816 ns/op | 24.862 ns/op | 1.04 |
| nodejs byteArrayEquals 48 bytes (pubkey) | 36.846 ns/op | 36.218 ns/op | 1.02 |
| nodejs byteArrayEquals 96 bytes (signature) | 33.114 ns/op | 32.518 ns/op | 1.02 |
| nodejs byteArrayEquals 1024 bytes | 40.850 ns/op | 39.780 ns/op | 1.03 |
| nodejs byteArrayEquals 131072 bytes (blob) | 1.7079 us/op | 1.6950 us/op | 1.01 |
| browser block root to RootHex using toHex | 140.92 ns/op | 139.08 ns/op | 1.01 |
| browser block root to RootHex using toRootHex | 128.48 ns/op | 126.00 ns/op | 1.02 |
| browser fromHex(blob) | 1.6744 ms/op | 1.4586 ms/op | 1.15 |
| browser fromHexInto(blob) | 670.16 us/op | 605.45 us/op | 1.11 |
| browser block root to RootHex using the deprecated toHexString | 531.86 ns/op | 438.40 ns/op | 1.21 |
| browser byteArrayEquals 32 bytes (block root) | 28.386 ns/op | 26.820 ns/op | 1.06 |
| browser byteArrayEquals 48 bytes (pubkey) | 39.994 ns/op | 37.677 ns/op | 1.06 |
| browser byteArrayEquals 96 bytes (signature) | 74.653 ns/op | 70.744 ns/op | 1.06 |
| browser byteArrayEquals 1024 bytes | 761.06 ns/op | 723.01 ns/op | 1.05 |
| browser byteArrayEquals 131072 bytes (blob) | 95.153 us/op | 91.405 us/op | 1.04 |
by benchmarkbot/action
nazarhussain
force-pushed
the
nh/ssz-engine-api
branch
from
2d17bb9 to
c5e1e38
Compare
nazarhussain
changed the title
## Summary Implements SSZ-REST Engine API transport on the consensus layer (client side), as specified in [ethereum/execution-apis#764](ethereum/execution-apis#764). - New CLI flag `--execution.sszRestUrl` to configure SSZ-REST endpoint - SSZ-encoded request/response bodies for all Engine API methods - Automatic fallback to JSON-RPC on network errors - Supports: `new_payload` (v1-v5), `forkchoice_updated` (v1-v3), `get_payload` (v1-v5), `exchange_capabilities` - Proper fork-based version selection for Deneb/Electra/Fulu --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace the hand-rolled byte-level encoders/decoders in sszRestEncoding.ts with @chainsafe/ssz ContainerType definitions for every Engine API request and response. This fixes a number of wire-format defects from #8994: - NewPayloadV1/V2 now carry the required Container offset prefix. - PayloadAttributes encoding matches the per-fork shape (V1 lacks withdrawals, V2 lacks parentBeaconBlockRoot, etc.) instead of always writing the V3 layout. - execution_requests is encoded as the spec's flat List[ByteList, 256] with proper SSZ list framing, not a flat concatenation of typed blobs. - GetPayloadResponse V2 (Shanghai) and the V5/V6 Osaka/Amsterdam shapes are now decodable. - GetBlobs V2 cell proofs (List[Bytes48, CELLS_PER_EXT_BLOB]) decode correctly; the previous fixed-stride scan only worked for V1. - Fork → version mapping is centralized in newPayloadVersion, getPayloadVersion, forkchoiceUpdatedVersion, and getBlobsVersion, fixing the prior fulu→v5 mismapping for newPayload and the v4 gap for forkchoiceUpdated. PayloadAttributes containers are redefined locally because ssz.{fork}.PayloadAttributes from @lodestar/types declares suggestedFeeRecipient with a JSON-only stringType that throws on SSZ serialize. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sszRest The SSZ-REST Engine API transport from #8994 was constructed unconditionally and probed on every Engine call, then silently fell back to JSON-RPC on network errors. Until ethereum/execution-apis#764 stabilises and the ELs we test against advertise support consistently, this probing is wasted traffic against vanilla EL deployments and can mask transient infra issues. Add a `sszRest` flag to ExecutionEngineHttpOpts and a hidden `--execution.sszRest` CLI flag. The SszRestClient is only constructed when the flag is set; otherwise the JSON-RPC path is used exclusively. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the local hexToBytes helper in favour of fromHex from @lodestar/utils, matching the convention used elsewhere in the package. Addresses gemini-code-assist feedback on #8994. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…baseUrl CodeQL (js/polynomial-redos) flagged engineUrl.replace(/\/+$/, "") at the SszRestClient init site. The regex is O(N^2) on N trailing slashes due to greedy + backtracking against the `$` anchor. The input is operator-supplied (--execution.urls), so the alert is not exploitable in our threat model, but the fix is trivial and clears the security alert. Use a linear charCode scan instead, and drop the redundant duplicate strip inside SszRestClient (the caller already normalises). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add SSZ containers for ExecutionPayloadBodyV1, PayloadBodiesV1Response, and the two request shapes from execution-apis#764, plus encoder/decoder helpers and the two HTTP call sites. Advertise the new endpoints in supportedSszRestEndpoints so the EL knows we support them; both methods negotiate via engine_exchangeCapabilities and fall back to JSON-RPC on network errors. Payload bodies can be sizeable (transactions + withdrawals), so binary SSZ avoids the hex-encoding bloat of the JSON-RPC equivalent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add SSZ containers for ClientVersionV1, GetClientVersionV1Request, and GetClientVersionV1Response from execution-apis#764, plus the call site in the existing getClientVersion path. Split the response handling into fetchClientVersions (raw transport) and the surrounding code mapping (ClientCode enum + strip 0x prefix). Advertise POST /engine/v1/client/version in supportedSszRestEndpoints; the call negotiates via engine_exchangeCapabilities and falls back to JSON-RPC on network errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Re-add POST /engine/v1/blobs to supportedSszRestEndpoints (removed in e4d5d11) and flip the v1 SSZ-REST test to assert the new behaviour. Spec v1 returns List[BlobAndProofV1, MAX_BLOB_HASHES_REQUEST] with no per-element nullability, while the JSON-RPC v1 contract returns a same-length array with null for missing blobs. Map the gap by padding the SSZ response up to the request length with null, assuming the EL returns results in request order with any trailing positions missing. This is a Lodestar-side assumption since the spec is silent on response ordering for v1; revisit if interop testing surfaces ELs that return out-of-order results. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Osaka SSZ spec defines both v2 (all-or-nothing) and v3 (per-element nullable) blob endpoints. Lodestar wires only v2; record the four reasons inline on getBlobsVersion so a future reader doesn't have to reconstruct them: - IExecutionEngine.getBlobs post-Fulu is all-or-nothing by design - Transport-symmetric with the existing JSON-RPC v2 path - Matches the spec's own guidance for all-or-nothing consumers - Buffer-reuse optimisation in block production assumes all-or-nothing Plus a note on when to revisit (if a granular blob-fetch consumer appears) and that picking v2 has no interop cost since the major ELs (Nethermind, Erigon) serve both. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
nazarhussain
force-pushed
the
nh/ssz-engine-api
branch
from
13a1eee to
3534a7a
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #9382 +/- ##
============================================
- Coverage 52.55% 52.54% -0.01%
============================================
Files 848 848
Lines 60950 60927 -23
Branches 4487 4486 -1
============================================
- Hits 32034 32016 -18
+ Misses 28854 28849 -5
Partials 62 62 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3534a7aa12
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+56
to
+58
| const PayloadStatusV1 = new ContainerType( | ||
| {status: Uint8, latestValidHash: NullableHash, validationError: ValidationErrorBytes}, | ||
| {typeName: "PayloadStatusV1"} |
There was a problem hiding this comment.
This container uses latestValidHash as List[Bytes32, 1], but the SSZ-REST Engine schema defines PayloadStatusV1.latest_valid_hash as fixed Bytes32 (zero-hash sentinel for absence), and ForkchoiceUpdatedResponseV1.payload_id similarly as fixed Bytes8. Using nullable-list wrappers here changes the wire layout, so a compliant EL response will fail deserialization (or be misdecoded) and interop will break for newPayload/forkchoiceUpdated.
Useful? React with 👍 / 👎.
Comment on lines
+469
to
+474
| case 2: | ||
| return ExecutionPayloadStatus.SYNCING; | ||
| case 3: | ||
| return ExecutionPayloadStatus.ACCEPTED; | ||
| default: | ||
| throw Error(`Unknown payload status byte=${byte}`); |
There was a problem hiding this comment.
statusByteToEnum maps only values 0-3 and throws on any other byte, but PayloadStatusV1 also includes INVALID_BLOCK_HASH (value 4). When an EL returns that valid status for newPayload, the SSZ path will throw instead of returning a structured ExecutePayloadResponse, turning a protocol-level response into an unexpected exception path during block verification.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements SSZ-REST Engine API transport on the consensus layer (client side), as specified in
ethereum/execution-apis#764.
--execution.sszRestUrlto configure SSZ-REST endpointnew_payload(v1-v5),forkchoice_updated(v1-v3),get_payload(v1-v5),exchange_capabilities