feat: ssz engine API transport#8994
Conversation
Implement SSZ-REST transport for all Engine API methods: - new_payload (v3/v4/v5) - forkchoice_updated (v3) - get_payload (v3/v4/v5) - exchange_capabilities (v1) New CLI flag --execution.sszRestUrl enables SSZ-REST with automatic fallback to JSON-RPC on network errors. Includes proper fork-based version selection for Fulu (v5). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant upgrade to the execution layer communication by integrating the EIP-8161 SSZ-REST Engine API transport. This change allows for more efficient, binary-encoded interactions with the execution layer, enhancing performance and aligning with new Ethereum standards. The implementation includes a resilient fallback mechanism to JSON-RPC in case of network issues, ensuring continuous operation. It also provides a new configuration option for users to enable and specify the SSZ-REST endpoint. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements EIP-8161, adding SSZ-REST transport for the Engine API. The changes are well-structured, introducing a new SszRestClient, SSZ encoding/decoding logic, and integrating it into the existing ExecutionEngineHttp with a fallback to JSON-RPC. My review includes a few suggestions to improve code reuse and maintainability by replacing custom utility functions with existing ones from @lodestar/utils and refactoring duplicated logic.
| const version = | ||
| ForkSeq[fork] >= ForkSeq.fulu ? 5 : ForkSeq[fork] >= ForkSeq.electra ? 4 : ForkSeq[fork] >= ForkSeq.deneb ? 3 : ForkSeq[fork] >= ForkSeq.capella ? 2 : 1; |
There was a problem hiding this comment.
This version selection logic is duplicated in the getPayload method (lines 554-555). To improve maintainability and adhere to the DRY (Don't Repeat Yourself) principle, consider extracting this logic into a dedicated helper function. For example:
function getPayloadApiVersion(fork: ForkName): number {
const forkSeq = ForkSeq[fork];
if (forkSeq >= ForkSeq.fulu) return 5;
if (forkSeq >= ForkSeq.electra) return 4;
if (forkSeq >= ForkSeq.deneb) return 3;
if (forkSeq >= ForkSeq.capella) return 2;
return 1;
}You can then call this function here and in getPayload.
| constructor(opts: SszRestClientOpts) { | ||
| // Strip trailing slash for consistent path joining | ||
| this.baseUrl = opts.baseUrl.replace(/\/+$/, ""); | ||
| this.jwtSecret = opts.jwtSecretHex ? hexToBytes(opts.jwtSecretHex) : undefined; |
There was a problem hiding this comment.
For consistency and to leverage a more robust utility, please use the fromHex function from @lodestar/utils instead of the custom hexToBytes implementation. You'll need to add fromHex to your imports and can then remove the hexToBytes function (lines 139-146).
| this.jwtSecret = opts.jwtSecretHex ? hexToBytes(opts.jwtSecretHex) : undefined; | |
| this.jwtSecret = opts.jwtSecretHex ? fromHex(opts.jwtSecretHex) : undefined; |
| function hexToBytes20(hex: string): Uint8Array { | ||
| const stripped = hex.startsWith("0x") ? hex.slice(2) : hex; | ||
| if (stripped.length !== 40) { | ||
| throw Error(`Expected 20-byte hex address, got ${stripped.length / 2} bytes`); | ||
| } | ||
| const bytes = new Uint8Array(20); | ||
| for (let i = 0; i < 20; i++) { | ||
| bytes[i] = parseInt(stripped.substring(i * 2, i * 2 + 2), 16); | ||
| } | ||
| return bytes; | ||
| } |
There was a problem hiding this comment.
Instead of a custom hexToBytes20 implementation, you can use the existing fromHex utility from @lodestar/utils and add a length check. This promotes code reuse and consistency.
You will need to add import {fromHex} from "@lodestar/utils";.
function hexToBytes20(hex: string): Uint8Array {
const bytes = fromHex(hex);
if (bytes.length !== 20) {
throw Error(`Expected 20-byte hex address, got ${bytes.length} bytes`);
}
return bytes;
}…-REST
- new_payload → POST /engine/v{N}/payloads
- forkchoice_updated → POST /engine/v{N}/forkchoice
- get_payload → GET /engine/v{N}/payloads/{payload_id}
- get_blobs → POST /engine/v{N}/blobs
- Error responses now text/plain instead of JSON
- Added doGetRequest for GET support, Fulu version selection fix
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per execution-apis PR ChainSafe#764 spec: - PayloadStatus.latest_valid_hash: List[Hash32, 1] - ForkchoiceUpdatedResponse.payload_id: List[Bytes8, 1] - ForkchoiceUpdatedRequest.payload_attributes: List[PayloadAttributes, 1] Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
is this different from #8993 which is based on @barnabasbusa ssz engine api spec? |
SSZ-REST URL is now derived from the first engine URL (same host:port). The EL serves SSZ-REST on the engine port under /engine/* paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
effectively the same |
Giulio2002
changed the title
|
Implements the SSZ-REST Engine API transport spec: ethereum/execution-apis#764 |
|
Both this PR and #8993 implement the same spec (execution-apis PR #764) — SSZ-first Engine API transport with JSON-RPC fallback. As Giulio confirmed, effectively the same goal. Key differences in approach:
Happy to consolidate if there is a preferred direction. |
nazarhussain
changed the title
nazarhussain
merged commit 2d17bb9
into
ChainSafe:nh/ssz-engine-api
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66b6604dc2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const latestValidHashOffset = readUint32LE(data, 1); | ||
| const validationErrorOffset = readUint32LE(data, 5); | ||
|
|
||
| // Decode latestValidHash: List[Hash32, 1] — 0 bytes = absent, 32 bytes = present | ||
| let latestValidHash: string | null = null; | ||
| const hashLen = validationErrorOffset - latestValidHashOffset; | ||
| if (hashLen === 32) { | ||
| const hashBytes = data.subarray(latestValidHashOffset, validationErrorOffset); | ||
| latestValidHash = "0x" + bytesToHex(hashBytes); |
There was a problem hiding this comment.
EIP-8178 defines PayloadStatusV1 as status: uint8, fixed latest_valid_hash: Bytes32, then a variable validation_error; there are not two offsets after the status byte. When an SSZ-REST newPayload or forkchoice response is decoded here, bytes from the hash are interpreted as offsets, so latestValidHash and validationError are dropped or decoded as garbage for normal INVALID/SYNCING/ACCEPTED responses instead of reflecting the EL result.
Useful? React with 👍 / 👎.
| return decoded.map((item) => ({ | ||
| blob: item.blob, | ||
| proof: item.kzgProof, |
There was a problem hiding this comment.
When fork is Fulu this path selects /engine/v2/blobs, whose response items are BlobAndProofV2 values with a proofs list, not a single Deneb-style proof. Returning {blob, proof} means callers of the Fulu overload receive objects where proofs is undefined, so SSZ-REST blob retrieval cannot be verified/used once this path succeeds.
Useful? React with 👍 / 👎.
| const executionPayloadOffset = readUint32LE(data, 0); | ||
| const blockValue = readUint256LE(data, 4); | ||
| const blobsBundleOffset = readUint32LE(data, 36); | ||
| const shouldOverrideBuilder = data[40] !== 0; |
There was a problem hiding this comment.
This decoder assumes every getPayload response has a blobs-bundle offset and shouldOverrideBuilder byte, but SSZ v1 returns a raw ExecutionPayload and v2 only has execution_payload plus block_value before the payload bytes. For Bellatrix/Capella SSZ-REST responses, bytes 36/40 are part of the payload rather than header fields, so the payload is sliced with bogus offsets and deserialization fails instead of returning the built block.
Useful? React with 👍 / 👎.
## Summary Implements SSZ-REST Engine API transport on the consensus layer (client side), as specified in [ethereum/execution-apis#764](ethereum/execution-apis#764). - New CLI flag `--execution.sszRestUrl` to configure SSZ-REST endpoint - SSZ-encoded request/response bodies for all Engine API methods - Automatic fallback to JSON-RPC on network errors - Supports: `new_payload` (v1-v5), `forkchoice_updated` (v1-v3), `get_payload` (v1-v5), `exchange_capabilities` - Proper fork-based version selection for Deneb/Electra/Fulu --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace the hand-rolled byte-level encoders/decoders in sszRestEncoding.ts with @chainsafe/ssz ContainerType definitions for every Engine API request and response. This fixes a number of wire-format defects from #8994: - NewPayloadV1/V2 now carry the required Container offset prefix. - PayloadAttributes encoding matches the per-fork shape (V1 lacks withdrawals, V2 lacks parentBeaconBlockRoot, etc.) instead of always writing the V3 layout. - execution_requests is encoded as the spec's flat List[ByteList, 256] with proper SSZ list framing, not a flat concatenation of typed blobs. - GetPayloadResponse V2 (Shanghai) and the V5/V6 Osaka/Amsterdam shapes are now decodable. - GetBlobs V2 cell proofs (List[Bytes48, CELLS_PER_EXT_BLOB]) decode correctly; the previous fixed-stride scan only worked for V1. - Fork → version mapping is centralized in newPayloadVersion, getPayloadVersion, forkchoiceUpdatedVersion, and getBlobsVersion, fixing the prior fulu→v5 mismapping for newPayload and the v4 gap for forkchoiceUpdated. PayloadAttributes containers are redefined locally because ssz.{fork}.PayloadAttributes from @lodestar/types declares suggestedFeeRecipient with a JSON-only stringType that throws on SSZ serialize. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sszRest The SSZ-REST Engine API transport from #8994 was constructed unconditionally and probed on every Engine call, then silently fell back to JSON-RPC on network errors. Until ethereum/execution-apis#764 stabilises and the ELs we test against advertise support consistently, this probing is wasted traffic against vanilla EL deployments and can mask transient infra issues. Add a `sszRest` flag to ExecutionEngineHttpOpts and a hidden `--execution.sszRest` CLI flag. The SszRestClient is only constructed when the flag is set; otherwise the JSON-RPC path is used exclusively. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the local hexToBytes helper in favour of fromHex from @lodestar/utils, matching the convention used elsewhere in the package. Addresses gemini-code-assist feedback on #8994. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary Implements SSZ-REST Engine API transport on the consensus layer (client side), as specified in [ethereum/execution-apis#764](ethereum/execution-apis#764). - New CLI flag `--execution.sszRestUrl` to configure SSZ-REST endpoint - SSZ-encoded request/response bodies for all Engine API methods - Automatic fallback to JSON-RPC on network errors - Supports: `new_payload` (v1-v5), `forkchoice_updated` (v1-v3), `get_payload` (v1-v5), `exchange_capabilities` - Proper fork-based version selection for Deneb/Electra/Fulu --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace the hand-rolled byte-level encoders/decoders in sszRestEncoding.ts with @chainsafe/ssz ContainerType definitions for every Engine API request and response. This fixes a number of wire-format defects from #8994: - NewPayloadV1/V2 now carry the required Container offset prefix. - PayloadAttributes encoding matches the per-fork shape (V1 lacks withdrawals, V2 lacks parentBeaconBlockRoot, etc.) instead of always writing the V3 layout. - execution_requests is encoded as the spec's flat List[ByteList, 256] with proper SSZ list framing, not a flat concatenation of typed blobs. - GetPayloadResponse V2 (Shanghai) and the V5/V6 Osaka/Amsterdam shapes are now decodable. - GetBlobs V2 cell proofs (List[Bytes48, CELLS_PER_EXT_BLOB]) decode correctly; the previous fixed-stride scan only worked for V1. - Fork → version mapping is centralized in newPayloadVersion, getPayloadVersion, forkchoiceUpdatedVersion, and getBlobsVersion, fixing the prior fulu→v5 mismapping for newPayload and the v4 gap for forkchoiceUpdated. PayloadAttributes containers are redefined locally because ssz.{fork}.PayloadAttributes from @lodestar/types declares suggestedFeeRecipient with a JSON-only stringType that throws on SSZ serialize. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sszRest The SSZ-REST Engine API transport from #8994 was constructed unconditionally and probed on every Engine call, then silently fell back to JSON-RPC on network errors. Until ethereum/execution-apis#764 stabilises and the ELs we test against advertise support consistently, this probing is wasted traffic against vanilla EL deployments and can mask transient infra issues. Add a `sszRest` flag to ExecutionEngineHttpOpts and a hidden `--execution.sszRest` CLI flag. The SszRestClient is only constructed when the flag is set; otherwise the JSON-RPC path is used exclusively. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the local hexToBytes helper in favour of fromHex from @lodestar/utils, matching the convention used elsewhere in the package. Addresses gemini-code-assist feedback on #8994. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 participants
Summary
Implements SSZ-REST Engine API transport on the consensus layer (client side), as specified in ethereum/execution-apis#764.
--execution.sszRestUrlto configure SSZ-REST endpointnew_payload(v1-v5),forkchoice_updated(v1-v3),get_payload(v1-v5),exchange_capabilities