Skip to content

feat(validator): accept a JSON produceBlockV3 answer - #862

Open
MegaRedHand wants to merge 5 commits into
mainfrom
validator-json-produce-block
Open

MegaRedHand wants to merge 5 commits into
mainfrom
validator-json-produce-block

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

Closes #861.

Problem

ethereum-package's blobber (marioevz/blobber v1.1.0, validator_proxy/proxy.go) sits between a validator client and its beacon node. On /eth/v2/validator/blocks/{slot} and /eth/v3/validator/blocks/{slot} it rewrites the request's Accept to application/json, then relays the node's answer unchanged, headers included. produce_block asked for SSZ only, and read_ssz refused anything else ("asked for SSZ, node answered with application/json"), so every proposal by an ethlambda validator client behind blobber failed. get_ssz's doc gave the reason: "there is no JSON path for a block".

What this does

produceBlockV3 takes SSZ or JSON. The request now sends Accept: application/octet-stream;q=1.0,application/json;q=0.9 (the specification's own example) through a new get_block_v3, which dispatches on the answer's Content-Type:

  • application/octet-stream, or no content type, goes down the existing SSZ path unchanged: the fork from Eth-Consensus-Version, a required Eth-Execution-Payload-Blinded, then ProducedBlock::from_ssz / from_blinded_ssz.
  • application/json (with or without ; charset=...) is read as the beacon-APIs JSON answer:
    • the fork comes from the body's version, falling back on Eth-Consensus-Version;
    • blindedness comes from execution_payload_blinded, falling back on Eth-Execution-Payload-Blinded;
    • when a body field and its header are both present they must agree, and a disagreement is an InconsistentResponse. This applies to the fork, as the issue asks, and also to blindedness, since either one selects the container data is decoded as;
    • when neither the field nor the header is present, the answer is refused, as an SSZ answer missing the header is.
  • Any other content type is refused rather than guessed at.

JSON data is decoded into the same ProducedBlock the SSZ path yields, through new ProducedBlock::from_json / from_blinded_json. These share the fork match with the SSZ functions (a private Body enum, SSZ bytes or JSON value), so electra and fulu decode, and gloas and the pre-electra forks are refused by name exactly as before. Every check after decoding still runs: a blinded block while builder proposals are off is refused, and validate_produced_block checks the slot and proposer.

No parallel types. The block containers (electra::BeaconBlock, builder::BlindedBeaconBlock, KzgProof) already derive serde in ethlambda-types, and the beacon node serves its own produceBlockV3 JSON with them. The only addition is Deserialize on the validator's BlockContents and FuluBlockContents, with the same field attributes the node's FuluBlockContents uses (serde_helpers::seq for the proofs, ssz_hex_seq for the blobs). The root is still taken from the typed container, never from a hand-written mapping. execution_payload_value and consensus_block_value are not read, because the SSZ path does not read their headers either: the node, not this client, weighs one payload against another.

What stays SSZ-only.

  • produceBlockV4 (gloas, post_json_for_ssz), which blobber does not intercept.
  • The payload envelope fetch (get_ssz).
  • Every publish: blobber passes those through untouched.

Docs. I rewrote every place that said block production is SSZ-only:

  • the doc comments on get_ssz and post_json_for_ssz;
  • the block_contents module doc;
  • BeaconNodeApi::produce_block;
  • the aggregate doc in dto.rs;
  • the comment in the validator's Cargo.toml;
  • docs/cli.md.

Tests

Validator unit tests (beacon_node/http.rs, on the existing serve double):

  • produceBlockV3 sends the new Accept; produceBlockV4 still sends application/octet-stream alone.
  • A JSON answer decodes into the same ProducedBlock as its SSZ, root included, at electra and fulu. The block in data is written by the containers' own serde.
  • A blinded JSON answer decodes as blinded with builder proposals on and keeps the full block's root, and is refused with them off.
  • A body missing version and execution_payload_blinded falls back on the headers.
  • A body and headers disagreeing on the fork, or on blindedness, are refused.
  • No fork in either place is refused, and a content type that is neither SSZ nor JSON is refused.

End to end (ethlambda-rpc's validator_client_tests), against the real beacon router behind a blobber stand-in modelled on proxy.go: it rewrites Accept to application/json on produceBlockV3 and relays the answer and its headers unchanged.

  • Fulu block with two blobs: the client decodes the node's own JSON into the block the node's SSZ gives. It then signs the block and publishes it through the proxy, and the node takes the block and its 128 column sidecars.
  • Blinded builder block: the JSON blinded block equals the SSZ one, and publishing it hands it to the mock relay.

Both suites compile under make lint. I have not run them; CI runs the tests. The blobber stand-in is written from proxy.go, not the real binary, so a kurtosis run with blobber_enabled and an ethlambda validator client is still the real end-to-end check. Once this is in :unstable, ethereum-package's guard against that combination (add-ethlambda) can go.

ethereum-package's blobber rewrites Accept to application/json on produceBlockV3 and relays the node's answer unchanged, so a validator client that took SSZ alone lost every proposal behind it.

produceBlockV3 now asks for application/octet-stream;q=1.0,application/json;q=0.9 and dispatches on Content-Type. SSZ goes down the existing path. JSON is decoded into the same ProducedBlock through the containers' serde support in ethlambda-types: the fork from the body's version (falling back on Eth-Consensus-Version), blindedness from execution_payload_blinded (falling back on Eth-Execution-Payload-Blinded), a disagreement between body and header refused. produceBlockV4 and the payload envelope stay SSZ-only.
@MegaRedHand

Copy link
Copy Markdown
Collaborator Author

/review

@github-actions

Copy link
Copy Markdown

AI security review: 0 finding(s) to check

Reviewed by Opus, GPT, Kimi, GLM, DeepSeek.
Malicious-code check (tests, build and install scripts, hooks, editor tasks, dependency sources): nothing found.

No findings to check.

Leads from a script (not checked): 1

Found without a model, from the lines this PR adds. Some can be wrong, e.g. a path created at runtime or a name used through reflection.

  • validator_proxy/proxy.go does not exist in the repository (referenced at crates/validator/src/beacon_node/http.rs:467)

Cost of this review: ~$1.93 of a $30 budget (estimated from token counts).

"Blinded",
)?;
let bytes = Self::answer_bytes(url, response).await?;
let answer: ProduceBlockV3Response = serde_json::from_slice(&bytes)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real blobber drops the node's status code. In validator_proxy/proxy.go (v1.1.0, proxyHandler, lines 143-185) it copies the upstream headers and then io.Copys the body, but never calls w.WriteHeader(proxyRes.StatusCode), so Go answers 200. Any produceBlockV3 failure behind it (503 while syncing, 400, 500) therefore reaches this client as 200 application/json with a body like {"code":503,"message":"..."}, and lands here as Error::Decode("produceBlockV3 JSON answer: missing field data").

Decode is retryable, so failover still moves on, but a syncing node is no longer reported as BeaconNodeSyncing and the error points at a malformed answer instead of the node's own error.

Suggestion: when the body has code/message and no data, map it to what send_for_answer would have returned for that status (503 -> syncing, otherwise the BeaconNode error carrying the message).

relayed.lock().unwrap().push(content_type);
}
let mut response = Response::new(Body::from(answer.bytes().await.unwrap()));
*response.status_mut() = status;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real proxy.go doesn't do this: it never writes the upstream status, so every relayed answer is a 200 (see the comment on json_block_answer in http.rs). Since the stand-in is documented as modelled on proxy.go, I'd drop the status here as blobber does, and add a case where the node fails produceBlockV3 behind it (for example while syncing), so the 200-with-an-error-body shape is covered.

/// written by its container's own serde, the one an ethlambda beacon node
/// serves it with; the proofs and blobs as the hex strings the Beacon API
/// specifies.
fn block_contents_at(fork: ForkName) -> (serde_json::Value, Vec<u8>) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every JSON test builds data with our own serde_json::to_value, and the e2e test in ethlambda-rpc runs against the ethlambda node, so what is tested is ethlambda's JSON round-tripping through ethlambda's serde. Behind blobber in ethereum-package the node may be Lighthouse, Teku, Prysm and so on.

The types crate has some cover (tests/beacon_json.rs checks that no integer is a bare number, and the builder-specs fulu signed_blinded_beacon_block example is vendored), but nothing from another source exercises a full electra/fulu produceBlockV3 answer: the full ExecutionPayload (transactions, withdrawals), execution_requests, and kzg_proofs/blobs.

Could we vendor one real answer (captured from Lighthouse, or the beacon-APIs example) with its SSZ alongside, and assert both decode to the same ProducedBlock and root? That's the property this PR rests on.

pablodeymo and others added 3 commits November 20, 2026 19:37
blobber's proxyHandler copies the node's headers and body but never its
status, so a node failing produceBlockV3 behind it answers 200 with the
specification's ErrorMessage in the body. That read as a malformed block
answer, and a syncing node stopped being reported as one. A JSON block
answer that does not parse and is an ErrorMessage with a 4xx/5xx code is
now the error its status would have given (503 is BeaconNodeSyncing).

The blobber stand-in in the rpc tests now drops the status on
produceBlockV3 the way proxy.go does, with an end-to-end case for a node
failing block production behind it.

The JSON path is also checked against JSON this repository did not write:
builder-specs' fulu blinded block and execution payload bundle examples
(the bundle newly vendored), decoded as produceBlockV3 answers and held to
their own SSZ.
# Conflicts:
#	crates/validator/src/beacon_node/http.rs

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

validator: accept a JSON produceBlockV3 answer, so proposals work behind blobber

2 participants