feat: embedding reth directly into ream by Sahilgill24 · Pull Request #1472 · ReamLabs/ream

Sahilgill24 · 2026-06-26T00:11:36Z

What was wrong?

#1471

Current Progress

RethHandle is initialized, which contains the mpsc sender's to communicate with reth
The functions for communication b/w EL and CL are complete and tested.

To-Do

I have read CONTRIBUTING.md.
This PR title follows conventional commits.

Sahilgill24 · 2026-06-26T23:14:48Z

Added the Fork Choice updated function and its related helper functions.
Along with two tests

The reth node accepts tx's from the JSON-RPC method and the transaction is properly submitted in the pool
FCU returns the Valid status

Sahilgill24 · 2026-07-02T21:32:04Z

@zemse a few changes from my side

the build() function should only call resolve_kind, it does not need to call the send_new_payload(), we already have the payload id from the FCU updated. the send_new_payload() function would create a new Payload Id and cause a mismatch .
in the build() function , built.try_into_v3() would convert into ExecutionPayloadV3 which has no slot_number field but as we are using slot_number field (which is in V4) it would result in block_hash mismatch , we can just replace it with into_execution_data() and follow the V4 as in the main chain config.
Not something that will cause an error, but I don't think we need DB for even testing purposes, it's just extra import and build overhead, we can remove it ideally. What do you suggest for it or am I missing something related to the DB usage.

zemse · 2026-07-03T10:22:36Z

Not something that will cause an error, but I don't think we need DB for even testing purposes, it's just extra import and build overhead,

Yes we can remove the in memory db for the tests, if we can simply just remove the tests. Anyway those tests for reth are redundant since it is anyway tested on upstream, so as long as we are using their release tag we're good. I was going to remove them yesterday but I thought lets talk to you about it first.

zemse · 2026-07-03T12:04:59Z

-    Ok(built.try_into_v3()?)
+    Ok(built.into_execution_data())


We will also need to include blob sidecars in the build fn for the CL to do DA

we could keep them, we just would have to include built.sidecars().clone(); in the build function and can return it along with the ExecutionData, but currently as we were doing the main ream/reth integration I thought let's remove them.

I think we should make these when we open that issue

Sahilgill24 · 2026-07-03T15:34:53Z

We can remove the tests, but I thought it was better if we could just test out the functions directly and capture the output as well to verify, I think we can remove them once we are complete with it (it's more of a personal choice, helps me better in debugging and all :) )

when slot value is passed, reth generates a glamsterdam block

Resolve conflicts against master (33f99bb): - Feature tables: keep the branch's `reth` feature alongside master's new `optimized-leanvm` / `jemalloc` / `shadow-integration` features across bin/ream and the lean crates. ream-consensus-lean takes master's devnet4-removal and keeps `reth = ["dep:ream-execution-rpc-types"]`. - Store: keep both new fields, `reth_handle` and `block_production_strategy`, plus both builder methods. - Master split `build_block` into `load_build_context` / `select_*` / `seal_block`, so thread the cfg-gated `execution_payload` down into `select_round_based` and `seal_block` instead of leaving it stranded in `build_block`. - LeanChainService: keep `reth_handle` next to master's `aggregation_in_flight`, and keep the EL payload build path in `handle_produce_block` while adopting master's `wall_slot` prebuild. - Regenerate Cargo.lock and book/cli/ream/lean_node.md rather than hand-merging them. Verified: cargo check/clippy clean for both default features and `--features reth`; ream-fork-choice-lean, ream-chain-lean and ream-consensus-lean tests pass.

zemse · 2026-08-25T13:53:01Z


 [target.x86_64-unknown-linux-gnu]
-rustflags = ['-C', 'target-cpu=x86-64-v3']
+rustflags = ['-C', 'target-cpu=x86-64-v3', '-C', 'target-feature=+pclmulqdq']


Just for info - this build flag is added here because rocksdb from reth requires it

KolbyML · 2026-08-31T03:01:19Z

@Sahilgill24 @zemse please rebase this and I will look at it tomorrow

Resolve conflicts against master (837329b): - `IngestHandle::submit` / `submit_retention`: master fixed the same `make fmt` `map_err` naming rule this branch had worked around with an `if ... .is_err()` block, so drop the workaround and take master's `map_err(|err| ...)` version with the `debug!` log. - Master's new fork-choice store tests build `BlockBody` directly, so add the cfg-gated `execution_payload` field there the same way the `ream-consensus-lean` tests do; without it the crate stops compiling under `--features reth`. Verified: cargo check --workspace --all-targets is clean both with default features and with `--features reth`.

KolbyML

Here is some feedback

KolbyML · 2026-09-01T03:00:15Z

+    #[arg(
+        long,
+        default_value = "./reth-data",
+        help = "Set reth data directory (needs `--features reth`)"
+    )]
+    pub reth_datadir: PathBuf,
+
+    #[arg(long, help = "Set reth eth_* JSON-RPC address", default_value_t = DEFAULT_HTTP_ADDRESS)]
+    pub reth_rpc_address: IpAddr,
+
+    #[arg(long, help = "Set reth eth_* JSON-RPC port", default_value_t = DEFAULT_RETH_RPC_PORT)]
+    pub reth_rpc_port: u16,
+
+    #[arg(
+        long,
+        help = "Set reth RLPx (devp2p) address",
+        default_value_t = DEFAULT_RETH_P2P_ADDRESS,
+        requires = "reth_p2p_port"
+    )]
+    pub reth_p2p_address: IpAddr,
+
+    #[arg(
+        long,
+        help = "Set reth RLPx (devp2p) port, to gossip transactions with other execution layers. Unset means an isolated EL with no peers",
+        requires = "reth_p2p_secret"
+    )]
+    pub reth_p2p_port: Option<u16>,
+
+    #[arg(
+        long,
+        help = "Set 32-byte hex secp256k1 key pinning reth's enode identity, so peers can address it deterministically",
+        requires = "reth_p2p_port"
+    )]
+    pub reth_p2p_secret: Option<B256>,
+
+    #[arg(
+        long,
+        value_delimiter = ',',
+        help = "Comma-separated enode URLs of other execution layers to dial directly as a static trusted mesh (enode://<pubkey>@<ip>:<port>). Discovery stays off.",
+        requires = "reth_p2p_port"
+    )]
+    pub reth_trusted_peers: Vec<String>,


these should be behind a feature flag

KolbyML · 2026-09-01T03:01:09Z

@@ -465,6 +468,49 @@ pub async fn run_lean_node(config: LeanNodeConfig, executor: ReamExecutor, ream_
        config.http_allow_origin,
    );

+    // Start the reth EL embedded in the same process. `start` returns the cloneable
+    // `RethHandle` (given to the lean chain service) and also the owned node.
+    #[cfg(feature = "reth")]
+    let (reth_handle, mut reth_node) = {
+        let p2p = config
+            .reth_p2p_port
+            .zip(config.reth_p2p_secret)
+            .map(|(port, secret_key)| RethP2pConfig {
+                address: config.reth_p2p_address,
+                port,
+                secret_key,
+                trusted_peers: config.reth_trusted_peers.clone(),
+            });
+
+        let (handle, node) = RethHandle::start(RethNodeConfig {
+            runtime: Some(executor.runtime().handle().clone()),
+            datadir: config.reth_datadir.clone(),
+            http_rpc: Some(SocketAddr::new(
+                config.reth_rpc_address,
+                config.reth_rpc_port,
+            )),
+            p2p,
+        })
+        .await
+        .expect("failed to boot embedded reth execution layer");
+
+        info!(
+            "Embedded reth is started with genesis hash: {:?}",
+            node.node.chain_spec().genesis_hash()
+        );
+
+        (handle, node)
+    };
+
+    #[cfg(feature = "reth")]
+    chain_service.set_reth_handle(reth_handle).await;
+
+    #[cfg(feature = "reth")]
+    let reth_exit = &mut reth_node.node_exit_future;
+
+    #[cfg(not(feature = "reth"))]
+    let reth_exit = std::future::pending::<()>();


I think we can scope this better, instead of spamming feature flags every line

KolbyML · 2026-09-01T03:02:34Z

+/// Creates a `ForkchoiceState` used by the Engine API.
+///
+/// The fork choice state specifies the current head, justified, and finalized
+/// block hashes from ream.
+pub fn create_fork_choice_state(
+    head_block_hash: B256,
+    safe_block_hash: B256,
+    finalized_block_hash: B256,
+) -> ForkchoiceState {
+    ForkchoiceState {
+        head_block_hash,
+        safe_block_hash,
+        finalized_block_hash,
+    }
+}


this function serves no purpose we should remove the function, and inline the logic

KolbyML · 2026-09-01T03:02:44Z

+pub async fn update(
+    consensus_engine_handle: &ConsensusEngineHandle<EthEngineTypes>,
+    state: ForkchoiceState,
+    payload_attributes: Option<EthPayloadAttributes>,
+) -> eyre::Result<ForkchoiceUpdated> {
+    let updated = consensus_engine_handle
+        .fork_choice_updated(state, payload_attributes)
+        .await?;
+    Ok(updated)
+}


KolbyML · 2026-09-01T03:03:01Z

+/// Withdrawals are currently omitted and set to None.
+pub fn create_payload_attributes(
+    timestamp: u64,
+    prev_randao: B256,
+    suggested_fee_recipient: Address,
+    parent_beacon_block_root: Option<B256>,
+    slot_number: Option<u64>,
+    target_gas_limit: Option<u64>,
+) -> EthPayloadAttributes {
+    EthPayloadAttributes {
+        timestamp,
+        prev_randao,
+        suggested_fee_recipient,
+        withdrawals: Some(vec![]),
+        parent_beacon_block_root,
+        slot_number,
+        target_gas_limit,
+    }
+}
+
+/// Creates payload attributes for a Ream block proposal.
+///
+/// The timestamp is computed from the slot and genesis time. The
+/// `prev_randao` value is deterministically derived from the parent Lean
+/// block root and slot.
+pub fn create_ream_payload_attributes(
+    slot: u64,
+    parent_lean_block_root: B256,
+    genesis_time: u64,
+    seconds_per_slot: u64,
+) -> EthPayloadAttributes {
+    let prev_randao = {
+        // prev_randao = SHA256(parent_lean_block_root || slot)
+        let mut h = Sha256::new();
+        h.update(parent_lean_block_root.as_slice());
+        h.update(slot.to_le_bytes());
+        B256::from_slice(&h.finalize())
+    };
+
+    create_payload_attributes(
+        genesis_time + slot * seconds_per_slot,
+        prev_randao,
+        Address::ZERO,
+        Some(parent_lean_block_root),
+        None,
+        None,
+    )
+}


KolbyML · 2026-09-01T03:04:39Z

+pub fn custom_chain() -> Arc<ChainSpec> {
+    let custom_genesis = r#"
+{
+    "nonce": "0x42",
+    "timestamp": "0x0",
+    "extraData": "0x5343",
+    "gasLimit": "0x1C9C380",
+    "difficulty": "0x400000000",
+    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
+    "coinbase": "0x0000000000000000000000000000000000000000",
+    "alloc": {
+        "0x6Be02d1d3665660d22FF9624b7BE0551ee1Ac91b": {
+            "balance": "0x4a47e3c12448f4ad000000"
+        },
+        "0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266": {
+            "balance": "0x4a47e3c12448f4ad000000"
+        },
+        "0x70997970C51812dc3A010C7d01b50e0d17dc79C8": {
+            "balance": "0x4a47e3c12448f4ad000000"
+        },
+        "0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC": {
+            "balance": "0x4a47e3c12448f4ad000000"
+        }
+    },
+    "number": "0x0",
+    "gasUsed": "0x0",
+    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
+    "config": {
+        "ethash": {},
+        "chainId": 2600,
+        "homesteadBlock": 0,
+        "eip150Block": 0,
+        "eip155Block": 0,
+        "eip158Block": 0,
+        "byzantiumBlock": 0,
+        "constantinopleBlock": 0,
+        "petersburgBlock": 0,
+        "istanbulBlock": 0,
+        "berlinBlock": 0,
+        "londonBlock": 0,
+        "terminalTotalDifficulty": 0,
+        "terminalTotalDifficultyPassed": true,
+        "shanghaiTime": 0,
+        "cancunTime": 0,
+        "pragueTime": 0
+    }
+}
+"#;
+    let genesis: Genesis = serde_json::from_str(custom_genesis).expect("genesis failed");
+    Arc::new(genesis.into())
+}


ccustom chain? we should be more descriptive

KolbyML · 2026-09-01T03:04:45Z

+    let genesis: Genesis = serde_json::from_str(custom_genesis).expect("genesis failed");
+    Arc::new(genesis.into())


inline this

KolbyML · 2026-09-01T03:05:07Z

+pub async fn build(
+    payload_builder: &PayloadBuilderHandle<EthEngineTypes>,
+    payload_id: PayloadId,
+) -> eyre::Result<ExecutionData> {
+    // we can Ideally extract out WaitForPending to be more idiomatic
+    let built = payload_builder
+        .resolve_kind(payload_id, PayloadKind::WaitForPending)
+        .await
+        .ok_or_eyre("payload could not be resolved")??;
+
+    Ok(built.into_execution_data())
+}
+
+/// Imports an execution payload from a Lean block into the execution layer via
+/// `engine_newPayloadV4`.
+///
+/// Returns a `PayloadStatus` reporting whether the EL accepted the payload
+/// (`Valid`, `Invalid`, or `Syncing`), so the caller must inspect it.
+pub async fn import(
+    consensus_engine_handle: &ConsensusEngineHandle<EthEngineTypes>,
+    execution_data: ExecutionData,
+) -> eyre::Result<PayloadStatus> {
+    let status = consensus_engine_handle.new_payload(execution_data).await?;
+    Ok(status)
+}


these functions don't descure to exist

KolbyML · 2026-09-01T03:05:13Z

+/// V3 shape — Prague's EIP-7685 execution requests are carried in the
+/// `newPayloadV4` sidecar as a separate parameter, not inside the payload body.
+/// So the payload is always an `ExecutionPayload::V3` here
+pub fn to_ream_execution_payload(data: &ExecutionData) -> eyre::Result<ReamExecutionPayload> {


KolbyML · 2026-09-01T03:05:21Z

+        return Err(eyre!(
+            "expected a V3 execution payload body (Cancun or later active in the EL genesis)"
+        ));


KolbyML · 2026-09-04T06:44:40Z

@zemse are you still working on resolving the concerns or did you forget to push?

zemse · 2026-09-04T09:23:54Z

yes I have resolved most of them locally, couple of them are pending. I plan to do it tonight.

zemse · 2026-09-04T20:44:48Z

please review @KolbyML

KolbyML · 2026-09-10T05:30:17Z

@zemse ok fix the last conflict and I should be able to give the pr another look now

zemse · 2026-09-11T14:31:40Z

ive fixed the conflicts @KolbyML

KolbyML

looks good

feat: initialize reth-engine for embedding

2039189

Sahilgill24 force-pushed the feat/reth-integration-ream branch from 7a9109d to 2039189 Compare June 26, 2026 00:17

zemse reviewed Jun 26, 2026

View reviewed changes

Comment thread crates/common/reth/Cargo.toml Outdated

Comment thread crates/common/reth/Cargo.toml Outdated

Sahilgill24 added 3 commits June 26, 2026 07:19

feat: reth node setup and handle initialized

859c7d8

feat: adding fork choice updated function and relevant helper functions

46c65df

test: adding tests for handle and FCU

ed1c554

IvanBelyakoff mentioned this pull request Jun 29, 2026

engine-api integration with ream #1473

Open

zemse and others added 6 commits July 2, 2026 20:44

feat: payload module for building & importing

751809e

ref: reth handle

70bbbad

feat: add reth to bin/ream

6402693

feat: add build_payload & import_payload to RethHandle

7023796

feat: add data dir to the reth node

43ee479

feat: changes in payload building

b0c6c5d

feat: updating test to full payload flow for a proposer

3adae65

Sahilgill24 force-pushed the feat/reth-integration-ream branch from c4ab910 to 3adae65 Compare July 2, 2026 21:42

zemse reviewed Jul 3, 2026

View reviewed changes

zemse and others added 10 commits July 8, 2026 01:50

feat: add EL block in lean block body

6619876

ref: collapse RethHandle to a cloneable engine handle

78a7c13

feat: build EL payload inside handle_produce_block

c9a6dc2

pin EL to cancun v3 payloads

7583570

feat: pin EL to prague + add helper functions for payload building

3cd85c5

feat: update EL payload for both proposer and non-proposer

2d84b43

fix: invalid payload error

f374dfc

when slot value is passed, reth generates a glamsterdam block

feat: reth p2p to allow for tx mempool interop

ead1930

fix: missing import

f1d8030

zemse added 3 commits August 25, 2026 15:27

fix: docs, sort and format

5ae25a5

fix: rocksdb pclmul build flag and unused dev-dependency

f63f02b

chore: drop verbose comment from cargo config

851a965

Sahilgill24 marked this pull request as ready for review August 25, 2026 13:47

Sahilgill24 requested review from Kayden-ML, KolbyML and shariqnaiyer as code owners August 25, 2026 13:47

zemse reviewed Aug 25, 2026

View reviewed changes

KolbyML reviewed Sep 1, 2026

View reviewed changes

zemse added 3 commits September 5, 2026 02:01

refactor(reth): address review feedback

9d1146c

ref: better scope the reth feature cfg attributes

1b554a0

refactor(reth): fold payload attribute helpers into one

b874c72

Merge upstream/master into feat/reth-integration-ream

681faa0

KolbyML approved these changes Sep 13, 2026

View reviewed changes

KolbyML merged commit b003b25 into ReamLabs:master Sep 13, 2026
29 of 32 checks passed

		let genesis: Genesis = serde_json::from_str(custom_genesis).expect("genesis failed");
		Arc::new(genesis.into())

Conversation

Sahilgill24 commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was wrong?

Current Progress

To-Do

Uh oh!

Uh oh!

Uh oh!

Sahilgill24 commented Jun 26, 2026

Uh oh!

Sahilgill24 commented Jul 2, 2026

Uh oh!

zemse commented Jul 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sahilgill24 commented Jul 3, 2026

Uh oh!

zemse Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KolbyML commented Aug 31, 2026

Uh oh!

KolbyML left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KolbyML commented Sep 4, 2026

Uh oh!

zemse commented Sep 4, 2026

Uh oh!

zemse commented Sep 4, 2026

Uh oh!

KolbyML commented Sep 10, 2026

Uh oh!

zemse commented Sep 11, 2026

Uh oh!

KolbyML left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Sahilgill24 commented Jun 26, 2026 •

edited

Loading

zemse Aug 25, 2026 •

edited

Loading