Espresso 3b: TEE batcher (re-hosted)

[QuentinI]

Based on #448

Pulls in the Espresso/TEE batcher .

• In op-node: adds EspressoBatch type and marshaling logic for it. This is the datastructure that ends up posted to Espresso.
• In espresso package: adds the CLI flags and interfaces for the streamer.
• In the batcher: adds NSM helper in op-batcher/enclave/attestation.go and modifies the driver to add an Espresso path. Bulk of the changes is in espresso_-prefixed files.

This is #447, re-hosted from an in-repo branch (now properly stacked).

[QuentinI]

Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: op-batcher/batcher/service.go:390.

Original transcript:

---

@chatgpt-codex-connector — 2026-05-26:

Guard ChainSigner initialization behind Espresso mode

initTxManager now always calls initChainSigner, but bs.TxManager is created as *txmgr.SimpleTxManager, which does not implement opcrypto.ChainSigner; this makes service startup fail with "tx manager does not implement ChainSigner" even when --espresso.enabled=false. Because this path runs during normal batcher boot, it becomes a universal startup regression rather than an Espresso-only failure.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.

[QuentinI]

Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: espresso/cli.go:200.

Original transcript:

---

@chatgpt-codex-connector — 2026-05-26:

Validate fallback-auth lead time outside Espresso-enabled block

FallbackAuthLeadTime is validated only when c.Enabled is true, but fallback mode (--espresso.enabled=false) still uses this field in isFallbackAuthRequired; a negative duration therefore bypasses validation and is later cast to uint64, producing a huge lead that can force premature/always-on fallback auth gating. This creates hard-to-diagnose behavior for fallback batchers with a mis-set duration flag.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.

[QuentinI]

Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: op-batcher/batcher/espresso_active.go:52.

Original transcript:

---

@chatgpt-codex-connector — 2026-05-26:

Check configured signer address in active-batcher decision

isBatcherActive determines activity solely from activeIsEspresso vs EspressoEnabled, without verifying that this node’s Txmgr.From() matches the currently authorized batcher address for that mode. In fallback mode, a node with the wrong sender key will still consider itself active and keep attempting publishes that revert at BatchAuthenticator, causing avoidable failure loops and no progress if this is the only running batcher.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.

[QuentinI]

Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: op-batcher/batcher/espresso.go:1344.

Original transcript:

---

@jcortejoso — 2026-05-29:

Would be possible to include here daType and tx size?:

type txRef struct {
    id       txID
    isCancel bool
    isBlob   bool
    daType   DaType
    size     int
}

[piersy]

I think this breaks batcher startup entirely. initTxManager (op-batcher/batcher/service.go:389) always calls initChainSigner, which asserts bs.TxManager to opcrypto.ChainSigner. But bs.TxManager is always a *txmgr.SimpleTxManager, which has neither Sign nor SignTransaction, so the assertion always fails and every batcher start — espresso or not — errors with "tx manager does not implement ChainSigner". The driver tests miss it because they build BatchSubmitter directly and skip service init.

The only ChainSigner implementations here (clientSigner, privateKeySigner in op-service/crypto/espresso.go) aren't wired into production — their only caller is a test.

Fix: build a real ChainSigner instead of casting the tx manager, or at least gate initChainSigner behind cfg.Espresso.Enabled so non-espresso batchers still start.

[lukeiannucci]

i just added a flag for non espresso mode
700f549

[piersy]

throttlingLoop starts by default (LowerThreshold defaults to 3,200,000) and just ranges over unsafeBytesUpdated with no context select. The non-espresso path sends on that channel and closes it on exit, but the espresso loops never touch it.

Two consequences: DA throttling silently does nothing in TEE mode, and on shutdown throttlingLoop blocks forever on the never-closed channel, so the wg.Wait in StopBatchSubmitting (op-batcher/batcher/driver.go:298) never returns — admin_stopBatcher or SIGTERM hangs until SIGKILL.

Fix: feed and close unsafeBytesUpdated from the espresso path like the non-espresso one

[lukeiannucci]

This i may need you to take a close look:
abb7ee1

After every block loaded from the streamer I send it to the throttling loop now to check if it should throttle.

[piersy]

This looks like a data race on the streamer. reset runs in the queueing loop and calls clearState, which resets the streamer via resetEspressoStreamer (writes readPos). Meanwhile the loading loop calls Peek/Update/Next on the same streamer without holding channelMgrMutex, and BufferedEspressoStreamer has no internal locking. So the two goroutines touch readPos and the batches slice concurrently — -race will flag it.

Beyond the race, a reset landing mid-iteration rewinds readPos, so the loading loop re-peeks already-consumed batches, re-adds a duplicate block, and AddL2Block returns ErrReorg — which triggers another clearState, churning in a loop.

Fix: serialize streamer access. Either don't reset the streamer from the queueing loop (signal the loading loop to do it), or guard every streamer call in both loops with one mutex.

[lukeiannucci]

014f88d

I made the espressoBatchQueueingLoop requests the clearState as suggested.

[piersy]

The throttling loop is started here, before startEspressoLoops runs its fallible setup (registerBatcher, resolveTEEVerifierAddress). If startEspressoLoops returns an error, StartBatchSubmitting returns without unwinding, so this goroutine is left on the waitgroup. running also stays true, so admin_startBatcher won't restart, and StopBatchSubmitting's wg.Wait blocks on this leaked loop (it ranges over unsafeBytesUpdated, which the espresso path never closes — see the other comment), so stop deadlocks too. The batcher is wedged until a process restart.

The earlier error returns (waitForL2Genesis, waitNodeSync) don't have this problem because they happen before any goroutine is started, so StopBatchSubmitting's wg.Wait returns immediately and recovers cleanly.

Fix: start the throttling loop after the espresso/non-espresso loops instead of before them. The send to it is non-blocking (select with a default in sendToThrottlingLoop), so it doesn't need to be running first. Then an error return from startEspressoLoops leaves no goroutine on the waitgroup, matching the earlier returns and keeping the failure recoverable.

[lukeiannucci]

let me know:
62e97b8

[piersy]

ToBlock doesn't validate the batch, and that turns a malformed-but-signed batch into a livelock for every TEE batcher on the namespace.

The encode side (BlockToEspressoBatch) checks that the first tx is an L1-info deposit, but the decode side never re-checks it: UnmarshalEspressoTransaction only recovers the signer and RLP-decodes, and ToBlock just re-inserts b.L1InfoDeposit as txs[0] without checking IsDepositTx() or that the header matches the batch body. An authorized signer — which includes the non-TEE fallback batcher key, a hot key on an ordinary host — can post a batch that is validly signed but semantically malformed (first tx not a deposit, or L1-info that doesn't parse). It passes the signature check and the header-only peek.

In the loading loop this hits the wrong error path. When ToBlock fails, the loop calls Next() and continues — it skips the bad batch. When AddL2Block fails, it calls clearState + streamer Reset and breaks — it rewinds. Because ToBlock does no validation it succeeds on the malformed batch, so the failure only surfaces one step later in AddL2Block (via BlockToSingularBatch rejecting the non-deposit tx), which takes the reset path. Reset rewinds to the safe head without advancing past the batch, so the next tick re-peeks the same batch, ToBlock succeeds again, AddL2Block fails again, reset again — forever. The safe head never advances, and since the namespace is shared every TEE batcher consuming it is stuck the same way until someone intervenes.

This is exactly what the NOTE here already asks for — ToBlock should fail only on invalid batches, so a bad batch takes the skip path rather than the reset path. The implementation just doesn't do the checking.

Fix: validate in ToBlock (or UnmarshalEspressoTransaction) — reject the batch when !b.L1InfoDeposit.IsDepositTx(), when the L1-info data fails to parse, or when BatchHeader is inconsistent with the decoded SingularBatch (parent hash, number, timestamp). Then a malformed-but-signed batch fails in ToBlock and the loop steps over it instead of resetting onto it.

[lukeiannucci]

let me know what you think about this fix + tests
b531ec9

[piersy]

SimpleTxManager does not implement ChainSigner this cast will always fail

[piersy]

This eth_sign call can't work against op-signer, and I don't think op-signer should be extended to make it work either.

op-signer doesn't serve eth_sign. Its server registers only two namespaces — eth (eth_signTransaction) and opsigner (signBlockPayload, signBlockPayloadV2): https://github.com/ethereum-optimism/infra/blob/main/op-signer/service/service.go#L73-L82. There is no arbitrary-data signing method. And this SignerClient can only talk to op-signer in the first place: NewSignerClient dials with op-signer's mutual-TLS and then handshakes with a health_status ping before returning, so pointing it at a plain geth or another HSM front-end fails at construction. So the call here errors method-not-found at runtime against a real op-signer.

The reason op-signer has no such method is deliberate, and it's why I'd argue against adding one. An HSM-backed signer must never sign raw bytes the caller hands it. If it did, a compromised batcher could pass a 32-byte value that is really the sighash of an L1 transaction spending the funded key, or a block payload for equivocation, and the HSM would sign it. That's why every op-signer method reconstructs the thing being signed server-side from typed arguments and binds a domain tag and chain id into the hash — see BlockPayloadArgs (domain, chainId, payloadBytes) and Message().ToSigningHash(). The client never sends a bare hash. Adding an eth_sign that signs any digest would remove exactly that protection for a key that also signs L1 transactions.

There's a second, backend-independent problem: eth_sign applies the EIP-191 prefix ("\x19Ethereum Signed Message:\n32" || hash), but the verify side recovers over the raw digest (crypto.SigToPub(batchHash, sig) in op-node/rollup/derive/espresso_batch.go):

optimism/op-node/rollup/derive/espresso_batch.go

Lines 105 to 107 in 014f88d

	batchHash := crypto.Keccak256(batchData)
	signerKey, err := crypto.SigToPub(batchHash, signatureData)

So even a signer that did serve eth_sign would recover the wrong address and every batch would be rejected.

If remote HSM signing of Espresso batches is a requirement, the right shape is a purpose-built op-signer method modeled on signBlockPayload: the client sends typed args (a fixed domain tag, the L2 chain id / namespace, and the batch commitment), op-signer reconstructs the domain-separated digest and signs it with the HSM key, and op-node verifies the same digest. That also resolves the separate domain-separation gap (the batch digest is currently a bare keccak256(rlp(batch)) with no namespace binding). It is a change in the op-signer repo, so it can't land from this PR alone — until it exists, only the local private-key ChainSigner actually works. I'd suggest dropping this eth_sign helper and the clientSigner branch here rather than shipping a path that can't sign or verify.

[piersy]

This field appears unused, it's never read, except in a test. It's not set at construction time (line 83) and then only set during deserialisation (line 117). I'd suggest removing it.

[piersy]

This is an unauthenticated remote crash. Number() calls bigs.Uint64Strict(BatchHeader.Number), which panics if the value doesn't fit in uint64 (op-service/bigs/bigutil.go:24). UnmarshalEspressoTransaction does no bounds check on the header number, and RLP will happily decode a big.Int of any size into Header.Number, so a decoded batch can carry a number >= 2^64.

The reason an arbitrary attacker can reach it: posting to an Espresso namespace is permissionless — a namespace is just a tag, anyone can submit a transaction under it. The streamer defends against unauthorized batches by recovering the signer and checking it against the on-chain registered batcher, but that check happens too late. In espresso-streamers v1.1.0 the receive path processEspressoTransaction unmarshals the batch (op_streamer.go:557) and then calls CheckBatch (op_streamer.go:563), whose very first line calls batch.Number() (op_streamer.go:255) — while the authorized-signer check is ~40 lines later at op_streamer.go:296. So the panic fires before the signer is ever validated.

Concretely: anyone with a throwaway key signs a batch whose BatchHeader.Number exceeds 2^64 and posts it to the namespace. Unmarshal succeeds (the signature is well-formed, the recovered signer is just some junk address), then CheckBatch calls Number() and every consumer of the namespace — op-node derivation, the caff node — panics before it gets to reject the unauthorized signer. One message takes them down.

Hash() at line 41 has the same shape: it dereferences b.L1InfoDeposit.Hash(), and nothing guarantees L1InfoDeposit is non-nil after decode, so a batch that omits it triggers a nil-pointer panic.

Fix: bounds-check at the decode boundary in UnmarshalEspressoTransaction — reject a BatchHeader.Number that doesn't fit in uint64, and reject a nil L1InfoDeposit — returning an error so the batch is dropped at op_streamer.go:558 instead of reaching CheckBatch. This is the same decode-side validation the ToBlock/livelock issue needs, so both can land together.

[piersy]

In the Espresso transaction submitter there is a channel close/send race that shows up under the race detector and can panic in production on shutdown.

Each submit worker creates its own job channel and does defer close(ch) (espresso.go:638-639), then hands that channel to the scheduler through the worker queue. But it is the scheduler that sends on the channel — in scheduleSubmitTransactionJobs it does worker <- espressoTransactionJobAttempt{...} at espresso.go:567. When the context is cancelled these two goroutines race: the worker can return and close ch while the scheduler has already committed to the send. That is an unsynchronized close-and-send on the same channel. Running

go test ./op-batcher/batcher/ -run EspressoTransactionSubmitter -race

fails intermittently (3 of 8 local runs) with a data race between the close at espresso.go:639 and the send at espresso.go:567. Beyond the flaky test, if the scheduler select happens to pick the send case at the moment the channel closes it panics with "send on closed channel" and takes the batcher down instead of shutting it down cleanly. The verify worker has the same pattern at espresso.go:703-704.

The usual rule is that only the sender closes a channel. The simplest fix is to drop the defer close(ch) from both workers: nothing relies on the close for termination, since both the worker and the scheduler already exit on ctx.Done(), and the channel just becomes garbage once its references drop.

[piersy]

prevSyncStatus needs to be set somewhere when exiting the function, I'm not sure if it's on every return site or just some, but currently it's set once and never updated