feat(execution): payload size bottlenecks with lazy execution value hydration, safer materialization, and batched parallel execution

[icecrasher321]

Summary

• Parallel Block batching removing iteration limits
• Lazy loading for variable references involving base64 supported for isolated-vm js execution

Type of Change

• New feature

Testing

Tested manually

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		May 12, 2026 6:35pm

[cursor]

PR Summary

High Risk
High risk because it changes core workflow execution/streaming behavior and introduces new payload compaction/lazy materialization paths that affect API responses, pause/resume snapshots, and parallel orchestration state.

Overview
Adds platform-wide payload size mitigation. Workflow execute and function execute routes now compact oversized outputs/log payloads before returning or streaming them, propagate workspaceId/workflowId/executionId/userId into streaming/event writers, and surface large-value placeholders in the UI structured output.

Introduces lazy materialization for JS isolated-vm Functions. The function execute API wires new sim.files.read* and sim.values.read brokers (with byte limits and chunked reads) so functions can hydrate large files/value refs on-demand, with explicit handling for resource-limit errors.

Implements batched Parallel execution. Parallel subflows gain batchSize (1–20) persisted in DB and editable in the UI; the executor runs large parallels in serial batches while preserving global branch indexes, snapshot-serializes accumulated outputs/mappings for pause/resume, and updates edge routing to support a parallel_continue sentinel path.

Docs/SDK references updated. Async execution docs and SDK examples switch from taskId to jobId/statusUrl, and docs add guidance on oversized outputs and large input handling.

^{Reviewed by Cursor Bugbot for commit f9d4725. Configure here.}

[greptile-apps]

Greptile Summary

This PR introduces lazy large-value hydration for isolated-VM JavaScript execution, a new LargeValueRef spill/materialization pipeline, and batched parallel block execution to remove the previous hard branch-count ceiling.

• Large-value payload system (payloads/store.ts, cache.ts, materialization.server.ts, serializer.ts): block outputs exceeding 8 MB are serialised to durable storage and replaced inline with a LargeValueRef marker; a process-local LRU cache provides fast re-reads within the same execution.
• Lazy hydration in function blocks (resolver.ts, reference-async.server.ts, isolated-vm-worker.cjs): instead of inlining large values as JSON literals, the resolver emits await sim.values.read(globalThis[varName]) calls; a new IPC broker channel in the worker dispatches them back to the main process for materialisation.
• Batched parallel execution (parallel.ts): branches are now executed in configurable batches (default 20); accumulated outputs are compacted between batches using compactSubflowResults before the next batch is scheduled; loop and forEach iteration caps are replaced with soft defaults.

Confidence Score: 4/5

The core parallel batching and serializer pipeline are well-structured, but the sim.values.read broker in the function route only reads from durable storage, bypassing the in-memory cache. A transient storage upload failure leaves the value in memory without a key, causing the broker to always throw under storage pressure.

The sim.values.read broker calls readLargeValueRefFromStorage directly rather than materializeLargeValueRef. When persistValue fails silently the value has no key and lives only in the in-memory cache; the broker cannot find it and throws, so function blocks referencing large values produced during a storage disruption will fail.

apps/sim/app/api/function/execute/route.ts (sim.values.read broker handler) and apps/sim/executor/execution/state.ts (accumulatedOutputs type definition)

Important Files Changed

Filename	Overview
apps/sim/app/api/function/execute/route.ts	Adds sim.values.read and sim.files.* broker handlers for the isolated-VM function block; the sim.values.read handler calls readLargeValueRefFromStorage (storage-only) instead of materializeLargeValueRef, silently failing for memory-only refs when storage upload previously failed.
apps/sim/executor/orchestrators/parallel.ts	Adds batched parallel execution with configurable batch size, per-batch compaction of accumulated outputs, and scoped resetBatchExecutionState that only resets the incoming batch nodes.
apps/sim/executor/variables/resolver.ts	Makes all resolution methods async, adds lazy LargeValueRef injection for JS function blocks via formatLazyLargeValueReference, and improves the module-syntax scanner to avoid false-positives on comments/strings.
apps/sim/lib/execution/payloads/store.ts	New file implementing storeLargeValue (persist to storage + cache) and materializeLargeValueRef (cache-first, then storage fallback). Logic is sound.
apps/sim/lib/execution/payloads/cache.ts	Process-level singleton LRU cache for large values (256 MB cap). Evicts only recoverable entries under pressure.
apps/sim/lib/execution/isolated-vm-worker.cjs	Adds sim global to the isolate sandbox backed by a new brokerCallback IPC channel; uses sendIpcRequest to surface delivery failures immediately.
apps/sim/executor/variables/resolvers/reference-async.server.ts	New server-side async path navigator that materialises LargeValueRef at each step and hydrates base64 file fields on demand.
apps/sim/executor/orchestrators/loop.ts	Removes hard iteration caps, switches to DEFAULT_LOOP_ITERATIONS as a default, adds per-loop result compaction, and makes reference resolution async.
apps/sim/lib/execution/payloads/materialization.server.ts	New server-side materialization helpers with workspace/workflow/execution scope access-control checks.
apps/sim/lib/execution/payloads/serializer.ts	New async compaction pipeline that spills large values to storage and replaces them with LargeValueRef markers, with WeakSet cycle detection.
apps/sim/executor/execution/state.ts	Adds batch tracking fields to ParallelScope; accumulatedOutputs typed as Map<number, NormalizedBlockOutput[]> but post-compaction entries may be LargeValueRef objects — a type mismatch.

Sequence Diagram

sequenceDiagram
    participant Executor
    participant Serializer
    participant Store
    participant Cache
    participant Storage
    participant FunctionRoute
    participant IVMWorker
    participant Broker

    Executor->>Serializer: compactExecutionPayload(blockOutput)
    Serializer->>Store: storeLargeValue(value, json, size, ctx)
    Store->>Storage: persistValue(id, json) key
    Store->>Cache: cacheLargeValue(id, value, size, ctx)
    Store-->>Serializer: LargeValueRef
    Serializer-->>Executor: compacted output with LargeValueRef

    Executor->>Executor: resolveCodeWithContextVars()
    Note over Executor: LargeValueRef found - lazy injection
    Executor->>FunctionRoute: POST /api/function/execute with contextVariables

    FunctionRoute->>IVMWorker: executeInIsolatedVM with brokers
    IVMWorker->>IVMWorker: user code calls await sim.values.read(ref)
    IVMWorker->>Broker: IPC broker message sim.values.read
    Broker->>Storage: readLargeValueRefFromStorage(ref)
    Storage-->>Broker: materialised value
    Broker-->>IVMWorker: IPC brokerResponse
    IVMWorker-->>FunctionRoute: execution result

Loading

Comments Outside Diff (1)

1. apps/sim/app/api/function/execute/route.ts, line 806-820 (link)

   sim.values.read broker bypasses the in-memory cache

   readLargeValueRefFromStorage returns undefined when ref.key is absent (the path taken when persistValue failed silently due to a transient storage error). In that case the large value lives only in the in-memory cache, but this broker never consults it, so the call will always throw unavailableLargeValueError even though the data is right there in process.

   The fix is to use materializeLargeValueRef from @/lib/execution/payloads/store, which checks the in-memory cache first and only falls back to durable storage — matching how the variable resolver already materialises refs via navigatePathAsync.

_{Reviews (3): Last reviewed commit: "address more comments" | Re-trigger Greptile}

[icecrasher321]

@greptile

[icecrasher321]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f9d4725. Configure here.}

[icecrasher321]

@greptile

[icecrasher321]

@greptile

[icecrasher321]

@greptile