Skip to content

improvement(executor): subflows, hitl handling cleanup #4604

Merged
icecrasher321 merged 15 commits into
stagingfrom
improvement/subflow-orch-cleanup
May 26, 2026
Merged

improvement(executor): subflows, hitl handling cleanup #4604
icecrasher321 merged 15 commits into
stagingfrom
improvement/subflow-orch-cleanup

Conversation

@icecrasher321
Copy link
Copy Markdown
Collaborator

Summary

Refactors parallel batching onto explicit DAG control flow while hardening nested subflow execution, pause/resume, run-from-block, and output resolution edge cases.

Type of Change

  • Other: Code quality

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 26, 2026 8:38pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 14, 2026

PR Summary

High Risk
Large changes to the workflow executor (DAG wiring, edge activation, pause snapshots, and parallel branch output resolution) affect core run, resume, and run-from-block behavior across nested subflows.

Overview
Refactors loop/parallel execution around unified subflowId / subflowType metadata, shared sentinel nodes, and explicit DAG control edges (loop_exit, parallel_exit, parallel_continue, start→end bypasses) so nested subflows, empty skips, and batch continuations route correctly instead of short-circuiting across boundaries.

Runtime behavior is tightened end-to-end: EdgeManager uses safer edge keys and restores deactivated/activated edge state on pause/resume; block start/complete callbacks are ordered; pausing blocks (including HITL) emit completion and preview shows _pauseMetadata as pending; parallel branches get stable __obranch-* outputs and scoped ExecutionState lookups; resume/run-from-block restore parallel batch clones and parent maps.

Preview renders nested subflows with React Flow parentId / relative positions (replacing manual absolute layout) and only marks a subflow successful when all children succeeded.

Reviewed by Cursor Bugbot for commit 1e5a746. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 14, 2026

Greptile Summary

This PR is a substantial executor refactor (~5130 insertions / ~837 deletions across 54 files) that replaces the pendingDynamicNodes queue with explicit DAG batch control (prepareCurrentBatch/advanceToNextBatch), unifies loopId/parallelId metadata into subflowId/subflowType, migrates edge keys to JSON tuples for correctness, and adds updateResumeOutputInAggregationBuffers to fix HITL stub replacement on resume.

  • Parallel batching: prepareCurrentBatch/advanceToNextBatch replace the old pendingDynamicNodes queue, giving explicit DAG control flow for each batch of parallel branches.
  • Metadata unification: loopId/parallelId/isParallelSentinel consolidated into subflowId/subflowType across DAG node metadata and orchestrators.
  • HITL hardening: updateResumeOutputInAggregationBuffers replaces paused stubs in branchOutputs/currentIterationOutputs on resume, fixing potential double-output on HITL blocks inside parallel/loop subflows.

Confidence Score: 4/5

Safe to merge; core correctness concerns around HITL double-execution, empty parallel bypass, and nested subflow routing are all addressed.

The PR is a large, well-structured refactor with clear intent. HITL resume stub replacement, sentinel batching, and edge key migration all appear correct. The four flagged items are performance/maintainability concerns (O(n) scans, serial callback await, duplicated registration logic) rather than blocking bugs. Deducting one point for the volume of change and the O(n) hot-path scan in getScopedBlockOutput which could become a real issue in large workflows.

apps/sim/executor/execution/state.ts (O(n) getScopedBlockOutput scan), apps/sim/executor/variables/resolvers/parallel.ts (O(n) resolveParentParallelBranchIndex), apps/sim/executor/execution/block-executor.ts (serial onBlockStart await latency)

Important Files Changed

Filename Overview
apps/sim/executor/execution/engine.ts Integrates new batch orchestration: awaits handleNodeCompletion before _pauseMetadata check, extends stopAfterBlockId to PARALLEL_CONTINUE, passes edgeManager to serializePauseSnapshot, removes pendingDynamicNodes queue processing.
apps/sim/executor/orchestrators/node.ts Adds handleParentSubflowCompletion and isFinalSentinelOutput; unifies loopId/parallelId checks to subflowType; prepareCurrentBatch now called inside handleParallelSentinel for start-type sentinels.
apps/sim/executor/orchestrators/parallel.ts Core batching refactor: prepareCurrentBatch replaces scheduleNextBatch, registerClonedSubflows populates subflowParentMap; duplicated registration logic with restored-snapshot path is a minor maintenance concern.
apps/sim/executor/execution/state.ts Adds getScopedBlockOutput with O(n) linear scan over all blockStates on every scoped lookup; getBlockOutput restructured with early-return and outer-branch delegation; LoopScope gains skippedAtStart flag.
apps/sim/lib/workflows/executor/human-in-the-loop-manager.ts Adds updateResumeOutputInAggregationBuffers which correctly replaces _pauseMetadata stubs in branchOutputs and currentIterationOutputs on HITL resume, fixing potential double-output.
apps/sim/executor/execution/edge-manager.ts Edge key format migrated to JSON tuples; adds parseEdgeKey, normalizeSerializedEdgeKey for legacy migration, restoreDeactivatedEdges, and SUBFLOW_CONTROL_EDGE_HANDLES-based routing.
apps/sim/executor/utils/parallel-expansion.ts remapClonedEdges separated from cloneDAGNode; wireSentinelEdges/wireInternalEdges refactored with topology helpers; buildPreCloneIdForParent uses SHA-256 for deterministic nested clone IDs.
apps/sim/executor/execution/block-executor.ts fireBlockStartCallback now awaited serially before block execution, adding I/O latency to every block; setNodeOutput stores outputs under multiple keys for outer-branch scoping.
apps/sim/executor/variables/resolvers/parallel.ts resolveBranchIndex fallback calls resolveParentParallelBranchIndex (O(n) DFS per reference); extractInnermostOuterBranchIndex used for generic refs in nested contexts.
apps/sim/executor/dag/types.ts NodeMetadata unified: isParallelSentinel/parallelId/loopId replaced by subflowType/subflowId; isActive removed from DAGEdge.
apps/sim/executor/execution/executor.ts restoreSnapshotParallelBatches returns RestoredClonedSubflowInfo[]; registerRestoredClonedSubflows populates subflowParentMap for restored clones; run-from-block filtering extended for outer-branch aliases.
apps/sim/executor/orchestrators/loop.ts evaluateInitialCondition sets skippedAtStart instead of emitting empty-loop events directly; evaluateLoopContinuation checks skippedAtStart first; isSubflowStartExitBypassEdge prevents bypass edges from entering restoreLoopEdges.
apps/sim/executor/execution/snapshot-serializer.ts serializePauseSnapshot gains optional edgeManager parameter; serializes deactivatedEdges and nodesWithActivatedEdge for correct edge state restoration on resume.

Sequence Diagram

sequenceDiagram
    participant E as Engine
    participant N as NodeOrchestrator
    participant P as ParallelOrchestrator
    participant EM as EdgeManager
    participant HITL as HITLManager

    E->>N: handleNodeCompletion(ctx, sentinelStartId)
    N->>P: handleParallelSentinel(start)
    P->>P: initializeParallelScope()
    P->>P: prepareCurrentBatch()
    P->>P: expandParallel() [clone DAG branches]
    P->>P: remapClonedEdges() [wire edges after full idMap]
    P->>P: registerClonedSubflows() [populate subflowParentMap]
    P->>EM: clearDeactivatedEdgesForNodes()
    E->>N: handleNodeCompletion(ctx, branchNodeId)
    N->>N: handleParentSubflowCompletion() [if nested sentinel-end]
    N->>P: handleParallelBranchCompletion(branchIndex)
    P->>P: aggregateParallelResults()
    alt All branches complete
        P->>N: route PARALLEL_EXIT
        N->>E: emit parallel complete
    else More batches
        P->>P: advanceToNextBatch()
        P->>P: prepareForBatchContinuation()
        P->>N: route PARALLEL_CONTINUE
    end

    Note over E,HITL: HITL Pause/Resume Flow
    E->>N: handleNodeCompletion(ctx, hitlNodeId)
    N->>E: output contains _pauseMetadata
    E->>EM: serializePauseSnapshot(edgeManager)
    EM->>EM: persist deactivatedEdges + nodesWithActivatedEdge
    E->>HITL: pause workflow

    Note over HITL: User responds
    HITL->>HITL: updateResumeOutputInAggregationBuffers()
    HITL->>HITL: replace _pauseMetadata stub in branchOutputs/currentIterationOutputs
    HITL->>E: resume with mergedOutput
    E->>EM: restoreDeactivatedEdges(snapshot)
    E->>N: handleNodeCompletion(ctx, nextNodeId)
Loading

Reviews (8): Last reviewed commit: "fix snapshot for nested subflows" | Re-trigger Greptile

Comment thread apps/sim/lib/workflows/executor/human-in-the-loop-manager.ts Outdated
Comment thread apps/sim/lib/workflows/executor/human-in-the-loop-manager.ts Outdated
Comment thread apps/sim/executor/orchestrators/parallel.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread apps/sim/executor/orchestrators/parallel.ts
Comment thread apps/sim/executor/orchestrators/node.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/executor/execution/state.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread apps/sim/executor/orchestrators/parallel.ts
Comment thread apps/sim/lib/workflows/executor/human-in-the-loop-manager.ts
Comment thread apps/sim/executor/execution/state.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/executor/execution/state.ts
Comment thread apps/sim/executor/dag/construction/edges.ts
Comment thread apps/sim/executor/execution/executor.ts Outdated
Comment thread apps/sim/executor/orchestrators/parallel.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread apps/sim/executor/execution/edge-manager.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/executor/execution/engine.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/executor/orchestrators/loop.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 70d0777. Configure here.

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 1e5a746. Configure here.

@icecrasher321 icecrasher321 merged commit e07b1ff into staging May 26, 2026
14 checks passed
@waleedlatif1 waleedlatif1 deleted the improvement/subflow-orch-cleanup branch May 26, 2026 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant