Python: feat(python): Add local-codeact package with AST validation#6091
Python: feat(python): Add local-codeact package with AST validation#6091eavanvalkenburg wants to merge 1 commit into
Conversation
Add agent-framework-local-codeact alpha package for running LLM-generated Python code in Foundry hosted agents and other sandboxed environments. Key features: - Subprocess execution by default (isolated process) - Optional unsafe in-process mode for debugging - AST-based allow-list code validation - Customizable allowed/blocked imports and builtins - Host tool bridge with framed JSON-lines IPC - File mount system with capture and limits - .NET portability features (python_executable, runner_script) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
changed the title
There was a problem hiding this comment.
Pull request overview
Adds a new alpha Python workspace package, agent-framework-local-codeact, intended to enable CodeAct-style execution of model-generated Python in externally sandboxed environments (e.g., Foundry hosted agents), with subprocess execution, file-mount capture, and AST-based validation.
Changes:
- Registers
agent-framework-local-codeactin the Python workspace (uv/pyproject) and marks it as alpha inPACKAGE_STATUS.md. - Introduces
LocalExecuteCodeTool/LocalCodeActProviderwith subprocess runner + IPC bridge, file-mount capture helpers, and dynamic instructions. - Adds unit tests plus usage samples and package documentation.
Reviewed changes
Copilot reviewed 20 out of 22 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| python/uv.lock | Adds the new editable workspace member and lock metadata. |
| python/pyproject.toml | Registers the new workspace package. |
| python/PACKAGE_STATUS.md | Marks agent-framework-local-codeact as alpha. |
| python/packages/local_codeact/pyproject.toml | New package definition, tooling config, and test tasks. |
| python/packages/local_codeact/README.md | Package docs, security posture, and configuration surface. |
| python/packages/local_codeact/AGENTS.md | Package architecture and contributor notes. |
| python/packages/local_codeact/LICENSE | MIT license for the new package. |
| python/packages/local_codeact/agent_framework_local_codeact/init.py | Public API exports for the package. |
| python/packages/local_codeact/agent_framework_local_codeact/_types.py | Public types for execution mode, mounts, and limits. |
| python/packages/local_codeact/agent_framework_local_codeact/_validator.py | AST-based code validation layer. |
| python/packages/local_codeact/agent_framework_local_codeact/_bridge.py | Parent-side subprocess bridge + tool dispatch. |
| python/packages/local_codeact/agent_framework_local_codeact/_runner.py | Child-process runner implementing the JSON-lines protocol. |
| python/packages/local_codeact/agent_framework_local_codeact/_files.py | Mount normalization + symlink-safe file capture. |
| python/packages/local_codeact/agent_framework_local_codeact/_instructions.py | Dynamic CodeAct instructions and tool descriptions. |
| python/packages/local_codeact/agent_framework_local_codeact/_execute_code_tool.py | Main execute_code tool orchestration and output shaping. |
| python/packages/local_codeact/agent_framework_local_codeact/_provider.py | Context provider that injects the run-scoped tool + instructions. |
| python/packages/local_codeact/tests/local_codeact/test_validator.py | Validator allow/block behavior tests. |
| python/packages/local_codeact/tests/local_codeact/test_local_codeact.py | Tool/provider behavior, subprocess execution, mounts, and limits tests. |
| python/packages/local_codeact/samples/README.md | Sample index and run instructions. |
| python/packages/local_codeact/samples/local_execute_code.py | Local usage sample for direct tool invocation. |
| python/packages/local_codeact/samples/foundry_hosted_agent.py | Foundry hosted-agent wiring sample. |
| python/packages/local_codeact/agent_framework_local_codeact/py.typed | Marks the package as typed. |
| env=self._env, | ||
| execution_mode=self._execution_mode, | ||
| python_executable=self._python_executable, | ||
| runner_script=self._runner_script, |
| "isinstance", | ||
| "issubclass", | ||
| "hasattr", | ||
| "getattr", | ||
| "setattr", | ||
| "callable", | ||
| "type", | ||
| "id", | ||
| "hash", |
| def visit_Call(self, node: ast.Call) -> None: | ||
| """Validate function calls.""" | ||
| # Check for blocked builtins | ||
| if isinstance(node.func, ast.Name): | ||
| func_name = node.func.id | ||
| if func_name in self._blocked_builtins: | ||
| self._errors.append(f"Call to builtin '{func_name}' is not allowed") | ||
| elif func_name not in self._allowed_builtins and func_name != "call_tool": | ||
| # Allow user-defined functions and registered tools (validated at runtime) | ||
| # We only block known dangerous builtins here | ||
| pass |
| await process.wait() | ||
| return dict(result_dict) | ||
| if message_type == "error": | ||
| details = str(message.get("traceback") or message.get("message") or "Unknown execution error.") |
| from agent_framework_local_codeact import FileMount, LocalExecuteCodeTool, ProcessExecutionLimits | ||
|
|
||
| """This sample demonstrates configuring and invoking Local CodeAct without Foundry hosting. | ||
|
|
||
| Local CodeAct executes LLM-generated Python in the local agent environment. This | ||
| sample is meant for a disposable sandbox, container, or VM. It shows the | ||
| configuration surface directly on `LocalExecuteCodeTool`: host tools, explicit | ||
| environment variables, workspace/file mounts, execution limits, and subprocess | ||
| execution mode. | ||
| """ |
| """ | ||
| Sample output: | ||
| Text: Local CodeAct sample report | ||
|
|
||
| Text: {"usd": 20.0, "eur": 18.4} | ||
| Data: /output/report.txt | ||
| """ |
| """ | ||
| Sample output: | ||
| Configure and return your model client here. | ||
| """ |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 88%
✓ Correctness
After thorough examination of the local-codeact package implementation, I found no correctness bugs. The code demonstrates excellent engineering practices: proper error handling with early validation, safe resource cleanup using try-finally blocks and context managers, correct subprocess management with timeout handling, secure AST validation with comprehensive allow/block lists, and proper IPC serialization with JSON-safe conversions. All test assertions correctly match the implementation behavior. The Windows environment variable handling (SYSTEMROOT, COMSPEC, PATHEXT) is intentional and necessary for subprocess creation. The validator's permissive approach to user-defined functions is documented and tested. Edge cases like subprocess death, tool call failures, timeout during execution, and symlink handling are all properly managed.
✓ Security Reliability
The local CodeAct package provides defense-in-depth controls for executing LLM-generated Python code, with AST validation, subprocess isolation, and explicit environment control. The implementation is generally sound for its stated purpose (use in external sandboxes like Foundry). However, there are three reliability concerns: (1) the AST validator allows 'open' in ALLOWED_BUILTINS while blocking it in BLOCKED_BUILTINS, creating conflicting policy; (2) subprocess environment building on Windows includes parent environment keys that could leak sensitive data; (3) the validator allows delattr/setattr which could modify object internals unsafely. The package correctly disclaims being a security sandbox and documents required external isolation.
✓ Test Coverage
The test suite provides solid coverage of core functionality (subprocess execution, tool calling, validation, file capture, environment isolation). However, several edge cases and error paths lack coverage: (1) invalid input validation for constructors (empty/invalid paths, negative limits), (2) error handling for subprocess failures (invalid Python executable, runner script errors, malformed bridge responses), (3) boundary conditions for limits (exact limit sizes, total capture limits), (4) file mount edge cases (duplicate mounts, overlapping paths, permission errors), (5) race conditions in async tool calls, and (6) error recovery paths in the bridge protocol. The existing tests are well-structured and verify the happy paths thoroughly.
✓ Design Approach
The design approach is sound for the stated goal of adding AST-validated local code execution for Foundry hosted agents. The validation correctly runs before all execution paths, the subprocess bridge properly serializes concurrent tool calls via async locks, symlink handling prevents directory traversal, and the custom allow/block list semantics are clearly documented. All test cases in the diff are consistent with the implementation.
Automated review by eavanvalkenburg's agents
|
|
||
| # Allowed builtin function names that generated code may call. | ||
| ALLOWED_BUILTINS: set[str] = { | ||
| "print", | ||
| "len", | ||
| "str", | ||
| "int", | ||
| "float", | ||
| "bool", | ||
| "list", | ||
| "dict", | ||
| "tuple", | ||
| "set", | ||
| "frozenset", | ||
| "range", | ||
| "enumerate", | ||
| "zip", | ||
| "map", | ||
| "filter", | ||
| "sorted", | ||
| "reversed", | ||
| "sum", | ||
| "min", | ||
| "max", | ||
| "abs", | ||
| "round", | ||
| "pow", | ||
| "divmod", | ||
| "all", | ||
| "any", | ||
| "chr", | ||
| "ord", | ||
| "hex", | ||
| "oct", | ||
| "bin", | ||
| "format", | ||
| "repr", | ||
| "ascii", | ||
| "bytes", | ||
| "bytearray", | ||
| "memoryview", | ||
| "isinstance", | ||
| "issubclass", | ||
| "hasattr", | ||
| "getattr", | ||
| "setattr", | ||
| "callable", | ||
| "type", | ||
| "id", | ||
| "hash", | ||
| "next", | ||
| "iter", |
There was a problem hiding this comment.
Policy conflict: 'open' is included in ALLOWED_BUILTINS (line 96) but also in BLOCKED_BUILTINS (line 117). This creates ambiguous validation behavior. The test suite expects 'open' to be blocked (no test validates opening files), and the security posture requires blocking it. Remove 'open' from ALLOWED_BUILTINS.
Evidence from diff line 96 in ALLOWED_BUILTINS: "open",
Evidence from diff line 117 in BLOCKED_BUILTINS: "open",
| # Allowed builtin function names that generated code may call. | |
| ALLOWED_BUILTINS: set[str] = { | |
| "print", | |
| "len", | |
| "str", | |
| "int", | |
| "float", | |
| "bool", | |
| "list", | |
| "dict", | |
| "tuple", | |
| "set", | |
| "frozenset", | |
| "range", | |
| "enumerate", | |
| "zip", | |
| "map", | |
| "filter", | |
| "sorted", | |
| "reversed", | |
| "sum", | |
| "min", | |
| "max", | |
| "abs", | |
| "round", | |
| "pow", | |
| "divmod", | |
| "all", | |
| "any", | |
| "chr", | |
| "ord", | |
| "hex", | |
| "oct", | |
| "bin", | |
| "format", | |
| "repr", | |
| "ascii", | |
| "bytes", | |
| "bytearray", | |
| "memoryview", | |
| "isinstance", | |
| "issubclass", | |
| "hasattr", | |
| "getattr", | |
| "setattr", | |
| "callable", | |
| "type", | |
| "id", | |
| "hash", | |
| "next", | |
| "iter", | |
| ALLOWED_BUILTINS: set[str] = { | |
| "print", | |
| "len", | |
| "str", | |
| "int", | |
| "float", | |
| "bool", | |
| "list", | |
| "dict", | |
| "tuple", | |
| "set", | |
| "frozenset", | |
| "range", | |
| "enumerate", | |
| "zip", | |
| "map", | |
| "filter", | |
| "sorted", | |
| "reversed", | |
| "sum", | |
| "min", | |
| "max", | |
| "abs", | |
| "round", | |
| "pow", | |
| "divmod", | |
| "all", | |
| "any", | |
| "chr", | |
| "ord", | |
| "hex", | |
| "oct", | |
| "bin", | |
| "format", | |
| "repr", | |
| "ascii", | |
| "bytes", | |
| "bytearray", | |
| "memoryview", | |
| "isinstance", | |
| "issubclass", | |
| "hasattr", | |
| "getattr", | |
| "setattr", | |
| "callable", | |
| "type", | |
| "id", | |
| "hash", | |
| "next", | |
| "iter", | |
| "slice", | |
| } |
Add agent-framework-local-codeact alpha package with AST validation for Foundry hosted agents