-
Notifications
You must be signed in to change notification settings - Fork 434
Agentic refactoring and full MCP support #1502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gustavo-grieco
wants to merge
65
commits into
master
Choose a base branch
from
dev-agents
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
65 commits
Select commit
Hold shift + click to select a range
25562c6
first step to define agent interactions
gustavo-grieco 352ab79
first functional MCP command
gustavo-grieco 9005d53
removed redundant code
gustavo-grieco 1f6f93f
verify -> verification
gustavo-grieco 1bf5e2a
priorize_function command
gustavo-grieco c73d7ea
read_logs command
gustavo-grieco 582e6ad
better logs and coverage_report
gustavo-grieco c0677a5
hlint fixes
gustavo-grieco 159a7d2
improve show_coverage
gustavo-grieco 33358ad
fixed flake to build correctly
gustavo-grieco 45b90fe
fixed test compilation
gustavo-grieco d6cb38f
fixed flake to build correctly
gustavo-grieco f82da87
refactor MCP code
gustavo-grieco b33baec
refactor MCP code
gustavo-grieco b4de33a
fix tests
gustavo-grieco 8685e54
new command
gustavo-grieco 1c1bdde
new command
gustavo-grieco 67d103c
fixes
gustavo-grieco af90c03
simplify get coverage tool
gustavo-grieco 0eefe2d
new command
gustavo-grieco 773b2a3
make sure logs are available for the mcp
gustavo-grieco 9b33fc7
implemented status command
gustavo-grieco 5830818
fixes
gustavo-grieco b52d729
more fixes
gustavo-grieco 017b181
more fixes
gustavo-grieco a73c072
allow sequences to be prioritized
gustavo-grieco b2159a0
clean-up
gustavo-grieco 718f11d
new command
gustavo-grieco 15a7eaf
inject_fuzz_transactions validation
gustavo-grieco 9a85bb2
added target mcp command
gustavo-grieco 2d4a4cd
upgraded haskell-mcp-server
gustavo-grieco bc5b652
upgraded haskell-mcp-server
gustavo-grieco 1d277a9
added optimization values to the status command
gustavo-grieco b521a45
allow to intercalate random transactions durign priorized sequence
gustavo-grieco 9df353f
better handling of parsing injected transactions
gustavo-grieco 84c9005
insert transactions in a random part of the ones in the corpus
gustavo-grieco fb40b77
refactoring and probabily tweaking
gustavo-grieco 0559648
wrap the coverage MCP command output in code tags
gustavo-grieco 0b35c32
merge with master
gustavo-grieco 9739c03
fix
gustavo-grieco 8b0d246
merge
gustavo-grieco 3720fba
merge
gustavo-grieco af67e07
Merge remote-tracking branch 'origin' into dev-agents
gustavo-grieco cd27e92
merge
gustavo-grieco 3b0cd48
merge
gustavo-grieco f045f7f
added some tests
gustavo-grieco 0276afe
new operation and converted status to json output
gustavo-grieco 4385eea
simplify commands
gustavo-grieco 40592a7
add sample command
gustavo-grieco 8e781c4
missing file
gustavo-grieco 71291d8
updated hevm
gustavo-grieco e312deb
fixed zero division on verification mode
gustavo-grieco 155d108
fixes
gustavo-grieco 9dc0ef6
clean up
gustavo-grieco fc83148
avoid executing methods in foundry that are non-test ones when seqLen…
gustavo-grieco 763e705
code fix
gustavo-grieco 2e68e26
merge with master
gustavo-grieco 1975190
Add MCP integration test scaffolding from crytic/echidna#1509
datradito 8e47660
Add robust external MCP testing: conformance + Claude/Codex client sims
gustavo-grieco 87cd668
gitignore: ignore Python bytecode/pytest caches (tests/mcp)
gustavo-grieco 42f8d26
ci(mcp-tests): set least-privilege GITHUB_TOKEN permissions
gustavo-grieco b0c7d2c
ci(mcp-tests): use trailofbits Cachix + concurrency guard
gustavo-grieco ef7eb91
ci(mcp-tests): install crytic-compile + solc so echidna can compile
gustavo-grieco 844c167
flake: pin codex-fixed haskell-mcp-server (3bd3cdc)
gustavo-grieco 5307576
flake: correct haskell-mcp-server sha256 for 3bd3cdc
gustavo-grieco File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| name: MCP integration tests | ||
|
|
||
| # Out-of-process MCP testing: spins up a real Echidna campaign with the MCP | ||
| # server and drives it the way real agent clients do. Besides the tool/semantic | ||
| # tests it runs the client-compatibility suite: | ||
| # * test_mcp_conformance.py -- wire-protocol regression guard (202/405/handshake) | ||
| # * test_mcp_codex.py -- replays Codex's strict rmcp handshake | ||
| # * test_mcp_claude.py -- drives the server with the official `mcp` SDK | ||
| # Both client checks are transport-level and need no API keys, so they run on | ||
| # every push/PR. (A live-model smoke using examples/mcp_agent.py needs | ||
| # ANTHROPIC_API_KEY / Codex auth and is intentionally not part of CI.) | ||
|
|
||
| on: | ||
| push: | ||
| branches: ["**"] | ||
| pull_request: | ||
|
|
||
| # Least-privilege GITHUB_TOKEN: this workflow only checks out the repo and runs | ||
| # tests, so it needs no write scopes. | ||
| permissions: | ||
| contents: read | ||
|
|
||
| # A new push to the same ref cancels an in-flight run (avoids piling up slow builds). | ||
| concurrency: | ||
| group: mcp-tests-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| mcp-client-compat: | ||
| name: MCP client compatibility (Claude + Codex) | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install Nix | ||
| uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25 # v22 | ||
|
|
||
| # Pull prebuilt echidna/hevm/deps from the project's Cachix so the build | ||
| # takes minutes instead of compiling the whole closure from source. | ||
| - name: Configure Cachix | ||
| uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17 | ||
| with: | ||
| name: trailofbits | ||
| authToken: ${{ secrets.CACHIX_AUTH_TOKEN }} | ||
|
|
||
| - name: Build echidna | ||
| run: nix build .#echidna --out-link result | ||
|
|
||
| - name: Add echidna to PATH | ||
| run: echo "$GITHUB_WORKSPACE/result/bin" >> "$GITHUB_PATH" | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.11" | ||
|
|
||
| - name: Install test dependencies | ||
| run: pip install -r tests/mcp/requirements-test.txt | ||
|
|
||
| # Echidna shells out to crytic-compile + solc to compile the test contract. | ||
| - name: Install crytic-compile and solc | ||
| run: | | ||
| pip install crytic-compile solc-select | ||
| solc-select install 0.8.25 | ||
| solc-select use 0.8.25 | ||
|
|
||
| - name: Run MCP test suite (tools + conformance + Claude + Codex) | ||
| run: pytest tests/mcp -v | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # Echidna MCP Agent Example | ||
|
|
||
| An AI agent that connects to a live Echidna fuzzing campaign via the [MCP](https://spec.modelcontextprotocol.io/) server and autonomously guides the fuzzer. | ||
|
|
||
| ## Requirements | ||
|
|
||
| ``` | ||
| pip install langchain-anthropic langgraph httpx | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| **1. Start Echidna with the MCP server enabled:** | ||
|
|
||
| ```bash | ||
| echidna MyContract.sol --server 8080 --format text | ||
| ``` | ||
|
|
||
| The `--format text` flag is required — it disables the interactive TUI so the MCP server thread can run. | ||
|
|
||
| **2. Run the agent:** | ||
|
|
||
| ```bash | ||
| export ANTHROPIC_API_KEY=your_key_here | ||
| python examples/mcp_agent.py | ||
| ``` | ||
|
|
||
| ## What the agent does | ||
|
|
||
| The agent runs in a loop at a configurable interval. Each step it: | ||
|
|
||
| 1. **Observes** — calls `status` to read coverage, iterations, and corpus size. | ||
| 2. **Injects** — when coverage stagnates, asks Claude to suggest targeted transaction sequences and injects them via `inject_fuzz_transactions`. | ||
| 3. **Resets** — periodically calls `clear_fuzz_priorities` to prevent the fuzzer from getting stuck on a single function. | ||
|
|
||
| ## Available MCP tools | ||
|
|
||
| | Tool | Description | | ||
| |------|-------------| | ||
| | `status` | Campaign metrics: coverage, iterations, corpus size, last log lines | | ||
| | `target` | ABI of the target contract | | ||
| | `show_coverage` | Per-contract source coverage with line annotations | | ||
| | `dump_lcov` | Export coverage in LCOV format | | ||
| | `inject_fuzz_transactions` | Inject a semicolon-separated sequence of function calls | | ||
| | `clear_fuzz_priorities` | Reset function call weighting on all workers | | ||
| | `reload_corpus` | Reload transaction sequences from the corpus directory | | ||
|
|
||
| Transactions passed to `inject_fuzz_transactions` use Solidity call syntax and support `?` as a fuzzer wildcard: | ||
|
|
||
| ``` | ||
| transfer(0xdeadbeef00000000000000000000000000000001, 100);approve(?, ?) | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,166 @@ | ||
| #!/usr/bin/env python3 | ||
| """ | ||
| LangGraph agent for Echidna MCP integration. | ||
|
|
||
| Connects to a running Echidna campaign via its MCP server and autonomously | ||
| guides the fuzzer: analyzes coverage, injects targeted transactions, and | ||
| resets priorities when coverage stagnates. | ||
|
|
||
| Requirements: | ||
| pip install langchain-anthropic langgraph httpx | ||
|
|
||
| Usage: | ||
| # Start Echidna with MCP enabled: | ||
| echidna MyContract.sol --server 8080 --format text | ||
|
|
||
| # Run the agent: | ||
| ANTHROPIC_API_KEY=... python examples/mcp_agent.py | ||
| """ | ||
|
|
||
| import os | ||
| import time | ||
| import httpx | ||
| from typing import TypedDict, List | ||
| from langchain_anthropic import ChatAnthropic | ||
| from langgraph.graph import StateGraph, END | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # MCP client | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| def call_tool(tool: str, args: dict = None) -> str: | ||
| """Call an MCP tool and return the text response.""" | ||
| payload = { | ||
| "jsonrpc": "2.0", "id": 1, | ||
| "method": "tools/call", | ||
| "params": {"name": tool, "arguments": args or {}}, | ||
| } | ||
| resp = httpx.post("http://localhost:8080/mcp", json=payload, timeout=30) | ||
| result = resp.json().get("result", {}) | ||
| return result.get("content", [{}])[0].get("text", "") | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Agent state | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| class State(TypedDict): | ||
| coverage: int | ||
| iterations: int | ||
| stagnation: int | ||
|
|
||
|
|
||
| def parse_int(text: str, key: str) -> int: | ||
| for line in text.splitlines(): | ||
| if key in line: | ||
| try: | ||
| return int(line.split(":")[-1].strip().split()[0].replace(",", "")) | ||
| except ValueError: | ||
| pass | ||
| return 0 | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Graph nodes | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| def observe(state: State) -> State: | ||
| """Read current campaign status.""" | ||
| status = call_tool("status") | ||
| coverage = parse_int(status, "Coverage") | ||
| iterations = parse_int(status, "Iterations") | ||
| stagnation = state["stagnation"] + 1 if coverage == state["coverage"] else 0 | ||
| print(f" coverage={coverage} iterations={iterations} stagnation={stagnation}") | ||
| return {"coverage": coverage, "iterations": iterations, "stagnation": stagnation} | ||
|
|
||
|
|
||
| def inject(state: State) -> State: | ||
| """Ask the LLM to suggest targeted transactions and inject them.""" | ||
| llm = ChatAnthropic(model="claude-sonnet-4-5", temperature=0.7) | ||
| coverage_report = call_tool("show_coverage") | ||
|
|
||
| prompt = ( | ||
| "You are a smart-contract security expert helping an Echidna fuzzer.\n" | ||
| "The coverage report is:\n\n" | ||
| f"{coverage_report[:2000]}\n\n" | ||
| "Suggest 3 Solidity function calls that would increase coverage.\n" | ||
| "Reply with one call per line, e.g.: transfer(0xABCD..., 100)\n" | ||
| ) | ||
| suggestions = llm.invoke(prompt).content.strip().splitlines() | ||
|
|
||
| for tx in suggestions[:3]: | ||
| tx = tx.strip() | ||
| if tx and "(" in tx and not tx.startswith("#"): | ||
| print(f" injecting: {tx}") | ||
| call_tool("inject_fuzz_transactions", {"transactions": tx}) | ||
|
|
||
| return {**state, "stagnation": 0} | ||
|
|
||
|
|
||
| def reset(state: State) -> State: | ||
| """Clear function priorities to encourage exploration.""" | ||
| print(" clearing priorities") | ||
| call_tool("clear_fuzz_priorities") | ||
| return state | ||
|
|
||
|
|
||
| def route(state: State) -> str: | ||
| if state["stagnation"] >= 3: | ||
| return "inject" | ||
| if state["iterations"] > 0 and state["iterations"] % 100_000 == 0: | ||
| return "reset" | ||
| return END | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Build and run | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| def build_graph() -> StateGraph: | ||
| g = StateGraph(State) | ||
| g.add_node("observe", observe) | ||
| g.add_node("inject", inject) | ||
| g.add_node("reset", reset) | ||
| g.set_entry_point("observe") | ||
| g.add_conditional_edges("observe", route, {"inject": "inject", "reset": "reset", END: END}) | ||
| g.add_edge("inject", END) | ||
| g.add_edge("reset", END) | ||
| return g.compile() | ||
|
|
||
|
|
||
| def main(): | ||
| if not os.getenv("ANTHROPIC_API_KEY"): | ||
| print("Set ANTHROPIC_API_KEY before running.") | ||
| return | ||
|
|
||
| # Verify connectivity | ||
| try: | ||
| status = call_tool("status") | ||
| if not status: | ||
| raise RuntimeError("empty response") | ||
| print("Connected to Echidna MCP server.") | ||
| except Exception as e: | ||
| print(f"Cannot reach MCP server: {e}") | ||
| print("Start Echidna with: echidna MyContract.sol --server 8080 --format text") | ||
| return | ||
|
|
||
| graph = build_graph() | ||
| state: State = {"coverage": 0, "iterations": 0, "stagnation": 0} | ||
|
|
||
| duration = int(input("Duration in minutes [10]: ") or 10) | ||
| interval = int(input("Check interval in seconds [60]: ") or 60) | ||
| end_time = time.time() + duration * 60 | ||
|
|
||
| step = 0 | ||
| while time.time() < end_time: | ||
| step += 1 | ||
| print(f"\n--- step {step} ---") | ||
| state = graph.invoke(state) | ||
| time.sleep(interval) | ||
|
|
||
| print(f"\nDone. Final coverage: {state['coverage']} iterations: {state['iterations']}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.