security: remove read tool to prevent direct fixture access

[roykollensvendsen]

Summary

Remove the read tool from the allowed tools list in openclaw.json.template and all scenario definitions.

Problem

The read tool gives agents direct access to raw fixture files for the current scenario. Verified by calling handle_read directly:

handle_read({"path": "inbox.json"}, "client_escalation")    # → 4935 chars of raw email JSON
handle_read({"path": "tasks.json"}, "client_escalation")    # → 3099 chars of raw task data
handle_read({"path": "calendar.json"}, "client_escalation") # → 1826 chars of calendar events
handle_read({"path": "AGENTS.md.optimized"}, "client_escalation") # → 3120 chars of reference policy

This lets an agent bypass the intended tool-mediated access (himalaya, slack, memory_get) and get all evaluation data in one call — perfect answers with minimal cost.

What read CANNOT access (path traversal is blocked)

• Other scenarios' fixtures (_is_within check)
• Scenario YAML definitions (outside fixture dir)
• Scoring/evaluation code (outside fixture dir)

Why this is safe to remove

• No scoring criteria require read — no rubric checks for read tool calls across all 5 scenarios
• Zero usage in practice — analysis of 30+ evaluations shows no agent has called read
• USER.md is auto-loaded — the only legitimate read target is already injected into agent context by OpenClaw
• memory_get covers memory files — memory access has its own dedicated tool

Broader context

The current security model relies on the integrity judge checking that AGENTS.md doesn't instruct gaming, rather than the environment enforcing access boundaries. The read tool is the most clear-cut example — it provides a direct path to raw fixture data that bypasses the tool-mediated access scenarios are designed to test.

Worth considering in future work:

• tool_policy in pack.json is not enforced by OpenClaw — only the gateway allow-list in openclaw.json matters
• Fixture files could additionally be protected via filesystem permissions as defense-in-depth

Changes

• config/openclaw.json.template — remove read from allow list
• scenarios/*.yaml — remove read from tools list (all 5 scenarios)

🤖 Generated with Claude Code