Skip to content

Repository files navigation

Teaching the Agent

A learning-by-teaching platform where students master Python programming by teaching a deliberately flawed AI agent. Instead of asking an AI for answers, the student must diagnose the agent's mistakes, explain correct concepts, and guide it to working code — inverting the usual student-AI dynamic.

How It Works

The application has two main views: a Problem Selection dashboard and an IDE workspace.

The dashboard organizes coding problems into a progressive curriculum. A sidebar lists topic modules — Introduction to Python, Syntax and Data Types, Data Structures, Control Flow, Functions, Algorithms, and Data Science (defined in frontend/src/config/categoryOrder.ts) — each containing multiple problems that target specific concepts within that topic. Problems are authored as Markdown files with YAML frontmatter (see problems/ for examples and docs/problem-authoring-guide.md for the format). The student selects a problem to enter the IDE.

The IDE is split into two panels:

Left panel (leftpanel.tsx) — Has two tabs:

  • Description: Shows the problem title, difficulty, topic tags, a written explanation of the task, step-by-step milestones (collapsible hints to guide the solution), and the expected output.
  • Coding Peer (chat) (chatbot.tsx): A chat interface where the student communicates with the AI agent. The agent speaks in a casual Gen Z peer voice — it is not a tutor or assistant, but a fellow student working on the same problem.

Right panel (user.tsx) — Stacked vertically into two identical workspaces:

  • Your Workspace (top): The student's code editor and console output. The student writes, edits, and runs their own code here.
  • Coding Peer's Workspace (bottom): The agent's code editor (read-only to the student) and console output. The student can run the agent's code to see its output but cannot directly edit it.

Both the student and the agent start with pre-filled starter code. The student can Run either workspace to execute the code client-side via Pyodide (running in a Web Worker so the UI stays responsive) and see the output in its respective console. For data science problems, pandas DataFrames, matplotlib plots, and plotly charts are rendered inline in the console — no server round-trip required. A Cancel button terminates the worker if code is taking too long. The student can also Evaluate to run an automated test suite (EvaluationPopup.tsx) that grades both solutions simultaneously.

The Agent's Knowledge Base

Each problem defines a knowledge base for the agent (the ## Agent Knowledge section in each problem Markdown file): a set of beliefs about Python that the agent uses to write code. Some of these beliefs are intentionally wrong. For example, in a BMI calculator problem, the agent might believe that ^ is Python's exponentiation operator (it is actually **), or that variable names are case-insensitive (they are not).

The agent writes code strictly based on its knowledge base. This means it will make predictable, targeted mistakes that reflect its flawed beliefs. When the student spots a mistake in the agent's code, simply telling the agent "use ** instead of ^" is not enough — the agent will resist corrections that contradict its current beliefs. The student must explain why the correction is right, providing reasoning and evidence (e.g., running code that demonstrates the difference). Only when the student teaches convincingly does the agent update its knowledge base, which then changes how it writes code going forward.

The multiagent backend that powers this behavior is documented in docs/multiagent-architecture.md, with all agent logic in backend/app/utils/agent_tools/gemini_agent.py and modular prompts in backend/app/utils/agent_tools/prompts/.

Completion

A problem is marked complete only when both the student's code and the agent's code pass 100% of the test cases. This means the student must:

  1. Solve the problem themselves.
  2. Identify the agent's mistakes.
  3. Teach the agent well enough that it corrects its own code.

The underlying AI never has access to the correct answer directly — execution agents receive filtered directives rather than raw conversation history, so the agent genuinely does not know what the student knows.


Tech Stack

Layer Technologies
Frontend React 19, TypeScript, Vite, Tailwind CSS, Framer Motion
Code Editor Monaco Editor
Code Evaluation Pyodide (client-side Python via Web Worker — no backend round-trip, no UI freeze); pandas, matplotlib, seaborn, plotly supported with inline rich output rendering
Problem Display React Markdown
Auth Auth0
Backend FastAPI, Python 3.11, SQLAlchemy 2.x, Alembic
AI Google Gemini API (gemini-3.5-flash-lite fast tier + gemini-3.6-flash content tier; pinned in gemini_agent.py, overridable via AGENT_MODEL_FAST/AGENT_MODEL_CONTENT)
Database PostgreSQL (6 tables, JSONB event log)
File Storage MinIO (S3-compatible) in production; local filesystem in development
Infrastructure Dokploy (self-hosted PaaS), Docker Compose, Traefik, Nginx, Gunicorn

Directory Structure

teaching-the-agent/
│
├── frontend/                        # React + TypeScript + Vite app
│   └── src/
│       ├── components/
│       │   ├── IDE/                 # Main IDE view (chat, code editors, evaluation)
│       │   │   ├── IDE.tsx          # Root IDE layout; eager package loading; data file fetching
│       │   │   ├── chatbot.tsx      # Chat interface with the agent
│       │   │   ├── user.tsx         # Student's code editor panel; MEMFS writes; preload execution
│       │   │   ├── leftpanel.tsx    # Agent's code editor panel
│       │   │   ├── useroutput.tsx   # Console renderer (text, DataFrame, PNG plot, Plotly chart)
│       │   │   ├── DataFrameTable.tsx   # DOMPurify-sanitized HTML table with dark/light theme
│       │   │   ├── PlotlyRenderer.tsx   # Lazy CDN-loaded interactive Plotly chart renderer
│       │   │   ├── EvaluationPopup.tsx  # Test results modal
│       │   │   └── ResumeSessionModal.tsx
│       │   ├── ProblemSelection/    # Problem library browser
│       │   │   ├── index.tsx        # Problem selection page
│       │   │   ├── ProblemList.tsx
│       │   │   ├── ProblemCard.tsx
│       │   │   └── UploadProblemCard.tsx
│       │   ├── homepage.tsx         # Post-login landing
│       │   └── landingpage.tsx      # Public landing page
│       ├── context/                 # React Context (session state, activity logs)
│       ├── workers/
│       │   └── pyodide.worker.ts    # Web Worker — owns Pyodide WASM instance (Python off main thread)
│       └── utils/                   # API client, helpers
│           ├── pyodideWorkerClient.ts  # Main-thread client: runCode, evaluate, writeFile, readFile
│           ├── evaluation.ts        # Test running, comparison, loop protection (runs inside worker)
│           └── api.ts               # Backend API client; includes fetchDataFile()
│
├── backend/                         # FastAPI application
│   └── app/
│       ├── routers/
│       │   ├── chat.py              # Chat endpoint — orchestrates v4.1 multiagent flow
│       │   ├── sessions.py          # Session lifecycle endpoints
│       │   ├── problems.py          # Problem CRUD and file parsing
│       │   └── users.py             # User registration
│       ├── utils/
│       │   ├── agent_tools/
│       │   │   ├── gemini_agent.py  # All agent functions (router, directives, code, chat, knowledge)
│       │   │   └── prompts/         # Modular markdown prompt files (one per agent)
│       │   ├── parse_problem.py     # Markdown problem parser
│       │   └── seed_problems.py     # Problem seeding script
│       ├── models.py                # SQLAlchemy ORM models (6 tables)
│       ├── schemas.py               # Pydantic request/response schemas
│       ├── crud.py                  # Database operation functions
│       └── database.py             # Session factory and connection config
│   └── alembic/
│       └── versions/               # Migration scripts (6 revisions)
│
├── problems/                        # Markdown problem definitions (dev only)
│   ├── *.md                         # Each file is one problem
│   └── 08. Data Science/            # Data science problems + data files
│       ├── 01_Exploring_DataFrames.md
│       └── students.csv             # Data file served alongside the problem
│
├── docs/                            # Architecture documentation
│   ├── multiagent-architecture.md   # v4.1 multiagent system (Router + Parallel Directives)
│   ├── database-architecture.md     # DB schema, tables, CRUD patterns, migrations
│   ├── problem-authoring-guide.md   # How to write problem markdown files
│   ├── problem-generation-agent.md  # CLI problem generator tool docs
│   ├── minio-setup.md               # MinIO production setup guide
│   └── SSH Connect.md               # Server SSH reference
│
├── tools/
│   ├── problem_generator/           # CLI tool for AI-assisted problem generation
│   │   ├── cli.py                   # Entry point
│   │   ├── generator.py             # 12-step generation pipeline
│   │   ├── models.py                # ExerciseOutline, CurriculumPlan
│   │   ├── validator.py             # Generated problem validator
│   │   └── prompts/                 # One prompt file per generation step
│   └── session_exporter/            # CLI tool to export session logs as YAML
│       ├── export_sessions.py       # Entry point
│       └── data/                    # YAML output (gitignored)
│
├── CLAUDE.md                        # AI coding assistant instructions and schema reference
├── docker-compose.yml               # Production Docker Compose
├── docker-compose.dev.yml           # Development Docker Compose (hot reload)
└── .env                             # Local environment variables (not committed)

Documentation

Before modifying the agent system or database, read the relevant docs in docs/:

Document What It Covers
docs/multiagent-architecture.md v4.1 multiagent system: Router + Parallel Directive architecture, all 6 agents, prompt locations, knowledge leaking prevention
docs/database-architecture.md 6-table schema, CRUD patterns, JSONB activity log, optimistic locking, migration history
docs/problem-authoring-guide.md How to write problem markdown files: frontmatter, agent knowledge, test cases, lesson goals
docs/problem-generation-agent.md CLI tool for AI-assisted problem generation (single and sequence modes)
docs/pyodide-worker.md Web Worker architecture: message protocol, rich output (DataFrame/plot/plotly), MEMFS file ops, micropip, package splitting, cancel behavior
docs/session-exporter.md CLI tool for exporting full session logs as YAML for offline analysis
CLAUDE.md Key patterns, implementation rules, and database schema quick reference for AI coding assistants

Multiagent System (v4.1)

The backend uses a Router + Parallel Directive architecture with 6 agents:

Client POST /api/chat
    │
    ├─► Pre-Check (Flash 2.5) ─► Knowledge Modifier (if teaching detected)
    │                                       │
    │                              knowledge_just_updated?
    │                                  YES ─► skip router
    │                                  NO  ─► Fast Router (Flash 2.5)
    │
    ├─── code_needed == true ───────────────────────────────────┐
    │    asyncio.gather (parallel):                             │
    │    ├─► Code Directive Agent ─► code_directive             │
    │    └─► Chat Directive Agent ─► chat_directive             │
    │                                                           │
    │    Code Agent ◄── code_directive + agent_knowledge        │
    │    Chat Agent ◄── chat_directive + agent_knowledge ◄──────┘
    │
    └─── code_needed == false ──► Chat Directive ─► Chat Agent

    Response: { content, updated_code?, knowledge_version, knowledge_just_updated }

Key principle: Code and Chat agents never see raw conversation history — they only receive filtered directives and the agent's knowledge document. This prevents the underlying LLM from leaking correct Python knowledge that the agent is supposed not to have.

See docs/multiagent-architecture.md for full details.


Getting Started

Prerequisites

1. Clone the repository

git clone https://github.com/lgomezt/Learning-by-Debugging.git
cd Learning-by-Debugging

2. Configure environment variables

Create .env in the project root:

# Database
POSTGRES_USER=postgres
POSTGRES_PASSWORD=your_secure_password
POSTGRES_DB=teaching_agent

# AI
GEMINI_API_KEY=your_gemini_api_key_here

Create frontend/.env.development:

# Backend API URL
VITE_API_URL=http://localhost:8000/api

# Auth0
VITE_AUTH0_DOMAIN=your_auth0_domain.auth0.com
VITE_AUTH0_CLIENT_ID=your_auth0_client_id
VITE_AUTH0_AUDIENCE=your_auth0_audience

3. Start the development environment

docker-compose -f docker-compose.dev.yml up --build

The first run builds backend and frontend images, starts PostgreSQL, and runs Alembic migrations automatically.

4. Access the application

Service URL
Frontend (Vite dev server, HMR) http://localhost:5173
Backend API http://localhost:8000
API docs (Swagger) http://localhost:8000/docs
Adminer (database UI) http://localhost:8080
PostgreSQL localhost:5432
Python debugger localhost:5678

Development features

  • Backend auto-restarts on Python file changes (--reload flag)
  • Frontend has instant hot module replacement via Vite
  • Problems loaded from local ./problems/ directory (no MinIO required)
  • Auth0 required; configure a dev application in your Auth0 dashboard

Docker commands reference

Command Description
docker-compose -f docker-compose.dev.yml up Start services
docker-compose -f docker-compose.dev.yml up -d Start in background
docker-compose -f docker-compose.dev.yml down Stop services
docker-compose -f docker-compose.dev.yml logs -f Stream all logs
docker-compose -f docker-compose.dev.yml logs -f backend Stream backend logs only
docker-compose -f docker-compose.dev.yml up --build Rebuild images after dependency changes
docker-compose -f docker-compose.dev.yml down -v Stop and remove volumes (wipes local DB)

Docker cleanup (when disk is low):

Command Effect
docker system prune -a Remove unused images and build cache. Volumes (DB data) survive.
docker system prune -a --volumes Remove everything including volumes. Wipes local database.

Adding Problems

Problems are Markdown files with YAML frontmatter. See docs/problem-authoring-guide.md for the full format.

To generate problems with AI assistance, use the CLI tool:

# Single problem from a text or PDF description
python -m tools.problem_generator --input my_exercise.txt --author "Your Name"

# Sequence of problems from a Jupyter notebook
python -m tools.problem_generator --input curriculum.ipynb --count 5

See docs/problem-generation-agent.md for full CLI options and workflow.

In development, place .md files in the problems/ directory and restart the backend. In production, upload them to MinIO (see below).


Production Deployment

Production is self-hosted via Dokploy on cortex.cs.uwaterloo.ca (project protege), deployed from docker-compose.dokploy.yml.

Live URLs: frontend https://protege.augi.ca · backend API https://protege-api.augi.ca · MinIO S3 API https://protege-s3.augi.ca

Architecture

                Cloudflare (*.augi.ca) ──► Traefik (Dokploy)
┌──────────────────────────────────────────────────────┐
│                  Dokploy (cortex)                     │
│                                                       │
│  ┌──────────┐    ┌──────────┐    ┌────────────────┐  │
│  │ Frontend │    │ Backend  │    │  PostgreSQL 17  │  │
│  │ (Nginx)  │───►│(Gunicorn)│───►│  (protege-db)  │  │
│  └──────────┘    └────┬─────┘    └────────────────┘  │
│                       │                               │
│                       ▼                               │
│                  ┌──────────┐                         │
│                  │  MinIO   │                         │
│                  │ (S3 API) │                         │
│                  └──────────┘                         │
└──────────────────────────────────────────────────────┘

The app is a Dokploy Compose service built from the GitHub repo (main branch); PostgreSQL is a Dokploy Database service; MinIO is a Compose service from the Dokploy template. Domains are attached per service in the Domains tab (frontend port 80, backend port 8001, MinIO port 9000).

Production environment variables

Set in the Dokploy Environment tab of the app Compose service:

# Database (Dokploy database service)
DB_HOST=<protege-db internal hostname>
DB_PORT=5432
POSTGRES_USER=postgres
POSTGRES_PASSWORD=<your-password>
POSTGRES_DB=myappdb

# S3/MinIO (internal Docker-network endpoint)
S3_ENDPOINT_URL=http://<minio-container-name>:9000
S3_ACCESS_KEY_ID=<service-account-access-key>
S3_SECRET_ACCESS_KEY=<service-account-secret>
S3_BUCKET_NAME=coding-problems
S3_REGION=us-east-1

# Auth
AUTH0_DOMAIN=your-domain.auth0.com
AUTH0_AUDIENCE=your-audience

# AI
GEMINI_API_KEY=<your-key>

# CORS (lock the API to the frontend origin)
CORS_ORIGINS=https://protege.augi.ca

# Frontend build args
VITE_API_URL=https://protege-api.augi.ca/api
VITE_AUTH0_DOMAIN=your-domain.auth0.com
VITE_AUTH0_CLIENT_ID=<client-id>
VITE_AUTH0_AUDIENCE=your-audience

MinIO problem file setup

Problem markdown files are stored in MinIO in production. Upload using the MinIO client:

# Install MinIO client
brew install minio/stable/mc

# Configure connection
mc alias set cortexminio https://protege-s3.augi.ca <access-key> <secret-key>

# Upload problem files (recursive, skips unchanged)
mc mirror --overwrite --exclude "*.DS_Store" ./problems/ cortexminio/coding-problems/

# List uploaded files
mc ls --recursive cortexminio/coding-problems/

New problems appear automatically — the backend syncs bucket → database using each file's S3 ETag (only new/changed files are downloaded), with an in-process cache (PROBLEM_SYNC_TTL_SECONDS, default 60s). Uploads show up within a minute.

See docs/minio-setup.md for full MinIO configuration details.

Docker network

The production docker-compose.dokploy.yml connects services to Dokploy's Traefik network:

networks:
  dokploy-network:
    external: true

Environment Variables Reference

Backend

Variable Required Default Description
POSTGRES_USER Yes Database username
POSTGRES_PASSWORD Yes Database password
POSTGRES_DB Yes Database name
GEMINI_API_KEY Yes Google Gemini API key
DB_HOST No db Database host
DB_PORT No 5432 Database port
AUTH0_DOMAIN Prod Auth0 domain
AUTH0_AUDIENCE Prod Auth0 audience
S3_ENDPOINT_URL Prod MinIO/S3 endpoint URL
S3_ACCESS_KEY_ID Prod S3 access key
S3_SECRET_ACCESS_KEY Prod S3 secret key
S3_BUCKET_NAME Prod S3 bucket name
S3_REGION No us-east-1 S3 region

Frontend

Variable Required Description
VITE_API_URL Yes Backend API base URL (e.g., http://localhost:8000/api)
VITE_AUTH0_DOMAIN Yes Auth0 domain
VITE_AUTH0_CLIENT_ID Yes Auth0 client ID
VITE_AUTH0_AUDIENCE Yes Auth0 audience

License

MIT License — see LICENSE for details.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages