PhD Advisor Search (unmaintained)

An open-source platform connecting prospective PhD students with faculty advisors through AI-powered matching.

Why open source? This project was originally built as a SaaS product, but PhD students generally didn't want to pay for it. Rather than let the code rot, I'm releasing it for the community to self-host, learn from, or build upon.

Overview

PhD Advisor Search helps prospective graduate students find faculty advisors whose research interests, mentorship style, and lab culture align with their goals. The platform aggregates public faculty data from university websites, enriches it with academic metrics, and provides intelligent search and matching capabilities.

Key Features

• Semantic Search: Find advisors by research interests using keyword filters and vector similarity matching
• Faculty Profiles: View lab members, research summaries, open positions, and publication metrics
• Data Aggregation: Automated scraping pipeline to discover and extract faculty data from university websites
• Subscription System: Tiered access model (free/paid) with Stripe integration
• Admin Dashboard: Manage scraping jobs, view analytics, and monitor system health

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Frontend (SvelteKit)                     │
│              TypeScript + Tailwind CSS + SSR                │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                   Backend (Rust + Rocket)                   │
│         REST API + PostgreSQL + pgvector + Stripe           │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│               Infrastructure (Docker + OpenTofu)            │
│     Local: Docker Compose │ Cloud: AWS ECS + RDS + CDN      │
└─────────────────────────────────────────────────────────────┘

Technology Stack

Layer	Technologies
Frontend	SvelteKit 2, TypeScript, Tailwind CSS, Vitest, Playwright
Backend	Rust, Rocket 0.5, SQLx, pgvector, Argon2, JWT
Database	PostgreSQL 16 with pgvector extension
Payments	Stripe (checkout, webhooks, customer portal)
Scraping	Firecrawl, Parallel Web Systems API, OpenRouter (LLM)
Email	Resend API
Infra	Docker Compose, OpenTofu/Terraform, AWS (ECS, RDS, CloudFront)

Project Structure

advisor_search/
├── backend/                 # Rust API server
│   ├── src/
│   │   ├── api/            # HTTP endpoints (auth, search, admin, subscriptions)
│   │   ├── models/         # Domain entities (Advisor, Student, Position, etc.)
│   │   ├── repositories/   # Database access layer
│   │   ├── scraping/       # Data extraction pipeline (Firecrawl, Parallel, OpenAlex)
│   │   ├── search/         # Keyword filtering + semantic ranking
│   │   ├── scheduler/      # Background jobs (OpenAlex sync, subscription monitor)
│   │   └── notifications/  # Email alerts via Resend
│   └── migrations/         # SQL migrations
├── frontend/               # SvelteKit web app
│   ├── src/
│   │   ├── routes/         # Pages (landing, search, advisor, admin, pricing)
│   │   ├── lib/components/ # Reusable UI components
│   │   └── lib/stores/     # Auth state management
│   └── tests/              # E2E tests (Playwright)
├── infra/                  # Infrastructure as Code
│   ├── modules/            # OpenTofu modules (database, backend, frontend, ALB)
│   ├── environments/       # Environment-specific configs (dev, staging, prod)
│   └── fail2ban/           # DDoS protection configs
├── monitoring/             # Uptime monitoring service (Rust)
├── docker-compose.yml      # Local development environment
└── justfile                # Task runner commands

Getting Started

Prerequisites

• Docker and Docker Compose
• Rust (latest stable)
• Node.js (v18+)
• just (task runner)

Quick Start

1. Clone the repository

   git clone https://github.com/yourusername/advisor_search.git
   cd advisor_search

2. Set up environment variables

   cp .env.example .env
   # Edit .env with your API keys and secrets

3. Start the database

   just up
   just migrate

4. Run the application

   just run

   This starts both the backend (port 8000) and frontend (port 5173).

5. Open in browser

   • Frontend: http://localhost:5173
   • API: http://localhost:8000/api/health

Environment Variables

See .env.example for the full list. Key variables:

Variable	Required	Description
DATABASE_URL	Yes	PostgreSQL connection string
JWT_SECRET	Yes	Secret for signing JWT tokens
STRIPE_SECRET_KEY	No	Stripe API key (for payments)
FIRECRAWL_API_KEY	No	Firecrawl API key (for scraping)
OPENROUTER_API_KEY	No	OpenRouter API key (for LLM enrichment)
RESEND_API_KEY	No	Resend API key (for email notifications)

Available Commands

Run just to see all available commands. Key commands:

# Development
just run              # Start backend + frontend
just up               # Start Docker services
just down             # Stop Docker services
just migrate          # Run database migrations
just db-shell         # Connect to PostgreSQL

# Testing
just test             # Run all tests
just test-backend     # Run Rust tests (includes clippy + fmt check)
just test-frontend    # Run Vitest unit tests
just test-e2e         # Run Playwright E2E tests

# Building
just build            # Build backend + frontend
just docker-build     # Build Docker image

# Infrastructure
just infra-init       # Initialize OpenTofu
just infra-plan       # Plan infrastructure changes
just infra-apply      # Apply infrastructure changes

System Design

Data Model

The core entities are:

• Advisor: Faculty member with name, institution, department, h-index, and citation metrics
• AdvisorResearch: Research categories, keywords, summary, and embedding vector
• Student: Lab members (PhD students, postdocs, alumni) linked to advisors
• OpenPosition: Hiring information for labs
• User: Platform users with subscription tier (free/paid)
• Subscription: Stripe customer and subscription state

Search System

Two-layer search architecture:

1. Keyword Filtering (fast): SQL-based filtering by categories, keywords, institution, lab size, open positions
2. Semantic Ranking (relevant): Vector similarity search using pgvector to rank by research alignment

Scraping Pipeline

Two-stage data extraction:

1. University Discovery: Uses Firecrawl's map API or Parallel Web Systems to discover faculty from university websites
2. Lab Info Extraction: Deep scrapes individual lab pages to extract students, positions, and research details

LLM enrichment (via OpenRouter) generates research summaries and categorizes faculty research areas.

Subscription Model

• Free Tier: Search results with anonymized/blurred advisor names and institutions
• Paid Tier: Full profiles, AI-powered matching explanations, lab culture insights

Stripe handles checkout, webhooks, and customer portal.

Deployment

Local (Docker Compose)

docker-compose up -d

This starts PostgreSQL, Caddy (reverse proxy), and fail2ban.

Production (GitHub Actions CI/CD)

The project uses a blue-green deployment strategy via GitHub Actions (.github/workflows/ci.yml). On push to main:

1. Build: Docker images are built with GitHub Actions cache and tagged with :green and :SHA
2. Deploy Green: New containers start on alternate ports (8001, 3001) alongside the running blue stack
3. Migrate: Database migrations run against the shared PostgreSQL instance
4. Test: Playwright E2E tests run against the green stack
5. Promote: If tests pass, Caddy switches traffic to green; old blue containers are removed

Breaking Migrations: Commits containing [breaking] or migrations with DROP/RENAME/ALTER TYPE trigger maintenance mode:

• Database backup is created
• Traffic is routed to a maintenance page
• Blue stack stops, migration runs, new version starts
• Maintenance mode disables

E2E Testing in CI: Before promotion, Playwright tests run inside a Docker container on the server:

• Tests execute against the green stack (frontend-green:3000, backend-green:8000)
• Uses playwright.prod.config.ts for production-specific settings
• If tests fail, the green stack is torn down and blue remains active
• Test fixtures and config are transferred to the server during deployment

Rollback: Manual rollback available via scripts/rollback.sh on the server.

Requirements:

• Server with Docker and Docker Compose
• GitHub secrets: SSH_PRIVATE_KEY, SERVER_HOST, GIT_CRYPT_KEY
• git-crypt for encrypted .env.prod file

The infra/ directory contains OpenTofu modules for:

• RDS PostgreSQL with pgvector extension
• ECS Fargate for the backend API
• S3 + CloudFront for the frontend
• Application Load Balancer with HTTPS

just infra-init env=prod
just infra-plan env=prod
just infra-apply env=prod

See infra/README.md for detailed deployment instructions.

API Endpoints

Public

GET  /api/health              # Health check
GET  /api/search              # Search advisors (supports filtering and pagination)
GET  /api/search/categories   # List available research categories
GET  /api/advisors/:id        # Get advisor details (tier-gated)

Authenticated

POST /api/auth/register       # Create account
POST /api/auth/login          # Get JWT token
GET  /api/user/me             # Current user profile
POST /api/subscriptions/checkout  # Create Stripe checkout session
POST /api/subscriptions/portal    # Stripe customer portal

Admin

POST /api/admin/scrape        # Start scraping job
GET  /api/admin/scrape/:id    # Get job status
GET  /api/admin/universities  # List universities for scraping
GET  /api/admin/alerts        # System alerts
GET  /api/admin/users         # User analytics

Testing

# All tests
just test

# Backend only (includes linting)
just test-backend

# Frontend unit tests
just test-frontend

# E2E tests (requires running services)
just test-e2e

# Integration tests with Stripe webhooks
just test-integration

Monitoring

The monitoring/ directory contains a standalone uptime monitoring service that:

• Polls /api/health every 60 seconds
• Alerts via email (Resend) after 3 consecutive failures
• Sends recovery notifications when service comes back online

Current State

Implemented

• User registration and JWT authentication
• Search with keyword filters and pagination
• Tier-based result blurring (free vs paid)
• Stripe checkout and webhook handling
• University discovery and lab info scraping
• Background job scheduling
• Email notifications
• Admin dashboard

Not Fully Implemented

• Semantic vector search (pgvector infrastructure ready, needs embedding generation)
• Full lab profile database (researcher genealogy, career outcomes)
• Complete E2E test coverage

Contributing

Contributions are welcome. Please:

1. Fork the repository
2. Create a feature branch
3. Write tests for new functionality
4. Ensure just test passes
5. Submit a pull request

License

MIT License. See LICENSE for details.

Acknowledgments

• Firecrawl for web scraping APIs
• OpenAlex for academic metadata
• pgvector for vector similarity search
• Stripe for payment processing