NVIDIA AI-Q Blueprint

🏆 BENCHMARK NOTE 🏆

To obtain results consistent with the nvidia-aiq DeepResearch Bench leaderboard and DeepResearch Bench II benchmark repository results, please use the drb1 and drb2 branches, respectively.

Table of Contents

• Overview
• What's New
• Software Components
• Target Audience
• Prerequisites
• Architecture
• Getting Started
  • Clone the Repository
  • Automated Setup
  • Obtain API Keys
  • Set Up Environment Variables
• Configuration Files
• Ways to Run the Agents
  • Command-line interface (CLI)
  • Web UI
  • Async Deep Research Jobs
  • MCP Server
  • Benchmarks
  • Jupyter Notebooks
• Evaluating the Workflow
  • Available Benchmarks
  • Running Evaluations
• Development
• Roadmap
• Security Considerations
• License

Overview

The NVIDIA AI-Q Blueprint is an enterprise-grade research agent built on the NVIDIA NeMo Agent Toolkit and uses LangChain Deep Agents. It gives you both quick, cited answers and in-depth, report-style research in one system, with benchmarks and evaluation harnesses so you can measure quality and improve over time.

Key features:

• Orchestration node — One node classifies intent (meta vs. research), produces meta responses (for example, greetings, capabilities), and sets research depth (shallow vs. deep).
• Shallow research — Bounded, faster researcher with tool-calling and source citation.
• Structured deep research — Advisory source routing, structured planning, concurrent researcher workers, bounded source-tool batching, and a dedicated writer produce citation-backed reports and other requested output shapes.
• Report follow-up — Ask questions about a completed report, create a child-job cosmetic rewrite, or run delta research with the parent report as context.
• Workflow configuration — YAML configs define agents, tools, LLMs, and routing behavior so you can tune workflows without code changes.
• Modular workflows — All agents (orchestration node, shallow researcher, deep researcher, clarifier) are composable; each can run standalone or as part of the full pipeline.
• Skills, sandbox execution, and durable outputs — Built-in research/synthesis skills run code through a provider-neutral sandbox contract. Modal is fresh per job; the experimental OpenShell profile uses one shared, pre-provisioned sandbox and is not a multi-tenant isolation boundary. Opt-in rich-file capture checkpoints manifest-declared files after successful sandbox commands, finalizes on success/failure, stores bytes in SQL or S3-compatible storage, and delivers metadata to the Files tab live and on replay.
• Portable Agent Skills — aiq-deploy selects, starts, and validates an AI-Q deployment; aiq-research calls routed chat and async research from compatible coding harnesses.
• Data source registry — UI toggles and request payloads can select web, paper, enterprise, collaboration, and knowledge-layer sources per message.
• Expanded sources — Paper search supports Serper, SerpAPI, and SearchAPI; You.com adds web, contents, general-research, and finance-research tools; Nimble adds configurable web search; focused profiles demonstrate DuckDuckGo news, Polymarket, OpenSearch, and Azure AI Search knowledge retrieval.
• Production API and auth — REST endpoints, async job ownership, per-user OAuth-protected MCP sources, token validator entry points, and provider lifecycle hooks support authenticated deployments; a separate public MCP server exposes stateless research tools for trusted networks.
• Opt-in policy controls — NeMo Guardrails middleware covers selected workflow and agent boundaries, and narrow application-level encryption can protect final async output plus selected artifact-event content.
• Observability, profiling, and cost analysis — NAT-exported async traces preserve task, named-agent, and model/tool hierarchy across concurrent researchers. Tokenomics reports combine profiler traces with pricing configuration for cost, latency, and cache analysis.
• Evaluation harnesses — Built-in benchmarks (for example, FreshQA, DeepResearch) and evaluation scripts to measure quality and iterate on prompts and agent architecture.
• Frontend options — Run through CLI, web UI, or async jobs. Refer to Getting started and Ways to run the agents.
• Deployment options - Deployment assets for Docker Compose and Helm; the repository source chart honors the Helm release namespace for every namespaced resource.

What's New

Recent changes include:

• Structured, concurrent deep research — Advisory source routing, structured planning, concurrent researcher workers, bounded source-tool batching, and a dedicated writer replace the earlier three-role flow and move plan ownership out of the clarifier.
• Work that continues from a completed report — Users can ask questions against an existing report, create child-job rewrites, or run delta research with the parent report as context.
• Portable skills, sandboxes, and durable files — Provider-neutral sandbox execution, the new aiq-deploy skill, expanded aiq-research workflows, opt-in artifact capture, SQL or S3-compatible storage, and live or replayed Files-tab access turn generated files into durable outputs.
• Sources, integrations, and policy controls — OpenSearch and Azure AI Search join the knowledge backends; You.com adds four search and research tools, Nimble adds configurable web search, and the standalone public MCP server exposes submit/poll/report operations; per-user MCP OAuth, opt-in NeMo Guardrails middleware, and narrowly scoped async-content encryption add deployment controls without making them universal defaults.
• Operations and user experience — Async traces preserve the agent hierarchy, the source Helm chart honors the selected release namespace, and the UI improves concurrent-research activity, session recovery, and WebSocket reliability.

See the changelog for detailed release history; the linked feature docs describe configuration and current limitations.

Software Components

The checked-in default CLI and web profiles use these core components:

• NVIDIA NeMo Agent Toolkit 1.8.0
• LangChain Deep Agents 0.6.5 or newer
• NVIDIA Nemotron 3 Super 120B A12B for intent, shallow research, source routing, and deep-research roles in the default profiles
• NVIDIA nemotron-mini-4b-instruct (document summary, if used)
• NVIDIA llama-nemotron-embed-vl-1b-v2 (embedding model for llamaindex knowledge layer implementation, if used)
• NVIDIA nemotron-nano-12b-v2-vl (vision-language model for llamaindex knowledge layer implementation, if used)
• Tavily Search API for web search
• Serper, SerpAPI, or SearchAPI for Google Scholar paper search

Focused profiles can instead use GPT-OSS-120B for deep-research orchestration, planning, and writing, or GPT-5.2 for those roles in the frontier-model example. Refer to Configuration Files; there is no single all-features profile.

Target Audience

This project is for:

• AI researchers and developers: People building or extending agentic research workflows
• Enterprise teams: Organizations needing tool-augmented research with citation-backed research
• NeMo Agent Toolkit users: Developers looking to understand advanced multi-agent patterns

Prerequisites

• Python 3.11–3.13
• uv package manager
• NVIDIA API key from NVIDIA AI (for NIM models)
• Node.js 22+ and npm (optional, for web UI mode)

Optional requirements:

• Tavily API key (for web search functionality)
• A paper search API key for one of the supported providers: Serper (SERPER_API_KEY), SerpAPI (SERPAPI_API_KEY), or SearchAPI (SEARCHAPI_API_KEY)

Note: Configure at least one data source (Tavily web search, Serper search tool, or knowledge layer) to enable research functionality.

If these optional API keys are not provided, the agent continues to operate without the corresponding search capabilities. Refer to Obtain API Keys for details.

Hardware Requirements

When using NVIDIA API Catalog (the default), inference runs on NVIDIA-hosted infrastructure and there are no local GPU requirements. The hardware references below apply only when self-hosting models via NVIDIA NIM.

Component	Default Model	Self-Hosted Hardware Reference
LLM (intent, shallow research, source routing, researcher)	nvidia/nemotron-3-super-120b-a12b	Nemotron 3 Super model card
LLM (deep-research orchestrator, planner, writer)	nvidia/nemotron-3-super-120b-a12b in default profiles; GPT-OSS-120B or GPT-5.2 in focused profiles	Configuration Files
Document summary (optional)	nvidia/nemotron-mini-4b-instruct	Nemotron Mini 4B
Text embedding	nvidia/llama-nemotron-embed-vl-1b-v2	NeMo Retriever embedding support matrix
VLM (image/chart extraction, optional)	nvidia/nemotron-nano-12b-v2-vl	Vision language model support matrix
Knowledge layer (Foundational RAG, optional)	--	RAG Blueprint support matrix

For detailed installation instructions, refer to Installation -- Hardware Requirements.

Architecture

AI-Q uses a LangGraph-based state machine with the following key components:

• Orchestration node: Classifies intent (meta vs. research), produces meta responses when needed, and sets depth (shallow vs. deep) in one step
• Shallow research agent: Bounded tool-augmented research optimized for speed
• Deep research agent: Multi-phase research with planning, iteration, and citation management

Each agent can be run individually or as part of the orchestrated workflow. For detailed architecture documentation, refer to Architecture.

Getting Started

Clone the Repository

git clone https://github.com/NVIDIA-AI-Blueprints/aiq.git && cd aiq

Automated Setup

Run the setup script to initialize the environment:

./scripts/setup.sh

This script:

• Creates a Python virtual environment with uv
• Installs all Python dependencies (core, frontends, benchmarks, data sources)
• Installs UI dependencies (if Node.js is available)

Manual Installation

For selective installation, install packages individually:

# Create and activate virtual environment
uv venv --python 3.13 .venv
source .venv/bin/activate

# Install core with development dependencies
uv pip install -e ".[dev]"

# Install frontends (pick what you need)
uv pip install -e ./frontends/cli          # CLI frontend
uv pip install -e ./frontends/debug        # Debug console
uv pip install -e ./frontends/aiq_api      # Unified API (includes debug)

# Install benchmarks (pick what you need)
uv pip install -e ./frontends/benchmarks/freshqa

# Install data sources (pick what you need)
uv pip install -e ./sources/tavily_web_search
uv pip install -e ./sources/google_scholar_paper_search
uv pip install -e ./sources/nimble_web_search
uv pip install -e ./sources/you_com
uv pip install -e "./sources/knowledge_layer[llamaindex,foundational_rag]"

Obtain API Keys

API	Environment Variable	Purpose	Required
NVIDIA API	NVIDIA_API_KEY	LLM inference through NIM	Yes
Tavily	TAVILY_API_KEY	Web search	No (if not specified, agent continues without web search)
Nimble	NIMBLE_API_KEY	Configurable web search	No (required only when Nimble search is configured)
You.com	YDC_API_KEY	Web, contents, and research APIs	No (required only when You.com tools are configured)
Serper	SERPER_API_KEY	Academic paper search	No (if not specified, agent continues without paper search)

Obtain an NVIDIA API Key

1. Sign in to NVIDIA Build
2. Click on any model, then select "Deploy" > "Get API Key" > "Generate Key"

Obtain a Tavily API Key

1. Sign in to Tavily
2. Navigate to your dashboard
3. Generate an API key

Obtain a You.com API Key

Follow the You.com quickstart to create an API key and add it to deploy/.env as YDC_API_KEY. Refer to You.com API Suite for tool configuration.

Obtain a Paper Search API Key

Paper search supports three interchangeable providers. Set the provider field on the paper_search function in your workflow config (defaults to serper):

Provider	Environment Variable	Sign-up
Serper (default)	SERPER_API_KEY	serper.dev
SerpAPI	SERPAPI_API_KEY	serpapi.com
SearchAPI	SEARCHAPI_API_KEY	searchapi.io

Refer to sources/google_scholar_paper_search/README.md for configuration details.

Set Up Environment Variables

Create a .env file in deploy/ directory:

cp deploy/.env.example deploy/.env

Replace your API keys.

Note: Depending on your usecase, deep research report quality can be enhanced by enabling searching across academic research papers. We use Serper for this. If you want to use paper search, follow the steps in the Customization guide to enable it.

Configuration Files

The configs/ directory holds YAML workflow configs that define agents, tools, LLMs, and routing. Use the one that matches your run mode and data sources:

Config	Models	Description
config_cli_default.yml	Nemotron 3 Super	CLI chat pipeline with Tavily and clarification; no knowledge backend. Paper search is a commented opt-in.
config_web_default_llamaindex.yml	Nemotron 3 Super; Nemotron Mini summary	Default web/API chat pipeline with LlamaIndex/ChromaDB and Tavily. Paper search is commented out.
config_web_frag.yml	Nemotron 3 Super	Web/API and Helm base with Foundational RAG plus Tavily. Requires separately deployed RAG query and ingestion services.
config_web_opensearch.yml	Nemotron 3 Super; NVIDIA embedding model	Web/API with built-in OpenSearch knowledge retrieval plus Tavily; supports self-hosted, es, and aoss authentication modes.
config_web_azure_ai_search.yml	Nemotron 3 Super; NVIDIA embedding model	Web/API with Azure AI Search knowledge retrieval plus Tavily; supports API-key and Azure identity authentication.
config_frontier_models.yml	GPT-5.2; Nemotron 3 Super; Nemotron Mini summary	LlamaIndex profile using GPT-5.2 for orchestration/planning/writing and Nemotron Super for routing/research. Requires OPENAI_API_KEY.
config_web_default_guardrails.yml	GPT-OSS-120B; Nemotron 3 Super; Nemotron Mini summary	LlamaIndex profile with workflow Guardrails attached and async deep-agent Guardrails selected; shallow middleware is defined but not attached.
config_web_frag_mcp_auth.yml	Nemotron 3 Super	Foundational RAG plus an opt-in protected per-user OAuth MCP source example. Requires a real MCP endpoint and shared token store.
config_domain_routing_and_skills.yml	Nemotron 3 Super; Nemotron Mini summary	Direct deep-research profile with domain routing, DuckDuckGo news, Polymarket, enabled Serper paper search, LlamaIndex, built-in skills, and a fresh per-job Modal sandbox.
config_openshell.yml	GPT-OSS-120B; Nemotron 3 Super; Nemotron Mini summary	Experimental web/API skills profile with artifact capture over one shared, pre-provisioned OpenShell sandbox; trusted single-operator use only.
config_mcp.yml	Nemotron 3 Super 120B	Standalone MCP server. Public NIM + Tavily research with PostgreSQL-backed stateless submit/poll/report. Requires NVIDIA_API_KEY, TAVILY_API_KEY, and AIQ_CHECKPOINT_DB.

Ways to Run the Agents

The frontends/ directory contains different interfaces for interacting with the agents. You can also run agents directly through the NeMo Agent Toolkit CLI.

Command-line interface (CLI)

The CLI provides an interactive research assistant in your terminal:

# Activate the virtual environment
source .venv/bin/activate

# Run with the convenience script
./scripts/start_cli.sh

# Verbose logging
./scripts/start_cli.sh --verbose

# Or run directly with the NeMo Agent Toolkit CLI (dotenv loads deploy/.env into the environment)
dotenv -f deploy/.env run nat run --config_file configs/config_cli_default.yml --input "How do I install CUDA?"

The CLI frontend source is in frontends/cli/.

Web UI

For a full web-based experience:

./scripts/start_e2e.sh

This starts:

• Backend API server at http://localhost:8000
• Frontend UI at http://localhost:3000

The web UI source is in frontends/ui/. Refer to frontends/ui/README.md for more details.

Web UI with Docker Compose

You can also run the backend and UI with Docker Compose:

cd deploy/compose

# No-auth local setup (LlamaIndex default)
docker compose --env-file ../.env -f docker-compose.yaml up -d --build

# To select a different backend config, set BACKEND_CONFIG in deploy/.env, for example:
# BACKEND_CONFIG=/app/configs/config_web_frag.yml

For more details, refer to:

• deploy/compose/README.md

Async Deep Research Jobs

For public endpoints, SSE replay, report follow-up, and durable artifact access, refer to the REST API documentation.

MCP Server

Expose AI-Q to MCP clients through the standalone, stateless Streamable HTTP server:

: "${NVIDIA_API_KEY:?Set NVIDIA_API_KEY}"
: "${TAVILY_API_KEY:?Set TAVILY_API_KEY}"
uv sync --project mcp --frozen
AIQ_CHECKPOINT_DB=postgresql://localhost/aiq_jobs \
  uv run --project mcp --frozen aiq-mcp-server

The endpoint defaults to http://localhost:9001/mcp and advertises exactly submit_query, poll_query, and get_final_report. This public server intentionally has no authentication; job UUIDs are bearer capabilities and the endpoint must not be exposed directly to an untrusted network. See Expose AI-Q as an MCP Server for the exact JSON protocol, health contracts, security model, and container deployment.

MCP is an independent uv project with its own mcp/uv.lock. The root lock remains compatible with NAT's cryptography<47 constraint, while the frozen MCP release/container profile pins cryptography==48.0.1 as a security hardening measure. The supported distribution paths are this source checkout and the release container; the MCP package's local path dependency closure is not published as a standalone wheel.

Benchmarks

To run agents in evaluation mode, refer to the Evaluating the Workflow section.

Jupyter Notebooks

The docs/notebooks/ directory contains a three-part series that walks through the blueprint from first run to full customization. Run them in order:

#	Notebook	What it covers	Prerequisites
0	Getting Started with AI-Q	Full blueprint overview — environment setup, orchestrated workflow (intent routing, shallow and deep research), and Docker Compose deployment	NVIDIA_API_KEY; optionally TAVILY_API_KEY, SERPER_API_KEY
1	Deep Researcher — Web Search	Deep researcher in depth — Python API, nat run, and end-to-end evaluation against the DeepResearch Bench with nat eval	Notebook 0 completed; NVIDIA_API_KEY, TAVILY_API_KEY, SERPER_API_KEY; OpenAI or Gemini key for the judge model
2	Deep Researcher — Customization	Extending the deep researcher — adding paper search, assigning different LLMs per agent role, editing prompts, and enabling the knowledge layer	Notebooks 0 and 1 completed; NVIDIA_API_KEY, TAVILY_API_KEY, SERPER_API_KEY

Evaluating the Workflow

The frontends/benchmarks/ directory contains evaluation pipelines for assessing agent performance.

Available Benchmarks

Benchmark	Description	Location
Deep Research Bench	RACE and FACT evaluation for research quality	frontends/benchmarks/deepresearch_bench/
FreshQA	Factuality evaluation on time-sensitive questions	frontends/benchmarks/freshqa/

Running Evaluations

Step 1: Install the dataset

The dataset files are not included in the repository. We have included a script to retrieve them from the Deep Research Bench Github Repository and format them for the NeMo Agent Toolkit evaluator.

To download the dataset files, run the following script:

python frontends/benchmarks/deepresearch_bench/scripts/download_drb_dataset.py

Step 2: Generate reports using NAT evaluation harness

dotenv -f deploy/.env run nat eval --config_file frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.yml

Step 3: Convert the output into a compatible format

python frontends/benchmarks/deepresearch_bench/scripts/export_drb_jsonl.py --input <path to your workflow_output.json> --output <path to the output file you want to create with .jsonl extension>

Step 4: Run evaluation

Follow instructions in the Deep Research Bench Github Repository to run evaluation and obtain scores.

Optional: Phoenix Tracing

If your config enables Phoenix tracing, start the Phoenix server before running nat eval.

Start server (separate terminal):

source .venv/bin/activate
phoenix serve

For detailed benchmark documentation, refer to:

• Deep Research Bench README
• FreshQA README

Development

For development, contribution, and documentation, refer to:

• Development and Contributing: Setup, testing, PR workflow, sign-off/DCO
• Tutorial Notebooks: Getting started overview, deeper dive, and customization notebooks
• Architecture: Component details and data flow
• Customization: Configuration and customization options
• Knowledge Layer Setup: RAG backends and document ingestion
• Agent Skills: Install the portable AI-Q research skill in compatible coding harnesses
• Skills and Sandbox Example: Run deep research with built-in skills and Modal sandbox execution
• Profiling and Cost Analysis: Generate tokenomics and latency reports from NAT profiler traces
• Docs index: Full documentation list and component docs
• Changelog: Version history and changes

Roadmap

The checkboxes below track implementation in the current branch; a checked item does not by itself indicate availability in a published release.

• NeMo Guardrails Integration: Opt-in middleware for selected workflow, shallow-researcher, and deep-researcher boundaries.
• NVIDIA Dynamo Integration: Reduce latency via priority scheduling at scale.
• Per-user MCP OAuth: Connect each signed-in user to protected MCP data sources through the UI.
• Skills & Sandboxing: Support built-in deep-research skills through a provider-neutral contract. Modal is fresh per job; the experimental OpenShell profile is shared and not a multi-tenant isolation boundary.
• Report Follow-up and Rewriting: Answer against a completed report, submit cosmetic child rewrites, and run delta research with parent-report context.
• Durable Sandbox Artifacts: Opt-in manifest checkpoints after successful sandbox commands, terminal harvesting on success/failure, SQL or S3-compatible byte storage, and live/replayed Files-tab access.
• Custom Skill Management: Add UI and lifecycle controls for user-provided skill bundles.
• Dynamic Model Routing: Allow sub-agents to automatically select the optimal model per task.
• Resource Management: Implement configurable token caps and tool-call budgets.
• Expanded Web Search: Additional integration examples including Perplexity and You.com.
• Multimedia Output: Embed audio, video, and images directly into reports.
• Voice-to-Text Input: Integrate NVIDIA Riva for hands-free accessibility.

Security Considerations

• The AI-Q Blueprint is shared as a reference and is provided "as is". The security in the production environment is the responsibility of the end users deploying it. When deploying in a production environment, please have security experts review any potential risks and threats; define the trust boundaries, implement logging and monitoring capabilities, secure the communication channels, integrate AuthN & AuthZ with appropriate access controls, keep the deployment up to date, ensure the containers/source code are secure and free of known vulnerabilities.
• A robust frontend that handles AuthN & AuthZ is highly recommended. Missing AuthN & AuthZ will result in ungated access to customer models if directly exposed e.g. the internet, resulting in either cost to the customer, resource exhaustion, or denial of service.
• AI-Q includes opt-in NeMo Guardrails middleware for selected workflow and agent boundaries. Guardrails are not enabled universally; operators must attach and test the policies required for their deployment.
• Optional async job content encryption protects final job output and selected artifact-event content only. It is off by default and is not full database-level job-content encryption.
• The AI-Q Blueprint doesn't require any privileged access to the system.
• Deep research skills can invoke sandboxed code execution for analysis workflows. Keep sandbox credentials, quotas, lifecycle cleanup, and network policy aligned with your deployment's trust boundaries; do not use the shared OpenShell example as isolation between mutually untrusted jobs.
• End users are responsible for ensuring the availability of their deployment.
• End users are responsible for building, and patching, the container images to keep them up to date.
• The end users are responsible for ensuring that OSS packages used by the developer blueprint are current.
• The logs from middleware, backend, and demo app are printed to standard out. They can include input prompts and output completions for development purposes. The end users are advised to handle logging securely and avoid information leakage for production use cases.

License

This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use, found in LICENSE-THIRD-PARTY.

GOVERNING TERMS: AIQ blueprint software and materials are governed by the Apache License, Version 2.0