This page covers everything you need to launch, observe, checkpoint, and recover a prime-rl training run — the RL trainer (and the distillation algorithms that run through it) and the SFT trainer. For multi-node and cluster layouts, see Scaling. For the loss math and algorithm knobs, see Algorithms. For standalone evals, see Eval.
AI agents working in this repo: the equivalent runbooks are at
skills/training/— top-level routing inskills/training/SKILL.md, launch details inskills/training/start-run/SKILL.md, and check-in / restart procedures inskills/training/monitor-run/SKILL.md.
| Command | Purpose | Notes |
|---|---|---|
uv run rl |
Wraps the trainer, orchestrator, and inference server in one launch from a merged TOML. | The default for any RL run. Runs locally for single-node experiments; submits to SLURM for single- or multi-node when [slurm] is set (see Scaling § SLURM). |
uv run sft |
Supervised fine-tuning on a HF dataset. | Launches torchrun internally; never call torchrun directly. |
uv run inference |
vLLM server. | Always use this entrypoint over vllm serve — it adds /update_weights, /load_lora_adapter, and /init_broadcaster. |
uv run orchestrator |
Standalone orchestrator process. | Pair with a separately-launched inference server and one env-server per source. |
uv run eval |
Multi-env evals against a live inference server. | One epoch per source, pinned (or adaptive) concurrency, cursor checkpoints + --resume, dashboard + optional platform upload; see Eval. |
uv run env-server |
Standalone env server for one environment. | The rl launcher starts these automatically (one per train/eval source; each binds an OS-assigned loopback port and publishes it to configs/attempt_N/resolved/envs/<split>/<name>.address for the orchestrator); only needed when running the orchestrator standalone, or for sources with an explicit serve.address — those are externally managed (e.g. on their own host) and the launcher expects the server to already run there. |
The minimal RL run trains an SFT-warmed Qwen3-0.6B on the reverse-text task — the env is bundled with the verifiers submodule, so nothing else needs to be installed:
uv run rl @ examples/basic/reverse-text/rl.tomlA condensed view of the knobs you'll most often tune. For trainer-side parallelism, sampling, optimizer, and loss knobs see Scaling and Algorithms.
Data and algorithm:
| Knob | What it does |
|---|---|
orchestrator.batch_size |
Tasks per trainer step. |
orchestrator.constant_trainer_batch_size |
Keep trainer batches constant when samples have no training signal, such as zero advantage on all tokens. Enabled by default. Disable it for faster collection with variable trainer batch sizes. |
orchestrator.train.group_size |
Rollouts generated per task. |
orchestrator.max_off_policy_steps |
Maximum staleness of a trained rollout (default 8): the version a batch trains on minus the oldest version that generated the rollout, queue time included. Episodes past the bound are dropped; a group shares one dispatch version, so its episodes age out together. The main off-policy dial on long agentic rollouts — bump for throughput, lower for tighter on-policyness. Watch off_policy/* and mismatch_kl/all/mean when tuning. |
[orchestrator.train.algo] |
Training algorithm — its type names it (grpo default, max_rl, rae, hierarchical_grpo, opd, opsd, sft, echo). See Algorithms. |
trainer.model.freeze_moe_router |
Freeze MoE router gates (default true for RL, false for SFT). Router updates in RL shift the trainer's expert choices away from the ones inference sampled, amplifying the train-inference mismatch, so RL keeps the pretrained router. Set false to train it. |
[[orchestrator.train.source]] |
Training sources. List multiple tables for multi-env training; weight them via ratio. See Configuration § Training sources. |
[[orchestrator.eval.source]] + orchestrator.eval.interval |
Eval environments and cadence (default every 100 steps). |
Monitoring:
| Knob | What it does |
|---|---|
log.level |
Process log level for trainer + orchestrator (info default; falls back to $PRIME_LOG_LEVEL). Set per-process via trainer.log.level / orchestrator.log.level, or globally on the rl entrypoint to propagate to both. |
orchestrator.log.vf_level |
Env-worker / verifiers log level (info default; debug is noisy but useful for env debugging). |
--monitors.wandb (+ --monitors.wandb.project, --monitors.wandb.name) |
Enable Weights & Biases logging. See Weights & Biases. |
--monitors.prime |
Stream metrics and episodes to the Prime Intellect platform (Prime Lab). See Platform monitoring. |
Run management:
| Knob | What it does |
|---|---|
--output-dir outputs |
Directory that groups related runs. Each run writes its artifacts to its own run directory <output_dir>/<run_name> (<run_dir> below). Defaults to $PRL_OUTPUT_DIR if set, else outputs. |
--run.name <name> |
Run name, also the run directory name under <output_dir> (override the directory separately via --run.dir). Auto-generated as <envs>--<model>--<short-id> when unset, so every launch gets a fresh, readable run directory. Set an explicit name for a predictable path — required to resume the run later. |
--clean |
Wipe the run directory before starting. Useful when re-running a named run during iteration. |
--max-steps N |
Stop after N trainer steps. Overrides the config value. |
--dry-run |
Resolve + validate the full config, write per-process configs to <run_dir>/configs/latest/resolved/, and exit without launching. The fastest way to debug a misbehaving config. |
The RL entrypoint supports several training algorithms, switched via [orchestrator.train.algo]'s type (see Algorithms for the full reference, model references, and per-algorithm customization):
algo.type |
Frozen model | Use case |
|---|---|---|
grpo (default) |
None | Standard group-relative RL |
max_rl |
None | MaxRL: GRPO with mean-normalized advantages (maximum-likelihood RL) |
rae |
None | SPIRAL's role-conditioned advantage estimation: reward minus a per-agent EMA baseline, for multi-agent self-play envs (e.g. kuhn-poker) |
hierarchical_grpo |
None | GRPO for proposer-solver envs: compare solvers only with attempts on the same proposed problem, and compare proposers with the other proposals in the group |
opd |
Required, must be vLLM (needs prompt_logprobs) |
On-policy distillation: the policy generates rollouts, the trainer minimizes per-token reverse KL to a reference model |
sft |
Required, any OpenAI-compatible endpoint | Hard-distill: a frozen model generates rollouts, the policy trains on its tokens |
opsd |
None — the live policy is its own reference (no deployment) | SDFT: the model is its own reference conditioned on expert demonstrations |
echo |
None | GRPO plus cross-entropy on env-observation tokens |
A new algorithm is a named class in code, not a config — see Algorithms § Authoring an Algorithm.
Frozen models are declared inline on the algorithm, named where the model is used — [orchestrator.train.algo.teacher] for opd (the frozen model scored against), [orchestrator.train.algo.sampling.source] for sft (the model it samples from) — each with name + base_url. opsd declares no frozen model: it self-distills against the live policy. The rl entrypoint only manages policy inference — start frozen-model servers yourself and point base_url at them:
CUDA_VISIBLE_DEVICES=1 uv run inference \
--vllm.model <frozen-model> --server.port 8001The standalone uv run sft entrypoint is the more traditional SFT path — pure dataset-based, no orchestrator. Use the sft algorithm only when you want a frozen model to generate the supervision on the fly.
Pulled from the console logs and mirrored to W&B.
Progress (orchestrator):
reward/{all,env}/mean— main signal. Should trend upward over hundreds of steps.seq_len/{all,env}/meanandis_truncated/{all,env}/mean— rollout length and truncation rate.is_timeout/{all,env}/mean— share of rollouts stopped by a stage deadline (setup/agent/finalize/scoring_timeout). Training counts a timed-out rollout as an error, so it stays out of the batch; eval scores it.num_turns/{all,env}/mean— for multi-turn envs.empty_rollouts/{all,env},errored_rollouts/{all,env}— non-zero is fine in small numbers; sustained > 5% is a smell.eval/{env}/{avg@k,pass@k}— eval scores when[orchestrator.eval]is set.
Stability (trainer):
mismatch_kl/{all,env}/{mean,std,max}— KL between trainer's current policy and the (older) inference policy that generated the rollouts. A sustained, growing mean is the early-warning sign for off-policy collapse.entropy/{all,env}/mean— too low means mode-collapse; too high means the model isn't committing.is_masked/mean— fraction of tokens masked by the IPO trust region.optim/grad_norm— spikes precede divergence; check the loss config or lower the LR.
Performance (trainer + orchestrator step independently):
| Source | Metric | Reading |
|---|---|---|
| trainer | time/wait_for_batch |
high → orchestrator bottleneck |
| orchestrator | time/wait_for_policy |
high → trainer bottleneck |
The trainer warns when batch wait time exceeds active trainer time. Add inference nodes when this warning persists. The orchestrator warns when policy wait time exceeds active orchestrator time. Add trainer nodes when this warning persists. The orchestrator also warns when it discards more than half of an episode window. This warning reports stale, errored, and no-signal counts.
uv run sft runs supervised fine-tuning from a HF dataset. It shares model loaders, FSDP setup, checkpointing, and the chat-template plumbing with the RL trainer, so a typical workflow is SFT → RL → SFT → … without any reformatting.
Two accepted layouts:
- Prompt-completion: a HF dataset with
promptandcompletioncolumns (TRL format). The trainer masks out the prompt and computes loss only over the completion. - Messages: a HF dataset with a single
messagescolumn containing a list of chat turns. The trainer interprets the whole conversation as one sample, applies role-based loss masking, and trains over all assistant turns.
If both columns are present, messages takes precedence. A dataset that stores these under other names maps them in [data.columns]; the trainer renames the columns at load time:
[data.columns]
messages = "conversation"
tools = "schemas"Per-message loss selection. An optional message_loss_mask column contains one boolean or integer 0/1 per message. For example, [0, 0, 0, 1] on a user/assistant/user/assistant conversation trains only the final assistant turn while retaining the full context. For prompt-completion rows, the mask covers the concatenated prompt and completion messages. Zero excludes a message; one preserves the normal role and renderer loss settings. A missing or null mask leaves those settings unchanged. The renderer applies selection without changing token IDs, including ownership of assistant closing tokens. Invalid mask lengths or entries raise an error.
To read a differently named column, set data.columns.message_loss_mask = "selection".
Tool definitions and renderer controls. For tool-use SFT, add a tools column in OpenAI function-calling format. Each row's value can be either a list of dicts or a JSON-encoded string of a list.
Renderer-backed SFT reads template controls from the typed [renderer] config in the SFT TOML. For example:
[renderer]
name = "qwen3"
enable_thinking = falseDataset columns can set renderer template controls per row, on top of the [renderer] config. [data.columns.renderer] maps renderer fields to columns and defaults to reasoning_effort = "reasoning_effort", so a reasoning_effort column sets that field on renderers that have it (for example gpt-oss, qwen3.8, deepseek-v4). Rows with a null value use the configured renderer unchanged. This lets one run mix reasoning efforts, with the [renderer] value as the default for rows that do not set one:
# The renderer field is reasoning_effort; this dataset stores it in an effort column.
[data.columns.renderer]
reasoning_effort = "effort"If a model needs another template control, add it to that model's renderer config in renderers (for example a new field on the relevant *RendererConfig) and consume it in the renderer implementation.
Custom renderers. A renderer can also live outside the renderers package. Set name = "custom" and point import_path at the class, as my_module.Class or path/to/file.py:Class; relative file paths resolve against the launch directory. The class sets config_class to its own BaseRendererConfig subclass, which validates the remaining [renderer] keys and declares its template controls:
[renderer]
name = "custom"
import_path = "my_renderers.glm53_no_thinking.NoThinkingGLM53Renderer"
clear_thinking = trueThe same config selects the renderer for RL under [orchestrator.renderer]. The renderers README shows the class behind this example.
Renderer-backed tokenization. SFT tokenization is renderer-only. The renderers package owns message-to-token conversion and loss attribution end-to-end, so position-dependent chat templates (for example templates that strip past <think> blocks across user turns) do not corrupt the loss mask. [renderer] defaults to name = "auto"; set a typed renderer config only when you need model-specific template controls. Hand-coded renderers ship for Qwen3, Qwen3.5, GLM-5, GLM-4.5, Kimi K2/K2.5, MiniMax M2, DeepSeek V3, Nemotron 3, GPT-OSS, and VLM families such as Qwen3-VL/Qwen3.5.
VLM training requires a custom PrimeRL implementation. Training a model with [model.vlm] set (SFT or RL) only works for models with a registered PrimeRL VLM class (currently Qwen3.5 dense and MoE).
See Algorithms § Multi-Turn Trajectories for the full picture.
The minimal SFT run trains Qwen3-0.6B on the reverse-text SFT dataset:
uv run sft @ examples/basic/reverse-text/sft.toml --monitors.wandbMulti-GPU and multi-node use torchrun under the hood (the sft entrypoint manages this for you — see Scaling § SFT and Torchrun for non-default layouts; multi-node SFT goes through SLURM).
--monitors.prime streams the run to the Prime Intellect platform — see Platform Monitoring.
uv run sft can evaluate the model on rollout-based envs as it trains, reusing the RL orchestrator's eval machinery. Configure an [eval] block — the same shape as [orchestrator.eval]: multiple [[eval.source]] envs with per-source interval / select / group_size / sampling overrides — plus an [inference] block for the vLLM server:
[eval]
interval = 25
[eval.select]
limit = 32
[[eval.source]]
name = "reverse-text"
env.taskset.id = "reverse-text"
env.agent.harness.id = "null"
env.agent.runtime.type = "subprocess"
[inference]
[deployment]
num_train_gpus = 1 # trainer
num_infer_gpus = 1 # inferenceThe launcher starts the inference server, one env server per eval source, and an online-eval process next to the trainer (it logs to logs/attempt_<n>/eval.log). NCCL is the default weight transport. The trainer broadcasts weights at startup (fail-fast) and at every step an eval env is due, Every broadcast runs the same four-stage handshake in broadcasts/step_{n}: the trainer offers the version (.sender_ready) and blocks, the online-eval process acknowledges (.receiver_ready), then the trainer transfers (.started) and commits (.finished). It runs the due envs sequentially per broadcast, so every epoch measures exactly one policy version. Set [weight_broadcast] type = "filesystem" to reload weights from disk instead. LoRA and externally managed inference use filesystem broadcast automatically. The base model is evaluated before the first step (disable with eval.skip_first_step), and the final broadcast always fires every env. In-flight eval episodes are cancelled by default when the next checkpoint is ready, so stale evals do not delay a weight update. Set eval.cancel_on_new_checkpoint = false to drain every triggered epoch instead. The trainer can idle while it waits for slow evals. They are sized by the same adaptive concurrency controller as the orchestrator; bound it with [eval.concurrency] (min_inflight / max_inflight; set them equal for fixed concurrency).
On a multi_node deployment, one SLURM job reserves deployment.num_train_nodes + deployment.num_infer_nodes nodes. The first num_infer_nodes run the inference pool, router, env servers, and online-eval process. The remaining nodes run the trainer. The inference pool runs one vLLM engine per DP rank behind one router, with gpus_per_node / inference.vllm.tensor_parallel_size engines per node:
[deployment]
type = "multi_node"
num_train_nodes = 2 # trainer nodes
num_infer_nodes = 1 # inference pool + online evals
[inference.vllm]
tensor_parallel_size = 8
[slurm]
job_name = "my-run"The shared script passes the trainer rank-0 hostname directly to the online-eval process for NCCL weight broadcasts. Each transfer is synchronous, but eval rollout execution overlaps with later training steps. The allocation remains active while the final eval finishes. Without max_steps, online-eval never sees a final broadcast, so the job remains active until walltime. Trainer and online-eval log to one shared W&B run. The trainer creates it, and online-eval finalizes it.
| Knob | What it controls |
|---|---|
data.name |
HF dataset name or local path |
data.batch_size |
Tokens per trainer step (packed) |
data.seq_len |
Per-sample sequence length |
loss_mask.* |
Which roles contribute to loss (system / user / assistant / tool). |
val.interval |
Run validation every N steps; val.data mirrors data |
eval.interval |
Run online evals every N steps; see Online Evals |
Pulled from the console log and mirrored to W&B.
Progress and loss:
loss/mean,loss/perplexity— main signal. Should decrease through the run.val/loss,val/perplexity— validation metrics when[val]is set, logged everyval.intervalsteps.eval/{env}/...— online eval metrics when[eval]is set, logged at each evaluated checkpoint step.progress/epoch,progress/num_samples,progress/num_tokens— dataset progress.progress/<subset>/ratio_{samples,tokens}— when training on multiple HF subsets/splits, the realized mixing ratio.
Stability and optimization:
optim/grad_norm— spikes precede divergence.optim/lr— LR schedule.- For MoE:
max_vio/mean,max_vio/max— mean and max over the step's microsteps of the largest load-balancing violation across layers and EP groups, computed from expert token counts summed across each EP group.routing_confidence/mean— mean routing confidence. - For MoE:
expert_load/{cv,max_mean,cold_frac}/{mean,max}— expert load per layer over the whole step, from token counts summed across all ranks, as the mean and max over MoE layers.cvis std/mean of the per-expert token counts,max_meanthe busiest expert's count over the mean, andcold_fracthe fraction of experts receiving under 0.1x the mean. A trainable router that drifts shows up as a risingcv/cold_frac(MiMo-V2.6 §5.4).
Performance:
| Metric | Reading |
|---|---|
perf/throughput, perf/throughput_per_gpu |
tokens/s overall and per GPU |
perf/mfu |
MFU |
perf/peak_memory |
peak GPU memory (GiB) |
time/step, time/forward_backward, time/save_ckpt |
step breakdown |
Checkpointing is split across processes because the orchestrator and trainer can be on different machines and on different steps at any given time. Inference is stateless.
| Process | What's saved | Where |
|---|---|---|
| Trainer | FSDP-sharded model (DCP), optimizer, scheduler, progress | <run_dir>/checkpoints/step_{n}/trainer/ |
| Orchestrator | Progress, per-env data state | <run_dir>/checkpoints/step_{n}/orchestrator/ |
| Inference | nothing — re-pushed from the latest checkpoint on restart | n/a |
Checkpointing is off by default to save disk. Enable it with --ckpt:
uv run rl @ rl.toml --ckpt # default: end-of-training only
uv run rl @ rl.toml --ckpt.interval 25 # every 25 steps
uv run rl @ rl.toml --ckpt.interval 25 --ckpt.keep-last 3 # rolling window of 3
uv run rl @ rl.toml --ckpt.interval 25 --ckpt.keep-interval 100 # …plus permanent every 100Re-run the same launch command and pass --resume (latest checkpoint) or --resume.step <N>. Resuming reuses the run directory, so the run needs a name you can point back at — launch with --run.name (or pass the first run's auto-generated name). Make sure --max-steps is at least the target final step, not the remaining delta:
# First run: steps 1–10
uv run rl @ rl.toml --max-steps 10 --ckpt --run.name my-run
# Resume from the latest checkpoint: continue to step 20
uv run rl @ rl.toml --max-steps 20 --ckpt --resume --run.name my-run
# ...or from a specific step
uv run rl @ rl.toml --max-steps 20 --ckpt --resume.step 10 --run.name my-run
# ...or fork another run's checkpoint into a fresh run
uv run rl @ rl.toml --max-steps 20 --ckpt --run.name my-fork \
--resume.dir outputs/my-run/checkpoints/step_10Trainer checkpoints are DCP-sharded; export them to HF-format safetensors with tools/convert_dcp_to_bf16.py. The script reads the model config from the run's resolved config and writes sharded safetensors plus config/tokenizer assets to <ckpt_dir>/weights (or a second positional arg). It exports full fine-tunes only — LoRA checkpoints are rejected.
# single process (1 GPU)
uv run python tools/convert_dcp_to_bf16.py outputs/my-run/checkpoints/step_10
# multi-rank for faster gathers and models that don't fit one GPU
uv run torchrun --nproc-per-node 8 tools/convert_dcp_to_bf16.py outputs/my-run/checkpoints/step_10The exported directory loads directly into uv run inference --vllm.model <dir> or any HF consumer. Quantize it to blockwise FP8 (DeepSeek/GLM format, loads natively in vLLM) with tools/convert_bf16_to_fp8.py <dir>, or go straight from the checkpoint with tools/convert_dcp_to_fp8.py <ckpt_dir> (each rank quantizes its gathered slice, writes only <ckpt_dir>/weights-FP8 — no intermediate bf16 export); dequantize an fp8-only release (e.g. GLM-5-FP8) for training with tools/convert_fp8_to_bf16.py <dir>.
Each launch writes its command, input TOML, and resolved JSON files to
<run_dir>/configs/attempt_<n>/. Resumed runs keep the earlier configs.
command.txt uses shell-safe quoting.
configs/latest points to the current attempt.
The launcher tees every process's stdout/stderr into <run_dir>/logs/attempt_<n>/ — every launch (fresh or resumed) gets its own numbered attempt directory, and logs/latest symlinks to the current one. The full layout (single-node runs skip the node_*.log and router.log files — there the router logs into inference.log):
<run_dir>/logs/latest/ # symlink -> attempt_<n>, one per launch
├── trainer.log # rank 0 only; symlink → trainer/node_0.log on multi-node
├── orchestrator.log # single instance, single file
├── eval.log # `uv run eval` process, or the SFT online-eval process
├── inference.log # symlink → inference/node_0.log on multi-node
├── trainer/
│ ├── node_*.log # per-node trainer stdout (multi-node only)
│ └── torchrun/<rdzv>/attempt_0/<rank>/{stdout,stderr}.log # per-rank
├── inference/
│ ├── node_*.log # per-node inference stdout (multi-node only)
│ └── router.log # the single global router (multi-node only)
└── envs/{train,eval}/<env_name>.log # one env server process per source (broker + its workers)
Env logs are the first place to look for env-side errors (most user code lives there). Verbosity is controlled by orchestrator.log.vf_level. For multi-rank trainer debugging, drop into logs/latest/trainer/torchrun/<rdzv>/attempt_0/<rank>/{stdout,stderr}.log — verbose and per-rank.
Live tailing from a single point (works on the head node for multi-node runs over a shared filesystem):
tail -F <run_dir>/logs/latest/{trainer,orchestrator,eval,inference}.log
tail -F <run_dir>/logs/latest/trainer/node_*.log # multi-node only
tail -F <run_dir>/logs/latest/inference/router.log # multi-node onlyuv run dashboard [output_dir ...] (default outputs/) serves a local web dashboard at http://localhost:7788 with six views per run: an overview (the W&B overview sections, read from the file monitor's metrics.jsonl), metrics (every key in that file, one pane each, with a regex filter), the resolved configs, a rollout trace viewer with per-token overlays (advantage, entropy, trainer/sampling mismatch, IPO stable mask, loss and content masks), merged component logs, and markdown reports from <run>/reports/. The trace viewer separates the transcript, the wall-clock timeline of physical prefix branches, a wall-clock terminal replay of model and tool activity, and a top-to-bottom semantic graph of labeled model-call relationships. Replay uses recorded model-call and message timestamps; because providers do not persist per-token timestamps, it reveals response text evenly across the measured call span and labels that cadence as inferred. Agent and context labels in the semantic graph show the latest and peak prompt lengths first, with cumulative token processing available below on hover and click. It only reads the run dirs, so it is safe to point at a live run; pass several output directories to track parallel experiments. A taken port automatically bumps to the next free one, so several dashboards coexist on one node.
A coding agent on the same machine can drive the open dashboard: POST /api/view with an on-disk address ({"run", "tab", "step", "kind", "subset", "episode", "highlight": [...]}) navigates every connected tab there and paints quote-anchored highlights in the trace viewer. Reports cite traces with [^id] markers whose JSON definitions carry the same address plus a verbatim quote; the dashboard re-checks each quote against the trace files and marks the citation verified or broken, so answers stay grounded in what is actually on disk. The dashboard skill documents the full contract.
W&B is off by default (the file monitor, which writes the metrics and the trace stream under the run's monitors/file/ directory, is on by default):
uv run rl @ rl.toml --monitors.wandb # default project, random name
uv run rl @ rl.toml --monitors.wandb.project my-proj --monitors.wandb.name run-42
uv run rl @ rl.toml --no-monitors.file # disable the local metric/trace filesThe trainer and orchestrator log into a single shared W&B run, so all metrics from both processes land in one place. Shared mode requires the W&B SDK ≥ 0.19.9 and is incompatible with monitors.wandb.offline = true.
prime-rl deliberately logs a large number of metrics for maximum observability: every rollout metric is emitted per subset (all/effective), per statistic (mean/max/min/p10/p90), and per environment alongside a cross-env aggregate, so a multi-env run can emit thousands of series. To keep that navigable, every training run (RL and SFT) gets an auto-created overview saved view curating the handful of metrics that matter into train, eval, stability, and performance sections (with per-env breakdowns). The view is created once per project and adapts to the run's environments; if a later run uses a different set of environments, a new versioned view (overview-v2, …) is created instead of overwriting the first.
Register a run on the Prime Intellect platform (Prime Lab) and stream training metrics and episodes to the platform dashboard. Bare flag uses defaults:
uv run rl @ rl.toml --monitors.prime
uv run sft @ sft.toml --monitors.prime # same flag on the SFT trainerOr set it in TOML:
[monitors.prime]
name = "my-experiment"The monitor is a thin layer over the prime-runs SDK (installed as prime-runs[train]): it registers the run, streams per-step metrics, uploads every 10th step's episodes (full conversations with rewards and advantages) to the run's sample viewer, and closes the run out. Finalization is best-effort SDK delivery: a process that exits without finishing is reported crashed when the SDK's atexit hook runs, but an abrupt kill (SIGKILL, OOM, node loss) skips the report and a lost finalize is not retried — the platform owns the terminal state of attached (RUN_ID) and managed runs. Uploaded episodes are keyed to the platform run by the SDK; the orchestrator's own run id (the launcher's PRL_RUN_ID) stays on W&B and in the local records.
On sft the monitor registers the run with the dataset-batched training fields (max_steps, batch_size, seq_len) and streams the trainer's per-step metrics and validation losses; SFT has no rollouts, so no episodes are uploaded.
Requires PRIME_API_KEY (prime login or the env var) and a team (PRIME_TEAM_ID, or the team selected with prime login) enabled for external runs. A configured monitor must work: a missing key or a team outside the allowlist fails the launch. PRIME_RUNS_MODE=disabled keeps the monitor configured but opens no platform run; RUN_ID=<id> attaches to an external run a launcher already created instead of registering a new one. Currently internal-only.
- Start small. Run
examples/basic/reverse-text/rl.tomlend-to-end on 2 GPUs before scaling. If the smoke run finishes cleanly, your install is good. - Batch size ≥ 64. Smaller batches give noisy gradient estimates and the trainer's overhead-per-step dominates throughput. 64 is the practical floor; 128–512 is the range for quick ablations; production RL often runs at 1024+.
- Group size ≥ 8. Bigger groups (
orchestrator.train.group_size) make it more likely that a task produces a mix of high- and low-reward rollouts, which is what gives the trainer a usable signal — if all rollouts in a group succeed or all fail, the within-group advantage collapses to zero and the trainer learns nothing from that task. Bigger groups also tighten advantage normalization. 8 is the floor; 16–32 is common. - Runs never share a directory. Every launch writes to its own run directory
<output_dir>/<run_name>, auto-named<envs>--<model>--<short-id>by default. Name runs you want to find again or resume with--run.name <name>; re-using a name blocks unless you resume or pass--clean. - Use
--dry-runbefore SLURM. Validators (e.g. CP needs flash-attention) fail fast in dry-run and slow in queue.