[RobotSail]

Summary

Adds 1 new learning to the "Working With This Codebase Specifically" section of AGENTS.md:

• AdamW beta2=0.95 default: Both SFT (instructlab-training) and OSFT (mini_trainer) backends default to beta2=0.95, not PyTorch's standard 0.999. This non-standard default affects training dynamics and can confuse users porting hyperparameters from other frameworks. All four AdamW parameters (beta1, beta2, eps, weight_decay) are now exposed as first-class kwargs on sft() and osft().

Sources

• training_hub PR #108 — docs: document AdamW optimizer hyperparameters for SFT and OSFT (merged 2026-06-30)

Relationship to open PRs

Independent — no overlap with open PRs #96, #97, #99, #100, #101, #102, or #105. Those cover NVIDIA driver compat, Unsloth import order, datasets/pyarrow, trl 1.x packing, and other topics. This PR covers the AdamW beta2 default, which none of the open PRs address.

Analysis

22 merged PRs analyzed across 3 repos since last merged AGENTS.md update (PR #84, June 3). One new non-obvious learning identified:

• AdamW beta2=0.95 default — genuinely surprising since PyTorch defaults to 0.999

Other recent merges (PR #104 trl packing, PR #110 datasets pin, PR #83 ITS rollout, PR #106 validation loss docs) are either already covered by open PR #105 or are code/docs-derivable facts that don't meet the bar for AGENTS.md.

Test plan

• No code changes — AGENTS.md only
• Verified no duplicates with existing entries
• Verified no overlap with open PRs Update AGENTS.md with learnings from recent work (June 2026) #96-Update AGENTS.md with learnings from recent work (June 26, 2026) #105
• New entry follows existing format conventions

🤖 Generated with Claude Code

Summary by CodeRabbit

• Documentation
  • Updated the codebase guidance to note the default optimizer settings used by the SFT and OSFT backends.
  • Clarified that key optimizer parameters can now be configured directly through the main training entry points.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 82fd1d05-295b-4c73-9d3f-46c4d50ea81a

📥 Commits

Reviewing files that changed from the base of the PR and between b2e28dd and 5cfab88.

📒 Files selected for processing (1)

• AGENTS.md

---

📝 Walkthrough

Walkthrough

Adds one bullet to AGENTS.md noting that the SFT and OSFT backends default AdamW's beta2 to 0.95 instead of PyTorch's 0.999, and that beta1, beta2, eps, and weight_decay are exposed as kwargs on sft() and osft().

Changes

AGENTS.md AdamW documentation

Layer / File(s)	Summary
AdamW beta2 default note AGENTS.md	Adds a bullet documenting the non-standard beta2=0.95 default and the optimizer hyperparameter kwargs available on sft()/osft().

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

• Red-Hat-AI-Innovation-Team/training_hub#84: Also updates AGENTS.md with AdamW/optimizer guidance for the SFT/OSFT backends.
• Red-Hat-AI-Innovation-Team/training_hub#108: Documents the same SFT/OSFT AdamW hyperparameters (beta1, beta2=0.95, eps, weight_decay) as first-class options in API documentation.

Suggested reviewers

• training-hub-agent

Poem

A rabbit checks the docs with glee,
beta2 is 0.95, not 0.999!
A tiny note, a helpful sign,
so trainers know the default line.
🐇✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main documentation update about AdamW's non-standard beta2 default in AGENTS.md.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch learnings/june-30-2026

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}