ppo-critic can train - #2023

Open

Hyperion-shuo wants to merge 15 commits into

InternLM:mainfrom

Hyperion-shuo:ss/ppo-critic

Commits on Jun 22, 2026

longer rollout worker health check timeout
shenshuo
committed
6bf90e9

Commits on Jul 13, 2026

rollout is veto level add geometric and kl3.
shenshuo
committed
a952b11

Commits on Jul 14, 2026

ppo critic initial impelementation
shenshuo
committed
6ea950c

Commits on Jul 16, 2026

feat(critic): configure attention freezing and value head init
shenshuo
committed
091e670
feat(ppo): add actor and critic training controls
shenshuo
committed
338233a
feat(ppo): apply length-adaptive GAE to actor
shenshuo
committed
fb7bdce
fix(critic): clarify reduced training metrics
shenshuo
committed
0fc1ff3
discard burn in rollout
shenshuo
committed
7d70713

Commits on Jul 21, 2026

feat(engine): add pre-clip gradient norm observer
shenshuo
committed
f214781
feat(ppo): optionally train actor on uniform-reward groups
shenshuo
committed
79c5f8b
feat(ppo): log global flattened actor explained variance
shenshuo
committed
5aaa73a

Commits on Jul 27, 2026

hf interval save critic weight
shenshuo
committed
156c729

Commits on Jul 29, 2026

fix critic hf save
shenshuo
committed
bdc6191
enable close entropy control
shenshuo
committed
a386fc0

Commits on Jul 30, 2026

fix(rollout): debounce worker health checks
shenshuo
committed
15b6499