Skip to content

Questions on SFT Token Size for Unseen Robot Configurations and Request for FK/IK & Third‑Party Robot Training Scripts #312

Open

Description

@Yvonne-OH

We have reviewed the open-source scripts and found only two training modes available: SFT on captioned video (ti2v) and policy training. Following the relevant paper, we reproduced FK & IK training modes as well as multiview training modules on our end. These modules yield positive results on the existing UMI single-arm Droid robot configuration. However, the performance degrades significantly when transferred to other dual-arm robot configurations.

We have several questions:

How many tokens are required for SFT when adapting to robot configurations that are not included in your dataset?
Could you release data processing and inference scripts for other robot configurations (e.g., Agibot, Robodojo)? This would help us conduct cross-validation.
Could you open-source the training scripts for FK and IK modes, instead of only providing scripts for policy and ti2v modes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions