GAD: Geometry Attention Distillation for Consistent Video Generation

Junfeng Ni^1,2,3,*, Ziyu Zhu^2,*, Fan Jiang^2,*,†,✉, Yu Liu¹, Chiyu Wang²,
Jiawei Zhang², Zhaoxu Sun², Mu Xu², Siyuan Huang^3,✉, Yixin Chen^3,✉
* Equal contribution   ^† Project leader   ^✉ Corresponding author
¹Tsinghua University   ²AMAP, Alibaba Group
³State Key Laboratory of General Artificial Intelligence, BIGAI

Overview

Video diffusion models can generate visually compelling short clips, but often struggle to preserve scene geometry and object structure over time. We introduce Geometry Attention Distillation (GAD), a training framework that transfers cross-frame correspondences from a frozen geometry model into video-model attention.

GAD extracts correspondence targets from VGGT-Ω attention on clean training videos. It applies confidence-aware pairwise and trackwise supervision to a student attention layer selected according to the diffusion timestep. This timestep-routed supervision helps the video model form reliable cross-frame associations for both static surfaces and moving objects.

The geometry teacher and distillation objectives are used only during training. Inference retains the original video-model architecture and sampling procedure, with no additional inference modules or computational overhead.

Experiments on camera-controlled video generation show improved long-range scene consistency and dynamic object consistency across multiple video backbones. On Wan2.2 5B, GAD improves cross-frame matching accuracy by at least 30% over the Wan baseline on both long-range benchmarks.

Release Plan

• Release the project page
• Release the paper
• Release VGGT-Ω attention-label extraction code
• Release GAD training and inference code
• Release model checkpoints
• Release example data and usage documentation

Acknowledgements

This project builds on DiffSynth-Studio, Wan, VideoX-Fun, and VGGT-Ω. We thank the authors for making their work publicly available.

Citation

If you find GAD useful for your research, please cite:

@article{ni2026gad,
  title={GAD: Geometry Attention Distillation for Consistent Video Generation},
  author={Ni, Junfeng and Zhu, Ziyu and Jiang, Fan and Liu, Yu and Wang, Chiyu and Zhang, Jiawei and Sun, Zhaoxu and Xu, Mu and Huang, Siyuan and Chen, Yixin},
  journal={arXiv preprint arXiv:2610.18779},
  year={2026}
}