Repository navigation
Commit da76e36
committed
fix(group_offloading): use Ascend NPU stream when available
`apply_group_offloading` with `use_stream=True` only created a stream
for CUDA or Intel XPU devices, and raised
"Using streams for data transfer requires a CUDA device, or an Intel
XPU device." on every other accelerator. On Ascend NPU, `torch.npu.Stream()`
is available (torch_npu exposes it), so stream-based onload/offload should
be supported there too.
Adds an `npu` branch consistent with the existing CUDA/XPU checks, and
updates the error message to mention the Ascend NPU device.1 parent d77d530 commit da76e36
1 file changed
Lines changed: 5 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
670 | 670 | | |
671 | 671 | | |
672 | 672 | | |
| 673 | + | |
| 674 | + | |
673 | 675 | | |
674 | | - | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
675 | 679 | | |
676 | 680 | | |
677 | 681 | | |
| |||
0 commit comments