Skip to content

Commit da76e36

Browse files
committed
fix(group_offloading): use Ascend NPU stream when available
`apply_group_offloading` with `use_stream=True` only created a stream for CUDA or Intel XPU devices, and raised "Using streams for data transfer requires a CUDA device, or an Intel XPU device." on every other accelerator. On Ascend NPU, `torch.npu.Stream()` is available (torch_npu exposes it), so stream-based onload/offload should be supported there too. Adds an `npu` branch consistent with the existing CUDA/XPU checks, and updates the error message to mention the Ascend NPU device.
1 parent d77d530 commit da76e36

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

‎src/diffusers/hooks/group_offloading.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -670,8 +670,12 @@ def apply_group_offloading(
670670
stream = torch.cuda.Stream()
671671
elif hasattr(torch, "xpu") and torch.xpu.is_available():
672672
stream = torch.Stream()
673+
elif hasattr(torch, "npu") and torch.npu.is_available():
674+
stream = torch.npu.Stream()
673675
else:
674-
raise ValueError("Using streams for data transfer requires a CUDA device, or an Intel XPU device.")
676+
raise ValueError(
677+
"Using streams for data transfer requires a CUDA device, an Intel XPU device, or an Ascend NPU device."
678+
)
675679

676680
if not use_stream and record_stream:
677681
raise ValueError("`record_stream` cannot be True when `use_stream=False`.")

0 commit comments

Comments
 (0)