[Bug] CUDA sparse-conv backend fails to compile with CUDA 12.8 / torch 2.8 / GCC 11+
Summary
MinkowskiEngine master (0.5.4, commit 02fc608) does not compile its CUDA
backend on a modern toolchain: CUDA 12.8 (thrust/CCCL), torch 2.8, GCC 11 or 13.
A set of small compatibility fixes is available (see below) that lets it build and
run the sparse-conv backend on an RTX 5070 Ti (sm_120).
Environment
- OS: Ubuntu 22.04
- Host compiler: GCC 11.4 (also reproduced on GCC 13)
- CUDA: 12.8 (
nvcc 12.8.93, supports compute_120)
- PyTorch: 2.8.0+cu128
- Build:
SETUPTOOLS_USE_DISTUTILS=stdlib, TORCH_CUDA_ARCH_LIST="12.0",
CC=gcc-13 CXX=g++-13
Steps to reproduce
git clone https://github.com/NVIDIA/MinkowskiEngine.git
cd MinkowskiEngine
pip install torch==2.8.0+cu128 numpy<2 setuptools wheel
export CUDA_HOME=/usr/local/cuda-12.8 TORCH_CUDA_ARCH_LIST="12.0" SETUPTOOLS_USE_DISTUTILS=stdlib
export CC=gcc-13 CXX=g++-13
python setup.py build_ext --inplace
Errors encountered (in order after each previous fix)
-
namespace "thrust" has no member "sort_by_key" (also unique, sequence,
reduce, etc.) in src/spmm.cu, src/coordinate_map_gpu.cu, pooling kernels.
New thrust in CUDA 12.8 requires explicit includes; MinkowskiEngine relied
on the old transitive includes.
-
identifier "CUDF_FUNC_RANGE" is undefined in
src/3rdparty/concurrent_unordered_map.cuh. The bundled cudf nvtx macro was
disabled when its headers are removed.
-
std::__to_address overload ambiguity in <bits/shared_ptr_base.h>
(both GCC 11 and GCC 13), triggered by
concurrent_unordered_map::create() returning a std::unique_ptr that is
converted to std::shared_ptr. This is the hard blocker.
Root causes
- MinkowskiEngine (2021) predates the CUDA 12.x thrust/CCCL change that made
algorithm headers non-transitive.
- The bundled
nvtx3.hpp/ranges.hpp collide with torch's nvtx3.
default_allocator / c10_allocator were only minimally declared (no
pointer/rebind/operator== etc.), which modern libstdc++ shared_ptr
fails on.
concurrent_unordered_map::create() returns unique_ptr; the
unique_ptr -> shared_ptr conversion hits a libstdc++ std::__to_address
overload-ambiguity bug on both GCC 11 and 13.
Proposed fix (verified, builds on RTX 5070 Ti)
- Add explicit thrust includes (
<thrust/sort.h>, <thrust/unique.h>,
<thrust/sequence.h>, <thrust/reduce.h>, <thrust/remove.h>,
<thrust/count.h>, <thrust/copy.h>, <thrust/fill.h>,
<thrust/device_vector.h>, <thrust/iterator/zip_iterator.h>,
<thrust/tuple.h>, <thrust/execution_policy.h>).
- No-op the bundled nvtx3 headers and add
#define CUDF_FUNC_RANGE() ((void)0).
- Make both allocators fully conformant (
pointer, const_pointer, size_type,
difference_type, rebind, operator==/!=).
- Change
concurrent_unordered_map::create() to return std::shared_ptr<Self>
(pointer + deleter) instead of std::unique_ptr.
After the fix, a real sparse convolution runs on GPU:
>>> MinkowskiEngine 0.5.4 GPU sparse conv LIVE on RTX 5070 Ti (CUDA 12.8, sm_120)
>>> is_cuda_available: True, cuda_version: 12080
A ready-made patch is in the fix/build-cuda12.8-gcc13 branch (commit 6fd7c64)
of my fork. I verified it compiles and runs on CUDA 12.8 / torch 2.8 / sm_120.
Notes
- This affects downstream users: FUSER (
Jiang-HB/FUSER) pins
MinkowskiEngine==0.5.4 and, on a Blackwell GPU, needs torch>=2.7 — which
then requires this compatibility patch to compile. See the related
FUSER requirements.txt issue #1.
[Bug] CUDA sparse-conv backend fails to compile with CUDA 12.8 / torch 2.8 / GCC 11+
Summary
MinkowskiEnginemaster (0.5.4, commit02fc608) does not compile its CUDAbackend on a modern toolchain: CUDA 12.8 (thrust/CCCL), torch 2.8, GCC 11 or 13.
A set of small compatibility fixes is available (see below) that lets it build and
run the sparse-conv backend on an RTX 5070 Ti (sm_120).
Environment
nvcc 12.8.93, supportscompute_120)SETUPTOOLS_USE_DISTUTILS=stdlib,TORCH_CUDA_ARCH_LIST="12.0",CC=gcc-13 CXX=g++-13Steps to reproduce
Errors encountered (in order after each previous fix)
namespace "thrust" has no member "sort_by_key"(alsounique,sequence,reduce, etc.) insrc/spmm.cu,src/coordinate_map_gpu.cu, pooling kernels.New thrust in CUDA 12.8 requires explicit includes; MinkowskiEngine relied
on the old transitive includes.
identifier "CUDF_FUNC_RANGE" is undefinedinsrc/3rdparty/concurrent_unordered_map.cuh. The bundled cudf nvtx macro wasdisabled when its headers are removed.
std::__to_addressoverload ambiguity in<bits/shared_ptr_base.h>(both GCC 11 and GCC 13), triggered by
concurrent_unordered_map::create()returning astd::unique_ptrthat isconverted to
std::shared_ptr. This is the hard blocker.Root causes
algorithm headers non-transitive.
nvtx3.hpp/ranges.hppcollide with torch'snvtx3.default_allocator/c10_allocatorwere only minimally declared (nopointer/rebind/operator==etc.), which modern libstdc++shared_ptrfails on.
concurrent_unordered_map::create()returnsunique_ptr; theunique_ptr -> shared_ptrconversion hits a libstdc++std::__to_addressoverload-ambiguity bug on both GCC 11 and 13.
Proposed fix (verified, builds on RTX 5070 Ti)
<thrust/sort.h>,<thrust/unique.h>,<thrust/sequence.h>,<thrust/reduce.h>,<thrust/remove.h>,<thrust/count.h>,<thrust/copy.h>,<thrust/fill.h>,<thrust/device_vector.h>,<thrust/iterator/zip_iterator.h>,<thrust/tuple.h>,<thrust/execution_policy.h>).#define CUDF_FUNC_RANGE() ((void)0).pointer,const_pointer,size_type,difference_type,rebind,operator==/!=).concurrent_unordered_map::create()to returnstd::shared_ptr<Self>(pointer + deleter) instead of
std::unique_ptr.After the fix, a real sparse convolution runs on GPU:
A ready-made patch is in the
fix/build-cuda12.8-gcc13branch (commit6fd7c64)of my fork. I verified it compiles and runs on CUDA 12.8 / torch 2.8 / sm_120.
Notes
Jiang-HB/FUSER) pinsMinkowskiEngine==0.5.4and, on a Blackwell GPU, needstorch>=2.7— whichthen requires this compatibility patch to compile. See the related
FUSER
requirements.txtissue #1.