Skip to content

Creating a recipe for CUDA-enabled pytorch #18

Open
Open
Creating a recipe for CUDA-enabled pytorch#18

Description

@odgaard
odgaard
opened on May 21, 2025

Hi! Thank you for your incredible work with this layer.
After successfully building the python3-pytorch recipe today I discovered that it seems like a CPU-only install.
Have you made any attempts at getting a GPU-accelerated pytorch recipe working? Do you think it would be feasible to implement this based on your existing recipe?

Activity

  1. odgaard
    changed the title [-]Creating a recipe for GPU-enabled pytorch[/-] [+]Creating a recipe for CUDA-enabled pytorch[/+] on May 21, 2025

  2. zboszor commented on May 21, 2025

    @zboszor
    zboszor
    on May 21, 2025
    Owner

    I assume you are building Yocto for x86-64 as I do.

    Intel's MKL and oneDNN (MKLDNN) is already enabled in pytorch if meta-intel is added to the set of layers. This may make the GPU used implicitly, which is not the same as a fully accelerated build.

    I haven't found a complete Yocto layer which provides CUDA with selectable version to provide support for obsolete hardware and is up to date with styhead and walnascar support.

    Eventually I would want to go with chipStar (see #8 ) to fake CUDA which can work with any sufficiently mature OpenCL implementation. At least all its samples work with the Intel Compute Runtime's OpenCL driver. But chipStar is not there yet to build pytorch completely and you need Intel's icpx SYCL compiler from meta-intel to build chipStar's optional HIP libraries that may be needed to compile pytorch with chipStar's nvcc replacement. Unfortunately, icpx is not exposed as a native compiler executable by the Intel packages.

    The XPU method of GPU acceleration in pytorch also needs icpx, so it's not enabled either.

    So we're stuck with CPU-only at the moment. But at least the AVX2 and AVX512 stuff works.

    FWIW, OpenVINO can be built using PACKAGECONFIG += "opencl" and it does provide GPU acceleration with a proper OpenCL driver.

  3. zboszor commented on May 21, 2025

    @zboszor
    zboszor
    on May 21, 2025
    Owner

    I have just seen the new AdaptiveCpp 25.02.0 release with "portable CUDA" which may be what we need for a "CUDA enabled" pytorch build using SYCL/OpenCL behind the scenes.

  4. cmcquinn commented on Aug 18, 2025

    @cmcquinn
    cmcquinn
    on Aug 18, 2025

    I am interested in using this layer with meta-tegra. I might fork this layer, create some way to enable CUDA via a bbappend and then open a PR with my changes. Has anyone ever tried using this layer with meta-tegra before?

  5. zboszor commented on Aug 22, 2025

    @zboszor
    zboszor
    on Aug 22, 2025
    Owner

    I am interested in using this layer with meta-tegra. I might fork this layer, create some way to enable CUDA via a bbappend and then open a PR with my changes. Has anyone ever tried using this layer with meta-tegra before?

    I have not used meta-tegra before but please, go ahead and try it.

    See the python3-pytorch_%.bbappend for how it was done to support MKL when building for Intel.

    You can add USE_CUDA=1 etc. in a meta-tegra-specific python3-pytorch_%.bbappend here under dynamic-layers/tegra and don't forget to change BBFILES_DYNAMIC in layer.conf.

  6. asterix commented on Nov 21, 2025

    @asterix
    asterix
    on Nov 21, 2025

    @cmcquinn any luck with using this layer with meta-tegra and building pytorch with CUDA enabled? I'm trying to do the same. Currently, I can't even build pytorch due to broken python3-onnx dependencies in this layer (for arm64).

  7. cmcquinn commented on Nov 21, 2025

    @cmcquinn
    cmcquinn
    on Nov 21, 2025

    @cmcquinn any luck with using this layer with meta-tegra and building pytorch with CUDA enabled? I'm trying to do the same. Currently, I can't even build pytorch due to broken python3-onnx dependencies in this layer (for arm64).

    It turns out there the is already a recipe for pytorch in the meta-tegra-community repo which I ended up using.

  8. zboszor commented on Nov 21, 2025

    @zboszor
    zboszor
    on Nov 21, 2025
    Owner

    @cmcquinn any luck with using this layer with meta-tegra and building pytorch with CUDA enabled? I'm trying to do the same. Currently, I can't even build pytorch due to broken python3-onnx dependencies in this layer (for arm64).

    FYI, the python3-onnx 1.20.0rc1 recipe builds just fine now and a lot of recipes were also fixed, including pytorch.

    There are only two recipes that don't build:

    • mediamtx, because it builds a rpicamdownloader binary and tries to run it on the host (doesn't work when the CPU arch is different)
    • python3-kornia-rs, due to a rust 1.90 bug (but it builds fine for x86-64)

  9. asterix commented on Nov 21, 2025

    @asterix
    asterix
    on Nov 21, 2025

    @zboszor Thanks! I should've mentioned more details in my comment. I'll do so now. My issues are all on scarthgap because that's the stable version I need to stick with for various reasons. If the forward fixes are portable to scarthgap, that'd be amazing because scarthgap is the latest LTS along with kernel 6.6, also being LTS.

    A lot of these python packages' recipes are useful as ML tools, which are quite often hosted on aarch64 + Nvidia GPU platforms. Training and edge inferencing both.

    Here's the protobuf linking error when building 1.16.2 python3-onnx.

    Built target gen_onnx_data_proto
| [ 22%] Built target onnx_proto
| [ 97%] Built target onnx
| [ 98%] Linking CXX shared module onnx_cpp2py_export.cpython-312-aarch64-linux-gnu.so
| /yocto/nvidia/orin-nano/tegra-demo-distro/build/tmp/work/armv8a-oe4t-linux/python3-onnx/1.16.2/recipe-sysroot-native/usr/bin/aarch64-oe4t-linux/../../libexec/aarch64-oe4t-linux/gcc/aarch64-oe4t-linux/13.4.0/ld: /yocto/nvidia/orin-nano/tegra-demo-distro/build/tmp/work/armv8a-oe4t-linux/python3-onnx/1.16.2/recipe-sysroot-native/usr/lib/libprotobuf.so: error adding symbols: file in wrong format
| collect2: error: ld returned 1 exit status
| make[2]: *** [CMakeFiles/onnx_cpp2py_export.dir/build.make:169: onnx_cpp2py_export.cpython-312-aarch64-linux-gnu.so] Error 1
| make[1]: *** [CMakeFiles/Makefile2:223: CMakeFiles/onnx_cpp2py_export.dir/all] Error 2
| make: *** [Makefile:136: all] Error 2

    @cmcquinn Yep, meta-tegra-community recipe is what I'm looking at too. I'm not quite on whinlatter, but on scarthgap, so I have to copy out from that layer although I use that layer for other recipes. pytorch + CUDA is the "desired" combination for any Nvidia platform at it seems insane dependency management hell even with Nvidia's own Jetpack SDK 6.2.1 + Pytorch installation instructions. That's why I turned here, but meta-python-ai didn't look optioned for Nvidia runtime OOB.

  10. zboszor commented on Nov 22, 2025

    @zboszor
    zboszor
    on Nov 22, 2025
    Owner

    @zboszor Thanks! I should've mentioned more details in my comment. I'll do so now. My issues are all on scarthgap because that's the stable version I need to stick with for various reasons. If the forward fixes are portable to scarthgap, that'd be amazing because scarthgap is the latest LTS along with kernel 6.6, also being LTS.

    A lot of these python packages' recipes are useful as ML tools, which are quite often hosted on aarch64 + Nvidia GPU platforms. Training and edge inferencing both.

    Here's the protobuf linking error when building 1.16.2 python3-onnx.

    Built target gen_onnx_data_proto
| [ 22%] Built target onnx_proto
| [ 97%] Built target onnx
| [ 98%] Linking CXX shared module onnx_cpp2py_export.cpython-312-aarch64-linux-gnu.so
| /yocto/nvidia/orin-nano/tegra-demo-distro/build/tmp/work/armv8a-oe4t-linux/python3-onnx/1.16.2/recipe-sysroot-native/usr/bin/aarch64-oe4t-linux/../../libexec/aarch64-oe4t-linux/gcc/aarch64-oe4t-linux/13.4.0/ld: /yocto/nvidia/orin-nano/tegra-demo-distro/build/tmp/work/armv8a-oe4t-linux/python3-onnx/1.16.2/recipe-sysroot-native/usr/lib/libprotobuf.so: error adding symbols: file in wrong format
| collect2: error: ld returned 1 exit status
| make[2]: *** [CMakeFiles/onnx_cpp2py_export.dir/build.make:169: onnx_cpp2py_export.cpython-312-aarch64-linux-gnu.so] Error 1
| make[1]: *** [CMakeFiles/Makefile2:223: CMakeFiles/onnx_cpp2py_export.dir/all] Error 2
| make: *** [Makefile:136: all] Error 2

    I'll try to fix the scarthgap build, just as the master/whinlatter branches are fixed for aarch64 now.

  11. rmullick commented on Aug 8, 2026

    @rmullick
    rmullick
    on Aug 8, 2026

    Hi all, just wondering if there's any solution to this issue been found?

    I'm also trying to build pytorch on scarthgap branch with CUDA enabled. I got stuck with a Caffe2 build issue. I have -DBUILD_CAFFE2=ON and -DCAFFE2_USE_CUDA=ON. I got an error saying "cuda_profiler_api.h" not found. Now I'm trying with -DCAFFE2_USE_CUDA=OFF option. I figured it happened cause the underlying 'cuda-profilerpackage is onlytegraspecific version which is provided by themeta-tegra` layer. So, I'm skipping the DCAFFE2_USE_CUDA option.

    Does someone know how important to include CAFFE2? With a quick check I get to know, its not much important in recent version of pytorch.

    Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions