CUDA 4.0 compatibility

[jw3126]

cudnn was moved out of CUDA.jl

https://github.com/JuliaGPU/CUDA.jl/tree/master/lib/cudnn

So we need CUDNN_jll or cuDNN as an additional optional dependency.

[GunnarFarneback]

Multiple optional dependencies can be implemented with nested @requires. Something like

    @require CUDA="052768ef-5323-5732-b1bb-66c8b64840ba" begin
        CUDA.functional() && include("cuda.jl")
        @require cuDNN="02a925ec-e4fe-4b08-9a7e-0d78e3d38ccd" begin
            CUDA.functional() && include("cudnn.jl")
        end
    end

To support both CUDA 3 and CUDA 4 requires some more conditionals in the current cuda.jl code. E.g. CUDA.libcufft has become a string in CUDA 4, so CUDA.libcufft() yields a Method Error. I would have made a PR if I had understood the role of the CUDA sub-libraries and the significance of the calls being made in cuda.jl.

[jw3126]

This is a bit of a tricky problem. The situation is as follows:

• To run on cuda libonnxruntime needs a bunch of other libs like libcudnn
• Not all of these libs can easily be provided by binary builder, see Replaced artifacts with JLL artifacts #19
• In CUDA.jl 3.0 all the required libs where shipped with CUDA.jl. So a pragmatic solution to our problem was to @require CUDA.jl and then figure out the paths to all libs shipped with CUDA.
• CUDA.jl v4 does not ship all the libs anymore. So we would need a nested requires or perhaps CUDNN_jll as a direct dependency and CUDA.jl via requires.

Right now I do not need CUDA.jl v4 and onnxruntime gpu simultaneously. So personally I won't fix this soon, but a PR would be awesome of course.

[GunnarFarneback]

Not all of these libs can easily be provided by binary builder

If I understand https://www.juliabloggers.com/cuda-jl-4-0/ correctly, this is a solved problem in CUDA.jl 4. If so I guess most of the stuff happening in cuda.jl isn't really needed with CUDA 4, except we can't control which version of CUDA gets loaded through Requires.

or perhaps CUDNN_jll as a direct dependency and CUDA.jl via requires.

CUDNN_jll brings in a couple more CUDA jlls and I suspect those are uncomfortably big for someone who only wants to run on CPU.

[jw3126]

Not all of these libs can easily be provided by binary builder

If I understand https://www.juliabloggers.com/cuda-jl-4-0/ correctly, this is a solved problem in CUDA.jl 4. If so I guess most of the stuff happening in cuda.jl isn't really needed with CUDA 4, except we can't control which version of CUDA gets loaded through Requires.

Oh, this is awesome news. I see two paths forward:

Option 1: Get rid of Requires.jl Create a tiny ONNXRuntimeCUDA package that depends on all the jll packages that expose the CUDA libs.

Option 2: Get rid of Requires.jl obtain the cuda libs as lazy artifacts

What do you think?

or perhaps CUDNN_jll as a direct dependency and CUDA.jl via requires.

CUDNN_jll brings in a couple more CUDA jlls and I suspect those are uncomfortably big for someone who only wants to run on CPU.

Good point.

[stemann]

The version of CUDA can be controlled with the CUDA_Runtime_jll preference version: https://cuda.juliagpu.org/stable/installation/overview/#Specifying-the-CUDA-version

Can't Preferences.jl be exploited similarly here? E.g. to add a preference to ONNXRuntime_jll to select e.g. CPU or CUDA-artifacts - and let the rest sort itself out.

Cf. #19 (comment)

[GunnarFarneback]

#32 implements support for CUDA.jl versions 4 and 5, but only for Julia 1.9 and later.