fix vulkan building

[Dts0]

Fix #7

Now you can build it without make -f Makefile.sync clean sync

cmd:

# Build for CPU
cmake --preset CPU
cmake --build --parallel --preset CPU
cmake --install build --component CPU --strip

# Build for Vulkan
cmake --preset Vulkan
cmake --build --parallel --preset Vulkan
cmake --install build --component Vulkan --strip

# Build Ollama binary
go build -trimpath -buildmode=pie -o dist/bin/ollama

If a error occur while running, you can run it with sudo or enable CAP_PERFMON .

[isoos]

Thank you for doing this, I was able to run ollama + vulkan locally with ~21% improvement over CPU inference with an AMD iGPU (8700G vs 780M).

[pmonck]

I compiled this on 22.04, but it's still not working for me. Vulkan is detected on both of my GPUS, but ollama looks for rocm and falls back to CPU when it doesn't find the rocm library. Do I need to install rocm even if I don't intend to use it?

time=2025-02-18T22:11:00.296Z level=INFO source=routes.go:1237 msg="Listening on 127.0.0.1:11434 (version 0.0.0)" time=2025-02-18T22:11:00.296Z level=INFO source=gpu.go:255 msg="looking for compatible GPUs" time=2025-02-18T22:11:00.514Z level=INFO source=gpu.go:200 msg="vulkan: load libvulkan and libcap ok" time=2025-02-18T22:11:00.698Z level=INFO source=gpu.go:422 msg="error looking up vulkan GPU memory" error="device is a CPU" time=2025-02-18T22:11:00.698Z level=WARN source=amd_linux.go:61 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2025-02-18T22:11:00.699Z level=WARN source=amd_linux.go:443 msg="amdgpu detected, but no compatible rocm library found. Either install rocm v6, or follow manual install instructions at https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install" time=2025-02-18T22:11:00.699Z level=WARN source=amd_linux.go:348 msg="unable to verify rocm library: no suitable rocm found, falling back to CPU" time=2025-02-18T22:11:00.725Z level=INFO source=types.go:137 msg="inference compute" id=GPU-2e542f98-b4e2-1707-d3e8-aa04138f97bc library=cuda variant=v12 compute=8.6 driver=12.8 name="NVIDIA GeForce RTX 3090" total="23.6 GiB" available="4.1 GiB" time=2025-02-18T22:11:00.725Z level=INFO source=types.go:137 msg="inference compute" id=0 library=vulkan variant="" compute=1.4 driver=1.4 name="NVIDIA GeForce RTX 3090" total="24.2 GiB" available="4.6 GiB" time=2025-02-18T22:11:00.725Z level=INFO source=types.go:137 msg="inference compute" id=1 library=vulkan variant="" compute=1.3 driver=1.3 name="AMD Radeon Graphics (RADV VEGA20)" total="32.0 GiB" available="31.4 GiB

[isoos]

@pmonck: running it as root helped me

[pmonck]

@pmonck: running it as root helped me

Thanks, that worked.
Does anyone know how to make it work running as 'ollama'?
I've tried "sudo setcap cap_perfmon=ep /usr/bin/ollama" , but it looks like there's another permissions issue somewhere.

[jhemmond]

@Dts0 Are these instructions valid for a Windows build?

[Dts0]

If you're looking for Windows, binaries, newer versions, or more detailed introductions, see :
#7

[jhemmond]

Thank you! I’ll take a look

…

On Sun, Mar 9, 2025 at 8:23 AM, dts ***@***.***(mailto:On Sun, Mar 9, 2025 at 8:23 AM, dts <<a href=)> wrote: If you're looking for Windows, binaries, newer versions, or more detailed introductions, see : [#7](#7) — Reply to this email directly, [view it on GitHub](#14 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AYIQWA4MLKXUWNEJHIPMIUT2TQXEXAVCNFSM6AAAAABXB6YNYOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBYHAZTANJSGA). You are receiving this because you commented.Message ID: ***@***.***> [Dts0] Dts0 left a comment [(whyvl/ollama-vulkan#14)](#14 (comment)) If you're looking for Windows, binaries, newer versions, or more detailed introductions, see : [#7](#7) — Reply to this email directly, [view it on GitHub](#14 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AYIQWA4MLKXUWNEJHIPMIUT2TQXEXAVCNFSM6AAAAABXB6YNYOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBYHAZTANJSGA). You are receiving this because you commented.Message ID: ***@***.***>