Failed to load dynlib/dll

[Inoix]

This issue appears related to issue #107, opened on February 19, but for me it is a regression in version 1.104.yr0-ROCm. Also, it appears similar to another issue in the main koboldcpp repository, but the root cause is unrelated as far as I can say.

Describe the Issue
KoboldCpp starts fine, but when loading the model with any backend it fails to load the corresponding dll.

Workaround

Because this issue happens only on the newer version and is related with dlls, I started tinkering, by replacing some of the dlls on newer koboldcpp with those from older one.
In other words, I opened older (KoboldCPP-v1.98.1.yr0-ROCm) version, went into it's unpacked files (MEI_something) in the appdata/local, copied all of the dlls to my desktop. Then, I opened the newer version, and started replacing some of the dlls in its _MEI folder with DLLs from the older version.
After some time, it turned out that if you add the missing hipblas.dll file to the newer version, it starts to work!

TL;DR: adding the missing hipblas.dll from an older version of koboldcpp fixes the issue.

Additional Information:
This issue persists across any backends (including failsafe one), models and flags, at least, as far as I've checked.
OS: Win 10
CPU: Ryzen 7 3700X
GPU: RX 6600
I don't have any antivirus or something similar, apart from Windows Defender, which seems placid.

Error log (powershell):

PS D:\models> ./koboldcpp_rocm.exe
***
Welcome to KoboldCpp - Version 1.104.yr0-ROCm
For command line arguments, please refer to --help
***
Auto Selected HIP Backend (flag=0)

Loading Chat Completions Adapter: C:\Users\Inn\AppData\Local\Temp\_MEI57922\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
System: Windows 10.0.19045 AMD64 AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD
Detected Available GPU Memory: 8176 MB
Detected Available RAM: 20977 MB
Initializing dynamic library: koboldcpp_hipblas.dll
Traceback (most recent call last):
  File "PyInstaller\loader\pyimod03_ctypes.py", line 53, in __init__
  File "ctypes\__init__.py", line 374, in __init__
OSError: [WinError 127] Не найдена указанная процедура

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "koboldcpp.py", line 8936, in <module>
  File "koboldcpp.py", line 7910, in main
  File "koboldcpp.py", line 8305, in kcpp_main_process
  File "koboldcpp.py", line 606, in init_library
  File "PyInstaller\loader\pyimod03_ctypes.py", line 55, in __init__
pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll 'C:\\Users\\Inn\\AppData\\Local\\Temp\\_MEI57922\\koboldcpp_hipblas.dll'. Most likely this dynlib/dll was not found when the application was frozen.
[13392] Failed to execute script 'koboldcpp' due to unhandled exception!

[YellowRoseCx]

that's odd! Does adding kobold_hipblas.dll from "koboldcpp_rocm_files.zip" work? https://github.com/YellowRoseCx/koboldcpp-rocm/releases/download/v1.104.yr0-ROCm/koboldcpp_rocm_files.zip

[Inoix]

that's odd! Does adding kobold_hipblas.dll from "koboldcpp_rocm_files.zip" work? https://github.com/YellowRoseCx/koboldcpp-rocm/releases/download/v1.104.yr0-ROCm/koboldcpp_rocm_files.zip

No. There is already a kobold_hipblas.dll (though, a different one), if I replace it, I still get the same error as above.

Changing the kobold_hipblas.dll to the one from "koboldcpp_rocm_files.zip" and adding the hipblas.dll works fine.

By the way, when using that kobold_hipblas.dll from "koboldcpp_rocm_files.zip" I get worse benchmark results (around KoboldCPP-v1.98.1.yr0-ROCm level), than when using the one that is in the .exe by default.

[ivivaitylin]

Just to add that I got the exact same issue:
`Loading Chat Completions Adapter: C:\Users<user>\AppData\Local\Temp_MEI396002\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
System: Windows 10.0.19045 AMD64 AMD64 Family 25 Model 33 Stepping 2, AuthenticAMD
Detected Available GPU Memory: 16368 MB
Detected Available RAM: 42105 MB
Initializing dynamic library: koboldcpp_hipblas.dll
Traceback (most recent call last):
File "PyInstaller\loader\pyimod03_ctypes.py", line 53, in init
File "ctypes_init_.py", line 374, in init
OSError: [WinError 127] The specified procedure could not be found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "koboldcpp.py", line 8936, in
File "koboldcpp.py", line 7910, in main
File "koboldcpp.py", line 8305, in kcpp_main_process
File "koboldcpp.py", line 606, in init_library
File "PyInstaller\loader\pyimod03_ctypes.py", line 55, in init
pyimod03_ctypes.install..PyInstallerImportError: Failed to load dynlib/dll 'C:\Users\\AppData\Local\Temp\_MEI396002\koboldcpp_hipblas.dll'. Most likely this dynlib/dll was not found when the application was frozen.
[31456] Failed to execute script 'koboldcpp' due to unhandled exception!`

If I go to the temp directory that this creates on launch, I can see the koboldcpp_hipblas.dll in there is 480Mb, but even if I overwrite it with the one from the b2_files.zip it still crashes when I try to load an LLM model.

Win10 22H2, 5900X, 7800XT.

[N1ckn1ght]

Does adding kobold_hipblas.dll from "koboldcpp_rocm_files.zip" work?

I can't even add this .dll to test, because koboldcpp_rocm creates new folder in Temp every time I try to use it.
By the way, yes, same issue.
Win10 22H2 (10.0.19045 Build 19045), 5950X, 9060XT.

upd: koboldcpp_rocm_b2 throws the same error.

[ivivaitylin]

Does adding kobold_hipblas.dll from "koboldcpp_rocm_files.zip" work?

I can't even add this .dll to test, because koboldcpp_rocm creates new folder in Temp every time I try to use it. By the way, yes, same issue. Win10 22H2 (10.0.19045 Build 19045), 5950X, 9060XT.

upd: koboldcpp_rocm_b2 throws the same error.

To add the dll you can run the kobold_rocm.exe which creates the temporary directory but doesn't load anything from it yet, then find that temp folder and add the dll files which it would then use for that run, but as soon as you close it the tmp directory gets deleted, so you'd have to do this every time.

However, as noted by henk717 here:
#149 (comment)

The main koboldcpp vulkan path is apparently at least as fast as this rocm branch and is also more stable, so it's probably best to transition over to that build now.