32bit Direct3D 9 games via DXVK will not launch [AMD 7900XTX]

[sonic2kk]

I have been using mesa-tkg-git from the Chaotic AUR for some time, but recently it seems that 32bit Direct3D 9 applications running via DXVK will not run. Direct3D 11 applications seem to work fine as I've been playing some for a while. I have not noticed this as I have mainly been playing native Vulkan applications without issue.

I am using an AMD 7900XTX GPU.

The issues I am experiencing happen with "Borderlands 2 (49520)" and "Geometry Wars: Retro Evolved (8400)". These titles now run correctly after switching back to stable Mesa on Arch Linux.

Inspecting the Proton Log for these titles, they share the same error with Proton 10-based builds of compatibility tools. Using GE-Proton10-15, the error looks like this:

info:  Game: Borderlands2.exe
info:  DXVK: v2.7-46-gbcb0abde99af4c8
info:  Build: x86 gcc 10.3.0
info:  Found built-in config:
info:    d3d9.floatEmulation = Strict
info:    d3d9.supportDFFormats = False
info:    d3d9.lenientClear = True
info:  Effective configuration:
info:    d3d9.lenientClear = True
info:    d3d9.supportDFFormats = False
info:    d3d9.floatEmulation = Strict
156.551:0138:013c:trace:loaddll:build_module Loaded L"C:\\windows\\system32\\winevulkan.dll" at 775B0000: builtin
info:  Vulkan: Found vkGetInstanceProcAddr in winevulkan.dll @ 0x775c20a0
info:  Extension providers:
info:    Platform WSI
info:    OpenVR
info:    OpenXR
info:  Enabled instance extensions:
info:    VK_KHR_win32_surface
156.551:0138:013c:err:vulkan:wine_vkCreateInstance Failed to create instance, res=-9
err:   DxvkInstance::createInstance: Failed to create Vulkan instance

This is from Borderlands 2, but Geometry Wars had the same error. I tried both with Proton 9.0-4, and the error message was slightly different, but shared between the two titles. I sadly do not have the logs for those anymore; they were lost throughout my testing.

I am not a graphics expert but the last two lines, especially wine_vkCreateInstance Failed to create instance, res=-9, are what make me suspect this is an issue with either 32bit applications and DXVK (perhaps specifically when trying to play Direct3D 9 titles), or the 32bit Vulkan drivers with Mesa from git.

Perhaps this issue is worded incorrectly, or missing some key details, and maybe this is specifically an upstream problem for either Mesa, Chaotic-AUR, or DXVK, so please don't hesitate to direct me! I have a workaround for now by rolling back to a release of Mesa, and am perfectly happy to just stick with this for a bit if this is a temporary breakage elsewhere. But if there is a package or step I am missing to get mesa-tkg-git to work again or otherwise something I am doing wrong, I'm equally happy to fix it and go back to mesa-tkg-git :-)

Thanks!

[sonic2kk]

Further investigation reveals that this issue has probably been present for a while before I really figured out the root cause. When trying to play Sacred Gold and Sacred 2 Gold (the Steam and GOG releases), they would not work. After testing them again since reverting to Mesa stable from the Arch repos they work just fine. I would've tested them around August time I believe. So it could be that this problem has been present for around a month and may be more broad than just "32bit Direct3D 9 titles".

[Tk-Glitch]

LTO recently triggered some issues upstream, and it might not be totally irrelevant here (I disabled it today), but I think the main issue was probably due to some changes regarding wayland-scanner binary mismatch between 32 and 64-bit on Arch. It got fixed a couple days ago and the few d3d9 32-bit games I tried started just fine on my 9070XT.

[Sterophonick]

Still having this issue sadly, tested on mesa commit b1370e1935cf958b440b67b9f471499af8bc8ac8 from earlier today. Is there anything I can potentially do to work around it?

[Tk-Glitch]

I'm still not able to reproduce the issue on my end. Are you both using chaotic-aur builds?

[Sterophonick]

No, I am using self-built, somewhat cut down. Here's my config file

customization.cfg.zip

Setting the target commit to 84c1064eef19ee05a7f7f41ddaa4df830973fad7 fixed it, this is the commit that 25.2.5 was finalized on

[Tk-Glitch]

My test above was against current HEAD.
Let's try something then.

Please switch _compiler="clang" to _compiler="gcc" and _rusticl32_bypass="true" to _rusticl32_bypass="false" before rebuilding against a commit that used to break for you.
I have a feeling it might be it. And it's annoying.

[Sterophonick]

Tried that, same issue.

[Tk-Glitch]

Okay, interesting. Maybe we don't want the native part?
Could you please try again, keeping _rusticl32_bypass="true" this time?
What a weird issue.

[Sterophonick]

Same issue, heh.