Skip to content

libobs-d3d11: Guard CreateTexture2D against SEH faults#13183

Closed
prgmitchell wants to merge 1 commit intoobsproject:masterfrom
prgmitchell:fixImageCrash
Closed

libobs-d3d11: Guard CreateTexture2D against SEH faults#13183
prgmitchell wants to merge 1 commit intoobsproject:masterfrom
prgmitchell:fixImageCrash

Conversation

@prgmitchell
Copy link
Member

Description

Catch structured exceptions around CreateTexture2D, return E_FAIL so this is handled gracefully instead of crashing OBS.

Motivation and Context

I saw reports of a crash when adding an image using an image source that is 16000x9000px. Although the Microsoft documentation states that the D3D size limit per side is 16384 for a 2D texture, large images like the one attached below cause an exception to be thrown in the driver call path for both AMD and NVIDIA drivers when testing.

TestImage

How Has This Been Tested?

Fixes the crash with the above image and logs the error as expected

19:10:35.641: CreateTexture2D raised structured exception 0xC0000005 (width=16000 height=9000 format=27) 19:10:35.642: device_texture_create (D3D11): Failed to create 2D texture (80004005)

This is an area of OBS I am not very familiar with but wanted to PR since I took the time to research and work on a fix, open to feedback of whether or not this approach is the correct way to fix this or if there are further implications to this change that I do not understand.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code has been run through clang-format.
  • I have read the contributing document.
  • My code is not on the master branch.
  • The code has been tested.
  • All commit messages are properly formatted and commits squashed where appropriate.
  • I have included updates to all appropriate documentation.

Catch structured exceptions around CreateTexture2D, return E_FAIL so this is handled gracefully.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant