Automatically detecting apps blocking Shadowplay

[leumasme]

I did some research on why Shadowplay even turns off when viewing some content, in an effort to figure out if this could be circumvented.
Shadowplay internally uses NvFBC (which is part of the Nvidia Capture SDK) to capture frames.
NVIDIA Capture SDK Programming Guide.pdf
Because of how NvFBC works, it would presumably be able to capture content that normally is not recordable

• this includes Netflix and other DRM-Protected content (see also: Widevine/Encrypted Media Extensions; not sure how exactly this is detected by NvFBC)
• but also Windows with the WDA_EXCLUDEFROMCAPTURE or MONITOR flag, see SetWindowDisplayAffinity
  • This is likely responsible for most or all cases of ShadowPlay disabling for no apparent reason. Some examples for windows with this flag are Password managers like KeePassXC, but also some Screensharing hints (like teams/zoom) that alert you that you're sharing your screen. They may be hidden from capture to make sure that they're not visible on the screenshare that they're warning about.

NvFBC will detect such "Protected Content" and refuse to capture frames if any is present, returning a NVFBC_ERROR_PROTECTED_CONTENT error.

Playback of protected content such as DVD or BluRay disks typically requires an
encrypted, secured path to the physical display output device. To prevent violation of
protected content license terms, NVFBC will not capture frames from the GPU
whenever a protected content session is active.
NVFBC indicates the presence of protected content by returning
NVFBC_ERROR_PROTECTED_CONTENT from calls to NVFBCEndGrabFrame(), and no frame
data is returned.

Logs about this happening can be found in C:\ProgramData\NVIDIA Corporation\ShadowPlay\CaptureCore.log, where ShadowPlay will log the Errors that NvFBC is returning

Exerpt from CaptureCore.log

CVideoCaptureFbc::CaptureFrame: CVideoFrameProvider::CaptureFrame failed 0XA
CVideoCapture::Process: CaptureFrame failed 0XFFFFFFFC
CVideoCapture::NvFBCH264GrabFrame: nvfbc grab frame failed
CVideoCapture::Capture: NvFBCH264GrabFrame failed with error = NVFBC_ERROR_PROTECTED_CONTENT
CCaptureCore::OnError: dwErrorType[121], uiErrorContext[0], hr[0X80004005], m_CaptureCoreCallback[0XAC97E7D2]
CCaptureSession::HandleCapCoreCB: CAPCORE_CBTYPE_ERROR
CCaptureSession::HandleErrorCB: error(0x00000079)
CCaptureSession::HandleError: uiErrorType[121], uiSubType[0], hr[0X80004005]
CCaptureSession::LogProtectedContentAppDetails: Protected Content is running by PID: 6060 App:
CCaptureSession::IsDVRDisableRequired YES : err(121:0) hr(0X80004005)
CCaptureSession::StopDVRCapture: IN
CircularBufferManager::Release: 0
CVideoCaptureFbc::CaptureFrame: CVideoFrameProvider::CaptureFrame failed 0XA
CVideoCapture::Process: CaptureFrame failed 0XFFFFFFFC
CVideoCapture::NvFBCH264GrabFrame: nvfbc grab frame failed
CVideoCapture::Capture: NvFBCH264GrabFrame failed with error = NVFBC_ERROR_PROTECTED_CONTENT

Note that this is also logging which PID contains the protected content

CCaptureSession::LogProtectedContentAppDetails: Protected Content is running by PID: 6060 App:

This means that it should be possible to detect the application that is blocking ShadowPlay, notify the user about it (of course resolve PID to something more readable first), and automatically whitelist it (optional).
As a simple approach, this could be done by monitoring the logfile for log messages of the right format and parsing them.

Related references:

• https://github.com/keylase/nvidia-patch
  • A project concerned with patching Nvidia Drivers / NvFBC, since NvFBC has some arbitrary limits (normally not usable on Consumer GPUs except by select applications like Shadowplay)
  • Related patch might be possible to disable protected content detection, but the impact on system stability and anticheat compatability from running an unsigned driver is questionable
• https://www.qiuqiuren.club/?p=296
  • A chinese article about the same, but it also provides some useful extra information that might allow bypassing the second protected content check with low system impact

  • This problem is actually caused by a dll in the nvfbc library, which will constantly call GetWindowDisplayAffinity at runtime to check which windows have declared anti-screenshots. Therefore, you can hook this API and return an API error or return fake information.

• Disabling the "Press [hotkey] to enable in-game overlay" notification that normally shows on every game launch
  • By modifying the js script serving the local Nvidia API server to supress a certain event from being propagated, this notification can be cleanly supressed
  • In C:\Program Files (x86)\NVIDIA Corporation\NvNode\NvShadowPlayAPI.js into the first line of the function EmitNotification add this:
    if (data && data.windowMsg == "showHotkeyMessage") return

[leumasme]

From looking at _nvspcaps64.dll strings, the log message for DRM content should be
CaptureSession::LogProtectedContentAppDetails : Protected content is played through secure channel eg. netflix in browser

[Verpous]

Impressive work, I'll look into it!

[leumasme]

Related patch might be possible to disable protected content detection

Made something for this to just stop shadowplay from disabling itself (two things actually). Modifying the relevant process in-memory is enough, without anticheat incompatability as I feared.
This works both with the old GeForce Experience, as well as the new NVIDIA App (Which seems to have removed NvNode entirely - does AlwaysShadow still work for it?).
It doesn't automatically enable ShadowPlay if it's not already enabled, though.

Either Standalone patcher (run once per pc restart, kinda janky, shadowplay needs to be enabled already): https://github.com/furyzenblade/ShadowPlay_Patcher/
Or via the Windhawk mod manager (keeps running as background service, has some other nice windows mods): https://windhawk.net/mods/shadowplay-do-not-disable

I also tried looking into how one could automatically enable shadowplay with the new nvidia app, but no success. The Overlay UI is still in JS (angular), but it seems to use proper IPC for communication and changing the UI JS files doesn't actually seem to affect the app, plus webpack/minification so I just gave up on investigating how it does IPC after a while.

[Verpous]

Your reverse engineering skills are clearly way beyond mine! I haven't updated to the NVIDIA app yet, and from what you're telling me I don't think I will rush to do it. Anyway as long as they haven't changed registry keys for a couple things, AlwaysShadow should still work.

[Tim098b]

Is this feature implemented in AlwaysShadow now?

[Verpous]

@Tim098b nope, not currently implemented or being worked on