How to take a screenshot from a background thread using WinRT?

Question

I'm having issues with WinRT ScreenCapture blocking/freezing my UI.

My app is looping through selected windows and takes a screenshot of each every few seconds. Every time a screenshot is taken the UI animation is lagging and sometimes the UI just freezes completely until I focus on another app which somehow releases the UI block.

Using this ScreenCapture repo as a reference - https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/tree/master/dotnet/WPF/ScreenCapture

In order to add screenshot capability, I've adjusted the BasicCapture.cs accordingly -

public class BasicCapture : IDisposable
{
 ...
 private Texture2D cpuTexture; // added this
 private bool screenshotReady; // added this
 public BasicCapture(IDirect3DDevice d, GraphicsCaptureItem i)
 {
 ...
 cpuTexture = CreateTexture2D(item.Size.Width, item.Size.Height); // added this
 }
 public void Dispose()
 {
 session?.Dispose();
 framePool?.Dispose();
 swapChain?.Dispose();
 d3dDevice?.Dispose();
 cpuTexture?.Dispose(); // added this
 }
 private Texture2D CreateTexture2D(int width, int height)
 {
 // create add texture2D 2D accessible by the CPU
 var desc = new Texture2DDescription()
 {
 Width = width,
 Height = height,
 CpuAccessFlags = CpuAccessFlags.Read,
 Usage = ResourceUsage.Staging,
 Format = Format.B8G8R8A8_UNorm,
 ArraySize = 1,
 MipLevels = 1,
 SampleDescription = new SampleDescription(1, 0),
 };
 return new Texture2D(d3dDevice, desc);
 }
 private void OnFrameArrived(Direct3D11CaptureFramePool sender, object args)
 {
 using (var frame = sender.TryGetNextFrame())
 {
 ...
 using (var backBuffer = swapChain.GetBackBuffer<SharpDX.Direct3D11.Texture2D>(0))
 using (var bitmap = Direct3D11Helper.CreateSharpDXTexture2D(frame.Surface))
 {
 d3dDevice.ImmediateContext.CopyResource(bitmap, backBuffer);
 
 if (screenshotReady == false)
 {
 // added this to copy the DirectX resource into the CPU-readable texture2D
 d3dDevice.ImmediateContext.CopyResource(bitmap, cpuTexture);
 screenshotReady = true;
 }
 }
 }
 }
 public async Task<Image> ScreenshotWindow()
 {
 screenshotReady = false;
 StartCapture();
 //await for screenshot to be captured
 while (screenshotReady == false) await Task.Delay(50);
 // get IDirect3DSurface from texture
 using var surf = Direct3D11Helper.CreateDirect3DSurfaceFromSharpDXTexture(cpuTexture);
 // build a WinRT's SoftwareBitmap from this surface/texture
 using var softwareBitmap = await SoftwareBitmap.CreateCopyFromSurfaceAsync(surf);
 using var InMemoryStream = new InMemoryRandomAccessStream(); 
 BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, InMemoryStream);
 encoder.SetSoftwareBitmap(softwareBitmap);
 await encoder.FlushAsync();
 using var stream = InMemoryStream.AsStream();
 return Image.Load(stream); 
 }
}

And to the BasicSampleApplication.cs Iv'e added this -

 public async Task<Image> ScreenshotWindow(GraphicsCaptureItem item)
 {
 capture = new BasicCapture(device, item);
 var surface = capture.CreateSurface(compositor);
 brush.Surface = surface;
 var img = await capture.ScreenshotWindow();
 StopCapture();
 return img;
 }

As I don't actually need to visually show the captured screen on my app, I instantiate the BasicSampleApplication without any visual properties, and I do it once on the app startup -

 // Create the compositor.
 compositor = new Compositor();
 _screenshotService = new BasicSampleApplication(compositor);

Now all that's left is to iterate through the selectedWindows and screenshot them -

 private async Task ScreenshotSelectedWindows()
 {
 while (ScreenshotsEnabled)
 {
 foreach (var wnd in _selectedWindows)
 {
 GraphicsCaptureItem item = CaptureHelper.CreateItemForWindow(wnd.WindowHandler);
 using var img = await _screenshotService.ScreenshotWindow(item);
 }
 await Task.Delay(TimeSpan.FromSeconds(10));
 }
 }

And now when I use ScreenshotSelectedWindows() it works, but as I initially said its making the UI animations lag, and sometimes the UI just freezes completely till I remove focus.

Iv'e tried calling this method from a background task -

Task.Run(()=>ScreenshotSelectedWindows());

For some reason doing that won't work, the OnFrameArrived callback is never triggered. Iv'e also tried to instantiate the BasicSampleApplication on the background thread but creating the compositor on a background thread throws an error System.UnauthorizedAccessException: 'Access is denied. '.

Any tips on how can I achieve that screenshot goal without blocking/freezing my UI?

Thanks!

Edit - I'll add a link to a GitHub issue as it seems to be referencing my issue specifically, which I fail to translate to my code :D

https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/issues/69#issuecomment-1498188216

• Simon Mourier: Difficult to say fom pieces of code like that, every details is important with this type of code. Do you have a simple full reproducible sample stackoverflow.com/help/minimal-reproducible-example
• Ben: Got it, will start working on reproducible code, might take a while as using this API on dotnet 6 requires multiple patches ... btw, following you last recommendation I looked into running that on a STA thread. It seems like I can't create the compositor outside the main app thread, it throws System.UnauthorizedAccessException: 'Access is denied.'. and if I try to use the compositor from another thread the OnFrameArrived is never triggered. Any idea how to create a compositor on another thread?
• Ben: I was able to create a compositor from a background thread following this solution - github.com/microsoft/CsWinRT/issues/617#issuecomment-740757111 But still, OnFrameArrived never triggers

Answer 1

Problem solved.

Creating a compositor on a background thread requires creating a dispatcher queue - https://github.com/microsoft/CsWinRT/issues/617#issuecomment-740757111

It's not actually necessary for my use case, OnFrameArrived wasn't triggered on a background thread because the frame pool needs to be created using - Direct3D11CaptureFramePool.CreateFreeThreaded

Everything is explained with better code samples than I shared in this Github Issue comment -

https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/issues/69#issuecomment-1498245106