Capturing the screen on Windows in C++ using OpenCV#

OpenCV is just a great computer vision tool with a wide variety of capabilities, that is available in both C++ and Python. In this first blog about OpenCV, I will be introducing a simple algorithm to capture the content of the screen on Windows using OpenCV in C++.

Build library and Includes#

In order to use OpenCV in C++, we have first to install the library. In my case, I am using Visual Studio 2019 so I had to build the library from source and correctly link it to my project. You can choose any build architecture that suits you, but most of my upcoming posts will be using the x86 architecture as I will be combining my code with win32 based GUIs. To install and link OpenCV correctly, I suggest referring to this very helpful video OpenCV 4 Building with CMake & Visual Studio 2017 Setup.

Once the project is correctly setup, we can start writing some code. So first we need to include the needed headers. In this case we need the OpenCV header (opencv2/opencv.hpp) and the Windows header (Windows.h) which we will use to get the desktop window handle:

1#pragma once
2#include <Windows.h>
3#include <opencv2/opencv.hpp>
4
5using nammespace cv;

Capture screenshot function#

Now let's write the main capture function, which will take a window handle to get its associated contextual device and return a cv::Mat object including the screenshot information. This function will first define handles to the device context and the associated Region of interest (defined using start-x, start-y, width and height). The bitmap and its header are then created and filled with the screen pixels data. This data is also passed to the Mat object. Finally the device contexts are deleted to avoid any memory leaks. For aesthetic & simplicity reasons, I chose to initialize the bitmap header in a separate function (BITMAPINFOHEADER createBitmapHeader(int, int)). The previously described steps looks as follows in C++:

 1BITMAPINFOHEADER createBitmapHeader(int width, int height)
 2{
 3   BITMAPINFOHEADER  bi;
 4
 5     // create a bitmap
 6     bi.biSize = sizeof(BITMAPINFOHEADER);
 7     bi.biWidth = width;
 8     bi.biHeight = -height;  //this is the line that makes it draw upside down or not
 9     bi.biPlanes = 1;
10     bi.biBitCount = 32;
11     bi.biCompression = BI_RGB;
12     bi.biSizeImage = 0;
13     bi.biXPelsPerMeter = 0;
14     bi.biYPelsPerMeter = 0;
15     bi.biClrUsed = 0;
16     bi.biClrImportant = 0;
17
18     return bi;
19}
20
21Mat captureScreenMat(HWND hwnd)
22{
23     Mat src;
24
25     // get handles to a device context (DC)
26     HDC hwindowDC = GetDC(hwnd);
27     HDC hwindowCompatibleDC = CreateCompatibleDC(hwindowDC);
28     SetStretchBltMode(hwindowCompatibleDC, COLORONCOLOR);
29
30     // define scale, height and width
31     int screenx = GetSystemMetrics(SM_XVIRTUALSCREEN);
32     int screeny = GetSystemMetrics(SM_YVIRTUALSCREEN);
33     int width = GetSystemMetrics(SM_CXVIRTUALSCREEN);
34     int height = GetSystemMetrics(SM_CYVIRTUALSCREEN);
35
36     // create mat object
37     src.create(height, width, CV_8UC4);
38
39     // create a bitmap
40     HBITMAP hbwindow = CreateCompatibleBitmap(hwindowDC, width, height);
41     BITMAPINFOHEADER bi = createBitmapHeader(width, height);
42
43     // use the previously created device context with the bitmap
44     SelectObject(hwindowCompatibleDC, hbwindow);
45
46     // copy from the window device context to the bitmap device context
47     StretchBlt(hwindowCompatibleDC, 0, 0, width, height, hwindowDC, screenx, screeny, width, height, SRCCOPY);  //change SRCCOPY to NOTSRCCOPY for wacky colors !
48     GetDIBits(hwindowCompatibleDC, hbwindow, 0, height, src.data, (BITMAPINFO*)&bi, DIB_RGB_COLORS);            //copy from hwindowCompatibleDC to hbwindow
49
50     // avoid memory leak
51     DeleteObject(hbwindow);
52     DeleteDC(hwindowCompatibleDC);
53     ReleaseDC(hwnd, hwindowDC);
54
55     return src;
56 }

The main call#

In order to test this, and for you to have an idea on how to use the previous code, in your future projects. Let's call it inside of a main function, encode the output as a PNG and save the captured screenshot to the hard drive. In code this looks like this:

 1int main()
 2{
 3      // capture image
 4      HWND hwnd = GetDesktopWindow();
 5      Mat src = captureScreenMat(hwnd);
 6
 7      // save img
 8      cv::imwrite("Screenshot.png", src);
 9
10  // clean-ups
11      buf.clear();
12      return 0;
13}

Just in case you need, in memory PNG data then just copy the data in the cv::Mat object to a vector like the following:

1// encode result in case you need in memory byte data
2std::vector<uchar> buf;
3cv::imencode(".png", src, buf);

The full code can be found in this gist: CaptureSceenshotUsingOpenCV.cpp. In case you prefer having `JPEG` data, then just replicate all the previous steps while replacing :code::`".png"` with :code::`".jpg"`.

Limitations#

  • The previous implementation is a bit limited. As it is somewhat slow comparing to the screen capture windows function associated with the capture screen button. This can be explained by the fact that unlike the windows function, OpenCV was not built for such a basic task.

  • Furthermore, in a multi-monitors setup, if you play with the DPI and the scaling settings of the screens, you will notice that the resulting screenshots can be cropped. This can be solved by setting the C++ project DPI-awareness to True. In Visual Studio 2019, this can be done under: Project > Project-Name Properties > Manifest Tool > Input and Output > DPI Awareness

  • Another limitations is that this code only allows for one screenshot of all screens, which is not always the best option. Some users might want to only capture a specific screen. This can be solved -as we will see in future posts- by manipulating the start-x, start-y, width and the height variables used in the capture function.

Conclusion#

To summarize, in this post we introduced a small example of how to capture the screen content using OpenCV and save it to the hard drive as an image or to the memory to use it inside your code. The code is fairly simple and supports both PNG and JPEG. On the other hand, the code is slightly slow and therefore using the native Windows solution might result in better performance. This option will be explored in my next posts, so stay tuned.

Share this blog#

Updated on 08 April 2022

👨‍💻 edited and review were on 08.04.2022

References and Further readings#

Root mean square normalization in Python   Capturing the screen on Windows in C++ using GDI+