X11 Screenshots & Fake Input Events

I was surprised at how easy it is to take screenshots and create fake inputs in X11. X11 was probably one of the first C libraries I ever looked at and I immediately went looking for something else. So I wonder if I just don't have this impression of it being over-complicated because I had no experience with anything when I first tried to use it.

I ended up with these four includes:

#include <X11/Xlib.h> #include <X11/X.h> #include <X11/Xutil.h> #include <X11/extensions/XTest.h>

Taking a screenshot is done like this:

Display display = XOpenDisplay(NULL); Window root = DefaultRootWindow(display); XWindowAttributes attributes; XGetWindowAttributes(display, root, &attributes); int screen_width = attributes.width; int screen_height = attributes.height; // x, y = top-left corner of region to capture // w, h = width and height of region to capture XImage *image = XGetImage(display, root, x, y, w, h, AllPlanes, ZPixmap); // After using image: XDestroyImage(image);

I then wrote this code to get the RGB values out of X11's image format:

unsigned int r_mask = image->red_mask; unsigned int g_mask = image->green_mask; unsigned int b_mask = image->blue_mask; int r_shift = 0, g_shift = 0, b_shift = 0; { int temp; temp = r_mask; while ((temp & 1) == 0) temp >>= 1, r_shift++; temp = g_mask; while ((temp & 1) == 0) temp >>= 1, g_shift++; temp = b_mask; while ((temp & 1) == 0) temp >>= 1, b_shift++; }; for (int y = 0; y < h; y++) { for (int x = 0; x < w; x++) { unsigned int pixel = XGetPixel(image, x, y); red_value = (pixel & r_mask) >> r_shift; green_value = (pixel & g_mask) >> g_shift; blue_value = (pixel & b_mask) >> b_shift; }; };

I do wonder why the bit shift values weren't something I could just pull out of the structure but instead have to calculate myself. I guess the bits can also be in MSB order, but I decided to just assume that will never happen.

My real surprise was with how easy the fake input events are to create. Using the "display" variable obtained above, you just do this:

XTestFakeButtonEvent(display, button, True, 0); // press mouse button XTestFakeButtonEvent(display, button, False, 0); // release mouse button XTestFakeMotionEvent(display, -1, x, y, 0); // move mouse to absolute position XTestFakeRelativeMotionEvent(display, dx, dy, 0); // relative mouse movement (what minecraft looks at when mouse is captured) XTestFakeKeyEvent(display, keycode, True, 0); // press a key XTestFakeKeyEvent(display, keycode, False, 0); // release a key

The mouse buttons are numbered 1 through 5, in order left, middle, right, and the two scroll wheel buttons. They keycodes seem to be just keyboard scan codes, so there was some work there in using 'xev' to find out what they all are, and then in writing a function to create the correct key events to type text, but I don't mind that so much. I like programming, but I hate figuring out how to do shit with libraries, and so if the choice is between writing my own code to turn text into keycodes or dealing with some obtuse library, I'd much rather write my own code to turn text into keycodes.

Before writing this in C, I was writing it in Perl using a program called "scrot" to take screen captures, some code I wrote in C to read the screen captures and decode Minecraft's on-screen text, and then a program called "xdotool" to generate the mouse and keyboard events, because I assumed that taking screen captures and generating input events would be stupidly difficult. However, that process was kinda slow, mostly because it took a quarter second for scrot to take a screen capture, and I couldn't find any program that does it faster, and scrot doesn't allow you to tell it to capture just part of the screen, nor did any program I found that takes screen captures. So I was quite surprised to see that not only is it easy to do these things but the call to take screen captures requires that you tell it what part of the screen you want to capture.

X11 Screenshots & Fake Input Events

Comments