finding RGB values for pixels on current frame

[Zoefschildpad]

Hi, I am about to start a little project where I apply filters on the screen outside my application. I think I worked out most of what I need to do but I can't seem to find a way to retrieve the last rendered frame (pixel by pixel in RGB if possible). How can I do this in c++ (such taht it works on windows if possible)?

[Duthomhas]

How are you accessing data outside your application? You should have ah HDC that you can use with GetPixel[Ex]() to get color information.

[Zoefschildpad]

Hmm... never heard of an HDC, well, guess I have some googling to do

[Duthomhas]

Er, how exactly are you getting the "screen" outside your application? And by "screen", what exactly do you mean?

[new hope]

an HDC is just a device context. After you create a handle to a window in order to work with graphics you have to create a handle to a device context and the graphics is rendered with the device context.

[Zoefschildpad]

I meant the physical computer screen, I want to make an application that modifies the rgb values of a part of the screen. What I need to know for that is how to:
- retrieve the rgb values of the pixels in that part of the screen.
- redraw those pixels. (I know how to draw pixels in opengl, but i don't know how to draw them outside a predefined window)

[Duthomhas]

I think you are out of your league at the moment.

If you are willing to put in the time to learn more about it, then start by googling around "windows gdi tutorial". Everything in Windows is a specific type of resource, including the "desktop" wallpaper (which is what I think you are talking about, as you have yet to be more specific).

Once you get that far, you need to learn about how the wallpaper image is accessed, stored, and modified.

If you are talking about something more advanced, like modifying the colors of everything in a certain region of the display (including other programs running in that spot, and icons displayed there, etc), then you'll need to get into things like hardware overlays, or transparent windows and message hooks, or etc.

Yes, you really have asked to open a whole can of worms. It is all relatively straightforward, but there is a very large learning curve involved.

[Zoefschildpad]

Right, so, what I need to do is make a program that consists of two parts. in it's most basic form there should be a button on the desktop (in a small window, I am planning on making this in QT since I've worked with that before). Pushing that button makes the other part appear. The other part is an m*n rectangular, draggable 'box' (we're thinking 300*300 or something, but the size shouldn't matter) inside which all pixels undergo a transformation. (r, g, b)->(r, r, r) for instance to get the red component. our goal is to let colorblind people use that to easily use it to quickly see the difference between colors. I hope that's more clear.

I don't suppose it's suddenly easier to do if the box is actually a normal window? I'd need to know what would have been drawn there had the window not been there, is that possible?

I'm starting to see that this might not be possible in a reasonable amount of time. (say, a couple days). So I think I might make a gimped version that only works on a single image inside the application. (which is acceptible for the course I'm doing, it's not a course on programming but on human-computer interaction)

Oh brilliant, I looked up a magnifying glass to see how that worked, it freezes the entire screen. It's open source so I think I'm going to try and see if I can edit that to make it do what I want rather than magnifying

[gpotw]

I don't suppose it's suddenly easier to do if the box is actually a normal window? I'd need to know what would have been drawn there had the window not been there, is that possible?

GetDC(NULL) returns the DC of the entire screen which you can use to draw directly to and from the screen, but since Windows controls the screen DC you can expect whatever you draw to be quickly erased. Even if you constantly update your display in a loop you will see alot of flicker as windows underneath are updated. It would be easiest to create an always-on-top borderless window and position it so it just fills the area of the screen you want to modify. I imagine the process something like this:

1) Create a bitmap and DC in memory, both compatible with the screen DC.
2) Use BitBlt to copy the region of the screen under the window into the DC (hide the window first).
3) Use GetDIBits on the bitmap to get an array of pixel data.
4) Modify the pixel array.
5) Use SetDIBits to save the pixel array back in the bitmap.
6) Use BitBlt to copy the modified bitmap to the window.

You can get alot of effects using different raster operations with BitBlt and by controlling the window opacity so maybe steps 3-5 are unnecessary?

[Duthomhas]

Ah, I understand what you want to do. Yes, you can do this fairly quickly.

Google around "layered windows". This will allow your window to 'capture' the underlying window data, which you can then modify offscreen, and paint to your window.

You can find a good overview at http://www.google.com/search?btnI=1&q=msdn+Layered+Windows

Your normal window will, of course, have the controls to adjust how the special window will modify the display.

The special window will need some careful attributes:
- stay on top
- no borders?
- do not display on the taskbar
- some careful hit testing to pass all messages through to windows underneath

Good luck!

[Zoefschildpad]

I'm not really having any luck finding out how to get information from underneath the window. I'm not sure what I should be looking for... Can anyone point me in the right direction?

[Zoefschildpad]

Right, sorry about that last post, that was me being mad at the world.

So I wrote some code, which I realize is probably bad, it'll probably perform bad, but that's not the issue at the moment. The problem is my window is black, and I don't know why. Can anyone tell me what is going wrong?

#include <windows.h>

HWND hwnd;
/*  Declare Windows procedure  */
LRESULT CALLBACK WindowProcedure (HWND, UINT, WPARAM, LPARAM);

void edit(COLORREF pixels[300][300]){
     
     }

void lens() //gets the bitmap data
{
  COLORREF pixels[300][300];
  HBITMAP hBitmap;
  BITMAP Bitmap;
  RECT r;
  HDC windDC = GetDC(hwnd);
  HDC ActiveDC = GetDC(NULL);
  HDC CopyDC = CreateCompatibleDC(ActiveDC);
  

  int scrWidth = 1920;
  int scrHeight = 1080;

  for (int i = 0; i < 300; i++){
      for(int j = 0; j < 300; j++){
          pixels[i][j] = GetPixel(CopyDC, i, j);
      }
  }
  edit(pixels);
  for (int i = 0; i < 300; i++){
      for(int j = 0; j < 300; j++){
          SetPixel(windDC, i, j, pixels[i][j]);
      }
  }
  ReleaseDC(NULL, ActiveDC);
  DeleteDC(CopyDC);
  DeleteDC(windDC);
}

/*  Make the class name into a global variable  */
char szClassName[ ] = "WindowsApp";

int WINAPI WinMain (HINSTANCE hThisInstance,
                    HINSTANCE hPrevInstance,
                    LPSTR lpszArgument,
                    int nFunsterStil)

{
                   /* This is the handle for our window */
    MSG messages;            /* Here messages to the application are saved */
    WNDCLASSEX wincl;        /* Data structure for the windowclass */

    /* The Window structure */
    wincl.hInstance = hThisInstance;
    wincl.lpszClassName = szClassName;
    wincl.lpfnWndProc = WindowProcedure;      /* This function is called by windows */
    wincl.style = CS_DBLCLKS;                 /* Catch double-clicks */
    wincl.cbSize = sizeof (WNDCLASSEX);

    /* Use default icon and mouse-pointer */
    wincl.hIcon = LoadIcon (NULL, IDI_APPLICATION);
    wincl.hIconSm = LoadIcon (NULL, IDI_APPLICATION);
    wincl.hCursor = LoadCursor (NULL, IDC_ARROW);
    wincl.lpszMenuName = NULL;                 /* No menu */
    wincl.cbClsExtra = 0;                      /* No extra bytes after the window class */
    wincl.cbWndExtra = 0;                      /* structure or the window instance */
    /* Use Windows's default color as the background of the window */
    wincl.hbrBackground = (HBRUSH) COLOR_BACKGROUND;

    /* Register the window class, and if it fails quit the program */
    if (!RegisterClassEx (&wincl))
        return 0;

    /* The class is registered, let's create the program*/
    hwnd = CreateWindowEx (
           0,                 /* Extended possibilites for variation */
           szClassName,         /* Classname */
           "Monocle",       /* Title Text */
           
           WS_EX_LAYERED, /* default window */
           CW_USEDEFAULT,       /* Windows decides the position */
           CW_USEDEFAULT,       /* where the window ends up on the screen */
           300,                 /* The programs width */
           300,                 /* and height in pixels */
           HWND_DESKTOP,        /* The window is a child-window to desktop */
           NULL,                /* No menu */
           hThisInstance,       /* Program Instance handler */
           NULL                 /* No Window Creation data */
           );

    /* Make the window visible on the screen */
    ShowWindow (hwnd, nFunsterStil);

    /* Run the message loop. It will run until GetMessage() returns 0 */
    while (GetMessage (&messages, NULL, 0, 0))
    {
        /* Translate virtual-key messages into character messages */
        TranslateMessage(&messages);
        /* Send message to WindowProcedure */
        DispatchMessage(&messages);
        lens();
        
    }

    /* The program return-value is 0 - The value that PostQuitMessage() gave */
    return messages.wParam;
}



/*  This function is called by the Windows function DispatchMessage()  */

LRESULT CALLBACK WindowProcedure (HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)                  /* handle the messages */
    {
        case WM_DESTROY:
            PostQuitMessage (0);       /* send a WM_QUIT to the message queue */
            break;
        default:                      /* for messages that we don't deal with */
            return DefWindowProc (hwnd, message, wParam, lParam);
    }

    return 0;
}

[gpotw]

Looks like the problem is line 27. You are copying pixels from an empty DC. Use GetDC(NULL) as the source DC, or better yet replace the whole loop with a single BitBlt:

http://msdn.microsoft.com/en-us/library/dd183370%28v=VS.85%29.aspx

[Zoefschildpad]

Ye, I was planning on replacing the loop as soon as I got this working, that way I can get it working with functions I know I understand before piling mistakes on mistakes and get a debugging nightmare. Anyway, I changed CopyDC in line 27 to GetDC(NULL) (I also tried ActiveDC which should be the same, with the same results) but now it doesn't draw anything and I get a (not responding). I guess an error message of some kind would be too much to ask...

[Zoefschildpad]

I'm getting closer, I hope. I want to know why this doesn't paint anything on the screen (lines 28-31) I copy a block from the screen into memory, and then copy it from my memory into the window, what is going wrong?
I took the GetObject function from another program, but I have no clue why he used the screen DC instead of the memory DC, is that right?
Is pointer arithmatic the best way to edit a bitmap pixel by pixel, or is there some nicer way I'm not seeing?

#include <windows.h>

HWND hwnd;
/*  Declare Windows procedure  */
LRESULT CALLBACK WindowProcedure (HWND, UINT, WPARAM, LPARAM);

void edit(COLORREF pixels[300][300]){
     
     }

void lens() //gets the bitmap data
{
  COLORREF pixels[300][300];
  COLORREF pixel;
  HBITMAP hBitmap;
  BITMAP Bitmap;
  RECT r;
  HDC windDC = GetDC(hwnd);
  HDC ActiveDC = GetDC(NULL);
//  HDC CopyDC = CreateCompatibleDC(hBitmap);
  HDC memDC = CreateCompatibleDC(ActiveDC);
  char* p;

  int scrWidth = 1920;
  int scrHeight = 1080;

  hBitmap = CreateCompatibleBitmap(ActiveDC, 300, 300);
  BitBlt(memDC, 0, 0, 300, 300, ActiveDC, 50, 50, SRCCOPY);
//  GetObject(ActiveDC,sizeof(BITMAP),&Bitmap); 
// edit(parameters)
  BitBlt(windDC, 0, 0, 300, 300, memDC,  0,  0,  SRCCOPY);
  ReleaseDC(NULL, ActiveDC);
//  DeleteDC(CopyDC);
  DeleteDC(windDC);
}

/*  Make the class name into a global variable  */
char szClassName[ ] = "WindowsApp";

int WINAPI WinMain (HINSTANCE hThisInstance,
                    HINSTANCE hPrevInstance,
                    LPSTR lpszArgument,
                    int nFunsterStil)

{
                   /* This is the handle for our window */
    MSG messages;            /* Here messages to the application are saved */
    WNDCLASSEX wincl;        /* Data structure for the windowclass */

    /* The Window structure */
    wincl.hInstance = hThisInstance;
    wincl.lpszClassName = szClassName;
    wincl.lpfnWndProc = WindowProcedure;      /* This function is called by windows */
    wincl.style = CS_DBLCLKS;                 /* Catch double-clicks */
    wincl.cbSize = sizeof (WNDCLASSEX);

    /* Use default icon and mouse-pointer */
    wincl.hIcon = LoadIcon (NULL, IDI_APPLICATION);
    wincl.hIconSm = LoadIcon (NULL, IDI_APPLICATION);
    wincl.hCursor = LoadCursor (NULL, IDC_ARROW);
    wincl.lpszMenuName = NULL;                 /* No menu */
    wincl.cbClsExtra = 0;                      /* No extra bytes after the window class */
    wincl.cbWndExtra = 0;                      /* structure or the window instance */
    /* Use Windows's default color as the background of the window */
    wincl.hbrBackground = (HBRUSH) COLOR_BACKGROUND;

    /* Register the window class, and if it fails quit the program */
    if (!RegisterClassEx (&wincl))
        return 0;

    /* The class is registered, let's create the program*/
    hwnd = CreateWindowEx (
           0,                 /* Extended possibilites for variation */
           szClassName,         /* Classname */
           "Monocle",       /* Title Text */
           
           WS_EX_LAYERED, /* default window */
           CW_USEDEFAULT,       /* Windows decides the position */
           CW_USEDEFAULT,       /* where the window ends up on the screen */
           300,                 /* The programs width */
           300,                 /* and height in pixels */
           HWND_DESKTOP,        /* The window is a child-window to desktop */
           NULL,                /* No menu */
           hThisInstance,       /* Program Instance handler */
           NULL                 /* No Window Creation data */
           );

    /* Make the window visible on the screen */
    ShowWindow (hwnd, nFunsterStil);

    /* Run the message loop. It will run until GetMessage() returns 0 */
    while (GetMessage (&messages, NULL, 0, 0))
    {
        /* Translate virtual-key messages into character messages */
        TranslateMessage(&messages);
        /* Send message to WindowProcedure */
        DispatchMessage(&messages);
        lens();
        
    }

    /* The program return-value is 0 - The value that PostQuitMessage() gave */
    return messages.wParam;
}



/*  This function is called by the Windows function DispatchMessage()  */

LRESULT CALLBACK WindowProcedure (HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
{
    switch (message)                  /* handle the messages */
    {
        case WM_DESTROY:
            PostQuitMessage (0);       /* send a WM_QUIT to the message queue */
            break;
        default:                      /* for messages that we don't deal with */
            return DefWindowProc (hwnd, message, wParam, lParam);
    }

    return 0;
}

[gpotw]

You need to select the bitmap into the memDC before using BitBlt. Right after CreateCompatibleBitmap put this: SelectObject(memDC, hBitmap); As for the best way to edit pixels I would say try a combination of different raster operations with BitBlt. If you create your window with the WS_EX_LAYERED extended style flag you can set its opacity and make it translucent by calling SetLayeredWindowAttributes. I feel like for your purpose you could simply fill the entire window a solid color and make it translucent.

If you really want to edit each individual pixel use GetDIBits to get at an array of pixel data, modify the array, then call SetDIBits to update the bitmap with the modified pixels. Looping through the entire bitmap like this is going to be slooow no matter how you do it.

[Zoefschildpad]

Ok I found out I missed a functioncall. I can now display the thing I have in memory onto the screen. Now 'all' I need to do is edit it.

Between the following 2 lines I need to somehow access every pixel and change them:

 BitBlt(memDC, 0, 0, 300,300, ActiveDC, 50, 50, SRCCOPY); 
  
  
BitBlt(windDC,  0,  0,  300,  300,  memDC,  0,  0,  SRCCOPY);

But I can't seem to find out how to access this data. If I understand this correctly, the bitmap should be stored in 4 bytes per pixel (r, g, b and 0) So what I think I need is a char (or byte) pointer to the first pixel. How can I get that?

[gpotw]

gpotw wrote:

If you really want to edit each individual pixel use GetDIBits to get at an array of pixel data, modify the array, then call SetDIBits to update the bitmap with the modified pixels. Looping through the entire bitmap like this is going to be slooow no matter how you do it.

Example:

BITMAP bmp;
GetObject(bmp1, sizeof(BITMAP), &bmp);
	
BITMAPINFOHEADER bi;
bi.biSize = sizeof(BITMAPINFOHEADER);
bi.biWidth = bmp.bmWidth;
bi.biHeight = bmp.bmHeight;
bi.biPlanes = 1;
bi.biBitCount = 24;
bi.biCompression = BI_RGB;
bi.biSizeImage = ((((bi.biWidth * bi.biBitCount) + 31) & ~31) >> 3) * bi.biHeight;

BYTE* bits = new BYTE[bi.biSizeImage];
int ret1 = GetDIBits(hdc1, bmp1, 0, bi.biHeight, bits, (BITMAPINFO*)&bi, DIB_RGB_COLORS);

[Zoefschildpad]

That was extremely helpful, it works! and it's easily fast enough for my purposes. 300*300 isn't that big. it shows a cpu usage of 0% so I think it's not a problem.

Now to make it do cool things...

[Zoefschildpad]

I think the forum lost a post :/

Anyway, it works. But it flickers when I mouse over it. I think that is because I made the window hide itself to get data from underneath, but it's actually extremely annoying. currently I lower the alpha to 0 just long enough for the thing to disappear. I think the problem is in when i paint a new frame, because I do that in the while(getmessage) loop. How can I fix that? I read somewhere that it should draw when WM_PAINT is received, but when I added that it didn't draw at all anymore (even though my function call was also still in the while loop)...