Click here to Skip to main content
1,837 members
Articles / Multimedia / C++
Article

Target Eye Revealed - Part 2

Rate me:
Please Sign up or sign in to vote.
5.00/5 (6 votes)
10 Sep 2013CPOL 16.5K   6  
Michael Haephrati describes Target Eye screen capturing mechanism

Voted Best C++ article of September 2012  

Introduction

Screen Capturing is one of the main building blocks in any surveillance and monitoring product. Target Eye Monitoring System, developed starting of 2000, had its unique JPG screenshots. This article is the 2nd one in the Target Eye Revealed series. The third article (following this one) is about the Shopping List mechanismFourth article is about Keyboard Capturing.    

Background

Image 1


Target Eye Monitoring System, which I have developed 12 years ago, was probably the first surveillance and monitoring tool for capturing activity of remote computers. The following description is taken from the original Business Plan of this venture:

Target Eye is a start-up company whose mission is to develop integrated software solutions for real-time monitoring of remote PCs, which are based on the company’s patent pending technologies (60/204,084 and 60/203,832). Our major product, Target Eye Monitoring System, is a software product that can continuously track, record, playback, analyze and report any activity performed on one or multiple remote PCs, in a way which is undetectable by their users. The software relies on a stream of rapidly captured, compressed full-screen images and continuous keystroke capturing to provide a comprehensive and accurate account of user activities, including local activities which do not generate any network traffic. In this way, the software can track and record activities, which are undetectable by systems relying on network traffic analysis. A smart agent module running on the monitored PCs uses a rule base to send alerts to the monitoring location(s) or perform pre-defined local operations. Monitoring can be performed from multiple locations. Major markets are law-enforcement. 

Target Eye's Building Blocks 

There are several terms and building blocks that sums up to the Target Eye Monitoring System: 

Target Eye Secret Agent - the covert part of the product that runs on the monitored computer 

Target Eye Operator - the authority that operates the Secret Agent (for example: a Law Enforcement agency).

A Mission - a request to perform an action. There are many possible actions (for example, capturing the screen, as described in this article or searching and finding files, as described in this article).   

Target Eye Compiler - The tool that is used by the Target Eye Operator to customize a Secret Agent for performing specific tasks.    

The Purpose of Screen Capturing

Screen Capturing serves one purpose which is to provide an accurate image of the activity that takes place on the monitored computer at a given moment. That is achieved by grabbing the image that is seen on the screen, and add to it a watermark which reflects any additional data that can help later on, to use the screenshot as an evidence: such additional data could be data and time stamp, timezone, a unique sequential number and any additional label such as the name of the active application or a certain key sequence that was captured as part of the "hot words" mechanism (for example: "Bin Laden"  

Image 2

The Target Eye Screenshots Files

Screen capturing files should be small in size and yet readable. The JPG format provides the mechanism to setup a the image quality using a numeric value between 1 and 100. This value also creates a unique look for the screenshots which makes it next to impossible to forge such screens or manipulate them, ensuring that each screenshot file is genuine. Target Eye uses its own unique imperfect screens.

Image 3

Taken by one of the first versions form May 2000. For original image press here.

Image 4

This is a reduced size screenshot taken from my own laptop at December 2003. You can actually see the Visual C++ 6.0 withe the Target Eye source code opened... To see the original image, click here.

Steps in capturing the screen

The following steps should be taken in order to capture a screen:

Get the DC

DC stands for Device Contents. The GetDC function retrieves a handle to a device context (DC) for the client area of a specified window or for the entire screen. To get the DC of a given Window, call:

HDC hDC = GetDC(hWnd);  

In our case, we pass "0" to GetDC, in order to capture the entire Desktop.

HDC hDC = GetDC(0);   

The returned handle is used subsequent GDI functions to draw in the DC. The device context is an opaque data structure, whose values are used internally by GDI.

Creating a Memory compatible DC

Now we create a compatible allocated memory which is compatible with the DC we capture by calling:

HDC CreateCompatibleDC(
  HDC hdc
);  

We also store the width and the height of the area to be captured:

int ScreenWidth = GetDeviceCaps(CI.m_hScreenDC, HORZRES);
int ScreenHeight = GetDeviceCaps(CI.m_hScreenDC, VERTRES);

Creating a Compatible Bitmap

By calling CreateCompatibleBitmap we create a bitmap compatible with the device that is associated with the specified device context.


HBITMAP CreateCompatibleBitmap(
  __in  HDC hdc,
  __in  int nWidth,
  __in  int nHeight
);  


SelectObject is used to copy the data from the screen to memory.

HGDIOBJ SelectObject(
  HDC hdc,   
  HGDIOBJ hgdiobj);   

Calling BitBlt

The BitBlt function is used a bit-block transfer of the captured area, by transferring the entire data per each color into the destination device context.

BOOL BitBlt(
  __in  HDC hdcDest,
  __in  int nXDest,
  __in  int nYDest,
  __in  int nWidth,
  __in  int nHeight,
  __in  HDC hdcSrc,
  __in  int nXSrc,
  __in  int nYSrc,
  __in  DWORD dwRop
);

As this stage we are ready to add the time stamp and any additional data on top of the captured image, as described in the following sections, but first, lets explain why and how we use ImgSource.

ImgSource - The SDK used by Target Eye's Screen Capturing

Target Eye's screenshots are created with the help of a graphic library named ImgSource by Smaller Animals, a privately-held corporation, founded in 1993, and incorporated in 1999, based in Research Triangle Park, NC. I have a full source code license for this library, as products like Target Eye (and customers who buy it, like Law Enforcement agencies) require having full source code of all components, with no "black boxes".

Chris Losinger, the President, has provided a lot of help along the way since 2000. It should be noted that ImgSource is not an open source library and is not a free product, but I strongly recommend it. Unregistered users can test the library but images will have a large red X drawn on them (including when it comes to the source code attached to this article). Along the many features this library has here is a partial list:

Image File Functions
  • Read images from files or memory.
  • Write image to files or memory.
  • Supports user-defined input and output managers - the ultimate in I/O flexibility
  • Optionally read and write most supported image file formats one line-at-a-time.
  • Read to a DIB in one function call
  • Get image dimensions and bit-depth from image files
  • Progress / cancel callback option provided for most file I/O routines and image processing options.
  • Supports JPG, GIF, BMP, TIFF, PhotoCD, Photoshop, WBMP, PNG, PCX, PAX, TLA, WMF, EMF, APM, PPM, PGM, PBM and TGA
  • JPG, PNG, PAX and TLA comment support
  • Multi-page TIFF I/O capabilities
  • Animated GIF I/O capabilities
  • JPEG_APPx marker support
  • Read EXIF data from JPGs and TIFFs
  • Write EXIF data to JPG
  • Read and write ICC profile data from JPG, PNG, TIFF
  • Read and write IPTC data from JPG, TIFF
  • Lossless JPG transformations
  • Read/write 1-bit G3/G4 (FAX) TIFFs
  • Custom memory allocation callbacks for image file reading
  • Much more...
Windows Bitmap Functions
  • Generate HBITMAPs from RGB buffers
  • Generate RGB buffers from HBITMAPs
  • Generate 8-bit and RGB buffers from DIBs
  • Generate DIBs from RGB, 8 or 1-bit images
  • Draw HBITMAP, DIB, RGB, RGBA or 8-bit images to an HDC in one call
  • Capture DCs to RGB buffers in one call
  • Draw transparent HBITMAP or RGBA or RGB images
  • Much more...
Image Processing Functions
  • Resize an image to any size
  • Apply an arbitrary convolution matrix to an image
  • Sharpen, blur and unsharp mask an image
  • Rotate an image, any angle
  • Detect image rotation (deskew) angle
  • Adaptive thresholding for converting grayscale to monochrome
  • Polygon and ellipse fill, flood fill or draw lines onto images
  • Apply a look up table (LUT) to an image
  • Generate grayscale images from RGB images
  • Generate 8-bit colormapped images from 24-bit RGB or 32-bit RGBA images (color quantize)
  • Convert between 24-bit and 8-bit images
  • Using any palette, generate an 8-bit image from an RGB image
  • Convert between 16-bit per component and 8-bit per component
  • Modify brightness / contrast / saturation of an image
  • Automagic brightness correction
  • Automagic image border detection
  • Automatic dust and scratch removal
  • Emboss images
  • Polygon warp and point to point warp functions
  • FFT and morphological operations
  • Despeckle grayscale and RGB images
  • Histogram stretch / equalization
  • Many alpha blending and image compositing/overlay functions
  • Crop images
  • Vertically and horizontally flip buffers
  • Count colors in RGB images 
  • Draw smoothed text onto images
  • Replace a color in an RGB image
  • Many of these functions can be used with 16-bit per component data
  • Much more...

The CaptureScreen function

Image 5

Target Eye's screenshots are generated by calling CaptureScreen() which is used to capture the entire screen into a JPG file kept in a local hidden location, and then send it to the FTP server.

The first version of Target Eye was released on May 2000 and had a simple screen capturing routine that saved the files in .bmp format, which is large in size, which makes it problematic if we wish to capture screens every minute... It was then replaced with a loosy JPG file, which did the job. The following function had gone a long way of many hours of QA, as Window's GDI consumes some memory, and if such routine is ran for days, due to bugs in GDI, the application will consume the computer's resources, but that was solved by carefully allocating and releasing the necessary resources and managing the memory allocation correctly.

static HDC ScreenDC=NULL;
static HDC hmemDC=NULL;
static HBITMAP hmemBM=NULL;
static HGLOBAL hRGB=NULL;
void SimpleCapture()
{    
    ScreenDC = GetWindowDC(0);
    if (ScreenDC != NULL)
    {
        hmemDC = CreateCompatibleDC(ScreenDC);   
        if (hmemDC != NULL)
        {
            int ScreenWidth = GetDeviceCaps(ScreenDC, HORZRES);
            int ScreenHeight = GetDeviceCaps(ScreenDC, VERTRES);
            hmemBM = CreateCompatibleBitmap(ScreenDC, ScreenWidth, ScreenHeight);
            if (hmemBM != NULL)
            {
                HGDIOBJ result=SelectObject(hmemDC, hmemBM);   // copy screen to memory DC
                if((ULONG)result!=0L && (ULONG)result!=GDI_ERROR)
                {
                    GdiFlush();
                    if((BitBlt(hmemDC, 0, 0, ScreenWidth, ScreenHeight, ScreenDC, 0, 0, SRCCOPY)))
                    {
                        
 
                        hRGB = ISHBITMAPToRGB(hmemBM, (unsigned int *)&ScreenWidth, (unsigned int *)&ScreenHeight, hmemDC, NULL);
                        if (hRGB)
                        {
                            // for grayscale
                            //.. RGB -> JPG
                            HISDEST hDest = ISOpenFileDest("c:\\test.jpg");
                            if(hDest!=NULL)
                            {
                                static ULONG ImageCount=0;
                                int Wrote=ISWriteJPG(hDest, (BYTE *)hRGB, ScreenWidth, ScreenHeight, 25, 1, 24, NULL);
                                ISCloseDest(hDest);
                                TRACE("Created Image %ld\n",ImageCount++);
                            }
                            
                            //.. clean up
                            GlobalFree(hRGB);
                            hRGB=NULL;
                            
                            
                        }
                        else
                        {
                            // handle error
                        }
 
                    }
                }
                DeleteObject(hmemBM);   
                hmemBM=NULL;
            }
            DeleteDC(hmemDC);  
            hmemDC=NULL;
        }
        DeleteDC(ScreenDC);
        ScreenDC=NULL;
    }
}  

As you can see, the "Quality" attribute set in this version is 25 (from 100).

Creating the Header

Since versions 2003 and 2004 of Target Eye, a unique Header was created with valuable information about the screenshot. The header is created after the screen images is captured to memory, and just before the creation of the JPG file.



// Now we create the header which is stamped on top of the screen
sprintf(s,"%ld",cycle);
CString FormattedTime=(CString)CTime::GetCurrentTime().FormatGmt(L" %A, %d.%m.%y Time Zone: %z %H:%M Label = ");
if(TextToWrite!="") FormattedTime+=TextToWrite;                                
FormattedTime += s;
HPEN hNewPen;
hNewPen=CreatePen(PS_SOLID, 1, RGB(233,238,45));
SelectObject(CI.m_hmemDC, hNewPen);
TextOut(CI.m_hmemDC,0,0,FormattedTime,FormattedTime.GetLength());

Naming the Files

A monitoring application should have a scheme for naming the various files it creates, including the screens captured. The function in the attached source code provides a very simple file naming scheme based on the text "Screen" followed by a sequential number and the ".jpg" extension.

CString ComposeFileName(int i)
{
 
    CString result;
    result.Format(L"%s%d%s",(CString)L"screen",i,(CString)L".jpg");
    return result;
}  

Target Eye uses a more sophisticated system which was based on counting the "cycles" (which are the times in which the application reaches the main event loop). The cycle occurs every X seconds, which are calculated as the minimal amount of time any information is requested to be captured. For example, if the operator requests that files will be copied every 5 minutes and screens will be captured every 10 minutes, then provided that there aren't any additional periodically information gathering requests, the cycle will be every 5 minutes. This is set using a Timer.

Using the Source Code

The source code attached to this article is based on a newer version of Target Eye which is 100% Win32API (with no use of MFC), and UNICODE support. The following screenshot was created with the attached code, with the exception that before publishing this source code I have removed my own personal License Key, so screenshots will have a red X on top of them.

Image 6

Running a computer game (Bear in the Big Blue House), borrowed from my daughter Emma for continues days, made it possible to ensure the screen capturing functionality is stable enough with no memory leaks.  

Image 7

 

 Michael Haephrati CodeProject MVP 2013   

 ©2000-2013 Michael Haephrati and Target Eye LTD<o:p>

All materials contained on this article are protected by International copyright law and may not be used, reproduced, distributed, transmitted, displayed, published or broadcast without the prior written permission given by Michael Haephrati and Target Eye LTD. You may not alter or remove any trademark, copyright or other notice from copies of the content.<o:p>

<o:p>  

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Michael Haephrati
United States United States
Michael Haephrati, born in 1964, an entrepreneur, inventor and a musician. Haephrati worked on many ventures starting from HarmonySoft, designing Rashumon, the first Graphical Multi-lingual word processor for Amiga computer.

Worked with Amdocs and managed several software projects, among them one for the Ministry of Tourism in New Zealand. During 1995-1996 he worked as a Contractor with Apple at Cupertino. After returning to Israel, worked as a Project Manager with Top Image Systems (mostly with JCC, Nicosia), and then at a research institute made the fist steps developing the credit scoring field in Israel. He founded Target Scoring and developed a credit scoring system named ThiS, based on geographical statistical data, participating VISA CAL, Isracard, Bank Leumi and Bank Discount (Target Scoring, being the VP Business Development of a large Israeli institute).
During 2000, he founded Target Eye, and developed the first remote PC surveillance and monitoring system, named Target Eye.

Other ventures included: Data Cleansing (as part of the DataTune system which was implemented in many organizations.


Also a Code Project Member since Sunday, March 16, 2003 (10 years, 5 months)
20 Sep 2013: Best C++ article of August 2013
25 Jan 2013: Code Project - Best C++ article of December 2012
31 Dec 2012: CodeProject MVP 2013

Comments and Discussions

 
-- There are no messages in this forum --