Debug Labs


Access Violation in External Code

This lab will demonstrate an Access External Code. In this scenario, an AV in the "Kernel32.dll"

 

1- Lab Setup
	

Note: This lab assumes that you have already installed the necessary lab files to run it.

1- Set the IIS Admin Service to interact with the Desktop, and verify the service is running under the system account.

- In "Administrative Tools" click "Services"

- Double Click "IIS Admin Service"

- Click the "Log On" Tab

- Verify that the radio button "Local System Account" is selected

- Check "Allow Services to interact with Desktop" checkbox

2- Set the "Image File Execution Options" for the CGI Executable "badCgi.exe"

- Start Registry Editor (Regedt32.exe).

- Locate the following registry key: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\]

- Create a subkey with the name "badCgi.exe"

- To the "badCgi.exe" subkey, add the REG_SZ value named Debugger and the full path to the debugger with the "-g" switch: eg: c:\debugger\windbg -g

 

3- Set the "CreateProcessAsUser" metabase prooperty to "false"

- Open the command prompt (cmd)

- Switch to "Inepub\Adminscripts" folder

- Type in: CScript.exe adsutil.vbs set w3svc/CreateProcessAsUser "false" and hit Enter

 

2- Run Scenario

1- Click the "AV CGI" button to cause an AV.
 Note that the windbg window will appear with the following information. 
	a- Comments about the exception
b- The value of the registers
c- The module where the AV happened
d- The assembly line that caused the AV

2- In windbg command line, type in "g" and hit enter to cause a 2nd chance AV.

 

3- Add the symbol path in windbg using the GUI or the ".sympath+" command. (Use the company wide symbol server \\symbols\symbols)

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in \\symbols\symbols and hit Enter

b) Using the ".sympath+" command:

Go to windbg command line and type in this command ".sympath+ \\symbols\symbols" and press Enter.

 

Note: For more information on setting the symbol path for windbg, please refer to the debugger documentation for the .sympath command. This will describe

the "downstream store" feature that lets you download symbols to a symbol server of your own.

 

                

 

1- Debug Annotation

 

1) Without symbols

 

a) The faulting Stack

 

The stack backtrace of the thread that AV'd would look like:

 

0:000> kvn

# ChildEBP RetAddr Args to Child WARNING: Stack unwind information not available. Following frames may be wrong.

00 0006fe48 0100163f 00000000 01001441 0006fe6c KERNEL32!GetSystemTime+0x34

WARNING: Stack unwind information not available. Following frames may be wrong.

01 0006ff70 01001daf 00000001 00232468 00232cd8 badcgi+0x163f

02 0006ffc0 77e9ca90 77f82b95 77f83920 7ffdf000 badcgi+0x1daf

03 0006fff0 00000000 01001cb0 00000000 000000c8 KERNEL32!BaseProcessStart+0x3d

 

Keep in mind that we are here debugging a custom CGI application that misbehaved. The web application relying on this CGI will eventually not run properly.

So, in this stack backtrace, it looks like the problem is in "Kernel32.dll" since the AV is there! But in actuality, It is most likely because of some bad parameters passed to an API

call in kernel32. to invistigate this further, we need to line up the symbols properly what function AV'd in frame 00.

 

 

b) The regiters values  

0:000> r

eax=00000000 ebx=00000003 ecx=000007d3 edx=0000000c esi=78002041 edi= 002324b0

eip=77e87b14 esp=0006fe30 ebp=0006fe48 iopl=0 nv up ei pl nz na pe nc

cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl= 00000202

KERNEL32!GetSystemTime+34:

77e87b14 668908 mov [eax],cx ds:0023:00000000= ????

 

c) The dissassembly code

 

0:000> u eip

KERNEL32!GetSystemTime+34:

77e87b14 668908 mov [eax],cx

77e87b17 668b4dea mov cx,[ebp-0x16]

77e87b1b 66894802 mov [eax+0x2],cx

77e87b1f 668b4df6 mov cx,[ebp-0xa]

77e87b23 66894804 mov [eax+0x4],cx

77e87b27 668b4dec mov cx,[ebp-0x14]

77e87b2b 66894806 mov [eax+0x6],cx

77e87b2f 668b4dee mov cx,[ebp-0x12]

 

2) With symbols including public symbols of "badCgi.exe"

 

First, we need to add the private symbol path(company wide symbols) as well as public symbols of our exe

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in SRV*C:\LabSymbols*\\symbols\symbols;C:\IISDebugLabs\symbols\Pub and hit Enter

b) Using the ".sympath+" command:

.sympath+ SRV*C:\LabSymbols*\\symbols\symbols;C :\IISDebugLabs\symbols\Pub

The stack backtrace of the thread that AV'd would look like:

 

 

0:000> .reload;kvn50 ......

# ChildEBP RetAddr Args to Child

00 0006fe48 0100163f 00000000 01001441 0006fe6c KERNEL32!GetSystemTime+0x34

01 0006ff70 01001daf 00000001 00232468 00232cd8 badcgi!doAV2+0xf

02 0006ffc0 77e9ca90 77f82b95 77f83920 7ffdf000 badcgi!mainCRTStartup+0xff (CONV: cdecl) [crtexe.c @ 338]

03 0006fff0 00000000 01001cb0 00000000 000000c8 KERNEL32!BaseProcessStart+0x3d (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 504]

 

Now, with the public symbols lined up, we see that the last call that AV'd was to the function "GetSystemTime" in "kernel32.dll". this API was called from

the function "doAV2".

 

Looking in MSDN for the API "GetSystemTime" shows:

 

GetSystemTime

The GetSystemTime function retrieves the current system date and time. The system time is expressed in Coordinated Universal Time (UTC).

 
void GetSystemTime(
  LPSYSTEMTIME lpSystemTime
);

 

So it looks like the GetSystemTime API takes as a parameter a pointer to a SYSTEMTIME structure to receive

the current system date and time which could not be NULL.

 

You may want to look closely at the "GetSystemTime" function.

The source code for this function is shown below:

VOID WINAPI GetSystemTime( LPSYSTEMTIME lpSystemTime)

..... .

....

.....

RtlTimeToTimeFields(&SystemTime,&TimeFields);

lpSystemTime->wYear = TimeFields.Year ;

lpSystemTime->wMonth = TimeFields.Month ;

lpSystemTime->wDayOfWeek = TimeFields.Weekday ;

lpSystemTime->wDay = TimeFields.Day ;

lpSystemTime->wHour = TimeFields.Hour ;

lpSystemTime->wMinute = TimeFields.Minute ;

lpSystemTime->wSecond = TimeFields.Second ;

lpSystemTime->wMilliseconds = TimeFields.Milliseconds; }

 

The first and only parameter to this function is a long pointer to the system time (LPSYSTEMTIME).

Back to the dump file, this first parameter is NULL as shown below:

 

00 0006fe48 0100163f 00000000 01001441 0006fe6c KERNEL32!GetSystemTime+0x34

 

Since "lpSystemTime" is NULL, any attempt to access its members will result in an AV.

 

 

3) with Private Symbols "badCgi.exe"

 

First, we need to add the private symbol path(company wide symbols) as well as private symbols of our exe

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in SRV*C:\LabSymbols*\\symbols\symbols;C:\IISDebugLabs\symbols\Pri a nd hit Enter

b) Using the ".sympath+" command:

.sympath+ SRV*C:\LabSymbols*\\symbols\symbols;C :\IISDebugLabs\symbols\Pri

The stack backtrace of the thread that AV'd would look like:

 

0:000> .reload;kvn50

......

# ChildEBP RetAddr Args to Child

00 0006fe48 0100163f 00000000 01001441 0006fe6c KERNEL32!GetSystemTime+0x34 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\datetime.c @ 177]

01 0006ff70 01001daf 00000001 00232468 00232cd8 badcgi!doAV2+0xf (CONV: stdcall) [C:\build\nT\PRIVATE\inet\iis\svcs\w3\gateways\cgiDBG\badcgi.cpp @ 207]

02 0006ffc0 77e9ca90 77f82b95 77f83920 7ffdf000 badcgi!mainCRTStartup+0xff (CONV: cdecl) [crtexe.c @ 338]

03 0006fff0 00000000 01001cb0 00000000 000000c8 KERNEL32!BaseProcessStart+0x3d (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 504]

 

Notice that all frames contain the source code information.

This shows that line 207 in the file badcgi.cpp is the one that cused this AV.

 

This how line 207 looks like:

 

DWORD doAV1(CHAR szQS[DEFAULT_LENGTH], int Variation)

{

if (1 == Variation)

{

SYSTEMTIME* pSysTime = NULL;

GetSystemTime( pSysTime ); // Line 207 would av in GetSystemTime

}

return 1;

}

 

 

Advanced Technique

 

Remember that our web application is running in the IIS out of process pooled application (dllhost.exe).

The CGI executable is loaded in its own memory space and it is basically spawned by "inetinfo.exe". So for all http requests involving directly the the CGI

without passing via another extension will be processed by both inetinfo and the CGI executable without passing via dllhost. (for more information, please

refer to the class docs Chapter:...).

 

To get a feeling on how the CGI processing is made, let do the following:

 

1) In the command prompt, switch to the folder where the debugger is installed and run the following command:

c:\ debuggers>windbg -pn inetinfo.exe

 

2) In WinDbg, let set a breakpoint in CreateProcess function of Kernel32

In the windbg command line, type in "bp KERNEL32!CreateProcessA"

 

3) In Windbg command line, type in "g" and hit "Enter"

 

4) Click the "AV CGI" button in this page.

 

Notice that the debugger breaks, the command line is now available to run commands.

 

The stack backtrace of the thread where the break point was hit looks like:

 

0:010> kp

ChildEBP RetAddr

0136ee98 65f1be75 KERNEL32!CreateProcessA(char* lpApplicationName = 00000000 "", char* lpCommandLine = 0136f4b0 ""C:\IISDebugLabs\www\App\badCgi.exe" ", _SECURITY_ATTRIBUTES* lpProcessAttributes = 00000000 , _SECURITY_ATTRIBUTES* lpThreadAttributes = 00000000 , int bInheritHandles = 1, unsigned long dwCreationFlags = 0xc, void* lpEnvironment = 0136eed4 , char* lpCurrentDirectory = 0136fba8 "C:\IISDebugLabs\www", _STARTUPINFOA* lpStartupInfo = 0136f5b4 , _PROCESS_INFORMATION* lpProcessInformation = 0136f658 ) [D:\nt\private\windows\base\client\process.c @ 715]

0136f678 65f1b925 w3svc!HTTP_REQUEST::ProcessCGI(EXEC_DESCRIPTOR* pExec = 00e9eb2c , STR* pstrPath = 00000000 , STR* pstrWorkingDir = 0136fd10 , int* pfHandled = 0136feb0 , int* pfFinished = 0136fedc , STR* pstrCmdLine = 0136f5fc )+0x4c3 [D:\nt\private\inet\iis\svcs\w3\server\cgi.cxx @ 1179]

0136fd48 65f2a610 w3svc!HTTP_REQUEST::ProcessGateway(EXEC_DESCRIPTOR* pExec = 00e9eb2c , int* pfHandled = 0136feb0 , int* pfFinished = 0136fedc , int fTrusted = 0)+0x34e [D:\nt\private\inet\iis\svcs\w3\server\cgi.cxx @ 919]

0136fec4 65f020d7 w3svc!HTTP_REQUEST::DoWork(int* pfFinished = 0136fedc )+0x43f [D:\nt\private\inet\iis\svcs\w3\server\httpreq.cxx @ 839]

0136fee4 65f01b47 w3svc!CLIENT_CONN::DoWork(unsigned long BytesWritten = 0, unsigned long CompletionStatus = 0, int fIOCompletion = 0)+0x1aa [D:\nt\private\inet\iis\svcs\w3\server\conn.cxx @ 781]

0136ff08 65f01aa5 w3svc!CreateClient(_CLIENT_CONN_PARAMS* ClientParam = 0136ff20 )+0x7b [D:\nt\private\inet\iis\svcs\w3\server\connect.cxx @ 310]

0136ff4c 6d701443 w3svc!W3OnConnectEx(void* patqContext = 012742d8 , unsigned long cbWritten = 0xe4, unsigned long err = 0x1131e8, _OVERLAPPED* lpo = 00e897b8 )+0x118 [D:\nt\private\inet\iis\svcs\w3\server\connect.cxx @ 215]

0136ff80 6d702203 ISATQ!AtqpProcessContext(ATQ_CONTEXT* pAtqContext = 00000000 , unsigned long cbWritten = 0xe4, _OVERLAPPED* lpo = 012742dc , int fRet = 1)+0x23a [D:\nt\private\inet\iis\svcs\infocomm\atq\atqsupp.cxx @ 883]

0136ffb4 77e887dd ISATQ!AtqPoolThread(unsigned long* param = abcdef01 )+0x1a8 [D:\nt\private\inet\iis\svcs\infocomm\atq\atqsupp.cxx @ 1127]

0136ffec 00000000 KERNEL32!BaseThreadStart(* lpStartAddress = 6d7091f7 , void* lpParameter = abcdef01 )+0x52 [D:\nt\private\windows\base\client\support.c @ 460]

 

Note that the second parameter to CreateProcess is the command line to trun that process which is in this case "C:\IISDebugLabs\www\App\badCgi.exe"

 

Let try to get the whole HTTP request that w3svc received.

 

For this, let dump the EXEC_DESCRIPTOR class.

 

0:010> dt EXEC_DESCRIPTOR 00e9eb2c

+0x000 _pstrURL : 0x00e9e370

+0x004 _pstrPhysicalPath : 0x00e9e5e0

+0x008 _pstrUnmappedPhysicalPath : 0x00e9e610

+0x00c _pstrGatewayImage : 0x00e9eac0

+0x010 _pstrPathInfo : 0x00e9e400

+0x014 _pstrURLParams : 0x00e9e3d0

+0x018 _pdwScriptMapFlags : 0x00e9eabc -> 0

+0x01c _pGatewayType : 0x00e9e060 300002 ( GATEWAY_CGI )

+0x020 _pRequest : 0x00e9e008

+0x024 _hChildEvent : (null)

+0x028 _fMustWaitForChildEvent : 0 +0x02c _pParentWamRequest : (null)

+0x030 _dwExecFlags : 0

+0x034 _pMetaData : 0x00e9ecd8

+0x038 _pPathInfoMetaData : (null)

+0x03c _pPathInfoURIBlob : (null)

+0x040 _pAppPathURIBlob : (null)

 

 

Now let dump the HTTP_REQUEST class.

 

0:010> dt HTTP_REQUEST 0x00e9e008

+0x000 __VFN_table : 0x65f08680

+0x008 _cFilesSent : 0

+0x00c _cFilesReceived : 0

......

......

+0x9a8 _bufClientRequest : BUFFER

......

......

+0xbac _pFileNameLock : (null)

+0xbb0 _fPossibleDefaultExecute : 0y0

=65f465b0 HTTP_REQUEST::sm_GetInfoFuncs : [26] 0x65f26a87 w3svc!HTTP_REQUEST::GetInfoA+0

+0xbb4 _strDLCString : STR

 

 

The "_bufClientRequest " points to the buffer where the HTTP request is stored.

 

0:010> da poi(0x00e9e008 + 0x9a8 )

006ae398 "GET /IISDebugLabs/App/badCgi.exe"

006ae3b8 "?s=av1_v2 HTTP/1.1..Accept: */*."

006ae3d8 ".Accept-Language: en-us..Accept-"

006ae3f8 "Encoding: gzip, deflate..User-Ag"

006ae418 "ent: Mozilla/4.0 (compatible; MS"

006ae438 "IE 6.0; Windows NT 5.0)..Host: l"

006ae458 "ocalhost..Connection: Keep-Alive"

006ae478 "...."

 

5) In the windbg command line, type in "g" and hit "Enter"

Notice a second windbg window is displayed, this is the one attached to the CGI executable "badCgi.exe".

 

6) In the Windbg instance attached to "badCgi.exe" type in "q" and hit "Enter".

 

7) Finaly, back to the main WinDbg window, press "Ctrl-Break" to break in to the debugger, type in "q" and hit "Enter"

 

 

 

 


- Microsoft Confidential -