Debug Labs


Access Violation in Component Code

This lab will demonstrate an Access Violation in a component. In this scenarion, an AV in the ISAPI filter  "badFil.dll"

 

1- Lab Setup
	

Note: This lab assumes that you have already installed the necessary lab files to run it.

1-Install the ISAPI Filter

 

- Right Click the Server Name

- Click "Properties"

- In the "WWW Service" Master Properties, click "Edit"

- In the "ISAPI Filters" tab, click "Add"

- In the "Filter Name" type in "BadFilter"

- In the "Executable", browse to C:\IISDebugLabs\www\App\badfil.dll

- Click "Apply" and click "Ok" all teh way out.

 

2- Restart the IIS Admin service

 

- Right Click the Server Name

- Click "Restart IIS" and Click "Ok"

 

3- Attach the debugger to inetinfo.exe

a) Using the GUI:

- Click "File"

- Click "Attach to Process"

- Select "inetinfo.exe" and click "OK"

b) Using the command prompt

- Switch to the folder where the debuggers is installed

- Type in the command "windbg.exe -pn inetinfo.exe" with no quotes

 

4- In windbg command line, type in "g" with no quotation marks and press Enter

 

 

2- Run Scenario

1- Click the "AV Filter" button the first time to cause a 1st chance AV in inetinfo.exe.
 Note that the debugger breaks and diplays 
	a- Comments about the exception
b- The value of the registers
c- The module where the AV happened
d- The assembly line that caused the AV
2- In the WinDbg command line, type in "g" and press "Enter", a 2nd chance AV is now encountered 

                 

 

1- Debug Annotation

 

1) Without symbols

 

a) The faulting Stack

 

The stack backtrace of the thread that AV'd would look like:

 

0:008>kvn50

*** ERROR: Symbol file could not be found. Defaulted to export symbols for badfil.dll

*** ERROR: Symbol file could not be found. Defaulted to export symbols for w3svc.dll

# ChildEBP RetAddr Args to Child

WARNING: Stack unwind information not available. Following frames may be wrong.

00 0118fbf4 65f0354f 00000001 00004000 08000002 badfil!HttpFilterProc+0x539

01 0118fc1c 65f034bc 000003e8 00004000 00000002 w3svc!STR::Copy+0x11a

02 0118fc50 65f02cad 0118fedc 0118fedc 00e74618 w3svc!STR::Copy+0x87

03 0118fca8 65f02ac1 006a8fb8 0000019b 0118fcd4 w3svc!CLIENT_CONN::OnSessionStartup+0xa69

04 0118fcec 65f029b5 006a8fb8 0000019b 0118fedc w3svc!CLIENT_CONN::OnSessionStartup+0x87d

05 0118fd2c 65f028f8 0118fea4 0118fedc 0118feb8 w3svc!CLIENT_CONN::OnSessionStartup+0x771

06 0118fd4c 65f027b3 0118fea4 0118fedc 0118feb8 w3svc!CLIENT_CONN::OnSessionStartup+0x6b4

07 0118fec4 65f020d7 0118fedc 01093568 0118ff20 w3svc!CLIENT_CONN::OnSessionStartup+0x56f

08 0118fee4 65f01b47 00000000 00000000 00000000 w3svc!HTTP_HEADERS::Reset+0x311

09 0118ff08 65f01aa5 0118ff20 77e873b2 01093568 w3svc+0x1b47

*** ERROR: Symbol file could not be found. Defaulted to export symbols for ISATQ.DLL -

0a 0118ff4c 6d701443 01093568 0000019b 00104cd0 w3svc+0x1aa5

0b 0118ff80 6d702203 00000000 0000019b 0109356c ISATQ!AtqReadSocket+0x228

0c 77deec86 ece868ff fb6877dc 6477db19 000000a1 ISATQ!AtqWriteSocket+0x495

0d 6aec8b55 00000000 00000000 00000000 00000000 0xece868ff

 

In this stack backtrace, it is clear that the culprit is "badFil.dll". So it is enough to tell the customer to contact the dll vendor to fix it. If it is an in-house component and the customer would like help

from Microsoft, they should open an advisory case to troubleshoot the issue further.

 

b) The regiters values

 

0:008> r eax=00000000 ebx=00000003 ecx=00000000 edx=7803b012 esi=78002041 edi=00e70978

eip=015a1a3b esp=0118f5e0 ebp=0118fbf4 iopl=0 nv up ei pl zr na po nc

cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00000246

badfil!HttpFilterProc+539: 015a1a3b

66c700d307 mov word ptr [eax],0x7d3 ds:0023:00000000=????

 

c) The disassmbly code

 

0:008> u eip

badfil!HttpFilterProc+539:

015a1a3b 66c700d307 mov word ptr [eax],0x7d3

015a1a40 6a01 push 0x1

015a1a42 58 pop eax

015a1a43 c20400 ret 0x4

015a1a46 837c240401 cmp dword ptr [esp+0x4],0x1

015a1a4b 7508 jnz badfil!HttpFilterProc+0x553 (015a1a55)

015a1a4d 6a00 push 0x0

015a1a4f ff1524105a01 call dword ptr [badfil+0x1024 (015a1024)]

 

2) With symbols including public symbols of "badFil.dll"

 

First, we need to add the public symbol path(company wide symbols) as well as symbols of our dll

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in SRV*C:\LabSymbols*http://msdl.microsoft.com/download/symbols;C:\IISDebugLabs\symbols\Pub and hit Enter

b) Using the ".sympath+" command:

0:008> .sympath+ SRV*C:\LabSymbols*http://msdl.microsoft.com/download/symbols

Symbol search path is: SRV*C:\LabSymbols*http://msdl.microsoft.com/download/symbols

 

0:008> .sympath+ C :\IISDebugLabs\symbols\Pub (where public symbols of badFil.dll are located)

Symbol search path is:SRV*C:\LabSymbols*http://msdl.microsoft.com/download/symbols; C :\IISDebugLabs\symbols\Pub

 

Note: For more information on setting the symbol path for windbg, please refer to the debugger documentation for the .sympath command. This will describe

the "downstream store" feature that lets you download symbols to a symbol server of your own.

Visit also http://www.microsoft.com/whdc/ddk/debugging/symbols.mspx

 

The stack backtrace of the thread that AV'd would look like:

 

 

0:008>kvn50

# ChildEBP RetAddr Args to Child

00 0118fbf4 65f0354f 00000001 00004000 08000002 badfil!doAV1+0x9

01 0118fc1c 65f034bc 000003e8 00004000 00000002 w3svc!FILTER_LIST::NotifyFilters+0x9d

02 0118fc50 65f02cad 0118fedc 0118fedc 00e74618 w3svc!HTTP_FILTER::NotifyPreProcHeaderFilters+0x4c

03 0118fca8 65f02ac1 006a8fb8 0000019b 0118fcd4 w3svc!HTTP_REQUEST::Parse+0x380

04 0118fcec 65f029b5 006a8fb8 0000019b 0118fedc w3svc!HTTP_REQ_BASE::OnCompleteRequest+0x36

05 0118fd2c 65f028f8 0118fea4 0118fedc 0118feb8 w3svc!HTTP_REQ_BASE::UnWrapRequest+0x1f4

06 0118fd4c 65f027b3 0118fea4 0118fedc 0118feb8 w3svc!HTTP_REQ_BASE::OnFillClientReq+0x7a

07 0118fec4 65f020d7 0118fedc 01093568 0118ff20 w3svc!HTTP_REQUEST::DoWork+0x99

08 0118fee4 65f01b47 00000000 00000000 00000000 w3svc!CLIENT_CONN::DoWork+0x1aa

09 0118ff08 65f01aa5 0118ff20 77e873b2 01093568 w3svc!CreateClient+0x7b

0a 0118ff4c 6d701443 01093568 0000019b 00104cd0 w3svc!W3OnConnectEx+0x118

0b 0118ff80 6d702203 00000000 0000019b 0109356c ISATQ!AtqpProcessContext+0x23a

0c 0118ffb4 77e887dd abcdef01 00000000 00012047 ISATQ!AtqPoolThread+0x1a8

0d 0118ffec 00000000 6d7091f7 abcdef01 00000000 KERNEL32!BaseThreadStart+0x52

 

 

Now, with the public symbols lined up, we see that the last call that AV'd was to the function "doAV1" that was called in reponse to a notification

"PreProcHeader".

 

Again, In this stack backtrace, it is clear that the culprit is "badFil.dll". So it is enough to tell the customer to contact the dll vendor to fix it. If it is an in-house component and the customer would like help

from Microsoft, they should open an advisory case to troubleshoot the issue further.

 

3) with Private Symbols "badFil.dll"

 

 

Point the debugger to use the private symbols

For company wide internal symbols, use \\symbols\symbols

For the component private symbols, use C:\IISDebugLabs\Symbols\Pri

 

Verify the component symbols are linet up.

 

0:008> lmv mbadfil

start end module name

015a0000 015a6000 badfil (pdb symbols) C:\IISDebugLabs\Symbols\Pri

Loaded symbol image file: badfil.dll

Image path: C:\IISDebugLabs\www\App\badfil.dll

Timestamp: Wed May 28 01:11:01 2003 (3ED444E5)

Checksum: 0000D9A0

File version: 5.0.0.0

Product version: 5.0.0.0

File flags: 0 (Mask 3F)

File OS: 40004 NT Win32

File type: 2.0 Dll

File date: 00000000.00000000

CompanyName: Microsoft Corporation

ProductName: Internet Information Services

InternalName: badFIL.dll

OriginalFilename: badFIL.dll

ProductVersion: 5.00.0984

FileVersion: 5.00.0984

FileDescription: badFIL Filter DLL

LegalCopyright: Copyright (C) Microsoft Corp. 1981-1999

 

 

The stack backtrace of the thread that AV'd would look like:

 

0:008>kvn50

# ChildEBP RetAddr Args to Child

00 0118fbf4 65f0354f 00000001 00004000 08000002 badfil!doAV1+0x9 (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\filDBG\badfil.cpp @ 512]

01 0118fc1c 65f034bc 000003e8 00004000 00000002 w3svc!FILTER_LIST::NotifyFilters+0x9d (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\httpfilt.cxx @ 968]

02 0118fc50 65f02cad 0118fedc 0118fedc 00e74618 w3svc!HTTP_FILTER::NotifyPreProcHeaderFilters+0x4c (FPO: [Non-Fpo]) (CONV: thiscall) [.\filter.hxx @ 473]

03 0118fca8 65f02ac1 006a8fb8 0000019b 0118fcd4 w3svc!HTTP_REQUEST::Parse+0x380 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\httpreq.cxx @ 1497]

04 0118fcec 65f029b5 006a8fb8 0000019b 0118fedc w3svc!HTTP_REQ_BASE::OnCompleteRequest+0x36 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\httpio.cxx @ 1289]

05 0118fd2c 65f028f8 0118fea4 0118fedc 0118feb8 w3svc!HTTP_REQ_BASE::UnWrapRequest+0x1f4 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\httpio.cxx @ 634]

06 0118fd4c 65f027b3 0118fea4 0118fedc 0118feb8 w3svc!HTTP_REQ_BASE::OnFillClientReq+0x7a (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\httpio.cxx @ 253]

07 0118fec4 65f020d7 0118fedc 01093568 0118ff20 w3svc!HTTP_REQUEST::DoWork+0x99 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\httpreq.cxx @ 639]

08 0118fee4 65f01b47 00000000 00000000 00000000 w3svc!CLIENT_CONN::DoWork+0x1aa (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\w3\server\conn.cxx @ 781]

09 0118ff08 65f01aa5 0118ff20 77e873b2 01093568 w3svc!CreateClient+0x7b (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\w3\server\connect.cxx @ 310]

0a 0118ff4c 6d701443 01093568 0000019b 00104cd0 w3svc!W3OnConnectEx+0x118 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\w3\server\connect.cxx @ 215]

0b 0118ff80 6d702203 00000000 0000019b 0109356c ISATQ!AtqpProcessContext+0x23a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\infocomm\atq\atqsupp.cxx @ 883]

0c 0118ffb4 77e887dd abcdef01 00000000 00012047 ISATQ!AtqPoolThread+0x1a8 (FPO: [EBP 0x0118ffec] [1,4,4]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\infocomm\atq\atqsupp.cxx @ 1127]

0d 0118ffec 00000000 6d7091f7 abcdef01 00000000 KERNEL32!BaseThreadStart+0x52 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 460]

 

As mentioned above, badFil.dll AV'd while acting on the "PreProcHeader" notification, we know that because

of frame 02 (w3svc!HTTP_FILTER::NotifyPreProcHeaderFilters). It is helpful sometimes to exact information like what request

caused this filter to AV.

In this special case, using the debugger "kp" command will show the exact request.

 

 

0:008> kpn #

ChildEBP RetAddr

00 0118fbf4 65f0354f badfil!doAV1(int Variation = 0x65f0354f)+0x9 [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\filDBG\badfil.cpp @ 512]

01 0118fc1c 65f034bc w3svc!FILTER_LIST::NotifyFilters(HTTP_FILTER* pFilter = 000003e8 , unsigned long NotificationType = 0x4000, _HTTP_FILTER_CONTEXT* phfc = 00000002 , void* NotificationData = 0118fc3c , int* pfFinished = 0118fedc , int fNotifyAll = 0)+0x9d [D:\nt\private\inet\iis\svcs\w3\server\httpfilt.cxx @ 968]

02 0118fc50 65f02cad w3svc!HTTP_FILTER::NotifyPreProcHeaderFilters(int* pfFinished = 0118fedc )+0x4c [.\filter.hxx @ 473]

03 0118fca8 65f02ac1 w3svc!HTTP_REQUEST::Parse(char* pchRequest = 006a8fb8 "GET /IISDebugLabs/Site/fil_av1.htm?sf=av1_v1 HTTP/1.1???", unsigned long cbData = 0x19b, unsigned long* pcbExtraData = 0118fcd4 , int* pfHandled = 0118fce8 , int* pfFinished = 0118fedc )+0x380 [D:\nt\private\inet\iis\svcs\w3\server\httpreq.cxx @ 1497]

04 0118fcec 65f029b5 w3svc!HTTP_REQ_BASE::OnCompleteRequest(char* pchRequest = 006a8fb8 "GET /IISDebugLabs/Site/fil_av1.htm?sf=av1_v1 HTTP/1.1???", unsigned long cbData = 0x19b, int* pfFinished = 0118fedc , int* pfContinueProcessingRequest = 0118feb8 )+0x36 [D:\nt\private\inet\iis\svcs\w3\server\httpio.cxx @ 1289]

05 0118fd2c 65f028f8 w3svc!HTTP_REQ_BASE::UnWrapRequest(int* pfCompleteRequest = 0118fea4 , int* pfFinished = 0118fedc , int* pfContinueProcessingRequest = 0118feb8 )+0x1f4 [D:\nt\private\inet\iis\svcs\w3\server\httpio.cxx @ 634]

06 0118fd4c 65f027b3 w3svc!HTTP_REQ_BASE::OnFillClientReq(int* pfCompleteRequest = 0118fea4 , int* pfFinished = 0118fedc , int* pfContinueProcessingRequest = 0118feb8 )+0x7a [D:\nt\private\inet\iis\svcs\w3\server\httpio.cxx @ 253]

07 0118fec4 65f020d7 w3svc!HTTP_REQUEST::DoWork(int* pfFinished = 0118fedc )+0x99 [D:\nt\private\inet\iis\svcs\w3\server\httpreq.cxx @ 639]

08 0118fee4 65f01b47 w3svc!CLIENT_CONN::DoWork(unsigned long BytesWritten = 0, unsigned long CompletionStatus = 0, int fIOCompletion = 0)+0x1aa [D:\nt\private\inet\iis\svcs\w3\server\conn.cxx @ 781]

09 0118ff08 65f01aa5 w3svc!CreateClient(_CLIENT_CONN_PARAMS* ClientParam = 0118ff20 )+0x7b [D:\nt\private\inet\iis\svcs\w3\server\connect.cxx @ 310] 0a 0118ff4c 6d701443 w3svc!W3OnConnectEx(void* patqContext = 01093568 , unsigned long cbWritten = 0x19b, unsigned long err = 0x104cd0, _OVERLAPPED* lpo = 00e73bf0 )+0x118 [D:\nt\private\inet\iis\svcs\w3\server\connect.cxx @ 215]

0b 0118ff80 6d702203 ISATQ!AtqpProcessContext(ATQ_CONTEXT* pAtqContext = 00000000 , unsigned long cbWritten = 0x19b, _OVERLAPPED* lpo = 0109356c , int fRet = 1)+0x23a [D:\nt\private\inet\iis\svcs\infocomm\atq\atqsupp.cxx @ 883]

0c 0118ffb4 77e887dd ISATQ!AtqPoolThread(unsigned long* param = abcdef01 )+0x1a8 [D:\nt\private\inet\iis\svcs\infocomm\atq\atqsupp.cxx @ 1127]

0d 0118ffec 00000000 KERNEL32!BaseThreadStart(* lpStartAddress = 6d7091f7 , void* lpParameter = abcdef01 )+0x52 [D:\nt\private\windows\base\client\support.c @ 460]

 

 

Notice in frame OO the line of code as well as the name and location of the source code. Going back to the source code in badFil.cpp:

 

Line 512 looks like

 

DWORD doAV1(int Variation)

{

if (1 == Variation)

{

SYSTEMTIME* pSysTime = NULL;

pSysTime->wYear = 2003; // av right here. --Line 512:

}

return 1;

}

 

 


- Microsoft Confidential -