Debug Labs


Access Violation in Component Code

This lab will demonstrate an Access Violation in a component. In this scenarion, an AV in the COM Component  "badCom.dll"

 

1- Lab Setup
	

Note: This lab assumes that you have already installed the necessary lab files to run it.

1- Verify the site is running in medium protection

2- Verify that the process "dllhost.exe" mapped to the out of process pooled application is running and note down its PID

To find the PID of the desired dllhost, do the follwing:

- Open Component Services

- Expand "Component Services" Node

- Expand "Computers" Node

- Expand "My Computer" Node

- Select "COM+ Applications"

- Click "Status View" icon and notice the PID column on the right hand pane

3- Attach windbg to the dllhost process with the PID found in step 2

a) Using the GUI:

- Click "File"

- Click "Attach to Process"

- Select "dllhost.exe" with the PID found in step 2 and click "OK"

b) Using the command prompt:

- Switch to thr folder where the debugging tools for windows is installed

- Type in "windbg -p <PID>" and hit Enter

 

5- In windbg command line, type in "g" with no quotation marks and press Enter

 

 

2- Run Scenario

1- Click the "Invoke Component" button to load the COM Component in the dllhost process memory space. You should see the COM Component "badCom.dll"among the list of modules loaded in dllhost.exe
2- Click the "AV Component " button the first time to cause a 1st chance AV in dllhost.exe.
 Note that the debugger breaks and diplays 
	a- Comments about the exception
b- The value of the registers
c- The module where the AV happened
d- The assembly line that caused the AV

					

                                                                           

 

1- Debug Annotation

 

1) Without symbols

 

a) The faulting Stack

 

The stack backtrace of the thread that AV'd would look like:

 

0:021> kvn

# ChildEBP RetAddr Args to Child

WARNING: Stack unwind information not available. Following frames may be wrong.

00 01e4f0f4 020328f0 00000001 76613d73 31765f31 badcom!DllUnregisterServer+0xe76

*** ERROR: Symbol file could not be found. Defaulted to export symbols for OLEAUT32.dll -

01 01e4f124 779d7b9d 00408910 00408a70 00000000 badcom!DllUnregisterServer+0xb73

02 01e4f140 77a22c42 00408910 00000024 00000004 OLEAUT32!DispCallFunc+0x15d

03 01e4f1d0 020325d0 000c0cdc 00408910 00000003 OLEAUT32!ClearCustData+0x588

*** ERROR: Symbol file could not be found. Defaulted to export symbols for -

04 01e4f1f8 6b61dff3 00408910 00000003 6b659388 badcom!DllUnregisterServer+0x853

05 01e4f274 6b62171a 004085f8 00408910 00000003 vbscript!DllRegisterServer+0x844c

06 01e4f954 ffffffff 0040b150 00000000 004001a0 vbscript!DllRegisterServer+0xbb73

07 77f80000 00000003 00000004 0000ffff 000000b8 0xffffffff

 

In this stack backtrace, it is clear that the culprit is "badCom.dll". So it is enough to tell the customer to contact the dll vendor to fix it. If it is an in-house component and the customer would like help

from Microsoft, they should open an advisory case to troubleshoot the issue further.

 

b) The regiters values

 

 

0:021> r

eax=00000000 ebx=00000003 ecx=00000000 edx=01e4f131 esi=78002041 edi=0040ab18

eip=02032bf3 esp=01e4f0f4 ebp=01e4f0f4 iopl=0 nv up ei pl zr na po nc

cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000 efl=00000246

badcom!DllUnregisterServer+e76:

02032bf3 66c700d307 mov word ptr [eax],0x7d3 ds:0023:00000000=????

 

c) The disassmbly code

 

0:021> u eip

badcom!DllUnregisterServer+e76:

02032bf3 66c700d307 mov word ptr [eax],0x7d3

02032bf8 33c0 xor eax,eax

02032bfa 5d pop ebp

02032bfb c20400 ret 0x4

02032bfe 55 push ebp

02032bff 8bec mov ebp,esp

02032c01 837d0801 cmp dword ptr [ebp+0x8],0x1

02032c05 7508 jnz badcom!DllUnregisterServer+0xe92 (02032c0f)

 

2) With symbols including public symbols of "badCom.dll"

 

First, we need to add the symbol path(company wide symbols) as well as symbols of our dll

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in SRV*C:\LabSymbols*\\symbols\symbols;C:\IISDebugLabs\symbols\Pub and hit Enter

b) Using the ".sympath+" command:

.sympath+ SRV*C:\LabSymbols*\\symbols\symbols;C :\IISDebugLabs\symbols\Pub

The stack backtrace of the thread that AV'd would look like:

 

 

 

0:021> kvn50

# ChildEBP RetAddr Args to Child

00 01e4f0f4 020328f0 00000001 76613d73 31765f31 badcom!doAV1+0xb

01 01e4f124 779d7b9d 00408910 00408a70 00000000 badcom!CScenarios::processSc+0xa6

02 01e4f140 77a22c42 00408910 00000024 00000004 OLEAUT32!tPushValJmpTab+0xf5 [D:\OA\src\dispatch\WIN32\i386\invoke.asm @ 340]

03 01e4f1d0 020325d0 000c0cdc 00408910 00000003 OLEAUT32!CTypeInfo2::Invoke+0x244 (CONV: stdcall) [D:\OA\src\typelib2\tiinvoke.cpp @ 629]

04 01e4f1f8 6b61dff3 00408910 00000003 6b659388 badcom!ATL::IDispatchImpl::Invoke+0x41

05 01e4f274 6b62171a 004085f8 00408910 00000003 vbscript!CatchIDispatchInvoke+0x9e (CONV: stdcall) [E:\vbscript\src\core\callout.h @ 111]

06 01e4f384 6b604038 0009001f 0000023f 004084f0 vbscript!InvokeDispatch+0x330 (FPO: [EBP 0x6b659388] [7,60,4]) (CONV: stdcall) [E:\vbscript\src\vbscript\var.cpp @ 1211]

07 01e4f3c8 6b603156 004084f0 00408910 00000003 vbscript!GetDispatchDispID+0x1cc (FPO: [EBP 0x01e4f954] [4,3,4]) (CONV: stdcall) [E:\vbscript\src\vbscript\var.cpp @ 972]

08 01e4f998 6b601ef7 00000009 01e4fa8c 00000000 vbscript!CScriptRuntime::Run+0x3b8d (CONV: thiscall) [E:\vbscript\src\vbscript\interp.cpp @ 3347]

09 01e4f9c8 6b601c5c 00000000 00000000 00000000 vbscript!CScriptRuntime::Init+0x2e1 (FPO: [EBP 0x6b658a10] [6,0,4]) (CONV: thiscall) [E:\vbscript\src\vbscript\interp.cpp @ 904]

0a 01e4fb08 6b606fc5 004084f0 6b606ef1 00408be8 vbscript!CScriptEntryPoint::Call+0x76 (FPO: [5,58,2]) (CONV: thiscall) [E:\vbscript\src\vbscript\entrypnt.cpp @ 40]

0b 01e4fb54 749f2e70 01911e90 6b610c9a 00000000 vbscript!CSession::GetGlobalEntryPoint+0x25 (FPO: [3,0,0]) (CONV: thiscall) [E:\vbscript\src\core\wrapper.cpp @ 485]

0c 01e4fb6c 749f3af0 01912060 01912060 012d1e90 asp!CResponse::Release+0x18 (FPO: [1,0,1]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\response.cpp @ 1716]

0d 01e4fb9c 74a22f5b 00000000 74a32c70 77e873a1 asp!CActiveScriptEngine::TryCall+0x15 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\scrptmgr.cpp @ 2040]

0e 01e4fbe0 749f3a72 00000000 01e4fc28 749f593d asp!CActiveScriptEngine::Call+0x10c (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\scrptmgr.cpp @ 2005]

0f 01e4fbec 749f593d 01e4fc4c 00000000 00000000 asp!CallScriptFunctionOfEngine+0x48 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 1394]

10 01e4fc28 749f54c8 01a11fa0 01591e90 01e4fc94 asp!ExecuteRequest+0x19e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 326]

11 01e4fc70 749fcf9b 01a11fa0 01591e90 01e4fc94 asp!Execute+0x17e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 190]

12 01e4fcc0 749fcc37 00000000 000be330 01e4fd58 asp!CHitObj::ViperAsyncCallback+0x452 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\hitobj.cpp @ 2487]

13 01e4fcd8 787c214a 00000000 77acd455 01b51e90 asp!CViperAsyncRequest::OnCall+0x3f (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\viperint.cpp @ 194]

14 01e4fce0 77acd455 01b51e90 77aa71a0 00000000 COMSVCS!STAActivityWorkHelper+0xa (FPO: [1,0,0]) (CONV: stdcall) [.\stathred.cpp @ 1321]

15 01e4fd24 77acd318 000751d8 00085860 787c2140 ole32!EnterForCallback+0x6a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1758]

16 01e4fe50 77acd1dd 00085860 787c2140 01b51e90 ole32!SwitchForCallback+0x12b (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1643]

17 01e4fe78 77aac61a 00085860 787c2140 01b51e90 ole32!PerformCallback+0x50 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1558]

18 01e4fed4 77ad1185 000757c0 787c2140 01b51e90 ole32!CObjectContext::InternalContextCallback+0xf5 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\context.cxx @ 3942]

19 01e4fef4 787beff3 000757c0 787c2140 01b51e90 ole32!CObjectContext::DoCallback+0x1a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\context.cxx @ 3798]

1a 01e4ff24 787c11ad 00010007 00000000 01e4ffec COMSVCS!STAActivityWork::DoWork+0x73 (FPO: [0,4,2]) (CONV: thiscall) [..\inc\STAThred.h @ 340]

1b 01e4ffb4 77e887dd 0088f364 00010007 00000000 COMSVCS!STAThread::STAThreadWorker+0x465 (FPO: [EBP 0x01e4ffec] [1,31,4]) (CONV: stdcall) [.\stathred.cpp @ 628]

1c 01e4ffec 00000000 787c0d48 0088f364 00000000 KERNEL32!BaseThreadStart+0x52 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 460]  

 

 

 

Now, with the public symbols line up, we see that the last call that AV'd was to the function "doAV1" that was called from the class "CScenarios" by the method

"ProcessSc()"

 

Again, In this stack backtrace, it is clear that the culprit is "badCom.dll". So it is enough to tell the customer to contact the dll vendor to fix it. If it is an in-house component and the customer would like help

from Microsoft, they should open an advisory case to troubleshoot the issue further.

 

Advanced Technique

 

Notice also that the call to our COM component was invoked by "wam.dll" (Web Application Manager). It is the component responsible for handing

over any request that needs to be processed by an ISAPI extension.

 

Maybe the most interesting finding in theis dump analysis -in addition determining the dll cause of the crash- is to determine the ASP page

the faulting thread was running. There are many ways to find that, let find the running ASP page by dumping the ASP template class.

In frame 11 The first parameter passed to "asp!execute()" is a pointer to the CTempate class.

Note: You could use the "kpn" command instead of "kvn" to see the parameters mane/value passed.

 

To dump the class CTemplate, do the following:

 

0:021> dt CTemplate 0x01a11fa0 -r

+0x000 __VFN_table : 0x749f6a40

+0x004 m_pLinkNext : 0x74a31978

+0x000 __VFN_table : 0x749f2fac

+0x004 m_pLinkNext : 0x01a11fa0

.......

.......

+0x098 m_rgpFilemaps : 0x01c72be0 -> 0x01a51e90

+0x000 m_szPathInfo : 0x01c72c08 "/IISDebugLabs/app/initcom.asp"

+0x004 m_szPathTranslated : 0x01c72c30 "C:\IISDEBUGLABS\WWW\APP\INITCOM.ASP"

+0x008 m_pfilemapParent : (null)

......

......

 

This shows that the faulting thread was running the ASP page: "C:\IISDEBUGLABS\WWW\APP\INITCOM.ASP"

 

To verify that the symbols are lined up properly for theh componsnt, issue this command.

 

 

 

0:021> lmv mbadcom

start end module name

02030000 02038000 badcom (pdb symbols) c:\iisdebuglabs\symbols\pub\badcom.pdb

Loaded symbol image file: badcom.dll

Image path: c:\IISDEB~1\www\app\badcom.dll

Timestamp: Tue Jun 10 22:45:51 2003 (3EE697DF) Checksum: 0000F229

File version: 1.0.0.1

Product version: 1.0.0.1

File flags: 0 (Mask 3F)

File OS: 4 Unknown Win32

File type: 2.0 Dll

File date: 00000000.00000000

CompanyName:

ProductName: badCOM

Module InternalName: badCOM

OriginalFilename: badCOM.DLL

ProductVersion: 1, 0, 0, 1

FileVersion: 1, 0, 0, 1

FileDescription: badCOM Module

LegalCopyright: Copyright 2003

 

 

3) with Private Symbols "badCom.dll"

 

 

First, we need to add the private symbol path(company wide symbols) as well as private symbols of our exe

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in SRV*C:\LabSymbols*\\symbols\symbols;C:\IISDebugLabs\symbols\Pri and hit Enter

b) Using the ".sympath+" command:

.sympath+ SRV*C:\LabSymbols*\\symbols\symbols;C :\IISDebugLabs\symbols\Pri

 

The stack backtrace of the thread that AV'd would look like:

 

0:021> kvn50

# ChildEBP RetAddr Args to Child

00 01e4f0f4 020328f0 00000001 76613d73 31765f31 badcom!doAV1+0xb (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\comDBG\scenarios.cpp @ 208]

01 01e4f124 779d7b9d 00408910 00408a70 00000000 badcom!CScenarios::processSc+0xa6 (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\comDBG\scenarios.cpp @ 199]

02 01e4f140 77a22c42 00408910 00000024 00000004 OLEAUT32!tPushValJmpTab+0xf5 [D:\OA\src\dispatch\WIN32\i386\invoke.asm @ 340]

03 01e4f1d0 020325d0 000c0cdc 00408910 00000003 OLEAUT32!CTypeInfo2::Invoke+0x244 (CONV: stdcall) [D:\OA\src\typelib2\tiinvoke.cpp @ 629]

04 01e4f1f8 6b61dff3 00408910 00000003 6b659388 badcom!ATL::IDispatchImpl::Invoke+0x41

05 01e4f274 6b62171a 004085f8 00408910 00000003 vbscript!CatchIDispatchInvoke+0x9e (CONV: stdcall) [E:\vbscript\src\core\callout.h @ 111]

06 01e4f384 6b604038 0009001f 0000023f 004084f0 vbscript!InvokeDispatch+0x330 (FPO: [EBP 0x6b659388] [7,60,4]) (CONV: stdcall) [E:\vbscript\src\vbscript\var.cpp @ 1211]

07 01e4f3c8 6b603156 004084f0 00408910 00000003 vbscript!GetDispatchDispID+0x1cc (FPO: [EBP 0x01e4f954] [4,3,4]) (CONV: stdcall) [E:\vbscript\src\vbscript\var.cpp @ 972]

08 01e4f998 6b601ef7 00000009 01e4fa8c 00000000 vbscript!CScriptRuntime::Run+0x3b8d (CONV: thiscall) [E:\vbscript\src\vbscript\interp.cpp @ 3347]

09 01e4f9c8 6b601c5c 00000000 00000000 00000000 vbscript!CScriptRuntime::Init+0x2e1 (FPO: [EBP 0x6b658a10] [6,0,4]) (CONV: thiscall) [E:\vbscript\src\vbscript\interp.cpp @ 904]

0a 01e4fb08 6b606fc5 004084f0 6b606ef1 00408be8 vbscript!CScriptEntryPoint::Call+0x76 (FPO: [5,58,2]) (CONV: thiscall) [E:\vbscript\src\vbscript\entrypnt.cpp @ 40]

0b 01e4fb54 749f2e70 01911e90 6b610c9a 00000000 vbscript!CSession::GetGlobalEntryPoint+0x25 (FPO: [3,0,0]) (CONV: thiscall) [E:\vbscript\src\core\wrapper.cpp @ 485]

0c 01e4fb6c 749f3af0 01912060 01912060 012d1e90 asp!CResponse::Release+0x18 (FPO: [1,0,1]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\response.cpp @ 1716]

0d 01e4fb9c 74a22f5b 00000000 74a32c70 77e873a1 asp!CActiveScriptEngine::TryCall+0x15 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\scrptmgr.cpp @ 2040]

0e 01e4fbe0 749f3a72 00000000 01e4fc28 749f593d asp!CActiveScriptEngine::Call+0x10c (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\scrptmgr.cpp @ 2005]

0f 01e4fbec 749f593d 01e4fc4c 00000000 00000000 asp!CallScriptFunctionOfEngine+0x48 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 1394]

10 01e4fc28 749f54c8 01a11fa0 01591e90 01e4fc94 asp!ExecuteRequest+0x19e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 326]

11 01e4fc70 749fcf9b 01a11fa0 01591e90 01e4fc94 asp!Execute+0x17e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 190]

12 01e4fcc0 749fcc37 00000000 000be330 01e4fd58 asp!CHitObj::ViperAsyncCallback+0x452 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\hitobj.cpp @ 2487]

13 01e4fcd8 787c214a 00000000 77acd455 01b51e90 asp!CViperAsyncRequest::OnCall+0x3f (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\viperint.cpp @ 194]

14 01e4fce0 77acd455 01b51e90 77aa71a0 00000000 COMSVCS!STAActivityWorkHelper+0xa (FPO: [1,0,0]) (CONV: stdcall) [.\stathred.cpp @ 1321]

15 01e4fd24 77acd318 000751d8 00085860 787c2140 ole32!EnterForCallback+0x6a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1758]

16 01e4fe50 77acd1dd 00085860 787c2140 01b51e90 ole32!SwitchForCallback+0x12b (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1643]

17 01e4fe78 77aac61a 00085860 787c2140 01b51e90 ole32!PerformCallback+0x50 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1558]

18 01e4fed4 77ad1185 000757c0 787c2140 01b51e90 ole32!CObjectContext::InternalContextCallback+0xf5 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\context.cxx @ 3942]

19 01e4fef4 787beff3 000757c0 787c2140 01b51e90 ole32!CObjectContext::DoCallback+0x1a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\context.cxx @ 3798]

1a 01e4ff24 787c11ad 00010007 00000000 01e4ffec COMSVCS!STAActivityWork::DoWork+0x73 (FPO: [0,4,2]) (CONV: thiscall) [..\inc\STAThred.h @ 340]

1b 01e4ffb4 77e887dd 0088f364 00010007 00000000 COMSVCS!STAThread::STAThreadWorker+0x465 (FPO: [EBP 0x01e4ffec] [1,31,4]) (CONV: stdcall) [.\stathred.cpp @ 628]

1c 01e4ffec 00000000 787c0d48 0088f364 00000000 KERNEL32!BaseThreadStart+0x52 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 460]  

 

 

Notice in frame OO and 01 the line of the code as well as the name and location of the source code. Going back to the source code in scenarios .cpp:

 

Line 199 looks like

 

 

STDMETHODIMP CScenarios::ProcessSc(VARIANT *pVar) 

{

.....

.....

if (!strncmp(szS, "av1", 3))

{

return doAV1(lpEcb, 1); //Line 199

}

.....

.....

}

 

Line 208 looks like

 

DWORD doAV1(int Variation)

{

if (1 == Variation)

{

SYSTEMTIME* pSysTime = NULL;

pSysTime->wYear = 2003; // Line 208: av right here.

}

return S_OK;

}

 

 

 


- Microsoft Confidential -