Debug Labs


Handle Leak in Component

This lab will demonstrate a handle leak. In this scenario, a handle leak in the COM Component  "badCom.dll"

 

1- Lab Setup
	

Note: This lab assumes that you have already installed the necessary lab files to run it.

1- Verify the site is running in medium protection

2- Verify that the process "dllhost.exe" mapped to the out of process pooled application is running and note down its PID

To find the PID of the desired dllhost, do the follwing:

- Open Component Services

- Expand "Component Services" Node

- Expand "Computers" Node

- Expand "My Computer" Node

- Select "COM+ Applications"

- Click "Status View" icon and notice the PID column on the right hand pane

 

 

3- Run Performace Monitor to monitor the process handle count.

a- Start Performance Monitor

b- Click the Add button

c- Under "Performance Object", Select "Process"

d- Under "Select Counters from List", select "Handle Count"

e- Under "Select Instances from List", select "dllhost#1"

 

 

 

4- Attach windbg to the dllhost process with the PID found in step 2

a) Using the GUI:

- Click "File"

- Click "Attach to Process"

- Select "dllhost.exe" with the PID found in step 2 and click "OK"

b) Using the command prompt:

- Switch to thr folder where the debugging tools for windows is installed

- Type in "windbg -p <PID>" and hit Enter

 

5- In windbg command line, type in "g" with no quotation marks and press Enter

 

 

2- Run Scenario

1- Click the "Invoke COM" button to load the COM Component the dllhost process memory space. You should see the COM Component "badCom.dll"among the list of modules loaded in dllhost.exe
2- Click the "Handle Leak" button many times to cause a handle leak.
 Note that in Performance Monitor, the "handle count" counter increases and never decreases. This is an indication that there is a handle leak. 
 3- Back in Windbg, press "Ctrl_Break". 

					

                                                                           

 

1- Debug Annotation

 

Once you start Performance Monitor to monitor handles count, note how many handles the process "dllhost#1" has.

In our scenario, it shows 434 handles. Verify this count in windbg right after you attach by issuing this command:

 

 

0:031> !handle

Handle 4

Type Section

Handle 8

Type Event

Handle c

Type Event

.......

 

 

357 Handles

Type Count

Event 173

Section 9

File 68

Port 14

Directory 3

Mutant 2

WindowStation 2

Semaphore 15

Key 19

Token 2

Thread 43

Desktop 1

IoCompletion 5

Timer 1

 

 

In windbg, type in "g", and back to Internet Explorer, keep pressing "Handle Leak" button. (or just paste the URL that causes the leak in a new

IE window and keep pressing F5). This is the URL "http://localhost/iisdebuglabs/site/initcom.asp?s=handleleak1_v1"

 

Now performance Monitor shows 435 handles open. Back to Windbg, press "Ctrl_break" and run the "!handle" command again.

Here is the output

In Windbg

 

0:031> !handle

Handle 4

Type Section

Handle 8

Type Event

Handle c

Type Event

.......

 

435 Handles

Type Count

Event 182

Section 9

File 68

Port 14

Directory 3

Mutant 2

WindowStation 2

Semaphore 16

Key 86

Token 2

Thread 44

Desktop 1

IoCompletion 5

Timer 1

 

 

 

Based on this, we can see that the handle that we are leaking is a handle to a registry key. the key handle count jumped from 19 to 86.

Issue again the "!handle 0 4" to include the handle type information in the display

 

0:031> !handle 0 4

.........

.........

.........

 

Handle 6a0

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

Handle 6a4

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

Handle 6a8

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

Handle 6ac

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

Handle 6b0

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

Handle 6b4

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

Handle 6b8

Name \REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions

.............

.............

.............

 

 

So this shows that the handle we are leaking is to he registry key "\REGISTRY\MACHINE\SYSTEM\ControlSet001\Control\ProductOptions"

Now let try to see who is leaking the handle to the registry key mentioned above. To do this, we are going to set a break point.

 

In windbg command line, type in this command:

 

0:003> bp ADVAPI32!RegOpenKeyExA

 

This will set a break point in RegOpenKeyExA. When the break point is hit, the debugger breaks.

 

In Windbg command line, type in "g"

In the browser, click  the "Leak Handle" button, and notice that the debugger breaks.

Issue a "kvn50" in the debugger's command line.

Having the private symbols lined up, here is how the stack would look like:

 

 

0:020> kvn50

# ChildEBP RetAddr Args to Child

00 01e4f0c8 02032e94 80000002 02031450 00000000 ADVAPI32!RegOpenKeyExA (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\screg\winreg\client\regkey.c @ 875]

01 01e4f0e4 02032a19 00000001 61683d73 656c646e badcom!doSlow1+0xb (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\comDBG\scenarios.cpp @ 385]

02 01e4f124 779d7b9d 0040bfb8 00408a08 00000000 badcom!CScenarios::processSc+0x1cf (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\comDBG\scenarios.cpp @ 158]

03 01e4f140 77a22c42 0040bfb8 00000024 00000004 OLEAUT32!tPushValJmpTab+0xf5 [D:\OA\src\dispatch\WIN32\i386\invoke.asm @ 340]

04 01e4f1d0 020325d0 000bf444 0040bfb8 00000003 OLEAUT32!CTypeInfo2::Invoke+0x244 (CONV: stdcall) [D:\OA\src\typelib2\tiinvoke.cpp @ 629]

05 01e4f1f8 6b61dff3 0040bfb8 00000003 6b659388 badcom!ATL::IDispatchImpl::Invoke+0x41 (CONV: stdcall) [c:\nt\public\sdk\inc\atl30\atlcom.h @ 4093]

06 01e4f274 6b62171a 00408590 0040bfb8 00000003 vbscript!CatchIDispatchInvoke+0x9e (CONV: stdcall) [E:\vbscript\src\core\callout.h @ 111]

07 01e4f384 6b604038 0009001f 0000023f 00406060 vbscript!InvokeDispatch+0x330 (FPO: [EBP 0x6b659388] [7,60,4]) (CONV: stdcall) [E:\vbscript\src\vbscript\var.cpp @ 1211]

08 01e4f3c8 6b603156 00406060 0040bfb8 00000003 vbscript!GetDispatchDispID+0x1cc (FPO: [EBP 0x01e4f954] [4,3,4]) (CONV: stdcall) [E:\vbscript\src\vbscript\var.cpp @ 972]

09 01e4f998 6b601ef7 00000009 01e4fa8c 00000000 vbscript!CScriptRuntime::Run+0x3b8d (CONV: thiscall) [E:\vbscript\src\vbscript\interp.cpp @ 3347]

0a 01e4f9c8 6b601c5c 00000000 00000000 00000000 vbscript!CScriptRuntime::Init+0x2e1 (FPO: [EBP 0x6b658a10] [6,0,4]) (CONV: thiscall) [E:\vbscript\src\vbscript\interp.cpp @ 904]

0b 01e4fb08 6b606fc5 00406060 6b606ef1 0040d240 vbscript!CScriptEntryPoint::Call+0x76 (FPO: [5,58,2]) (CONV: thiscall) [E:\vbscript\src\vbscript\entrypnt.cpp @ 40]

0c 01e4fb54 749f2e70 017d1e90 6b610c9a 00000000 vbscript!CSession::GetGlobalEntryPoint+0x25 (FPO: [3,0,0]) (CONV: thiscall) [E:\vbscript\src\core\wrapper.cpp @ 485]

0d 01e4fb6c 749f3af0 017d2060 017d2060 01191e90 asp!CResponse::Release+0x18 (FPO: [1,0,1]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\response.cpp @ 1716]

0e 01e4fb9c 74a22f5b 00000000 74a32c70 77e873a1 asp!CActiveScriptEngine::TryCall+0x15 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\scrptmgr.cpp @ 2040]

0f 01e4fbe0 749f3a72 00000000 01e4fc28 749f593d asp!CActiveScriptEngine::Call+0x10c (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\scrptmgr.cpp @ 2005]

10 01e4fbec 749f593d 01e4fc4c 00000000 00000000 asp!CallScriptFunctionOfEngine+0x48 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 1394]

11 01e4fc28 749f54c8 018d1fa0 01451e90 01e4fc94 asp!ExecuteRequest+0x19e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 326]

12 01e4fc70 749fcf9b 018d1fa0 01451e90 01e4fc94 asp!Execute+0x17e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\exec.cpp @ 190]

13 01e4fcc0 749fcc37 00000000 000bca88 01e4fd58 asp!CHitObj::ViperAsyncCallback+0x452 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\cmp\asp\hitobj.cpp @ 2487]

14 01e4fcd8 787c214a 00000000 77acd455 01a11e90 asp!CViperAsyncRequest::OnCall+0x3f (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\cmp\asp\viperint.cpp @ 194]

15 01e4fce0 77acd455 01a11e90 77aa71a0 00000000 COMSVCS!STAActivityWorkHelper+0xa (FPO: [1,0,0]) (CONV: stdcall) [.\stathred.cpp @ 1321]

16 01e4fd24 77acd318 00075130 00085860 787c2140 ole32!EnterForCallback+0x6a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1758]

17 01e4fe50 77acd1dd 00085860 787c2140 01a11e90 ole32!SwitchForCallback+0x12b (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1643]

18 01e4fe78 77aac61a 00085860 787c2140 01a11e90 ole32!PerformCallback+0x50 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\crossctx.cxx @ 1558]

19 01e4fed4 77ad1185 000757c0 787c2140 01a11e90 ole32!CObjectContext::InternalContextCallback+0xf5 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\context.cxx @ 3942]

1a 01e4fef4 787beff3 000757c0 787c2140 01a11e90 ole32!CObjectContext::DoCallback+0x1a (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\context.cxx @ 3798]

1b 01e4ff24 787c11ad 00010007 00000000 01e4ffec COMSVCS!STAActivityWork::DoWork+0x73 (FPO: [0,4,2]) (CONV: thiscall) [..\inc\STAThred.h @ 340]

1c 01e4ffb4 77e887dd 0088f364 00010007 00000000 COMSVCS!STAThread::STAThreadWorker+0x465 (FPO: [EBP 0x01e4ffec] [1,31,4]) (CONV: stdcall) [.\stathred.cpp @ 628]

1d 01e4ffec 00000000 787c0d48 0088f364 00000000 KERNEL32!BaseThreadStart+0x52 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 460]

 

 

 

Let see what registry we are trying to open here:

The 2nd parameter of (ADVAPI32!RegOpenKeyExA ) is actually a pointer to a null teminated string that hold the registry key name.

Dumping that address shows the following:

 

 

0:020> da 02031450

02031450 "SYSTEM\CurrentControlSet\Control"

02031470 "\ProductOptions"

 

So the handle to this key is actually what we are leaking. From the stack backtrace, you could see that (ADVAPI32!RegOpenKeyExA ) is called from (badCom!doHandleLeak1). This is enough to tell teh customer that the

component "badCom.dll" is the one causing the handle leak.

 

Here is how line 385 looks like in "Scenarios.cpp"

 

DWORD doHandleLeak1( int Variation)

{

if (1 == Variation)

{

HKEY hKey;

LONG lRet;

 

lRet = RegOpenKeyEx( HKEY_LOCAL_MACHINE, "SYSTEM\\CurrentControlSet\\Control\\ProductOptions", 0, KEY_QUERY_VALUE, &hKey ); //line 337

}

return S_OK;

}

 


- Microsoft Confidential -