Debug Labs


Heap Corruption/Buffer Overrun

This lab will demonstrate a heap corruption in a filter caused by a buffer overwrite. 

 

1- Lab Setup
	

Note: This lab assumes that you have already installed the necessary lab files to run it.

1- Verify the site is running in medium protection

2- Verify that the process "dllhost.exe" mapped to the out of process pooled application is running and note down its PID

To find the PID of the desired dllhost, do the follwing:

- Open Component Services

- Expand "Component Services" Node

- Expand "Computers" Node

- Expand "My Computer" Node

- Select "COM+ Applications"

- Click "Status View" icon and notice the PID column on the right hand pane

 

3- Attach windbg to the dllhost process with the PID found in step 2

a) Using the GUI:

- Click "File"

- Click "Attach to Process"

- Select "dllhost.exe" with the PID found in step 2 and click "OK"

b) Using the command prompt:

- Switch to thr folder where the debugging tools for windows is installed

- Type in "windbg -p <PID>" and hit Enter

4- In windbg command line, type in "g" with no quotation marks and press Enter

 

2- Run Scenario

1- Click the "Corrupt Heap" button 2 or 3 times.
2- Open a new instance of Internet Explorer, Copy and paste the following URL to the address bar of the browser
(http://localhost/iisdebuglabs/app/badext.dll)
3) Keep pressing  "F5" until the debugger breaks
 Note that the debugger breaks and diplays 
	a- Comments about the exception
b- The value of the registers
c- The module where the AV happened
d- The assembly line that caused the AV
2- In the WinDbg command line, type in "g" and press "Enter", a 2nd chance AV is now encountered 

                 

 

1- Debug Annotation

Definition:

 

Heap Corruption occurs when a thread allocates a block of heap memory of a given size and then writes to memory

addresses beyond the requested size of the heap block. Another common cause of heap corruption is writing to block

of memory that has been freed or freeing a block of memory already freed.

 

In this scenario, we will try to demonstrate heap corruption caused by overwriting to a heap block beyond the requested

size (Buffer Overwrite) 

 

 

1) Without Page Heap Enabled.

 

The faulting Stack

 

First, we need to add the Private symbol path(company wide symbols) as well as symbols of our Filter

a) Using the GUI:

- Click "File"

- Click "Symbol File Path"

- Type in SRV*C:\LabSymbols*\\symbols\symbols;C:\IISDebugLabs\symbols\Pri and hit Enter

b) Using the ".sympath+" command:

0:008> .sympath+ SRV*C:\LabSymbols*\\symbols\symbols

Symbol search path is: SRV*C:\LabSymbols*\\symbols\symbols

 

0:008> .sympath+ C :\IISDebugLabs\symbols\Pri (where private symbols of badFil.dll are located)

Symbol search path is:SRV*C:\LabSymbols* \\symbols\symbols; C :\IISDebugLabs\symbols\Pub

 

Note: For more information on setting the symbol path for windbg, please refer to the debugger documentation for the .sympath command. This will describe

the "downstream store" feature that lets you download symbols to a symbol server of your own.

 

The stack backtrace of the thread that AV'd would look like:

 

0:001> kvn

# ChildEBP RetAddr Args to Child

00 0088f3c8 77d31f0d 000afde8 00082cc8 000b1c38 RPCRT4!LRPC_BINDING_HANDLE::GetBuffer+0x122 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcclnt.cxx @ 301]

01 0088f3e0 77b2a3b6 000b1be8 00082cc8 000b161c RPCRT4!I_RpcGetBufferWithObject+0x6e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\bufapi.cxx @ 119]

02 0088f424 77b29851 000b1ba0 6d5f10e0 0088f454 ole32!CRpcChannelBuffer::ClientGetBuffer+0x2d5 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 2184]

03 0088f434 77aa70c4 000b161c 0088f51c 6d5f10e0 ole32!CRpcChannelBuffer::GetBuffer+0x17 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 2005]

04 0088f454 77aa6f60 000b161c 0088f51c 6d5f10e0 ole32!CAptRpcChnl::GetBuffer+0x11d (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\callctrl.cxx @ 552]

05 0088f4b4 77d94b21 000b161c 00000000 6d5f10e0 ole32!CCtxComChnl::GetBuffer+0x1a2 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\ctxchnl.cxx @ 529]

06 0088f4cc 77d93d5e 000b0a2c 0088f568 05030117 RPCRT4!NdrProxyGetBuffer+0x3b (FPO: [2,0,2]) (CONV: stdcall) [D:\nt\private\rpc\ndr20\proxy.c @ 1898]

07 0088f714 77d94c61 6d5f11b0 6d5f1230 0088f748 RPCRT4!NdrClientCall2+0x3c6 (FPO: [Non-Fpo]) (CONV: cdecl) [D:\nt\private\rpc\ndr20\cltcall.c @ 1281]

08 0088f730 77d493aa 00000018 00000004 0088f778 RPCRT4!ObjectStublessClient+0x76 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\ndr20\i386\stblsclt.c @ 454]

09 0088f740 65d86edd 000b0a2c 00da1ea0 00000001 RPCRT4!ObjectStubless+0xf [..\i386\stubless.asm @ 154]

0a 0088f778 65d88de1 00000001 0088f7b8 0088f7f4 wam!WAM_EXEC_INFO::CleanupAndRelease+0x91 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\wam\object\wamxinfo.cxx @ 1303]

0b 0088f7b4 77d339c0 004069a0 00000000 000000c8 wam!WAM::ProcessRequest+0x357 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\wam\object\wamobj.cxx @ 790]

0c 0088f7dc 77d93570 65d8223f 0088f7f8 00000005 RPCRT4!Invoke+0x30 [..\i386\stubless.asm @ 88]

0d 0088fa54 77d949ac 0009cf10 0009e0b8 0009a1a8 RPCRT4!NdrStubCall2+0x63d (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\ndr20\srvcall.c @ 1724]

0e 0088fab8 77b29e7f 0009cf10 0009a1a8 0009e0b8 RPCRT4!CStdStubBuffer_Invoke+0xec (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\ndr20\stub.c @ 1571]

0f 0088fafc 77b29d6e 0009a1a8 000ad00c 0009f0f0 ole32!SyncStubInvoke+0x61 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1136]

10 0088fb44 77aa7ae7 0009a1a8 000acf20 0009cf10 ole32!StubInvoke+0xa8 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1334]

11 0088fba8 77aa7a33 0009e0b8 00000000 0009cf10 ole32!CCtxComChnl::ContextInvoke+0xbb (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\ctxchnl.cxx @ 1204]

12 0088fbc4 77b29c89 0009a1a8 00000001 0009cf10 ole32!MTAInvoke+0x18 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\callctrl.cxx @ 1952]

13 0088fbf4 77b29a89 0009a160 0009e0b8 0009cf10 ole32!AppInvoke+0xb5 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1013]

14 0088fcb4 77b2a12c 000b2188 00000000 000acff0 ole32!ComInvokeWithLockAndIPID+0x29e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1578]

15 0088fcf4 77d33721 0009a160 000acff0 00079af0 ole32!ThreadInvoke+0x1b7 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 3835]

16 0088fd2c 77d33667 77b29f7b 00079af0 0088fe0c RPCRT4!DispatchToStubInC+0x84 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\dispatch.c @ 82]

17 0088fd84 77d33579 00000000 00000000 0088fe0c RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x100 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 543]

18 0088fda4 77d48e81 00079af0 00000000 0088fe0c RPCRT4!RPC_INTERFACE::DispatchToStub+0x5e (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 407]

19 0088fdd4 77d34bc6 00079af0 00079ab4 00000000 RPCRT4!RPC_INTERFACE::DispatchToStubWithObject+0xa9 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 795]

1a 0088fe10 77d346c5 000a3948 00081df8 80030001 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0x1c6 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 2665]

1b 0088fe28 77d422ff 000799f8 0088fe50 000a3948 RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0x10c (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 867]

1c 0088ff74 77d420d9 77d42528 00081df8 00070000 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x1eb (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 1175]

1d 0088ff78 77d42528 00081df8 00070000 0006f740 RPCRT4!RecvLotsaCallsWrapper+0x9 (FPO: [1,0,0]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 37]

1e 0088ffa8 77d424da 00080f10 0088ffec 77e887dd RPCRT4!BaseCachedThreadRoutine+0x4f (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 3586]

1f 0088ffb4 77e887dd 0007e7b8 00070000 0006f740 RPCRT4!ThreadStartRoutine+0x18 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\threads.cxx @ 230]

20 0088ffec 00000000 77d424c2 0007e7b8 00000000 KERNEL32!BaseThreadStart+0x52 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 460]

 

 

 

 

 

The faulting stack in a heap corruption issue will involve heap operations. In this example we don't see direct calls to NTDLL heap

management fuctions, but it does not mean that this faulting stack is because of heap corruption.

 

 

Let scan the process heap and look for unreadable heap entry:

 

0:009> !heap 0 -m

1: 00070000

Segment at 00070000 to 00170000 (00043000 bytes committed)

Flags: 00000002

ForceFlags: 00000000

Granularity: 8 bytes

Segment Reserve: 00100000

Segment Commit: 00002000

DeCommit Block Thres:00000200

DeCommit Total Thres:00002000

Total Free Size: 0000012c

Max. Allocation Size:7ffdefff

Lock Variable at: 00070608

Next TagIndex: 0000

Maximum TagIndex: 0000

Tag Entries: 00000000

PsuedoTag Entries: 00000000

Virtual Alloc List: 00070050

UCR FreeList: 00070598

FreeList Usage: 0000000c 00000000 00000000 00000000

FreeList[ 00 ] at 00070178: 000b2808 . 000b2808

FreeList[ 02 ] at 00070188: 000a3608 . 000b05b8

FreeList[ 03 ] at 00070190: 0009c828 . 000b0388

Segment00 at 00070640:

Flags: 00000000

Base: 00070000

First Entry: 00070680

Last Entry: 00170000

Total Pages: 00000100

Total UnCommit: 000000bd

Largest UnCommit:000bd000

UnCommitted Ranges: (1) 000b3000: 000bd000

unable to read heap entry at 000d2a88

 

2: 00170000

Segment at 00170000 to 00180000 (00001000 bytes committed)

Flags: 00008000

ForceFlags: 00000000

Granularity: 8 bytes

......

......

......

 

So this shows that heap is corrupt, back in the thread you would see some function called in which we pass this heap. 

 

 

What this tells us is that the heap at " 00070000 " is corrupted. The thread displayed above was somehow trying to manipulate the heap.

but it was unable to do so since the heap is corrupted and hence it AV'd. Which means that the thread is either a victim of heap corruption

(some other thread corrupted the heap and this one was caught while doing some heap operations in that specific heap address) or it might

be that this thread corrupted the heap previously. So basically, the only conclusion we could get from this is that this is a heap corruption

problem and we can't identify the cause of this corruption just by analyzing this crash.

 

To know who corrupted the heap, we need to Enable Page Heap (A memory management feature to track more closely all heap operations).

This second of this debug annotation shows how to do that and identify the culprit.

 

2) With Page Heap Enabled.

 

a) Turn On "Enable Page Heap"

 

- Go to the Command Prompt

- Switch to the folder where the debugging tools for windows is installed

- Type in "GFlags" and hit Enter

- In the "Image File Name", Enter "dllhost.exe" without quotes

- Click "Image File Options" radio button

- Click "Enable Page Heap" check box

- Click "Apply"

- Click "OK"

- Restart IIS

 

b) Run the scenario again

 

- Do Step 3 and 4 in the "Lab Setup" Section

- Click "Corrupt Heap" one time

	- Note that the debugger breaks and diplays 
		a- Comments about the exception
b- The value of the registers
c- The module where the AV happened
d- The assembly line that caused the AV
- In the WinDbg command line, type in "g" and press "Enter", a 2nd chance AV is now encountered 
            - Notice the "Page Heap" Debug output. 
                    
	c) Line up the Private Symbols                    

 

- Point the debugger to use the private symbols

- For company wide internal symbols, use \\symbols\symbols

- For the component private symbols, use C:\IISDebugLabs\Symbols\Pri

 

 

The faulting Stack

 

The stack backtrace of the thread that AV'd would look like:

 

 

0:001> kvn50

# ChildEBP RetAddr Args to Child

00 00c9f708 029213ef 022e6d70 00000001 022e6e3c badext!doHC1+0x24 (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\extDBG\badext.cpp @ 246]

01 00c9f728 65d82b06 022e6d70 022e6e3c 022e6d70 badext!HttpExtensionProc+0x108 (CONV: stdcall) [H:\nT\PRIVATE\inet\iis\svcs\w3\gateways\extDBG\badext.cpp @ 115]

02 00c9f73c 65d82a97 022e6d70 022e6e3c 022e6d70 wam!HSE_APPDLL::ExecuteRequest+0x93 (FPO: [1,0,3]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\wam\object\isplocal.cxx @ 450]

03 00c9f770 65d823a2 0057bf70 022e6ef2 022e6d70 wam!WAM::InvokeExtension+0x35 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\inet\iis\svcs\wam\object\wamobj.cxx @ 913]

04 00c9f7b4 77d339c0 005f3f80 00000000 000000c8 wam!WAM::ProcessRequest+0x240 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\inet\iis\svcs\wam\object\wamobj.cxx @ 643]

05 00c9f7dc 77d93570 65d8223f 00c9f7f8 00000005 RPCRT4!Invoke+0x30 [..\i386\stubless.asm @ 88]

06 00c9fa54 77d949ac 01a3efe0 01a66e98 019c0f28 RPCRT4!NdrStubCall2+0x63d (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\ndr20\srvcall.c @ 1724]

07 00c9fab8 77b29e7f 01a3efe0 019c0f28 01a66e98 RPCRT4!CStdStubBuffer_Invoke+0xec (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\ndr20\stub.c @ 1571]

08 00c9fafc 77b29d6e 019c0f28 0148af24 01106aa8 ole32!SyncStubInvoke+0x61 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1136]

09 00c9fb44 77aa7ae7 019c0f28 01486f38 01a3efe0 ole32!StubInvoke+0xa8 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1334]

0a 00c9fba8 77aa7a33 01a66e98 00000000 01a3efe0 ole32!CCtxComChnl::ContextInvoke+0xbb (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\ctxchnl.cxx @ 1204]

0b 00c9fbc4 77b29c89 019c0f28 00000001 01a3efe0 ole32!MTAInvoke+0x18 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\callctrl.cxx @ 1952]

0c 00c9fbf4 77b29a89 019c0ee0 01a66e98 01a3efe0 ole32!AppInvoke+0xb5 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1013]

0d 00c9fcb4 77b2a12c 0143cd28 00000000 0148af08 ole32!ComInvokeWithLockAndIPID+0x29e (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 1578]

0e 00c9fcf4 77d33721 019c0ee0 0148af08 019e6f98 ole32!ThreadInvoke+0x1b7 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\ole32\com\dcomrem\channelb.cxx @ 3835]

0f 00c9fd2c 77d33667 77b29f7b 019e6f98 00c9fe0c RPCRT4!DispatchToStubInC+0x84 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\dispatch.c @ 82]

10 00c9fd84 77d33579 00000000 00000000 00c9fe0c RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x100 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 543]

11 00c9fda4 77d48e81 019e6f98 00000000 00c9fe0c RPCRT4!RPC_INTERFACE::DispatchToStub+0x5e (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 407]

12 00c9fdd4 77d34bc6 019e6f98 019e6f5c 00000000 RPCRT4!RPC_INTERFACE::DispatchToStubWithObject+0xa9 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 795]

13 00c9fe10 77d346c5 019e4ec8 00bc0f68 80030001 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0x1c6 (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 2665]

14 00c9fe28 77d422ff 019e6ea0 00c9fe50 019e4ec8 RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0x10c (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 867]

15 00c9ff74 77d420d9 77d42528 00bc0f68 00070000 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x1eb (FPO: [Non-Fpo]) (CONV: thiscall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 1175]

16 00c9ff78 77d42528 00bc0f68 00070000 01000002 RPCRT4!RecvLotsaCallsWrapper+0x9 (FPO: [1,0,0]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\lpcsvr.cxx @ 37]

17 00c9ffa8 77d424da 00c4afe0 00c9ffec 77e887dd RPCRT4!BaseCachedThreadRoutine+0x4f (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\hndlsvr.cxx @ 3586]

18 00c9ffb4 77e887dd 00c4cfa0 00070000 01000002 RPCRT4!ThreadStartRoutine+0x18 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\rpc\runtime\mtrt\threads.cxx @ 230]

19 00c9ffec 00000000 77d424c2 00c4cfa0 00000000 KERNEL32!BaseThreadStart+0x52 (FPO: [Non-Fpo]) (CONV: stdcall) [D:\nt\private\windows\base\client\support.c @ 460]

 

 

So basically, this is the thread that corrupted the heap, It's shows now that the culprit is "badFill.dll".

Notice also the function call "doHC1" as well as the line of code in "badfil.cpp".

 

Verify the extension's symbols are lined up.

 

 

0:001> lmv mbadext

start end module name

02920000 02926000 badext M (pdb symbols) c:\iisdebuglabs\symbols\pri\badext.pdb

Loaded symbol image file: C:\IISDebugLabs\www\app\badext.dll

Image path: C:\IISDebugLabs\www\app\badext.dll

Timestamp: Sat Jun 21 16:11:39 2003 (3EF4BBFB)

Checksum: 00008518

File version: 5.0.0.0

Product version: 5.0.0.0

File flags: 0 (Mask 3F)

File OS: 40004 NT Win32

File type: 2.0 Dll

File date: 00000000.00000000

CompanyName: Microsoft Corporation

ProductName: Internet Information Services

InternalName: badEXT.dll

OriginalFilename: badEXT.dll

ProductVersion: 5.00.0984

FileVersion: 5.00.0984

FileDescription: badEXT Test

LegalCopyright: Copyright (C) Microsoft Corp. 1981-1999

 

 

 

Line 246 looks like

 

DWORD doHC1(EXTENSION_CONTROL_BLOCK *lpEcb, int Variation) //Buffer Overwrite

{

if (1 == Variation) // Straight forward to corrupt the heap

{

char *ptr, *tmp;

int i;

ptr = (char*)GlobalAlloc(GMEM_FIXED, 16);

tmp = ptr;

for (i = 0; i < 32; ++i)

*(tmp++) = 'a';

free(ptr); // line 246

}

}

 

 

 

 


- Microsoft Confidential -