IIS Hang repeatedly waiting on Isapi Extension?? W2k
am 19.10.2007 20:56:00 von RichBaileyProblem - IIS is repeatedly hanging (and HARD crashes server if reset is
attempted)
Details - Windows 2000 Advanced Server SP4 (latest patches) running 1
websites with roughly 1M hits per day (total requests) running asp.net 2.x
application. Hardware is X86 running 2 processors (HT enabled so it shows 4)
with 2GB ram (showing 1.7GB available) at the time of crash.
IIS stops responding to incoming web requests at random intervals (somtimes
30 minutes, sometimes hours or days). A dump using IISSTATE on DLLHost.exe
shows thread locks waiting on one thread which is specified to be the ISAPI
Extension. Similar Dumps of Inetinfo.exe show it waiting on calls to
DLLHost.exe (based on threadID). We can stress the system to over 4 times
its daily traffic and run it at 100% for hours with no crashes. No memory
leaks are visible.
Attempting to Stop IIS via command or GUI or running IISRESET hangs server
to black screen requiring hard reboot.
ANY help is appreciated. Can someone tell me what extension is hanging it
or why it maintains this lock (or how to stop it, since we have removed the
pages and it still does it).
See thread ID 1 for lock example and thread ID 17 for locking thread.
Opened log file 'C:\iisstate\output\IISState-1608.log'
***********************
Starting new log output
IISState version 3.3.1
Fri Oct 19 11:34:29 2007
OS = Windows 2000
Executable: dllhost.exe
PID = 1608
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 38c
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0006fd28 7c59a0a2 ntdll!ZwWaitForSingleObject+0xb
01 0006fd50 7c57b40f KERNEL32!WaitForSingleObjectEx+0x71
02 0006fd60 7ce7b194 KERNEL32!WaitForSingleObject+0xf
03 0006fd80 7ce7a991
ole32!CSurrogateProcessActivator::WaitForSurrogateTimeout+0x 4f
04 0006fd9c 01001230 ole32!CoRegisterSurrogateEx+0x169
05 0006ff24 010014c6 dllhost!WinMain+0xb0
06 0006ffc0 7c5989d5 dllhost!WinMainCRTStartup+0x156
07 0006fff0 00000000 KERNEL32!BaseProcessStart+0x3d
Thread ID: 1
System Thread ID: ac
Kernel Time: 0:0:0.0
User Time: 0:0:0.93
Other information: Thread is waiting for a lock to be released. Looking for
lock owner.
Owning thread System ID: 144
Thread Type: ISAPI Extension
# ChildEBP RetAddr
00 0086f6bc 77f8f295 ntdll!ZwWaitForSingleObject+0xb
01 0086f730 77f87f26 ntdll!RtlpWaitForCriticalSection+0x9e
02 0086f738 65d81d8c ntdll!RtlEnterCriticalSection+0x46
03 0086f75c 65d81ada wam!SE_TABLE::GetExtension+0x64
04 0086f7a8 77d79290 wam!WAM::ProcessRequest+0x1f9
05 0086f7d0 77d96aee RPCRT4!Invoke+0x30
06 0086fa48 77d90776 RPCRT4!NdrStubCall2+0x664
07 0086faac 7cef55fd RPCRT4!CStdStubBuffer_Invoke+0xc8
08 0086faf0 7cef58d8 ole32!SyncStubInvoke+0x61
09 0086fb38 7ce8833d ole32!StubInvoke+0xa8
0a 0086fb9c 7ce7a711 ole32!CCtxComChnl::ContextInvoke+0xbb
0b 0086fbb8 7cef54e2 ole32!MTAInvoke+0x18
0c 0086fbe8 7cef5c06 ole32!AppInvoke+0xb5
0d 0086fca8 7cef3360 ole32!ComInvokeWithLockAndIPID+0x297
0e 0086fce8 77d543a6 ole32!ThreadInvoke+0x1b7
0f 0086fd20 77d393c3 RPCRT4!DispatchToStubInC+0x32
10 0086fd78 77d39297 RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x100
11 0086fd98 77d39563 RPCRT4!RPC_INTERFACE::DispatchToStub+0x5e
12 0086fdc8 77d5ad2f RPCRT4!RPC_INTERFACE::DispatchToStubWithObject+0xa9
13 0086fe04 77d59239 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0x210
14 0086fe28 77d5978e RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0x11b
15 0086ff74 77d58e4a RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x229
16 0086ff78 77d3aeed RPCRT4!RecvLotsaCallsWrapper+0x9
17 0086ffa8 77d37de8 RPCRT4!BaseCachedThreadRoutine+0x4f
18 0086ffb4 7c57b3bc RPCRT4!ThreadStartRoutine+0x18
19 0086ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 2
System Thread ID: 3d8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for
lock owner.
Owning thread System ID: 144
Thread Type: Other
# ChildEBP RetAddr
00 008afe78 77f8f295 ntdll!ZwWaitForSingleObject+0xb
01 008afeec 77f87f26 ntdll!RtlpWaitForCriticalSection+0x9e
02 008afef4 77f8d635 ntdll!RtlEnterCriticalSection+0x46
03 008aff98 7c590905 ntdll!LdrUnloadDll+0x5f
04 008affa0 7ce85b19 KERNEL32!FreeLibraryAndExitThread+0x1b
05 008affb4 7c57b3bc ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x34
06 008affec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 3
System Thread ID: 290
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 008eff30 7ce8c5c3 USER32!NtUserGetMessage+0xb
01 008eff70 7ce8c511 ole32!CDllHost::STAWorkerLoop+0x40
02 008eff8c 7ce8c44c ole32!CDllHost::WorkerThread+0xc2
03 008eff90 7ce85ba7 ole32!DLLHostThreadEntry+0x9
04 008effa8 7ce85aff ole32!CRpcThread::WorkerLoop+0x22
05 008effb4 7c57b3bc ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x1a
06 008effec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 4
System Thread ID: 560
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Other
# ChildEBP RetAddr
00 009ffc54 7c585463 ntdll!NtRemoveIoCompletion+0xb
01 009ffc80 6de8b9d0 KERNEL32!GetQueuedCompletionStatus+0x27
02 009ffd94 6de8b908 TxfAux!WORK_QUEUE::WorkerLoop+0xa0
03 009fffb4 7c57b3bc TxfAux!WORK_QUEUE::ThreadLoop+0x58
04 009fffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 5
System Thread ID: 328
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINNT\system32\ODBC32.DLL -
Thread Type: Other
# ChildEBP RetAddr
00 00c9ff1c 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 00c9ff6c 7c59a180 KERNEL32!WaitForMultipleObjectsEx+0xea
02 00c9ff84 00a66804 KERNEL32!WaitForMultipleObjects+0x17
WARNING: Stack unwind information not available. Following frames may be
wrong.
03 00c9ffb0 77f89c9d ODBC32!ODBCSetTryWaitValue+0xf5
04 00c9ffb4 7c57b3bc ntdll!CsrNewThread+0xb
05 00000000 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 6
System Thread ID: 55c
Kernel Time: 0:0:0.125
User Time: 0:0:0.93
Other information: Thread is waiting for a lock to be released. Looking for
lock owner.
Owning thread System ID: 144
Thread Type: ISAPI Extension
# ChildEBP RetAddr
00 00cdf6bc 77f8f295 ntdll!ZwWaitForSingleObject+0xb
01 00cdf730 77f87f26 ntdll!RtlpWaitForCriticalSection+0x9e
02 00cdf738 65d81d8c ntdll!RtlEnterCriticalSection+0x46
03 00cdf75c 65d81ada wam!SE_TABLE::GetExtension+0x64
04 00cdf7a8 77d79290 wam!WAM::ProcessRequest+0x1f9
05 00cdf7d0 77d96aee RPCRT4!Invoke+0x30
06 00cdfa48 77d90776 RPCRT4!NdrStubCall2+0x664
07 00cdfaac 7cef55fd RPCRT4!CStdStubBuffer_Invoke+0xc8
08 00cdfaf0 7cef58d8 ole32!SyncStubInvoke+0x61
09 00cdfb38 7ce8833d ole32!StubInvoke+0xa8
0a 00cdfb9c 7ce7a711 ole32!CCtxComChnl::ContextInvoke+0xbb
0b 00cdfbb8 7cef54e2 ole32!MTAInvoke+0x18
0c 00cdfbe8 7cef5c06 ole32!AppInvoke+0xb5
0d 00cdfca8 7cef3360 ole32!ComInvokeWithLockAndIPID+0x297
0e 00cdfce8 77d543a6 ole32!ThreadInvoke+0x1b7
0f 00cdfd20 77d393c3 RPCRT4!DispatchToStubInC+0x32
10 00cdfd78 77d39297 RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x100
11 00cdfd98 77d39563 RPCRT4!RPC_INTERFACE::DispatchToStub+0x5e
12 00cdfdc8 77d5ad2f RPCRT4!RPC_INTERFACE::DispatchToStubWithObject+0xa9
13 00cdfe04 77d59239 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0x210
14 00cdfe28 77d5978e RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0x11b
15 00cdff74 77d58e4a RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x229
16 00cdff78 77d3aeed RPCRT4!RecvLotsaCallsWrapper+0x9
17 00cdffa8 77d37de8 RPCRT4!BaseCachedThreadRoutine+0x4f
18 00cdffb4 7c57b3bc RPCRT4!ThreadStartRoutine+0x18
19 00cdffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 7
System Thread ID: 2e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 00d1ff08 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 00d1ff58 7c59a180 KERNEL32!WaitForMultipleObjectsEx+0xea
02 00d1ff70 787f67ee KERNEL32!WaitForMultipleObjects+0x17
03 00d1ffb4 7c57b3bc COMSVCS!CEventDispatcher::PushEvents+0x4e
04 00d1ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 8
System Thread ID: 3dc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: Unable to locate ASP page
Remote call is either to a MTA object or object not initialized. Also,
possible utility thread.
DCOM call being made to Process ID: 940
Waiting on thread id: ffffffff
# ChildEBP RetAddr
00 00d5fb68 77d4f214 ntdll!NtRequestWaitReplyPort+0xb
01 00d5fb94 77d3b7a6 RPCRT4!LRPC_CCALL::SendReceive+0x124
02 00d5fba0 7cef6bee RPCRT4!I_RpcSendReceive+0x2c
03 00d5fbc0 7cef6ab9 ole32!ThreadSendReceive+0xef
04 00d5fbd8 7cef3ab6 ole32!CRpcChannelBuffer::SwitchAptAndDispatchCall+0x14f
05 00d5fc18 7cef692d ole32!CRpcChannelBuffer::SendReceive2+0x96
06 00d5fc28 7ce3cc2d ole32!CRpcChannelBuffer::SendReceive+0x11
07 00d5fc88 7ce87f7f ole32!CAptRpcChnl::SendReceive+0xa9
08 00d5fce0 77d91337 ole32!CCtxComChnl::SendReceive+0x124
09 00d5fcfc 77d93b47 RPCRT4!NdrProxySendReceive+0x4c
0a 00d5ff44 77d96f9c RPCRT4!NdrClientCall2+0x4f5
0b 00d5ff60 77d792ab RPCRT4!ObjectStublessClient+0x76
0c 00d5ff70 787f6732 RPCRT4!ObjectStubless+0xf
0d 00d5ffb4 7c57b3bc COMSVCS!CEventDispatcher::GetEventServerInfoThread+0x152
0e 00d5ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 9
System Thread ID: 558
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 00d9fee0 7c59a301 ntdll!ZwDelayExecution+0xb
01 00d9ff00 7c59a2cc KERNEL32!SleepEx+0x32
02 00d9ff0c 787d213a KERNEL32!Sleep+0xb
03 00d9ff7c 780085bc COMSVCS!PostData+0xf2
04 00d9ffb4 7c57b3bc MSVCRT!_endthreadex+0xc1
05 00d9ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 10
System Thread ID: 25c
Kernel Time: 0:0:0.421
User Time: 0:0:0.62
Thread Type: Other
# ChildEBP RetAddr
00 0092fe5c 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0092feac 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0092ff08 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0092ff24 6e5a5a7c USER32!MsgWaitForMultipleObjects+0x1d
04 0092ff7c 780085bc IISRTL!SchedulerWorkerThread+0xa7
05 0092ffb4 7c57b3bc MSVCRT!_endthreadex+0xc1
06 0092ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 11
System Thread ID: 294
Kernel Time: 0:0:0.500
User Time: 0:0:0.125
Thread Type: Other
# ChildEBP RetAddr
00 0104fe5c 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0104feac 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0104ff08 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0104ff24 6e5a5a7c USER32!MsgWaitForMultipleObjects+0x1d
04 0104ff7c 780085bc IISRTL!SchedulerWorkerThread+0xa7
05 0104ffb4 7c57b3bc MSVCRT!_endthreadex+0xc1
06 0104ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 12
System Thread ID: 330
Kernel Time: 0:0:0.218
User Time: 0:0:0.109
Thread Type: Other
# ChildEBP RetAddr
00 0108fe5c 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0108feac 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0108ff08 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0108ff24 6e5a5a7c USER32!MsgWaitForMultipleObjects+0x1d
04 0108ff7c 780085bc IISRTL!SchedulerWorkerThread+0xa7
05 0108ffb4 7c57b3bc MSVCRT!_endthreadex+0xc1
06 0108ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 13
System Thread ID: 578
Kernel Time: 0:0:0.390
User Time: 0:0:0.140
Thread Type: Other
# ChildEBP RetAddr
00 010cfe5c 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 010cfeac 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 010cff08 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 010cff24 6e5a5a7c USER32!MsgWaitForMultipleObjects+0x1d
04 010cff7c 780085bc IISRTL!SchedulerWorkerThread+0xa7
05 010cffb4 7c57b3bc MSVCRT!_endthreadex+0xc1
06 010cffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 14
System Thread ID: 610
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 0120ff50 7c585463 ntdll!NtRemoveIoCompletion+0xb
01 0120ff7c 6d702957 KERNEL32!GetQueuedCompletionStatus+0x27
02 0120ffb4 7c57b3bc ISATQ!AtqPoolThread+0x40
03 0120ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 15
System Thread ID: 5c0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 0124ff50 7c585463 ntdll!NtRemoveIoCompletion+0xb
01 0124ff7c 6d702957 KERNEL32!GetQueuedCompletionStatus+0x27
02 0124ffb4 7c57b3bc ISATQ!AtqPoolThread+0x40
03 0124ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 16
System Thread ID: 534
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: Unable to locate ASP page
No remote call being made
# ChildEBP RetAddr
00 013cfeb8 7c585463 ntdll!NtRemoveIoCompletion+0xb
01 013cfee4 77d818ff KERNEL32!GetQueuedCompletionStatus+0x27
02 013cff20 77d51484 RPCRT4!COMMON_ProcessCalls+0x9e
03 013cff74 77d512bd RPCRT4!LOADABLE_TRANSPORT::ProcessIOEvents+0x99
04 013cff78 77d3aeed RPCRT4!ProcessIOEventsWrapper+0x9
05 013cffa8 77d37de8 RPCRT4!BaseCachedThreadRoutine+0x4f
06 013cffb4 7c57b3bc RPCRT4!ThreadStartRoutine+0x18
07 013cffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 17
System Thread ID: 144
Kernel Time: 0:0:0.31
User Time: 0:0:0.62
Thread Type: ISAPI Extension
# ChildEBP RetAddr
00 0148f1c0 77f8c097 ntdll!ZwRaiseHardError+0xb
01 0148f210 77f8bb1d ntdll!LdrpCreateDllSection+0x9b
02 0148f290 77f8d324 ntdll!LdrpMapDll+0xd9
03 0148f53c 77f85b43 ntdll!LdrpLoadDll+0x187
04 0148f554 7c5907c6 ntdll!LdrLoadDll+0x17
05 0148f5e8 7c59034b KERNEL32!LoadLibraryExW+0x231
06 0148f5f8 65d87677 KERNEL32!LoadLibraryExA+0x1d
07 0148f730 65d87225 wam!HSE_APPDLL::LoadModule+0x20
08 0148f75c 65d81ada wam!SE_TABLE::GetExtension+0xdf
09 0148f7a8 77d79290 wam!WAM::ProcessRequest+0x1f9
0a 0148f7d0 77d96aee RPCRT4!Invoke+0x30
0b 0148fa48 77d90776 RPCRT4!NdrStubCall2+0x664
0c 0148faac 7cef55fd RPCRT4!CStdStubBuffer_Invoke+0xc8
0d 0148faf0 7cef58d8 ole32!SyncStubInvoke+0x61
0e 0148fb38 7ce8833d ole32!StubInvoke+0xa8
0f 0148fb9c 7ce7a711 ole32!CCtxComChnl::ContextInvoke+0xbb
10 0148fbb8 7cef54e2 ole32!MTAInvoke+0x18
11 0148fbe8 7cef5c06 ole32!AppInvoke+0xb5
12 0148fca8 7cef3360 ole32!ComInvokeWithLockAndIPID+0x297
13 0148fce8 77d543a6 ole32!ThreadInvoke+0x1b7
14 0148fd20 77d393c3 RPCRT4!DispatchToStubInC+0x32
15 0148fd78 77d39297 RPCRT4!RPC_INTERFACE::DispatchToStubWorker+0x100
16 0148fd98 77d39563 RPCRT4!RPC_INTERFACE::DispatchToStub+0x5e
17 0148fdc8 77d5ad2f RPCRT4!RPC_INTERFACE::DispatchToStubWithObject+0xa9
18 0148fe04 77d59239 RPCRT4!LRPC_SCALL::DealWithRequestMessage+0x210
19 0148fe28 77d5978e RPCRT4!LRPC_ADDRESS::DealWithLRPCRequest+0x11b
1a 0148ff74 77d58e4a RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x229
1b 0148ff78 77d3aeed RPCRT4!RecvLotsaCallsWrapper+0x9
1c 0148ffa8 77d37de8 RPCRT4!BaseCachedThreadRoutine+0x4f
1d 0148ffb4 7c57b3bc RPCRT4!ThreadStartRoutine+0x18
1e 0148ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 18
System Thread ID: 3cc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: ASP
Executing Page: Unable to locate ASP page
# ChildEBP RetAddr
00 01fafe70 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 01fafec0 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 01faff1c 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 01faff38 74a11121 USER32!MsgWaitForMultipleObjects+0x1d
04 01faff7c 78008454 asp!CMTACallbackThread::Thread+0x42
05 01faffb4 7c57b3bc MSVCRT!_endthread+0xc6
06 01faffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 19
System Thread ID: 188
Kernel Time: 0:0:0.93
User Time: 0:0:0.218
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 01fefe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 01fefe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 01fefed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 01fefef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 01feff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 01feffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 01feffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 20
System Thread ID: 354
Kernel Time: 0:0:0.265
User Time: 0:0:1.93
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 0202fe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0202fe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0202fed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0202fef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 0202ff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 0202ffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 0202ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 21
System Thread ID: 644
Kernel Time: 0:0:0.187
User Time: 0:0:1.93
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 0206fe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0206fe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0206fed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0206fef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 0206ff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 0206ffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 0206ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 22
System Thread ID: 64c
Kernel Time: 0:0:0.296
User Time: 0:0:1.46
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 020afe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 020afe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 020afed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 020afef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 020aff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 020affb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 020affec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 23
System Thread ID: 63c
Kernel Time: 0:0:0.250
User Time: 0:0:1.109
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 020efe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 020efe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 020efed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 020efef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 020eff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 020effb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 020effec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 24
System Thread ID: 264
Kernel Time: 0:0:0.218
User Time: 0:0:1.156
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 0212fe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0212fe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0212fed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0212fef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 0212ff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 0212ffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 0212ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 25
System Thread ID: 5dc
Kernel Time: 0:0:0.421
User Time: 0:0:0.906
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 0216fe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0216fe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0216fed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0216fef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 0216ff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 0216ffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 0216ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 26
System Thread ID: 5e0
Kernel Time: 0:0:0.312
User Time: 0:0:1.0
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 021afe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 021afe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 021afed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 021afef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 021aff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 021affb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 021affec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 27
System Thread ID: 5d8
Kernel Time: 0:0:0.265
User Time: 0:0:0.859
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 021efe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 021efe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 021efed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 021efef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 021eff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 021effb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 021effec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 28
System Thread ID: 5cc
Kernel Time: 0:0:0.109
User Time: 0:0:1.562
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 0222fe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0222fe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0222fed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0222fef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 0222ff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 0222ffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 0222ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 29
System Thread ID: 5d0
Kernel Time: 0:0:0.328
User Time: 0:0:1.312
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 0226fe28 7c59a26d ntdll!ZwWaitForMultipleObjects+0xb
01 0226fe78 77e1e9db KERNEL32!WaitForMultipleObjectsEx+0xea
02 0226fed4 77e1ea28 USER32!MsgWaitForMultipleObjectsEx+0x153
03 0226fef0 787c4511 USER32!MsgWaitForMultipleObjects+0x1d
04 0226ff1c 787c38c0 COMSVCS!STAThread::WaitForWork+0x33
05 0226ffb4 7c57b3bc COMSVCS!STAThread::STAThreadWorker+0x4e2
06 0226ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 30
System Thread ID: 5c8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 022cff9c 77fa238a ntdll!ZwDelayExecution+0xb
01 022cffb4 7c57b3bc ntdll!RtlpTimerThread+0x42
02 022cffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 31
System Thread ID: 5bc
Kernel Time: 0:0:0.0
User Time: 0:0:0.62
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\Program Files\Common Files\System\Ole DB\oledb32.dll -
Thread Type: Other
# ChildEBP RetAddr
00 0327ff5c 7c59a0a2 ntdll!ZwWaitForSingleObject+0xb
01 0327ff84 7c57b40f KERNEL32!WaitForSingleObjectEx+0x71
02 0327ff94 1f869d5e KERNEL32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be
wrong.
03 0327ffb4 7c57b3bc oledb32!DllGetClassObject+0x858b
04 0327ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 32
System Thread ID: 5c4
Kernel Time: 0:0:0.31
User Time: 0:0:0.46
Thread Type: Other
# ChildEBP RetAddr
00 032bff58 7c59a0a2 ntdll!ZwWaitForSingleObject+0xb
01 032bff80 7c57b40f KERNEL32!WaitForSingleObjectEx+0x71
02 032bff90 1f869d1e KERNEL32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be
wrong.
03 032bffb4 7c57b3bc oledb32!DllGetClassObject+0x854b
04 032bffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 33
System Thread ID: 5b8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for
lock owner.
Owning thread System ID: 144
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 032ffa34 77f8f295 ntdll!ZwWaitForSingleObject+0xb
01 032ffaa8 77f87f26 ntdll!RtlpWaitForCriticalSection+0x9e
02 032ffab0 77f8d635 ntdll!RtlEnterCriticalSection+0x46
03 032ffb54 7c5908d2 ntdll!LdrUnloadDll+0x5f
04 032ffb64 412482a2 KERNEL32!FreeLibrary+0x3b
05 032ffb7c 4124820d SQLSRV32!SQLFreeConnect+0x90
06 032ffb88 00a4a62c SQLSRV32!SQLFreeHandle+0x2b
WARNING: Stack unwind information not available. Following frames may be
wrong.
07 032ffbd8 00a4a15a ODBC32!SQLDisconnect+0x224
08 032ffbec 1f6bb9da ODBC32!SQLFreeConnect+0x18
09 032ffc0c 1f6bb916 msdasql!CODBCHandle::ReleaseHdbc+0x87
0a 032ffc30 1f6bb87f msdasql!CDataSource::UninitializeHdbc+0x69
0b 032ffc50 1f6bb750 msdasql!CDataSource::DoUninitialize+0xa3
0c 032ffc64 1f6bb729 msdasql!CDataSource::~CDataSource+0x12
0d 032ffc6c 1f6b2d54 msdasql!CDataSource::`scalar deleting destructor'+0x8
0e 032ffc7c 1f873574 msdasql!CDataSource::Release+0x26
0f 032ffca8 1f8735ae oledb32!DllGetClassObject+0x11da1
10 032ffcdc 1f873634 oledb32!DllGetClassObject+0x11ddb
11 032ffd30 78790c25 oledb32!DllGetClassObject+0x11e61
12 032ffd4c 78790bc0 COMSVCS!CHolder::ProcessDestroyList+0x35
13 032ffd84 78790a1c COMSVCS!CHolder::DestroyResources+0x110
14 032ffd90 7878e75f COMSVCS!CHolder::Ping+0x1c
15 032fffb4 7c57b3bc COMSVCS!PingThread+0xcf
16 032fffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 34
System Thread ID: 5ac
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 034bff88 7cdf4a9b ntdll!ZwWaitForMultipleObjects+0xb
01 034bffb4 7c57b3bc NETAPI32!NetbiosWaiter+0x71
02 034bffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 35
System Thread ID: 5b4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 034fff20 77fa096f ntdll!NtRemoveIoCompletion+0xb
01 034fffb4 7c57b3bc ntdll!RtlpWorkerThread+0x6b
02 034fffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 36
System Thread ID: 784
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Other information: Thread is waiting for a lock to be released. Looking for
lock owner.
Owning thread System ID: 144
Thread Type: Other
# ChildEBP RetAddr
00 0435fe78 77f8f295 ntdll!ZwWaitForSingleObject+0xb
01 0435feec 77f87f26 ntdll!RtlpWaitForCriticalSection+0x9e
02 0435fef4 77f8d635 ntdll!RtlEnterCriticalSection+0x46
03 0435ff98 7c590905 ntdll!LdrUnloadDll+0x5f
04 0435ffa0 7ce85b19 KERNEL32!FreeLibraryAndExitThread+0x1b
05 0435ffb4 7c57b3bc ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x34
06 0435ffec 00000000 KERNEL32!BaseThreadStart+0x52
Thread ID: 37
System Thread ID: 3bc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Other information: Thread is waiting for a lock to be released. Looking for
lock owner.
Owning thread System ID: 144
Thread Type: Other
# ChildEBP RetAddr
00 07b3fc24 77f8f295 ntdll!ZwWaitForSingleObject+0xb
01 07b3fc98 77f87f26 ntdll!RtlpWaitForCriticalSection+0x9e
02 07b3fca0 77f839a1 ntdll!RtlEnterCriticalSection+0x46
03 07b3fd1c 77f91b93 ntdll!LdrpInitialize+0x1a7
04 00000000 00000000 ntdll!KiUserApcDispatcher+0x7
Closing open log file C:\iisstate\output\IISState-1608.log