possible MDAC issue hanging web servers?

am 03.04.2006 22:39:57 von Jerome Cohen

Hello,

We have a web farm of
win2K machines, IIS 5, MDAC 2.6
server 2003 machines, IIS 6.0, MDAC 2.8

all connecting to Oracle DB, using ADO which use ODBC drivers. We use
MS ODBC for Oracle and Oracle ODBC. all connect using TCP connections
(thru SQLnet) to the DB server. the ODBC drivers use connection
pooling.

We noticed that sometimes web servers "hang", that is they will not
respond to web requests, and then will clear themselves after few
seconds to minutes.

when opening trace on the oracle client we noticed there is a distinct
"break" in the trace file, that is nothing is written for the period we
see the server stuck.

Microsoft acknowleged a bug in MDAC 2.5 which would cause a web server
to "hang", it looks exactly like our problem. is there any chance this
same problem exists on these MDACs? is there any fix for MDAC 2.8 SP1
or otherwise?

We have turned off connection pooling on half of the servers to see if
this solves the issue. can we estimate the impact to performance?

Re: possible MDAC issue hanging web servers?

am 04.04.2006 02:54:53 von Ken Schaefer

You need to use a tool like DebugDiag or IISState to find out what's going
on inside your application: Post the log files that are generated here if
you need help understanding what's going on.

DebugDiag (recommended):
http://www.microsoft.com/windowsserver2003/iis/diagnostictoo ls/default.mspx

IISState
http://www.adopenstatic.com/faq/IISConfigureIISState.aspx

Since you have a managed app, get the Windows Debug Toolkit
(www.microsoft.com/ddk) so that you can get SOS (Son of Strike) to get the
managed stack.

Cheers
Ken

--
IIS Blog: http://www.adOpenStatic.com/cs/blogs/ken

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Yoram Ayalon" wrote in message
news:1144096797.016915.199280@e56g2000cwe.googlegroups.com.. .
: Hello,
:
: We have a web farm of
: win2K machines, IIS 5, MDAC 2.6
: server 2003 machines, IIS 6.0, MDAC 2.8
:
: all connecting to Oracle DB, using ADO which use ODBC drivers. We use
: MS ODBC for Oracle and Oracle ODBC. all connect using TCP connections
: (thru SQLnet) to the DB server. the ODBC drivers use connection
: pooling.
:
: We noticed that sometimes web servers "hang", that is they will not
: respond to web requests, and then will clear themselves after few
: seconds to minutes.
:
: when opening trace on the oracle client we noticed there is a distinct
: "break" in the trace file, that is nothing is written for the period we
: see the server stuck.
:
: Microsoft acknowleged a bug in MDAC 2.5 which would cause a web server
: to "hang", it looks exactly like our problem. is there any chance this
: same problem exists on these MDACs? is there any fix for MDAC 2.8 SP1
: or otherwise?
:
: We have turned off connection pooling on half of the servers to see if
: this solves the issue. can we estimate the impact to performance?
:

Re: possible MDAC issue hanging web servers?

am 04.04.2006 02:54:53 von Ken Schaefer

Re: possible MDAC issue hanging web servers?

am 04.04.2006 16:05:57 von Jerome Cohen

thanks,

I have installed DebugDiag on one web server and waiting for the beast
to raise its head...

Re: possible MDAC issue hanging web servers?

am 04.04.2006 16:05:57 von Jerome Cohen

thanks,

I have installed DebugDiag on one web server and waiting for the beast
to raise its head...

Re: possible MDAC issue hanging web servers?

am 04.04.2006 21:47:16 von Jerome Cohen

Here are the log files from DebugDiag. am I on to something? anything
else you would need?

=========================================

1) threads 28 and 29
============================================================ ============
Type Description Recommendation
Information The following threads in
w3wp.exe__DefaultAppPool__PID__1900__Date__04_04_2006__Time_ 11_59_48AM__96__IIS
Hang Dump.dmp are making a database operation using ADO. The call to
ADO originated from vbscript!CatchIDispatchInvoke+46

( 28 29 )

100.00% of executing ASP Requests blocked

3.17% of threads blocked
------------------------------------------------------------ ------------------
Thread 28 - System ID 3992

Entry point msvcrt!_endthreadex+2f

Create time 4/4/2006 1:10:17 AM

Time spent in user mode 0 Days 0:1:36.359

Time spent in kernel mode 0 Days 0:0:36.703

This thread is making a database operation using ADO. The call to ADO
originated from vbscript!CatchIDispatchInvoke+46

Function Source

ntdll!KiFastSystemCallRet

ntdll!NtWaitForSingleObject+c

kernel32!WaitForSingleObjectEx+ac

kernel32!WaitForSingleObject+12

msorcl32!RequestOCIWork+61

msorcl32!OCI_olog+67

msorcl32!SQLInternalConnect+54f

msorcl32!SQLDriverConnect+33

odbc32!ToAnsi_SQLDriverConnect+c2

odbc32!SQLInternalDriverConnectW+221

odbc32!SQLDriverConnectW+a8e

msdasql!CODBCHandle::DriverConnect+22

msdasql!CODBCHandle::OHDriverConnect+96

msdasql!CHdbcNode::DriverConnect+22

msdasql!CImpIDBInitialize::Initialize+163

oledb32!CDBInitialize::DoInitialize+33

oledb32!CDBInitialize::Initialize+71

oledb32!CDCMPool::CreateResource+297

comsvcs!CHolder::SafeDispenserDriver::CreateResource+25

comsvcs!CHolder::AllocResource+328

oledb32!CDCMPool::DrawResource+107

oledb32!CDCMPoolManager::DrawResource+384

oledb32!CDPO::Initialize+13d

msado15!_ConnectAsync+37d

msado15!ConnectAsync+4f

msado15!CConnection::Open+c3f

msado15!CConnection::Invoke+c17

vbscript!CatchIDispatchInvoke+46

vbscript!IDispatchInvoke2+af

vbscript!IDispatchInvoke+59

vbscript!InvokeDispatch+13a

vbscript!InvokeByName+42

vbscript!CScriptRuntime::Run+2625

vbscript!CScriptEntryPoint::Call+5c

vbscript!CScriptRuntime::Run+20ce

vbscript!CScriptEntryPoint::Call+5c

vbscript!CSession::Execute+b4

vbscript!COleScript::ExecutePendingScripts+13e

vbscript!COleScript::SetScriptState+150

asp!CActiveScriptEngine::TryCall+19

asp!CActiveScriptEngine::Call+31

asp!CallScriptFunctionOfEngine+5b

asp!ExecuteRequest+17e

asp!Execute+249

asp!CHitObj::ViperAsyncCallback+3e8

asp!CViperAsyncRequest::OnCall+92

comsvcs!CSTAActivityWork::STAActivityWorkHelper+32

ole32!EnterForCallback+c4

ole32!SwitchForCallback+1a3

ole32!PerformCallback+54

ole32!CObjectContext::InternalContextCallback+159

ole32!CObjectContext::DoCallback+1c

comsvcs!CSTAActivityWork::DoWork+12d

comsvcs!CSTAThread::DoWork+18

comsvcs!CSTAThread::ProcessQueueWork+37

comsvcs!CSTAThread::WorkerLoop+17c

msvcrt!_endthreadex+a3

kernel32!BaseThreadStart+34

ASP request executing on thread 28

GET request for
/sw/swchannel/productcatalogcf_v2/internet/productcatalog.as p

Request alive for 0 day(s) 00:00:11

QueryString parentid=152

Request mapped to
\\fileserver\shared-wwwroot\sw\swchannel\productcatalogcf_v2 \internet\productcatalog.asp

ASP Application /LM/W3SVC/1/Root/sw

ASP Template
\\fileserver\SHARED-WWWROOT\SW\SWCHANNEL\PRODUCTCATALOGCF_V2 \INTERNET\PRODUCTCATALOG.ASP

------------------------------------------------------------ ------------------

Thread 29 - System ID 3356

Entry point msvcrt!_endthreadex+2f

Create time 4/4/2006 1:10:21 AM

Time spent in user mode 0 Days 0:0:14.562

Time spent in kernel mode 0 Days 0:0:5.140

This thread is making a database operation using ADO. The call to ADO
originated from vbscript!CatchIDispatchInvoke+46

Function Source

ntdll!KiFastSystemCallRet

ntdll!NtWaitForSingleObject+c

kernel32!WaitForSingleObjectEx+ac

kernel32!WaitForSingleObject+12

msorcl32!RequestOCIWork+61

msorcl32!OCI_olog+67

msorcl32!SQLInternalConnect+54f

msorcl32!SQLDriverConnect+33

odbc32!ToAnsi_SQLDriverConnect+c2

odbc32!SQLInternalDriverConnectW+221

odbc32!SQLDriverConnectW+a8e

msdasql!CODBCHandle::DriverConnect+22

msdasql!CODBCHandle::OHDriverConnect+96

msdasql!CHdbcNode::DriverConnect+22

msdasql!CImpIDBInitialize::Initialize+163

oledb32!CDBInitialize::DoInitialize+33

oledb32!CDBInitialize::Initialize+71

oledb32!CDCMPool::CreateResource+297

comsvcs!CHolder::SafeDispenserDriver::CreateResource+25

comsvcs!CHolder::AllocResource+328

oledb32!CDCMPool::DrawResource+107

oledb32!CDCMPoolManager::DrawResource+384

oledb32!CDPO::Initialize+13d

msado15!_ConnectAsync+37d

msado15!ConnectAsync+4f

msado15!CConnection::Open+c3f

msado15!CConnection::Invoke+c17

vbscript!CatchIDispatchInvoke+46

vbscript!IDispatchInvoke2+af

vbscript!IDispatchInvoke+59

vbscript!InvokeDispatch+13a

vbscript!InvokeByName+42

vbscript!CScriptRuntime::Run+2625

vbscript!CScriptEntryPoint::Call+5c

vbscript!CScriptRuntime::Run+1f02

vbscript!CScriptEntryPoint::Call+5c

vbscript!CSession::Execute+b4

vbscript!COleScript::ExecutePendingScripts+13e

vbscript!COleScript::SetScriptState+150

asp!CActiveScriptEngine::TryCall+19

asp!CActiveScriptEngine::Call+31

asp!CallScriptFunctionOfEngine+5b

asp!ExecuteRequest+17e

asp!Execute+249

asp!CHitObj::ExecuteChildRequest+12c

asp!CServer::Transfer+ae

oleaut32!DispCallFunc+16a

oleaut32!CTypeInfo2::Invoke+234

asp!CDispatchImpl::Invoke+55

vbscript!CatchIDispatchInvoke+46

vbscript!IDispatchInvoke2+af

vbscript!IDispatchInvoke+59

vbscript!InvokeDispatch+13a

vbscript!InvokeByName+42

vbscript!CScriptRuntime::Run+2625

vbscript!CScriptEntryPoint::Call+5c

vbscript!CSession::Execute+b4

vbscript!COleScript::ExecutePendingScripts+13e

vbscript!COleScript::SetScriptState+150

asp!CActiveScriptEngine::TryCall+19

asp!CActiveScriptEngine::Call+31

asp!CallScriptFunctionOfEngine+5b

asp!ExecuteRequest+17e

asp!Execute+249

asp!CHitObj::ExecuteChildRequest+12c

asp!CServer::Transfer+ae

oleaut32!DispCallFunc+16a

oleaut32!CTypeInfo2::Invoke+234

asp!CDispatchImpl::Invoke+55

vbscript!CatchIDispatchInvoke+46

vbscript!IDispatchInvoke2+af

vbscript!IDispatchInvoke+59

vbscript!InvokeDispatch+13a

vbscript!InvokeByName+42

vbscript!CScriptRuntime::Run+2625

vbscript!CScriptEntryPoint::Call+5c

vbscript!CSession::Execute+b4

vbscript!COleScript::ExecutePendingScripts+13e

vbscript!COleScript::SetScriptState+150

asp!CActiveScriptEngine::TryCall+19

asp!CActiveScriptEngine::Call+31

asp!CallScriptFunctionOfEngine+5b

asp!ExecuteRequest+17e

asp!Execute+249

asp!CHitObj::ViperAsyncCallback+3e8

asp!CViperAsyncRequest::OnCall+92

comsvcs!CSTAActivityWork::STAActivityWorkHelper+32

ole32!EnterForCallback+c4

ole32!SwitchForCallback+1a3

ole32!PerformCallback+54

ole32!CObjectContext::InternalContextCallback+159

ole32!CObjectContext::DoCallback+1c

comsvcs!CSTAActivityWork::DoWork+12d

comsvcs!CSTAThread::DoWork+18

comsvcs!CSTAThread::ProcessQueueWork+37

comsvcs!CSTAThread::WorkerLoop+17c

msvcrt!_endthreadex+a3

kernel32!BaseThreadStart+34

ASP request executing on thread 29

GET request for /sw.asp

Request alive for 0 day(s) 00:00:07

QueryString

Request mapped to d:\inetpub\wwwroot\sw.asp

ASP Template D:\INETPUB\WWWROOT\SW.ASP

2) threads 61 and 62
============================================================ ============
Warning The following threads in
w3wp.exe__DefaultAppPool__PID__1900__Date__04_04_2006__Time_ 11_59_48AM__96__IIS
Hang Dump.dmp are waiting on data to be returned from another server
via WinSock.

The call to WinSock originated from orantcp9!nttini+4634

( 61 62 )

3.17% of threads blocked

Ensure that any remote server this application may be calling is
functioning properly and there are no network issues between the two
servers. If the problem continues, please contact the application
vendor for further assistance

Thread 61 - System ID 1616

Entry point msvcrt!_endthreadex+2f

Create time 4/4/2006 11:59:46 AM

Time spent in user mode 0 Days 0:0:0.0

Time spent in kernel mode 0 Days 0:0:0.0

This thread is waiting on data to be returned from another server via
WinSock.

The call to WinSock originated from orantcp9!nttini+4634

Function Source

ntdll!KiFastSystemCallRet

ntdll!NtWaitForSingleObject+c

mswsock!SockWaitForSingleObject+3a

mswsock!WSPRecv+203

ws2_32!WSARecv+77

orantcp9!nttini+4634

orantcp9!nttini+1f94

oran9!ztvull+17bb3

oran9!nsrdr+140

oran9!nsdo+d31

oran9!nioqrc+3e4

oracommon9!ttcdrv+2e7

oran9!nioqwa+2d

ORACLIENT9!xaolog+2ef6e

ORACLIENT9!xaolog+2e97a

ORACLIENT9!xaolog+2cf4c

ORACLIENT9!xaolog+2dcec

ORACLIENT9!xaolog+2dc39

ORACLIENT9!upilog+64

ORACLIENT9!ocirlo+b0

ociw32!olog+5c

msorcl32!ServiceOCIWorkRequest+10e

msorcl32!OCIWorkerThreadFunc+57

msvcrt!_endthreadex+a3

kernel32!BaseThreadStart+34

------------------------------------------------------------ ------------------

Thread 62 - System ID 3892

Entry point msvcrt!_endthreadex+2f

Create time 4/4/2006 11:59:47 AM

Time spent in user mode 0 Days 0:0:0.0

Time spent in kernel mode 0 Days 0:0:0.0

This thread is waiting on data to be returned from another server via
WinSock.

The call to WinSock originated from orantcp9!nttini+4634

Function Source

ntdll!KiFastSystemCallRet

ntdll!NtWaitForSingleObject+c

mswsock!SockWaitForSingleObject+19d

mswsock!WSPRecv+203

ws2_32!WSARecv+77

orantcp9!nttini+4634

orantcp9!nttini+1f94

oran9!ztvull+17bb3

oran9!ztvull+1af62

oran9!nsdo+b1a

oran9!nscall+f08

oran9!nscall+4ee

oran9!niotns+6fa

oran9!osncon+474

ORACLIENT9!kpuatch+64e

ORACLIENT9!xaolog+2f567

ORACLIENT9!xaolog+55157

ORACLIENT9!xaolog+551b2

ORACLIENT9!xaolog+2cbb8

ORACLIENT9!xaolog+2dcec

ORACLIENT9!xaolog+2dc39

ORACLIENT9!upilog+64

ORACLIENT9!ocirlo+b0

ociw32!olog+5c

msorcl32!ServiceOCIWorkRequest+10e

msorcl32!OCIWorkerThreadFunc+57

msvcrt!_endthreadex+a3

kernel32!BaseThreadStart+34

Re: possible MDAC issue hanging web servers?

am 04.04.2006 21:47:16 von Jerome Cohen