(repost) intermittent failure to Server.CreateObject("MSXML2.DOMDocument.4.0") in global.a

hi,

our web site is in transition (a process that will take years in all --
there is too much needed code to re-write with a small team, particularly
given other priorities) from ASP to ASP.NET.

to help maintain configuration sanity, we set up our Application_OnStart()
in global.asa to read data base connection information out of the same
machine.config (and web.congif if appropriate) that our ASP.NET apps use.
(ASP doesn't automatically restart when these files change the way ASP.NET
does, but we don't change them often, know the facts and act accordingly.)

this has been working fine for a couple years. since Wednesday, however, a
number of servers have failed to complete this configuration step correctly
during their nightly reboot.

as far as we can tell the problem is at the step
Server.CreateObject("MSXML2.DOMDocument.4.0")

just before the script parses the machine.config file.

because the problem affected 6 of 12 servers the first night, 3 the 2nd and
3 again last night (each time a different sub-set of servers) (and because
an IISRESET appears *always* to solve the problem), i wonder whether there
might some kind of race condition happening. (maybe something having to do
with CreateObject in general or XML in particular sometimes isn't quite
ready when global.asa's Application_OnStart() fires?

what might have changed (just before Halloween!? :) that would cause this
race to occur much more frequently? (i can't rule out that this might have
happened occasionally in the past, but never 3 of 12 servers and never 3
nights in a row.)

i note that the www publishing service is set up depending on HTTP SSL, IIS
Admin, and RPC Services; a quick google gives me the impression these are
the recommended dependencies.

we run IIS6 on Windoes 2003 Server. (plus ASP.NET 2.0 Final, for what that
is worth -- we don't have any such problems with ASP.NET at the moment --
just with good ol' ASP)

i'd be grateful for any ideas.

cheers,

Tim

(accidentally posted this first in microsoft . public . de . inetserver .
iis . asp -- sorry)

Re: (repost) intermittent failure to Server.CreateObject("MSXML2.DOMDocument.4.0") in glob

"tbh" wrote in message
news:OVrweiVHIHA.1212@TK2MSFTNGP05.phx.gbl...
> hi,
>
> our web site is in transition (a process that will take years in all --
> there is too much needed code to re-write with a small team, particularly
> given other priorities) from ASP to ASP.NET.
>
> to help maintain configuration sanity, we set up our Application_OnStart()
> in global.asa to read data base connection information out of the same
> machine.config (and web.congif if appropriate) that our ASP.NET apps use.
> (ASP doesn't automatically restart when these files change the way ASP.NET
> does, but we don't change them often, know the facts and act accordingly.)
>
> this has been working fine for a couple years. since Wednesday, however, a
> number of servers have failed to complete this configuration step
correctly
> during their nightly reboot.
>
> as far as we can tell the problem is at the step
> Server.CreateObject("MSXML2.DOMDocument.4.0")
>
> just before the script parses the machine.config file.
>
> because the problem affected 6 of 12 servers the first night, 3 the 2nd
and
> 3 again last night (each time a different sub-set of servers) (and because
> an IISRESET appears *always* to solve the problem), i wonder whether there
> might some kind of race condition happening. (maybe something having to do
> with CreateObject in general or XML in particular sometimes isn't quite
> ready when global.asa's Application_OnStart() fires?
>
> what might have changed (just before Halloween!? :) that would cause this
> race to occur much more frequently? (i can't rule out that this might have
> happened occasionally in the past, but never 3 of 12 servers and never 3
> nights in a row.)
>
> i note that the www publishing service is set up depending on HTTP SSL,
IIS
> Admin, and RPC Services; a quick google gives me the impression these are
> the recommended dependencies.
>
> we run IIS6 on Windoes 2003 Server. (plus ASP.NET 2.0 Final, for what that
> is worth -- we don't have any such problems with ASP.NET at the moment --
> just with good ol' ASP)
>
> i'd be grateful for any ideas.

MSXML doesn't depend on any external services so it seems unlikely that
service dependancies have any impact. Also Application_Start only occurs on
the first request not on reboot.

You haven't detailed what actually goes wrong. Is there an error or does it
simply not behave as expected?

Try using MSXML3 instead does that work?

--
Anthony Jones - MVP ASP/ASP.NET

Re: (repost) intermittent failure to Server.CreateObject("MSXML2.DOMDocument.4.0") in glob

thanks, Anthony,

> MSXML doesn't depend on any external services so it seems unlikely that
> service dependancies have any impact.

good to know.

> Also Application_Start only occurs on the first request not on reboot.

on the first request after a reboot or iisreset, yes. but in our web
farm, each www server is queried by the load balancer at regular
intervals to see if it is online. that means shortly after a reboot,
once IIS is up and the application is running, the first request will
come from the load balancer. this first request needs to get some
things initialized.

> You haven't detailed what actually goes wrong. Is there an error or does it
> simply not behave as expected?

i gave all the details i had. the initial info was simply that the
servers weren't "really up". iis was running, but the initialization
steps had failed. we built in more logging of debug info and one retry
of the load-info-from-machine.config step.

this morning the problem didn't crop up, so I don't have more info.
maybe this single retry will be enough to solve the problem. we'll
keep an eye on it.

> Try using MSXML3 instead does that work?

thanks. i'll keep that in mind. we had 2 attempts built in, the first
protected by try, the 2nd not. based on the debug info we had and
looking at the code we deduced that both CreateObject() calls had
failed. we made the debug info more verbose now, so we'll have more
details if it fails again:

try {
fnRecordDebugInfo("try/catch to CreateObject() an
MSXML2.DOMDocument.4.0 Objekt");
oConf = Server.CreateObject("MSXML2.DOMDocument.4.0");
fnRecordDebugInfo("oConf ist ein MSXML2.DOMDocument.4.0 Objekt");
} catch (e) {
fnRecordDebugInfo("error: " + e.message);
fnRecordDebugInfo("try (no catch) to CreateObject() an
MSXML2.DOMDocument.3.0 Objekt");
oConf = Server.CreateObject("MSXML2.DOMDocument.3.0");
fnRecordDebugInfo("oConf ist ein MSXML2.DOMDocument.3.0 Objekt");
}

thanks again for your reply.

Tim Hanson

Re: (repost) intermittent failure to Server.CreateObject("MSXML2.DOMDocument.4.0") in glob

On Nov 3, 12:15 am, Tim wrote:
> thanks, Anthony,
>
> > MSXML doesn't depend on any external services so it seems unlikely that
> > service dependancies have any impact.
>
> good to know.
>
> > Also Application_Start only occurs on the first request not on reboot.
>
> on the first request after a reboot or iisreset, yes. but in our web
> farm, each www server is queried by the load balancer at regular
> intervals to see if it is online. that means shortly after a reboot,
> once IIS is up and the application is running, the first request will
> come from the load balancer. this first request needs to get some
> things initialized.
>
> > You haven't detailed what actually goes wrong. Is there an error or does it
> > simply not behave as expected?
>
> i gave all the details i had. the initial info was simply that the
> servers weren't "really up". iis was running, but the initialization
> steps had failed. we built in more logging of debug info and one retry
> of the load-info-from-machine.config step.
>
> this morning the problem didn't crop up, so I don't have more info.
> maybe this single retry will be enough to solve the problem. we'll
> keep an eye on it.
>
> > Try using MSXML3 instead does that work?
>
> thanks. i'll keep that in mind. we had 2 attempts built in, the first
> protected by try, the 2nd not. based on the debug info we had and
> looking at the code we deduced that both CreateObject() calls had
> failed. we made the debug info more verbose now, so we'll have more
> details if it fails again:
>
> try {
> fnRecordDebugInfo("try/catch to CreateObject() an
> MSXML2.DOMDocument.4.0 Objekt");
> oConf = Server.CreateObject("MSXML2.DOMDocument.4.0");
> fnRecordDebugInfo("oConf ist ein MSXML2.DOMDocument.4.0 Objekt");
> } catch (e) {
> fnRecordDebugInfo("error: " + e.message);
> fnRecordDebugInfo("try (no catch) to CreateObject() an
> MSXML2.DOMDocument.3.0 Objekt");
> oConf = Server.CreateObject("MSXML2.DOMDocument.3.0");
> fnRecordDebugInfo("oConf ist ein MSXML2.DOMDocument.3.0 Objekt");
> }
>
> thanks again for your reply.
>
> Tim Hanson


I suspect a couple of retries will be good enough.

What I have found is that randomly (well, I don't know the pattern, so
it appears random to me), ASP.Net ISAPI will lock the web.config file
for its purposes, which causes other code attempting to read/write to
them to fail. This is basically a random race.

I wonder if your MSXML code is simply the unfortunate victim of such
behavior, in which case the best recourse is to retry a couple of
times until you're able to read/write to the web.config file.

I can't explain why you didn't see this before, nor the relative
frequency, but I can tell you that the behavior does happen, so code
trying to read/write to web.config file *must* retry if they want to
avoid these random locked files.


//David
http://w3-4u.blogspot.com
http://blogs.msdn.com/David.Wang
//

Re: (repost) intermittent failure to Server.CreateObject("MSXML2.DOMDocument.4.0") in glob

Thanks, David,

that is *very* good to know and explains the apparent race. it's still
unclear to us why this got so much worse in recent days, but since we
added the re-try the problem hasn't reappeared.

thanks again,

Tim