disable WebDav requests in IIS6 - return 40x rather than 501 ?

hi,

we have disabled WebDav on our website, but when someone tries, say, a
PROPFIND request the result iis returns appears always to be a 501 error on
our IIS6 / w2k3 Server(s) instead of, say, 400 or 403, which would be better
in at least two respects:
1) the visiter would realize the request won't work on retry (and might
therefore stop trying sooner);
2) for diagnostic purposes we treat 4-- errors as less serious and more
likely "the other end's problem" than our problem: we prefer 5-- errors to
signal something serious we need to be aware of.

i've googled quite a bit and found a couple clues, but nothing specific and
our webmasters have not found a solution yet either, so I'd be grateful for
any clues.

cheers,

Tim

Re: disable WebDav requests in IIS6 - return 40x rather than 501 ?

On Apr 9, 8:06=A0am, "tbh" wrote:
> hi,
>
> we have disabled WebDav on our website, but when someone tries, say, a
> PROPFIND request the result iis returns appears always to be a 501 error o=
n
> our IIS6 / w2k3 Server(s) instead of, say, 400 or 403, which would be bett=
er
> in at least two respects:
> 1) the visiter would realize the request won't work on retry (and might
> therefore stop trying sooner);
> 2) for diagnostic purposes we treat 4-- errors as less serious and more
> likely "the other end's problem" than our problem: we prefer 5-- errors to=

> signal something serious we need to be aware of.
>
> i've googled quite a bit and found a couple clues, but nothing specific an=
d
> our webmasters have not found a solution yet either, so I'd be grateful fo=
r
> any clues.
>
> cheers,
>
> Tim


There are no IIS features that allow you to control the HTTP status
code returned by the server for a 501.

1. which part of "Bad Request", "Forbidden", or "Not Implemented"
tells a visitor to retry?
2. Your diagnostic decision tree for HTTP status code does not match
what RFC 2616 specifies for HTTP. In my experience, both 4xx and 5xx
status codes indicate serious server-side issues

For example:
- if your homepage is missing, you get a 404
- if your homepage has a wrong ACL, you get a 401
- if your homepage requires SSL, you get a 403
- etc

These are serious problems accessing your website which have nothing
to do with "the other end".

Meanwhile:
- if a server application has an error, you get a 500
- if a server application crashed too many times, you get a 503

These are also serious problems accessing your website which have
nothing to do with "the other end".


//David
http://w3-4u.blogspot.com
http://blogs.msdn.com/David.Wang
//

Re: disable WebDav requests in IIS6 - return 40x rather than 501 ?

thanks, David,

"David Wang" wrote in message
news:fe53ae17-a8d2-4fd7-a026-9d10e823323f@z24g2000prf.google groups.com...

> There are no IIS features that allow you to control the HTTP status
> code returned by the server for a 501.

that's unfortunate. will IIS7 per chance improve this? that IIS chooses 501
for a disallowed request seems pretty arbitrary and in my particular case
suboptimal; a system that allows the webmaster to change the default would
be, imho, a good thing.

> 1. which part of "Bad Request", "Forbidden", or "Not Implemented"
> tells a visitor to retry?

nothing specifically in these definitions, but common sense: if you try to
get money from a bank machine, your subsequent behavior might (I predict)
vary depending on whether the machine says "your card doesn't work at this
bank" or "sorry, i'm broken today".

in my experience robots get discouraged from returning by 4-- results, but
keep coming back for more 5-- results. non-trivial web sites have to worry
about the best approaches to discouraging excessive robot nonsense, which
can stress systems, clog log files, and make troubleshooting "real" problems
more difficult.

> 2. Your diagnostic decision tree for HTTP status code does not match
> what RFC 2616 specifies for HTTP. In my experience, both 4xx and 5xx
> status codes indicate serious server-side issues

I disagree and I think your examples bear me out.

> For example:
> - if your homepage is missing, you get a 404

my homepage is (god will) never missing. if a user gets a 404 for my
homepage, he misspelled it.

> - if your homepage has a wrong ACL, you get a 401
> - if your homepage requires SSL, you get a 403
> These are serious problems accessing your website which have nothing
> to do with "the other end".

most 401s we return are either legitimate (the user wasn't logged in or has
inadequate rights) or the result of bugs involving apparent race conditions
in Windows networked file systems (either bugs in our code because we are
violating subtle rules we don't understand or bugs in the underling OS,
files system, drivers, etc.); these "buggy" 401s are the one example i'm
aware of where my (admittedly very vague) rule of thumb that
- 4xx means browser/agent should think again what it might be doing
wrong before retrying the URL
- 5xx means our web site has a problem, hopefully a temporary one

> Meanwhile:
> - if a server application has an error, you get a 500
> - if a server application crashed too many times, you get a 503
>
> These are also serious problems accessing your website which have
> nothing to do with "the other end".

no argument here. if we return 5-- we have either written sub-optimal code
or we have, e.g., DB or other server problems we hope will go away soon. in
general we hope the user will be patient and try again later.

cheers,

Tim

Re: disable WebDav requests in IIS6 - return 40x rather than 501 ?

oops. should have proof-read more carefully, sorry. of course I meant
"underlying" not "underling" and I meant to finish my sentence here:

these "buggy" 401s are the one example i'm aware of where my (admittedly
very vague) rule of thumb that
- 4xx means browser/agent should think again what it might be doing
wrong before retrying the URL
- 5xx means our web site has a problem, hopefully a temporary one
[is wrong.]

Re: disable WebDav requests in IIS6 - return 40x rather than 501 ?

To have your desired behavior, you can enable WebDAV such that those
requests are now "handled" by IIS and you can install an ISAPI Filter
to reject WebDAV with whatever status code you want.

However, IIS is going to stick with the RFC specification that if it
does not "handle" a verb, it returns a 501.

The problem with allowing change of defaults is complexity and
misconfiguration, which IMHO, would be a bad thing. The number of
users having problems due to complexity and misconfiguration far out-
number the users that configure things perfectly, so a system with
configurable defaults only seems nice.

From my perspective, saying that "Bad request" or "Forbidden" makes a
client stop retrying while "Not Implemented" makes a client keep
trying does not make sense. Of course, users and robots can do
whatever they want because that is a client's right, and you may argue
that it is in the web server's best interest to be able to control
client behavior, but that is beyond the interests of the web server.
- Retry on "Bad request" makes no sense. The first request was deemed
bad. Would retry of same request come back with different result?
Nope.
- Retry on "Forbidden" makes no sense. HTTP is stateless. The first
request was rejected. Would retry of same request be allowed? Nope.
- Retry on "Not Implemented" makes no sense. The server did not handle
the verb in the first request. Would retry of same request and verb
suddenly be allowed? Nope.

I do not believe the status-code changing feature deal with your
problem at all. The issue is that clients can always overwhelm servers
in terms of bandwidth (there are more clients than servers), so
aggressive robots which do not follow protocol can definitely kill
your server no matter what you do.

You are noting some artifacts of behavior that seem to control current
robots. However, without robots following protocol, there is no
guarantee that robots won't change their behavior pattern in the
future such that status-code changing feature is now ineffective. And
introduce features for an unbounded problem simply introduces legacy
baggage with no definitive benefits.

IIS features tend to solve general classes of problems and can be
narrowed down to solve specific problems. I do not believe your
problem has a bounded behavior, so it would not be appropriate for an
IIS-level solution.

You are basically saying that you interpret 5xx errors as "likely
server-side issues that you want to be aware of" and "4xx errors as
likely user-side issues that are less worrisome". I am pointing out
that such simplification as justification for changing 501 Not
implemented into a 4xx error doesn't work.

> > For example:
> > - if your homepage is missing, you get a 404
>
> my homepage is (god will) never missing. if a user gets a 404 for my
> homepage, he misspelled it.

If your website has perfect deployment, never has a missing/broken
link, etc, then you can dismiss 404 as user-side typo. However, this
statement is rarely true in general, and IIS is interested in options
that serve a general good, not specific user's need. Thus, your
rationale does not fit the requirements for a feature.

> > - if your homepage has a wrong ACL, you get a 401
> > - if your homepage requires SSL, you get a 403
> > These are serious problems accessing your website which have nothing
> > to do with "the other end".

If your website has perfect deployment, never has a mis-ACL'd
resource, or misconfigure Authentication settings, or expiring
anonymous user accounts or perfect user privilege configuration by
policy, then you can dismiss 401 as user-side mis-authentication with
username/password. Once again, this statement is rarely true in
general -- one simply cannot dismiss 401 issues as user-side. Now, you
may run the perfect website to never have this problem, but IIS is
interested in options that serve the greater good, not specific user
need.

Basically, I understand that you have a real problem, but I do not see
any general-purpose web-server feature to resolve it. Allowing user
defined status code is NOT a solution that belongs in a web server as
a feature. I believe this is a problem that users can solve by
implementing their own code.


//David
http://w3-4u.blogspot.com
http://blogs.msdn.com/David.Wang
//

Re: disable WebDav requests in IIS6 - return 40x rather than 501 ?

thanks, David,

i think you've convinced me. if anything i would prefer the 400s and the
500s to be grouped differently (and that clearly is way too big for me). :)

so I guess I will have to think about the ISAPI filter you mention or put
more smarts in our dashboards, as it were. :)

thanks for sharing your thoughts.

cheers, Tim

Re: disable WebDav requests in IIS6 - return 40x rather than 501 ?

On Apr 10, 6:26=A0am, "tbh" wrote:
> thanks, David,
>
> i think you've convinced me. if anything i would prefer the 400s and the
> 500s to be grouped differently (and that clearly is way too big for me). :=
)
>
> so I guess I will have to think about the ISAPI filter you mention or put
> more smarts in our dashboards, as it were. :)
>
> thanks for sharing your thoughts.
>
> cheers, Tim


Thanks for listening.

Unfortunately, the status codes weren't organized with your thoughts
in mind. You should have given them feedback on the specification a
decade ago! ;-)

The way I view HTTP status codes is:
1xx -- everything is fine
2xx -- everything is fine
3xx -- requires more interaction, but everything is still fine
4xx -- warning, there is a problem. The cause could either be client
or server; answer depends on server configuration and client's request
5xx -- error, there is a problem. The cause could be either the
endpoint server or intervening network device; answer depends on
server configuration and network topology configuration


//David
http://w3-4u.blogspot.com
http://blogs.msdn.com/David.Wang
//