banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

In my htaccess file, I have some restrictions using the following
rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
any browser with baidu in it name.

How can I set up a rule to ban this particular browser: Mozilla/4.0
(compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
my site by this browser is spam. I thought of using ^.*SV1.*$ but
unfortunately, there are legitimate browsers with SV1 in parts of
their name.

Thanks for the help.

--F

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On May 16, 6:25 pm, "no.rep...@comcast.net"
wrote:
> In my htaccess file, I have some restrictions using the following
> rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> any browser with baidu in it name.
>
> How can I set up a rule to ban this particular browser: Mozilla/4.0
> (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> my site by this browser is spam. I thought of using ^.*SV1.*$ but
> unfortunately, there are legitimate browsers with SV1 in parts of
> their name.
>
> Thanks for the help.
>
> --F

when u say "ban" i assume you mean, "try to stop"
any real spammer will use a valid IE6 user agent. this method does not
work unfortunately.
better would be to use mod_security to rule the type of request out
quietly, and accept that any public website (or hospital or library)
will be a target, and take steps to deal with the results rather than
gradually close the doors to people based on the way they look, which
will only ban those with no skill to hurt your interests anyway.

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On 16 May 2007 10:25:21 -0700 no.reply7@comcast.net said
> In my htaccess file, I have some restrictions using the following
> rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> any browser with baidu in it name.
>
> How can I set up a rule to ban this particular browser: Mozilla/4.0
> (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> my site by this browser is spam. I thought of using ^.*SV1.*$ but
> unfortunately, there are legitimate browsers with SV1 in parts of
> their name.
>
> Thanks for the help.


Looks like a good UA to me, but if you want to stop it, try
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1; SV1\)$

Sig

--
http://koiclubsandiego.org/comment/?r=8
3186fdb515784af427d7bd83018826b9

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On May 16, 5:08 pm, sig wrote:
> On 16 May 2007 10:25:21 -0700 no.rep...@comcast.net said
>
> > In my htaccess file, I have some restrictions using the following
> > rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> > RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> > any browser with baidu in it name.
>
> > How can I set up a rule to ban this particular browser: Mozilla/4.0
> > (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> > my site by this browser is spam. I thought of using ^.*SV1.*$ but
> > unfortunately, there are legitimate browsers with SV1 in parts of
> > their name.
>
> > Thanks for the help.
>
> Looks like a good UA to me, but if you want to stop it, try
> RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1; SV1\)$
>
> Sig
>
> --http://koiclubsandiego.org/comment/?r=8
> 3186fdb515784af427d7bd83018826b9

I wished it worked but it didn't. It gave me a 500 error when I don't
use that browser.

Here is my problem: I get bombarded with bogus GET request with
sources such as
http://www.toyamall.com/
http://www.blackprofessional.co.uk/
http://www.jusoo.ch/
http://www.tscheiner-huette.info/
http://www.francorpconnect.com/
etc...etc...

A large portion come from Korea and all from the exact browser
mentioned above. Any thought on how to make it go away?

Any thought would be greatly appreciated.

Thanks.

--F

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On May 17, 5:06 pm, "no.rep...@comcast.net"
wrote:
> On May 16, 5:08 pm, sig wrote:
>
>
>
> > On 16 May 2007 10:25:21 -0700 no.rep...@comcast.net said
>
> > > In my htaccess file, I have some restrictions using the following
> > > rule: RewriteCond %{HTTP_USER_AGENT}. One example would be:
> > > RewriteCond %{HTTP_USER_AGENT} ^.*baidu.*$ [NC,OR], which restricts
> > > any browser with baidu in it name.
>
> > > How can I set up a rule to ban this particular browser: Mozilla/4.0
> > > (compatible; MSIE 6.0; Windows NT 5.1; SV1) ? 95% of requests made to
> > > my site by this browser is spam. I thought of using ^.*SV1.*$ but
> > > unfortunately, there are legitimate browsers with SV1 in parts of
> > > their name.
>
> > > Thanks for the help.
>
> > Looks like a good UA to me, but if you want to stop it, try
> > RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1; SV1\)$
>
> > Sig
>
> > --http://koiclubsandiego.org/comment/?r=8
> > 3186fdb515784af427d7bd83018826b9
>
> I wished it worked but it didn't. It gave me a 500 error when I don't
> use that browser.
>
> Here is my problem: I get bombarded with bogus GET request with
> sources such

sources? do you mean referers?
If so you are probably being log spammed. If you logs are public, the
referers are logged and your logs indexed by google, which increases
the ranking for these sites.
Implement a simple login/basic auth protection for your logs and they
will get bored.
However if you run an open proxy, and these requests are via your
server, lock it down immediately.


ashttp://www.toyamall.com/http://www.blackprofessional.co.uk /http://www.jusoo.ch/http://www.tscheiner-huette.info/http:/ /www.francorpconnect.com/
> etc...etc...
>
> A large portion come from Korea and all from the exact browser
> mentioned above. Any thought on how to make it go away?
>

does your website sell to Korea, if not ban Korea's entire IP range,
this is similar to DNSBL for spam emails.
Use a dns tool to locate the network provider's IP block for this
user, and block them.
Implement a DNSBL script to test IPs, and dynamically ban them by
altering a precondition file (or .htaccess with a series of Deny from
xxx.xxx rules.
Get a list of bogus referers for this IP, and add them to a ban list.


> Any thought would be greatly appreciated.

Do the requests have a particular preferred time, implement time
dependant blocking of the smallest effective IP range.
If your site is itself involved with undesirable content, I would
imagine your site will always be a target for other similar sites, I
imagine to that sites like those dont always operate ethically. If
it's clean as a whistle then its probably just log spamming.
As you talk about "SPAM" do you mean these requests are just bogus
nonsense requests, or do you mean they are scanning you, or do you
mean they are sending spam through your forms, if the former two, you
could use a firewall in between your server and the internet to block
certain types of requests, Im not sure if you read my previous post
but even a software web application firewall like mod_security is
effective.
If the latter, then implement some kind of turing test, like a
captcha, and code with security in mind to keep unsophisticated
spammers out.

But ultimately, if your site is public and a target for sophisticated
attacks, (which it doesnt appear to be) then theres little you can do.
>
> Thanks.
>
> --F

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On 17 May 2007 09:06:24 -0700 no.reply7@comcast.net said

>
> Here is my problem: I get bombarded with bogus GET request with
> sources such as
> http://www.toyamall.com/
> http://www.blackprofessional.co.uk/
> http://www.jusoo.ch/
> http://www.tscheiner-huette.info/
> http://www.francorpconnect.com/
> etc...etc...
>
> A large portion come from Korea and all from the exact browser
> mentioned above. Any thought on how to make it go away?
>
> Any thought would be greatly appreciated.

Order Deny,Allow
Deny from www.toyamall.com
Deny from www.blackprofessional.co.uk
Deny from www.jusoo.ch
Deny from www.tscheiner-huette.info
Deny from www.francorpconnect.com

Sig
--
http://koiclubsandiego.org/comment/?r=8

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On May 17, 8:03 pm, sig wrote:
> On 17 May 2007 09:06:24 -0700 no.rep...@comcast.net said
>
>
>
> > Here is my problem: I get bombarded with bogus GET request with
> > sources such as
> >http://www.toyamall.com/
> >http://www.blackprofessional.co.uk/
> >http://www.jusoo.ch/
> >http://www.tscheiner-huette.info/
> >http://www.francorpconnect.com/
> > etc...etc...
>
> > A large portion come from Korea and all from the exact browser
> > mentioned above. Any thought on how to make it go away?
>
> > Any thought would be greatly appreciated.
>
> Order Deny,Allow
> Deny fromwww.toyamall.com
> Deny fromwww.blackprofessional.co.uk
> Deny fromwww.jusoo.ch
> Deny fromwww.tscheiner-huette.info
> Deny fromwww.francorpconnect.com
>
> Sig
> --http://koiclubsandiego.org/comment/?r=8

this would only work if the requests are from these domains, but it
appears these are the refers because they have http:// in front.
Worrying about log files will get you ina fiddle wihtout really
acheiving anything, unless these guys are serious slurpers of
bandwidth/CPU or something else, who cares how many idiotic request
they make unless they are getting something out of it you dont want
them to. Just prevent the onvious forms of attacks and then you will
sleep at night no matter who gets on the end of your websites, which
are as I keep saying, public - potential millions of people can do
this, are you going to ban them all, and then stay up at night
worrying about your growing ban list, and how out of date its getting?!

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On May 17, 12:13 pm, shimmyshack wrote:
> On May 17, 8:03 pm, sig wrote:
>
>
>
>
>
> > On 17 May 2007 09:06:24 -0700 no.rep...@comcast.net said
>
> > > Here is my problem: I get bombarded with bogus GET request with
> > > sources such as
> > >http://www.toyamall.com/
> > >http://www.blackprofessional.co.uk/
> > >http://www.jusoo.ch/
> > >http://www.tscheiner-huette.info/
> > >http://www.francorpconnect.com/
> > > etc...etc...
>
> > > A large portion come from Korea and all from the exact browser
> > > mentioned above. Any thought on how to make it go away?
>
> > > Any thought would be greatly appreciated.
>
> > Order Deny,Allow
> > Deny fromwww.toyamall.com
> > Deny fromwww.blackprofessional.co.uk
> > Deny fromwww.jusoo.ch
> > Deny fromwww.tscheiner-huette.info
> > Deny fromwww.francorpconnect.com
>
> > Sig
> > --http://koiclubsandiego.org/comment/?r=8
>
> this would only work if the requests are from these domains, but it
> appears these are the refers because they have http:// in front.
> Worrying about log files will get you ina fiddle wihtout really
> acheiving anything, unless these guys are serious slurpers of
> bandwidth/CPU or something else, who cares how many idiotic request
> they make unless they are getting something out of it you dont want
> them to. Just prevent the onvious forms of attacks and then you will
> sleep at night no matter who gets on the end of your websites, which
> are as I keep saying, public - potential millions of people can do
> this, are you going to ban them all, and then stay up at night
> worrying about your growing ban list, and how out of date its getting?!- Hide quoted text -
>
> - Show quoted text -

My site is clean as a whistle so it might be log spamming. I'll look
into mod_secure. and yes, I was talking about referrers, not sources.

Thanks.

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; WindowsNT 5.1; SV1)

shimmyshack wrote:
> On May 17, 5:06 pm, "no.rep...@comcast.net"
> wrote:


> sources? do you mean referers?
> If so you are probably being log spammed. If you logs are public, the
> referers are logged and your logs indexed by google, which increases
> the ranking for these sites.
> Implement a simple login/basic auth protection for your logs and they
> will get bored.

I'm not seeing much evidence that they get bored. My logs have never
been public, but I see log spam on a regular basis. One of the joys of
running an internet facing web server.

Jim

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

Post removed (X-No-Archive: yes)

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; WindowsNT 5.1; SV1)

still me wrote:
> On Mon, 21 May 2007 10:52:16 -0400, Jim Hayter
> wrote:
>
>> I'm not seeing much evidence that they get bored. My logs have never
>> been public, but I see log spam on a regular basis. One of the joys of
>> running an internet facing web server.
>>
>> Jim
>
> Jim:
>
> Can you tell me more about what you mean by "log spam"? Are you
> saying that they are trying to get into your logs just so you can see
> their url, or that they are getting into your logs and then getting
> your logs indexed by google (for you) to increase their own visibility
> ?
>
>

I meant the latter. It is my impression that since some sites publish
their logs, there are those who "spam" the logs in order to get their
URLs indexed. I find GET requests with arguments added in that have
nothing to do with my sites. The arguments have text and URLs that
point to sites all over the world. Some is pr0n, a lot is
pharmaceutical, all of it looks like sites I'd never want to visit.

Jim

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

Post removed (X-No-Archive: yes)

Re: banning this browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

On May 24, 1:52 am, still me wrote:
> On Wed, 23 May 2007 15:33:21 -0400, Jim Hayter
>
> wrote:
>
> >I meant the latter. It is my impression that since some sites publish
> >their logs, there are those who "spam" the logs in order to get their
> >URLs indexed.
>
> Thanks. Quite honestly, I've never understood why those sites publish
> their logs. I can't imagine who cares to see their log files - and
> they just add more noise to web searches.

they are also a boon to hackers - session ids (such as those on admin
urls) and other nasties can often appear in logs, providing another
vector for exploitation.