404 redirects "stolen" by provider

Hello,

I am running a website under apache and php where I do redirects on 404
errors:

apache conf:
ErrorDocument 404 /subapp_members/search_user.php

This is done to allow ULRs with usernames like this:
www.server.com/username

The PHP script search_user.php looks in a db if the user name is
existent and if yes shows his member page. If the name is not existent
it displays an internal 404 message.

This worked perfectly for recent years until now. Some users complain
that they do see advertisement instead of the page. A research showed
that they are using a provider called "unitymedia". As soon as a site
has a page not found error it redirects them to their own advertisement
page. This is true for all pages on the net. e.b. ebay.com/testing shows
their advertisement.

Has somebody an idea on how to fix that from my site?

Thank you for any help,

Merlin

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: 404 redirects "stolen" by provider

--=-6+/1NbnPJ0XDo6Lomv6s
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

On Fri, 2010-04-09 at 21:29 +0200, Merlin Morgenstern wrote:

> Hello,
>
> I am running a website under apache and php where I do redirects on 404
> errors:
>
> apache conf:
> ErrorDocument 404 /subapp_members/search_user.php
>
> This is done to allow ULRs with usernames like this:
> www.server.com/username
>
> The PHP script search_user.php looks in a db if the user name is
> existent and if yes shows his member page. If the name is not existent
> it displays an internal 404 message.
>
> This worked perfectly for recent years until now. Some users complain
> that they do see advertisement instead of the page. A research showed
> that they are using a provider called "unitymedia". As soon as a site
> has a page not found error it redirects them to their own advertisement
> page. This is true for all pages on the net. e.b. ebay.com/testing shows
> their advertisement.
>
> Has somebody an idea on how to fix that from my site?
>
> Thank you for any help,
>
> Merlin
>


It looks like the ISP is looking at the header response codes and
capturing the 404 ones. I've seen other ISP's do this and return a
search results page of their own sponsored content.

I think the best way round this is to use the .htaccess redirect rules
instead to deliver the correct content:

RewriteRule ^(.+) /users.php?username=$1

Of course that's only a simple example, and you might need to tweak it a
bit.

This way, no extra header codes are sent to the user, and can't be
captured by anyones ISP.

Thanks,
Ash
http://www.ashleysheridan.co.uk



--=-6+/1NbnPJ0XDo6Lomv6s--

Re: 404 redirects "stolen" by provider

On Fri, Apr 9, 2010 at 3:29 PM, Merlin Morgenstern wrote:
> Hello,
>
> I am running a website under apache and php where I do redirects on 404
> errors:
>
> apache conf:
> ErrorDocument 404 /subapp_members/search_user.php
>
> This is done to allow ULRs with usernames like this:
> www.server.com/username
>
> The PHP script search_user.php looks in a db if the user name is existent
> and if yes shows his member page. If the name is not existent it displays an
> internal 404 message.
>
> This worked perfectly for recent years until now. Some users complain that
> they do see advertisement instead of the page. A research showed that they
> are using a provider called "unitymedia". As soon as a site has a page not
> found error it redirects them to their own advertisement page. This is true
> for all pages on the net. e.b. ebay.com/testing shows their advertisement.
>
> Has somebody an idea on how to fix that from my site?
>
> Thank you for any help,
>
> Merlin
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

Is there anything in the request headers from these users that the
server could use to identify that they came from this provider? If so,
you could serve the 404 response with a 200 header. Effectively,
instead of having the server reply "No such page" to those request, it
is saying "Yes, there is no such page." It's a bit broken, IMO, but
you used to have to do something similar for WebTV. Their browser
would simply show a 404 dialog instead of the custom error page.

If you are feeling particularly spiteful toward "unitymedia," you
could even take the opportunity to add a message to your not-found
page for those people recommending a different provider that doesn't
hijack requests for missing content to another site. :-)

Andrew

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: 404 redirects "stolen" by provider

Merlin Morgenstern wrote:
> Hello,
>
> I am running a website under apache and php where I do redirects on 404
> errors:
>
> apache conf:
> ErrorDocument 404 /subapp_members/search_user.php
>
> This is done to allow ULRs with usernames like this:
> www.server.com/username
>
> The PHP script search_user.php looks in a db if the user name is
> existent and if yes shows his member page. If the name is not existent
> it displays an internal 404 message.
>
> This worked perfectly for recent years until now. Some users complain
> that they do see advertisement instead of the page. A research showed
> that they are using a provider called "unitymedia". As soon as a site
> has a page not found error it redirects them to their own advertisement
> page. This is true for all pages on the net. e.b. ebay.com/testing shows
> their advertisement.
>
> Has somebody an idea on how to fix that from my site?
>
> Thank you for any help,
>


yup, use a 303 See Other first of all

ErrorDocument 303 /subapp_members/search_user.php

then continue as normal; if you don't find the user set the status
header to 404 and show your internal 404.

They may never see the internal 404; but in many browsers they won't any
ways as most delegate it through to a "can't find, search with the
google?? :-)" type page.

regards!

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Re: 404 redirects "stolen" by provider

--=-oHDydotJJLNGRmmEvrJP
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

On Fri, 2010-04-09 at 21:03 +0100, Nathan Rixham wrote:

> Merlin Morgenstern wrote:
> > Hello,
> >
> > I am running a website under apache and php where I do redirects on 404
> > errors:
> >
> > apache conf:
> > ErrorDocument 404 /subapp_members/search_user.php
> >
> > This is done to allow ULRs with usernames like this:
> > www.server.com/username
> >
> > The PHP script search_user.php looks in a db if the user name is
> > existent and if yes shows his member page. If the name is not existent
> > it displays an internal 404 message.
> >
> > This worked perfectly for recent years until now. Some users complain
> > that they do see advertisement instead of the page. A research showed
> > that they are using a provider called "unitymedia". As soon as a site
> > has a page not found error it redirects them to their own advertisement
> > page. This is true for all pages on the net. e.b. ebay.com/testing shows
> > their advertisement.
> >
> > Has somebody an idea on how to fix that from my site?
> >
> > Thank you for any help,
> >
>
>
> yup, use a 303 See Other first of all
>
> ErrorDocument 303 /subapp_members/search_user.php
>
> then continue as normal; if you don't find the user set the status
> header to 404 and show your internal 404.
>
> They may never see the internal 404; but in many browsers they won't any
> ways as most delegate it through to a "can't find, search with the
> google?? :-)" type page.
>
> regards!
>


Sending any headers other than a 200 is still inviting this dubious ISP
to redirect it's customers to its own pages in an effort to gain
revenue. The ideal solution is to use mod_rewrite within the .htaccess
file, as this is completely transparent and the end user is non the
wiser.

Thanks,
Ash
http://www.ashleysheridan.co.uk



--=-oHDydotJJLNGRmmEvrJP--

Re: 404 redirects "stolen" by provider

--------------050908090009090300090208
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


Am 09.04.2010 21:53, schrieb Ashley Sheridan:
> On Fri, 2010-04-09 at 21:29 +0200, Merlin Morgenstern wrote:
>> Hello,
>>
>> I am running a website under apache and php where I do redirects on 404
>> errors:
>>
>> apache conf:
>> ErrorDocument 404 /subapp_members/search_user.php
>>
>> This is done to allow ULRs with usernames like this:
>> www.server.com/username
>>
>> The PHP script search_user.php looks in a db if the user name is
>> existent and if yes shows his member page. If the name is not existent
>> it displays an internal 404 message.
>>
>> This worked perfectly for recent years until now. Some users complain
>> that they do see advertisement instead of the page. A research showed
>> that they are using a provider called "unitymedia". As soon as a site
>> has a page not found error it redirects them to their own advertisement
>> page. This is true for all pages on the net. e.b. ebay.com/testing shows
>> their advertisement.
>>
>> Has somebody an idea on how to fix that from my site?
>>
>> Thank you for any help,
>>
>> Merlin
>>
>>
>
> It looks like the ISP is looking at the header response codes and
> capturing the 404 ones. I've seen other ISP's do this and return a
> search results page of their own sponsored content.
>
> I think the best way round this is to use the .htaccess redirect rules
> instead to deliver the correct content:
> RewriteRule ^(.+) /users.php?username=$1
>
> Of course that's only a simple example, and you might need to tweak it
> a bit.
>
> This way, no extra header codes are sent to the user, and can't be
> captured by anyones ISP.
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>

This sounds like the best solution to me. The only problem is that my
regex knowledge is pretty limited. The command:
RewriteRule ^(.+) /subapp_members/search_user.php

leads to an internal server error, the syntax seams ok?! Could you help
with the correct rewrite rule?

The script search_user.php gets the user name and looks it up in the db:
$parameter = explode('/', $_SERVER[REQUEST_URI]);
$user_name = addslashes($parameter[1]);

So I would need to redirect all potential 404s to that script. How could
that rewriterule look like?

--------------050908090009090300090208--

Re: 404 redirects "stolen" by provider

On 9 April 2010 22:20, Merlin Morgenstern wrote:
> This sounds like the best solution to me. The only problem is that my regex
> knowledge is pretty limited. The command:
> RewriteRule ^(.+) /subapp_members/search_user.php
>

The above rule will try to redirect everything to
/subapp_members/search_user.php. If you're looking to allow
example.com/username, then use something like:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /subapp_members/search_user.php?member=$1 [L]

This is likely to not do what you want from it, but it's the closest I
can guess as to what you want.

Have a read of http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html

--

WWW: http://plphp.dk / http://plind.dk
LinkedIn: http://www.linkedin.com/in/plind
Flickr: http://www.flickr.com/photos/fake51
BeWelcome: Fake51
Couchsurfing: Fake51


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: 404 redirects "stolen" by provider

Am 09.04.2010 22:58, schrieb Peter Lind:
> On 9 April 2010 22:20, Merlin Morgenstern wrote:
>
>> This sounds like the best solution to me. The only problem is that my regex
>> knowledge is pretty limited. The command:
>> RewriteRule ^(.+) /subapp_members/search_user.php
>>
>>
> The above rule will try to redirect everything to
> /subapp_members/search_user.php. If you're looking to allow
> example.com/username, then use something like:
>
> RewriteCond %{REQUEST_FILENAME} !-f
> RewriteCond %{REQUEST_FILENAME} !-d
> RewriteRule ^(.*)$ /subapp_members/search_user.php?member=$1 [L]
>
> This is likely to not do what you want from it, but it's the closest I
> can guess as to what you want.
>
> Have a read of http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html
>
>


This will not work, as I do have a bunch of other redirects inside the
page.

What might work is a rule, that redirects urls that do not have a full
stop or slash inside. Is this possible? My regex knowledge is
unfortunatelly pretty limited.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: 404 redirects "stolen" by provider

On 9 April 2010 23:08, Merlin Morgenstern wrote:
>
> Am 09.04.2010 22:58, schrieb Peter Lind:
>>
>> On 9 April 2010 22:20, Merlin Morgenstern  wr=
ote:
>>
>>>
>>> This sounds like the best solution to me. The only problem is that my
>>> regex
>>> knowledge is pretty limited. The command:
>>> RewriteRule ^(.+) /subapp_members/search_user.php
>>>
>>>
>>
>> The above rule will try to redirect everything to
>> /subapp_members/search_user.php. If you're looking to allow
>> example.com/username, then use something like:
>>
>> RewriteCond %{REQUEST_FILENAME} !-f
>> RewriteCond %{REQUEST_FILENAME} !-d
>> RewriteRule ^(.*)$ /subapp_members/search_user.php?member=3D$1 [L]
>>
>> This is likely to not do what you want from it, but it's the closest I
>> can guess as to what you want.
>>
>> Have a read of http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html
>>
>>
>
>
> This will not work, as I do have a bunch of other redirects inside the pa=
ge.
>
> What might work is a rule, that redirects urls that do not have a full st=
op
> or slash inside. Is this possible? My regex knowledge is unfortunatelly
> pretty limited.
>

Try:

RewriteRule ^([^./]+)$ /yourfile.php?variable=3D$1 [L]

Apart from that, rewrite rules work in order. If a rule above this
triggers and has the L flag, those below won't get processed.

--=20

WWW: http://plphp.dk / http://plind.dk
LinkedIn: http://www.linkedin.com/in/plind
Flickr: http://www.flickr.com/photos/fake51
BeWelcome: Fake51
Couchsurfing: Fake51


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: 404 redirects "stolen" by provider

header('HTTP/1.1 200 Ok');
in /subapp_members/search_user.php will do the job

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: 404 redirects "stolen" by provider

Am 09.04.2010 23:15, schrieb Peter Lind:
> On 9 April 2010 23:08, Merlin Morgenstern wrote:
>
>> Am 09.04.2010 22:58, schrieb Peter Lind:
>>
>>> On 9 April 2010 22:20, Merlin Morgenstern wrote:
>>>
>>>
>>>> This sounds like the best solution to me. The only problem is that my
>>>> regex
>>>> knowledge is pretty limited. The command:
>>>> RewriteRule ^(.+) /subapp_members/search_user.php
>>>>
>>>>
>>>>
>>> The above rule will try to redirect everything to
>>> /subapp_members/search_user.php. If you're looking to allow
>>> example.com/username, then use something like:
>>>
>>> RewriteCond %{REQUEST_FILENAME} !-f
>>> RewriteCond %{REQUEST_FILENAME} !-d
>>> RewriteRule ^(.*)$ /subapp_members/search_user.php?member=$1 [L]
>>>
>>> This is likely to not do what you want from it, but it's the closest I
>>> can guess as to what you want.
>>>
>>> Have a read of http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html
>>>
>>>
>>>
>>
>> This will not work, as I do have a bunch of other redirects inside the page.
>>
>> What might work is a rule, that redirects urls that do not have a full stop
>> or slash inside. Is this possible? My regex knowledge is unfortunatelly
>> pretty limited.
>>
>>
> Try:
>
> RewriteRule ^([^./]+)$ /yourfile.php?variable=$1 [L]
>
> Apart from that, rewrite rules work in order. If a rule above this
> triggers and has the L flag, those below won't get processed.
>
>

Hello Peter,

thank you for the rewrite rule. This workes perfectly!! The other ideas
like sending 200 afterwards did not work as 404 has already been sent.

Thank you for you help!

Merlin

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php