I know I can prevent robots by creating a robot text file. but how do
you know what to put in. for example the folowing user_agent
Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)
problem what is the exact name to put in the text file as this in not
always the same.
Is their a another server variable to acompany
$_SERVER['HTTP_USER_AGENT'];
Desmond.
Gazing into my crystal ball I observed Desmond
in news:1181469765.276407.219910@k79g2000hse.googlegroups.com:
> I know I can prevent robots by creating a robot text file. but how do
> you know what to put in. for example the folowing user_agent
> Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)
>
> problem what is the exact name to put in the text file as this in not
> always the same.
>
> Is their a another server variable to acompany
> $_SERVER['HTTP_USER_AGENT'];
>
> Desmond.
>
I don't restrict by user agent, I just do it on a page or folder basis.
You could go kuku trying to account for every bot.
--
Adrienne Boswell at Home
Arbpen Web Site Design Services
http://www.cavalcade-of-coding.info
Please respond to the group so others can share
On 10 Jun, 17:08, Adrienne Boswell
> Gazing into my crystal ball I observed Desmond
> innews:1181469765.276407.219910@k79g2000hse.googlegroups.com :
>
> > I know I can prevent robots by creating a robot text file. but how do
> > you know what to put in. for example the folowing user_agent
> > Mozilla/5.0 (Twiceler-0.9http://www.cuill.com/twiceler/robot.html)
>
> > problem what is the exact name to put in the text file as this in not
> > always the same.
>
> > Is their a another server variable to acompany
> > $_SERVER['HTTP_USER_AGENT'];
>
> > Desmond.
>
> I don't restrict by user agent, I just do it on a page or folder basis.
> You could go kuku trying to account for every bot.
>
> --
> Adrienne Boswell at Home
> Arbpen Web Site Design Serviceshttp://www.cavalcade-of-coding.info
> Please respond to the group so others can share
I do do it on a page or folder basis. but I would like
On 10 Jun, 17:08, Adrienne Boswell
> Gazing into my crystal ball I observed Desmond
> innews:1181469765.276407.219910@k79g2000hse.googlegroups.com :
>
> > I know I can prevent robots by creating a robot text file. but how do
> > you know what to put in. for example the folowing user_agent
> > Mozilla/5.0 (Twiceler-0.9http://www.cuill.com/twiceler/robot.html)
>
> > problem what is the exact name to put in the text file as this in not
> > always the same.
>
> > Is their a another server variable to acompany
> > $_SERVER['HTTP_USER_AGENT'];
>
> > Desmond.
>
> I don't restrict by user agent, I just do it on a page or folder basis.
> You could go kuku trying to account for every bot.
>
> --
> Adrienne Boswell at Home
> Arbpen Web Site Design Serviceshttp://www.cavalcade-of-coding.info
> Please respond to the group so others can share
Ok let's put it another way. Could I then allow googl Yahoo (and a few
others) and kill off the rest