Simple server-side spam filter needed

I have a virtual server environment (Unix-based) which means I can run
scripts and cronjobs, but I can't put in a server-side spam filter that
needs a close integration with the hosting environment's email system.
I would like to implement a fairly simple spam filter that just
accesses the email through POP and deletes unwanted mail based on basic
criteria, such as the sender id.

Can anyone give me any pointers? At the moment, I'm doing this
client-side, but I really want to be able to have this run unattended
directly from the server.

If necessary, I'm capable of modifying some existing script to meet my
exact needs, but not good enough to start from scratch.

- Beth

Re: Simple server-side spam filter needed

On comp.mail.misc, in
<1120012812.268591.105310@z14g2000cwz.googlegroups.com>,
"jaegerhome@hotmail.com" wrote:

> I have a virtual server environment (Unix-based) which means I
> can run scripts and cronjobs, but I can't put in a server-side
> spam filter that needs a close integration with the hosting
> environment's email system. I would like to implement a fairly
> simple spam filter that just accesses the email through POP
> and deletes unwanted mail based on basic criteria, such as the
> sender id.
>
> Can anyone give me any pointers? At the moment, I'm doing this
> client-side, but I really want to be able to have this run
> unattended directly from the server.
>
> If necessary, I'm capable of modifying some existing script to
> meet my exact needs, but not good enough to start from scratch.
>
> - Beth
>

Like a lot of people, I no longer help those who post to
technical groups using google.

Anyone that can make use of the information on these groups
can use a newsreader and get an account on a regular newsserver.
Or login to their home box if they are work, etc.

They are almost always trolls and spammers and crackers
trying to cover their tracks.

inetnum: 203.120.0.0 - 203.120.255.255
netname: PI-SG
descr: Pacific Internet Limited
descr: 89 Science Park Drive
descr: #04-09 Singapore 118261
country: SG
admin-c: WH1-AP
tech-c: WH1-AP
tech-c: AB19-AP
mnt-by: APNIC-HM
mnt-lower: PIPL-NOC-AP
remarks: This object can only be modified by APNIC hostmaster
remarks: If you wish to modify this object details please
remarks: send email to hostmaster@apnic.net with your organisation
remarks: account name in the subject line.
changed: hm-changed@apnic.net 20030902
status: ALLOCATED PORTABLE
source: APNIC

person: Wong Kok Hoou
address: Pacific Internet Pte Ltd
address: 89 Science Park Drive
address: #04-09/12, The Rutherford
address: Singapore 118261
phone: +65-771-0880
fax-no: +65-773-6812
e-mail: hoou@pacific.net.sg
nic-hdl: WH1-AP
mnt-by: PIPL-NOC-AP
changed: operations@pacific.net.sg 19991014
source: APNIC

person: Alan Boey
address: Pacific Internet Limited (Singapore)
address: Network Operations Centre
address: 89 Science Park Drive
address: #02-05/06, The Rutherford
address: Singapore 118261
phone: +65-6771-0557
fax-no: +65-6872-1011
e-mail: alanb@pacific.net.sg
nic-hdl: AB19-AP
remarks: Spam and Security: abuse@pacific.net.sg
remarks: Network Issues : noc@pacific.net.sg
mnt-by: PIL-NOC-AP
changed: alanb@pacific.net.sg 20020209
source: APNIC


AC

Re: Simple server-side spam filter needed

On Tue, 28 Jun 2005, jaegerhome@hotmail.com wrote:
> I have a virtual server environment (Unix-based) which means I can run
> scripts and cronjobs, but I can't put in a server-side spam filter that
> needs a close integration with the hosting environment's email system.
> I would like to implement a fairly simple spam filter that just
> accesses the email through POP and deletes unwanted mail based on basic
> criteria, such as the sender id.

It's probably better to do server-side spam filtering in the delivery
mechanism, instead of adding something using POP.

If this is for an enterprise, consider using something like Sophos'
PureMessage product.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Re: Simple server-side spam filter needed

Thanks, Mark. It's for personal use - I have the only account on this
particular mail server concerned. I'm using my email client software to
do this at the moment (it's one that can delete without downloading),
but I want to have something that will monitor and delete without me
needing to log in. I looked at products like Spam Assassin or the
server version of Mailwasher, but I think they need a tighter
integration to the mail server than I can get. I suspect Sophos may be
outside my budget - I'm looking at free or cheap.

Alan, my apologies. I agree that my email, unlike your reply, looks as
though it could have been written by a troll or spammer or cracker
trying desperately to remain anonymous. I will try not to give this
impression again in future, as I am sure that you are usually extremely
helpful. Thank you also for the information on who my ISP is, thus
adding to the body of knowledge available in this newsgroup.

- Beth

Re: Simple server-side spam filter needed

This is a MIME GnuPG-signed message. If you see this text, it means that
your E-mail or Usenet software does not support MIME signed messages.

--=_mimegpg-commodore.email-scan.com-30119-1120047390-0002
Content-Type: text/plain; format=flowed; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
X-Mime-Autoconverted: from 8bit to quoted-printable by mimegpg

Usenet Beavis writes:

> Like a lot of people, I no longer help those who post to
> technical groups using google.

Lucky for them.

> Anyone that can make use of the information on these groups
> can use a newsreader and get an account on a regular newsserver.
> Or login to their home box if they are work, etc.

Beavis, you obviously never had a meaningful job for even a single day of
your life.

Although there are certainly rare exceptions of employers that don't mind,
if you ever got caught monkeying around on company time and logging in to
your personal box from work, at most companies you will be fired on the
spot.

As typical, whatever â€=9Chelpâ€=9D Beavis gives, the right answer a=
lways lies 180
degrees to the opposite.

> They are almost always trolls and spammers and crackers
> trying to cover their tracks.

And your evidence of that, Beavis, isâ€=A6?

> inetnum: 203.120.0.0 - 203.120.255.255

Oh, look! Beavis knows how to use whois!


--=_mimegpg-commodore.email-scan.com-30119-1120047390-0002
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQBCwpEex9p3GYHlUOIRAqbDAJ0brCDqN8YFDc4+DjgF2AbDeC68gQCe LQ4v
d4uB1HSO4ZwaYFegvMIoeM4=
=V7QY
-----END PGP SIGNATURE-----

--=_mimegpg-commodore.email-scan.com-30119-1120047390-0002--

Re: Simple server-side spam filter needed

On Wed, 29 Jun 2005, jaegerhome@hotmail.com wrote:
> Thanks, Mark. It's for personal use - I have the only account on this
> particular mail server concerned. I'm using my email client software to
> do this at the moment (it's one that can delete without downloading),
> but I want to have something that will monitor and delete without me
> needing to log in. I looked at products like Spam Assassin or the
> server version of Mailwasher, but I think they need a tighter
> integration to the mail server than I can get.

I'm confused. When you say "a tighter integration to the mail server than
I can get", are you referring to an administrative limitation (as in you
don't have the authority to change the software), or are you referring to
a limitation in your technical ability?

For a personal mail server, Spam Assassin or Mailwasher would be suitable.
Normally, if a mail server only has a single user account, that user has
the necessary administrative access to that server (of course there are
exceptions).

If you're worried about your technical ability, don't. Determine what it
is that you need first, do the technical work to the best of your ability,
and ask questions when you hit a brick wall. Try to read through the
instructions; you may luck out and find that after doing each step
according to the instructions "it just works". [Trust me, it happens.]

If you have a question, try to be as specific and narrowly focused on the
problem as possible. Put another way, a report such as:
When I give the command "gorp zap", I get an error message
saying "gorp error 492 in zap". What do I do now?
is more useful than
gorp doesn't work
or
how do I use gorp

If it turns out that you need more hand-holding (let's face it, not
everybody is a computer nerd), your best bet may be to find a local expert
who can visit and set things up for you. For something as simple as this,
the fee may be as modest as a beer at the local pub.

> Alan, my apologies.

There is no need for you to apologize to him.

He has mental health issues. Anything said by him must be considered with
that fact in mind. The best thing to do is ignore him and not further
enable his bad behavior.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Re: Simple server-side spam filter needed

Mark Crispin wrote:
>
> I'm confused. When you say "a tighter integration to the mail server than
> I can get", are you referring to an administrative limitation (as in you
> don't have the authority to change the software), or are you referring to
> a limitation in your technical ability?
>
Administrative limitation, or at least it looks that way. It's a
third-party web hosting service that provides email (POP and SMTP), but
the email service is common to all of their customers rather than
within the virtual server environment. I can create accounts, set
quotas and define aliases but that's all. The hosting environment
itself allows me to run Unix scripts and appears to allow cron jobs.

At a technical level, I can handle basic Unix scripting and cgi stuff,
but I don't know where to start with email. If it's possible to do what
I want using scripts, I'm hoping to find some basic sample script for
retrieving and deleting mail, and could probably then work out the rest
myself.

Comments noted about this group's resident noise-generator.

- Beth

Re: Simple server-side spam filter needed

On Wed, 29 Jun 2005, jaegerhome@hotmail.com wrote:
> It's a
> third-party web hosting service that provides email (POP and SMTP), but
> the email service is common to all of their customers rather than
> within the virtual server environment. I can create accounts, set
> quotas and define aliases but that's all. The hosting environment
> itself allows me to run Unix scripts and appears to allow cron jobs.

Ah, OK, I understand now; something like a Cobalt box.

Check with your web hosting service to see if they already have a spam
filtering service. That would probably work a lot better for you than to
try to do something on your own.

As for doing something on your own, if you can set up procmail for your
account, then you can get into all the fun and laughter of creating
procmail scripts. procmail is a start (there are better choices, but
these are less widely available), and there are lots of tools designed to
work with procmail.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Re: Simple server-side spam filter needed

Mark Crispin wrote:
> Ah, OK, I understand now; something like a Cobalt box.
>
> Check with your web hosting service to see if they already have a spam
> filtering service. That would probably work a lot better for you than to
> try to do something on your own.
>
> As for doing something on your own, if you can set up procmail for your
> account, then you can get into all the fun and laughter of creating
> procmail scripts. procmail is a start (there are better choices, but
> these are less widely available), and there are lots of tools designed to
> work with procmail.
>
> -- Mark --

Not Cobalt, but same concept, yes. The hosting company does bayesian
spam filtering, but I want blacklist/whitelist too, which they don't
offer.

I must admit - I hadn't considered going to the extent of using
procmail, but yes, it probably would do what I wanted, and more.
Fortunately, my urgency on this decreased between yesterday and today
so I can afford to go through the fun and laughter stage. Thanks for
your help.

- Beth

Re: Simple server-side spam filter needed

Sorry, Alan, but I forgot to switch from google to a regular newsreader
for my follow-ups. I will try to do better next time. As compensation,
you may wish to note that my ISP today is Starhub Ltd., that my shoe
size is 9, and that my computer monitor faces south. Please feel free
to cut and paste this and use it to demonstrate your superior
investigative talents.
- Beth

Re: Simple server-side spam filter needed

jaegerhome@hotmail.com wrote:
> Mark Crispin wrote:
> >
> > I'm confused. When you say "a tighter integration to the mail server than
> > I can get", are you referring to an administrative limitation (as in you
> > don't have the authority to change the software), or are you referring to
> > a limitation in your technical ability?
> >
> Administrative limitation, or at least it looks that way. It's a
> third-party web hosting service that provides email (POP and SMTP), but
> the email service is common to all of their customers rather than
> within the virtual server environment. I can create accounts, set
> quotas and define aliases but that's all. The hosting environment
> itself allows me to run Unix scripts and appears to allow cron jobs.
>
> At a technical level, I can handle basic Unix scripting and cgi stuff,
> but I don't know where to start with email. If it's possible to do what
> I want using scripts, I'm hoping to find some basic sample script for
> retrieving and deleting mail, and could probably then work out the rest
> myself.

Okey, the picture is a bit clearer.

1. Put ~/.procmailrc in your home directory over at hosting
company. You can filter incoming emails any way you like, or
simply dump it to /dev/null. If they are unix, then they
probably use Procmail for local delivery.

2. If they don't allow ~/.procmailrc, then you have to log into POP
server, and do it through that. Of course, you can log in
remotely, or from locally at their hosting machine. For this
solution, you may want to browse
http://freshmeat.net/projects/popchecksh/
http://home.eol.ca/~parkw/index.html#spam
which is designed to delete (without downloading) Microsoft
Swen/Netsky/Sober worms from my remote POP account.

--
William Park , Toronto, Canada
ThinFlash: Linux thin-client on USB key (flash) drive
http://home.eol.ca/~parkw/thinflash.html
BashDiff: Super Bash shell
http://freshmeat.net/projects/bashdiff/

Re: Simple server-side spam filter needed

Thanks - I'm going to go with procmail as suggested.

- Beth

Re: Simple server-side spam filter needed

> Thanks - I'm going to go with procmail as suggested.

I highly recommend SpamProbe
http://spamprobe.sourceforge.net/

It's very easy to use via procmail. Once properly trained it will
demonstrate fantastic accuracy - I'm seeing over 99% accuracy.

--
Jem Berkes
Software design for Windows and Linux/Unix-like systems
http://www.sysdesign.ca/

Re: Simple server-side spam filter needed

Jem Berkes wrote:
>
> I highly recommend SpamProbe
> http://spamprobe.sourceforge.net/
>
> It's very easy to use via procmail. Once properly trained it will
> demonstrate fantastic accuracy - I'm seeing over 99% accuracy.
>
Nice but, weird as it may sound, I don't want Bayesian in this case. My
hosting service provides a fairly good Bayesian filter, marking the
mail as spam rather than deleting it, which I think is probably the
best approach for an ISP.

I'm intending to do automated deletion, and therefore want absolute
control over what is or is not deleted. I don't think a Bayesian
solution would meet that need, as it can't give an absolute guarantee
that it won't give false positives. My rules will be fairly simple, so
I don't have any problem with a manual rules solution.

- Beth