procmail trys to store in wrong folder

I have qmail with maildir mailboxes. I want to use bogofilter to
filter my spam and have procmail put it in a subfolder in my mailbox.
Below is what I have but is not working:

-------.qmail-------:
|preline /usr/bin/procmail
/var/qmail/mailnames/netkev.com/kevin/.procmailrc || exit 111

---------.procmailrc------:
MAILDIR=/var/qmail/mailnames/####.com/kevin
DEFAULT=/var/qmail/mailnames/####.com/kevin/Maildir/
LOGFILE = /var/qmail/mailnames/####.com/kevin/procmail-####.com.log
VERBOSE=yes
DEBUG=yes

:0fw
| bogofilter -u -l -d /var/qmail/mailnames/####.com/kevin/

:0e
{ EXITCODE=75 HOST }

# file the mail to .Junk subdirectory of maildir if it's spam.

:0
* ^X-Bogosity: Spam, tests=bogofilter
/var/qmail/mailnames/####.com/kevin/Maildir/.Junk/


------log-----:
May 14 22:22:40 freshinstall qmail: 1116134560.889505 status: local
0/10 remote 0/20
May 14 22:22:43 freshinstall qmail: 1116134563.888593 starting delivery
96: msg 3306811 to local 4-#####@#####.com
May 14 22:22:43 freshinstall qmail: 1116134563.888678 status: local
1/10 remote 0/20
May 14 22:22:43 freshinstall bogofilter[5089]: X-Bogosity: Unsure,
spamicity=0.520000, version=0.94.11
May 14 22:22:43 freshinstall qmail: 1116134563.912116 delivery 96:
deferral: procmail:_Couldn't_create_"/var/mail/popuser"/
May 14 22:22:43 freshinstall qmail: 1116134563.912205 status: local
0/10 remote 0/20

As you can see, executing procmail and procmail executes bogofilter, it
scores it and then procmail tries to put the mail in some other
mailbox? The user, "popuser" is somehow selected because the directory,
/var/qmail/mailnames/####.com/kevin is owned by popuser for other
reasons that must stay that way.

Put simply, I want my mail to goto:
/var/qmail/mailnames/####.com/kevin/Maildir/ if not spam, and
/var/qmail/mailnames/####.com/kevin/Maildir/.Junk/ if it is spam

Re: procmail trys to store in wrong folder

On comp.mail.misc, in
<1116135170.717597.3570@o13g2000cwo.googlegroups.com>,
"netkev@gmail.com" wrote:

> I have qmail with maildir mailboxes. I want to use bogofilter
> to filter my spam and have procmail put it in a subfolder in my
> mailbox. Below is what I have but is not working:
>

> -------.qmail-------:
>|preline /usr/bin/procmail
> /var/qmail/mailnames/netkev.com/kevin/.procmailrc || exit 111
>

That's different than most ~/.forward entries to pipe mail
from an MTA to procmail that I've seen. But if it works...

> ---------.procmailrc------:
SHELL=/bin/bash # my guess
PATH= # echo $PATH
> MAILDIR=/var/qmail/mailnames/####.com/kevin
> DEFAULT=/var/qmail/mailnames/####.com/kevin/Maildir/
> LOGFILE = /var/qmail/mailnames/####.com/kevin/procmail-####.com.log
> VERBOSE=yes
> DEBUG=yes # don't know this one and can't find a reference
# VERBOSE=yes IS "debug" to my knowledge.
>
>:0fw
>| bogofilter -u -l -d /var/qmail/mailnames/####.com/kevin/
>
#>:0e
#> { EXITCODE=75 HOST }

>
> # file the mail to .Junk subdirectory of maildir if it's spam.
>
>:0
> * ^X-Bogosity: Spam, tests=bogofilter
> .Junk/ # relative to $MAILDIR as defined above
>
>

:0
${DEFAULT}



HTH

AC

--
alanconnor AT earthlink DOT net
Use your real return address or I'll never know you
even tried to mail me. http://tinyurl.com/2t5kp

Re: procmail trys to store in wrong folder

On comp.mail.misc, in <1116135170.717597.3570@o13g2000cwo.googlegroups.com>, "netkev@gmail.com" wrote:
>
>
> I have qmail with maildir mailboxes. I want to use bogofilter to
> filter my spam and have procmail put it in a subfolder in my mailbox.
> Below is what I have but is not working:
>
> -------.qmail-------:
>|preline /usr/bin/procmail
> /var/qmail/mailnames/netkev.com/kevin/.procmailrc || exit 111
>
> ---------.procmailrc------:
> MAILDIR=/var/qmail/mailnames/####.com/kevin
> DEFAULT=/var/qmail/mailnames/####.com/kevin/Maildir/
> LOGFILE = /var/qmail/mailnames/####.com/kevin/procmail-####.com.log
> VERBOSE=yes
> DEBUG=yes
>
>:0fw
>| bogofilter -u -l -d /var/qmail/mailnames/####.com/kevin/
>
>:0e
> { EXITCODE=75 HOST }

I get that now, I think: If the mail is so screwed up that
bogofilter won't accept it, dump it.

If so, sounds good.

But what's the purpose of setting the EXITCODE to 75?
Just to have an entry in verbose logging so that you
can see a mail was dumped?

{ LASTFOLDER="Bogofilter Dumped Bad Mail" HOST }

will make an entry with non-verbose logging.



AC

Re: procmail trys to store in wrong folder

Thanks for the response Alan. I made the necessary changes but I still
get the same error :( For some reason, procmail really wants to
deliver to /var/mail/popuser despite my definition of default to be
somewhere else.

P.S. I beleive the error code is to notify qmail that we failed and to
re-queue the mail for another time.

Re: procmail trys to store in wrong folder

Did some more research and found the following in the procmail mailing
lists. Apparently, if you call procmail with -m, then it will not
default to the owning user's mailbox.
"According to the procmailrc manpage,
ORGMAIL /var/mail/$LOGNAME
(Unless -m has been specified, in
which case it is unset)"

However, this only elminated the error msg, the email still doesnt get
delivered. Now the error is just: "May 14 22:22:43 freshinstall qmail:
1116134563.912116 delivery 96:
deferral:".

I think I found more descriptive evidence of the problem though, I was
posting the log for qmail. Here's the verbose procmail log:
procmail: [3206] Sun May 15 09:56:33 2005
procmail: Executing
"bogofilter,-u,-l,-d,/var/qmail/mailnames/####.com/kevin/"
procmail: [3206] Sun May 15 09:56:33 2005
procmail: Program failure (2) of "bogofilter"
procmail: Rescue of unfiltered data succeeded
procmail: [3206] Sun May 15 09:56:33 2005
procmail: Assigning "EXITCODE=75"
procmail: Assigning "HOST"
>From mfvhc@heart.org Sun May 15 16:56:33 2005
Subject: Purchase All Your Medicines Now Online. No prior
Prescription.
Folder:
0

This leads me to beleive that it thinks bogofilter is failing but the
bogofilter output in the qmail log looks like it successfully scored
the mail.

Re: procmail trys to store in wrong folder

On Sun, May 15 2005, netkev@gmail.com wrote:

> Did some more research and found the following in the procmail mailing
> lists. Apparently, if you call procmail with -m, then it will not
> default to the owning user's mailbox.
> "According to the procmailrc manpage,
> ORGMAIL /var/mail/$LOGNAME
> (Unless -m has been specified, in
> which case it is unset)"
>
> However, this only elminated the error msg, the email still doesnt get
> delivered. Now the error is just: "May 14 22:22:43 freshinstall
> qmail:
> 1116134563.912116 delivery 96:
> deferral:".

This is a fairly common problem when using Maildir with procmail. You
may need to recompile procmail so that ORGMAIL is set to $HOME. See the
procmail section of Life With Qmail:



and this message from the procmail mailing list (if you haven't already
seen it):



HTH.

--
Neil

Re: procmail trys to store in wrong folder

On comp.mail.misc, in
<1116176371.697841.238560@f14g2000cwb.googlegroups.com>,
"netkev@gmail.com" wrote:

> Did some more research and found the following in the procmail
> mailing lists. Apparently, if you call procmail with -m, then
> it will not default to the owning user's mailbox.

> "According to the procmailrc manpage, ORGMAIL
> /var/mail/$LOGNAME (Unless -m has been specified, in which case
> it is unset)"
>
> However, this only elminated the error msg, the email still
> doesnt get delivered. Now the error is just: "May 14 22:22:43
> freshinstall qmail: 1116134563.912116 delivery 96: deferral:".

Did you set ORGMAIL at the top of the procmailrc?

>
> I think I found more descriptive evidence of the problem though, I was
> posting the log for qmail. Here's the verbose procmail log:
> procmail: [3206] Sun May 15 09:56:33 2005
> procmail: Executing
> "bogofilter,-u,-l,-d,/var/qmail/mailnames/####.com/kevin/"
> procmail: [3206] Sun May 15 09:56:33 2005
> procmail: Program failure (2) of "bogofilter"
> procmail: Rescue of unfiltered data succeeded
> procmail: [3206] Sun May 15 09:56:33 2005
> procmail: Assigning "EXITCODE=75"
> procmail: Assigning "HOST"
>>From mfvhc@heart.org Sun May 15 16:56:33 2005
> Subject: Purchase All Your Medicines Now Online. No prior
> Prescription.
> Folder:
> 0

Are you sure that a lot of this isn't a bogofilter problem?
Have you tried commenting out the bogofilter filter recipe
for a couple of runs?

>
> This leads me to beleive that it thinks bogofilter is failing
> but the bogofilter output in the qmail log looks like it
> successfully scored the mail.
>

Weird. And the tail end of the log there looks perfect.

Better take Neil's advice.

What's your MAIL variable set to and is it exported?

echo $MAIL

There's also a MAILDIR environment variable in bash.

export MAILDIR=

Have you checked to see if these are set in (debian paths)
/etc/profile /etc/bash.bashrc /etc/login.defs or in any
of the boot time scripts in /etc/init.d?

How about /etc/procmailrc or the like?

Good luck! Keep us posted. This challenge and it's
solution should go into the Usenet Archives for this
group.

AC

Re: procmail trys to store in wrong folder

Found solution through the bogofilter and procmail mailing lists(I had
3 problems):
..procmailrc
First problem was that my rule that caught any errors from from
executing bogofilter was instructed to re-send the e-mail back to qmail
for a rety later on.
:0e
{ EXITCODE=75 HOST }
I had set the HOST variable at the top and it was not the one set
during compile of procmail (i use an rpm) so procmail was just exiting
w/o errors(which is a bug i think). I deleted the HOST=###.com and now
procmail was feeding me debug info and more errors about bogofilter
failing.

Second problem was that bogofilter was executing correctly, but
procmail was saying that bogofilter exited with error 2. The problem
was that the latest version of bogofilter no longer uses just exit 0 or
1. It now uses, 0 for ham, 1 for spam and 2 for unsure. Since I had a
small set of emails(this is a new server) t o train it with, most of
the e-mails fell into that category and bogo was exiting with error 2,
which procmail thinks is bad. The solution was to add "-p -e" to the
bogofilter command options which makes it use only 0 or 1. Unsure and
ham get 0, spam gets 1 I beleive.

Third problem was that I was misundersand the way that procmail deals
with the MAILDIR variable. I read that when using the maildir mailbox
format(different from the MAILDIR variable btw), you must append a
forward-slash to the end to signify to procmail that it is to read or
write using the maildir mailbox format. This is correct but my problem
was that I was doing the following:
MAILDIR=/var/qmail/mailnames/###.com/kevin/Maildir
....
:0
* ^X-Bogosity: Spam, tests=bogofilter
/.Junk/

since i wanted it to dump the mail in
/var/qmail/mailnames/##.com/kevin/Maildir/.Junk. This is incorrect
because procmail does a change directory on the MAILDIR variable which
puts it in the directory: /var/qmail/mailnames/##.com/kevin/Maildir.
/.Junk/ is relative to / and it ofcourse cannot write there :).
Instead, I changed it to .Junk/ and it works. Also note that if you
want to dump it directly into
/var/qmail/mailnames/##.com/kevin/Maildir, you set MAILDIR as I did and
then in the rule, use ./ which ofcourse makes sense.

Flame protection: Obviously, I was very confused despite my degree in
computer science and 6 years experience using linux. I did read the
man pages and scoured google for 2 days before posting. Ofcourse, the
man pages make sense now that everything works but before, they did
not. If anyone still feels the need to flame me, go ahead. My problem
is solved so I won't be back here to read it :).

I'll post my finished .qmail and .procmailrc files for prosperity. If
I can save one other person from tearing their hair out and yelling at
their fiance in frustration...
..qmail
------------
|preline /usr/bin/procmail -m
/var/qmail/mailnames/##.com/kevin/.procmailrc || exit 111
---------
apparently, this executes procmail using the specified .procmailrc, w/o
trying to write to the owning user's mailbox. If it fails, it exists
with 111 which tells qmail to leave the message in the queue and try
later. The -m is necessary in my case since the user who owns the
qmail mailbox _directory_ is NOT the user who owns the mailbox.

..procmailrc
-------------
SHELL=/bin/sh #unsure if this is necessary but it wont hurt
MAILDIR=/var/qmail/mailnames/##.com/kevin/Maildir #dont put a
trailing / here
DEFAULT=/var/qmail/mailnames/##.com/kevin/Maildir/ #this is where mail
goes if it doesnt match any rules so you need the trailing / to tell it
you have a maildir style mbox
#LOGFILE = /var/qmail/mailnames/##.com/kevin/procmail-#.com.log
#defintitely put this in during testing and remove the #
#VERBOSE=yes #defintitely put this in during testing and remove the
#
#DEBUG=yes #defintitely put this in during testing and remove
the #

:0fw
| bogofilter -u -l -p -e -d
/var/qmail/mailnames/##.com/kevin/.bogofilter #score spam using
specified directory where wordlist is and return 0 or 1

:0e
#{ EXITCODE=75 HOST } #this sends mail back to qmail but I decided Id
rather get it in a directory so know somthing is wrong
..Failed/ #if theres an error with bogo, put the mail in subdir Failed

# file the mail to junk if it's spam.
:0
* ^X-Bogosity: Spam, tests=bogofilter
..Junk/
-------------------

Last but not least, my qmail training command lines for the maildir
mbox style
------------
bogofilter -d /var/qmail/mailnames/##.com/kevin/.bogofilter -nMB
/var/qmail/mailnames/##.com/kevin/Maildir/.Trash/ #train for ham
bogofilter -d /var/qmail/mailnames/##.com/kevin/.bogofilter -sMB
/var/qmail/mailnames/##.com/kevin/Maildir/.Junk/ #train for spam
---------

I'm proud to say that my log for this morning shows I've filtered 43
spams (67% of total spam). I checked and there are no false-positives.
I move all spams that got through into my Junk directory and since I
re-train every night, things will just get better and better.

Re: procmail trys to store in wrong folder

On comp.mail.misc, in <1116349294.971214.256710@g47g2000cwa.googlegroups.com>, "netkev@gmail.com" wrote:



Thanks a million for posting your solution(s) and so thoroughly
commenting them, netkev.

I don't think that YOU need to worry about being flamed here.
The vermin are after ME. (Have been for years because I use a
Challenge-Response mail filter which trolls and spammers cannot
beat)

AC


--
alanconnor AT earthlink DOT net
Use your real return address or I'll never know you
even tried to mail me. http://tinyurl.com/2t5kp

~