procmail whitelist recipe problem

So I'm a newbie with regards procmail but I'm having an odd issue
trying to get a whitelist to work. Here is my recipe

:0 w
* ? test (formail -zxFrom: | grep -i -f $HOME/whitelist)
{
LOG="WHITELISTED $? $FROM
"
:0
!xxxxx@xxxxxx.com
}

I have one address in the whitelist but everything is being logged as
whitelisted/matched with a return code of 0 being displayed in the log
message. When I run the test from the command line it returns a 1 for
everything that is not in the whitelist file. Why on earth is this
always returning true? Been baffling me for a couple of days as I've
tried many permutations of the test.

Any help much appreciated.

Damian

Re: procmail whitelist recipe problem

On Fri, 5 Jan 2007 dpenney@gmail.com wrote:

> So I'm a newbie with regards procmail but I'm having an odd issue
> trying to get a whitelist to work. Here is my recipe
>
> * ? test (formail -zxFrom: | grep -i -f $HOME/whitelist)
>
> I have one address in the whitelist but everything is being logged as
> whitelisted/matched with a return code of 0 being displayed in the log
> message. When I run the test from the command line it returns a 1 for
> everything that is not in the whitelist file. Why on earth is this
> always returning true? Been baffling me for a couple of days as I've
> tried many permutations of the test.
>

It is not a procmail issue but a grep issue. You may have only one
address in the file but you also have a blank line. That is the cause of
your problem.


--
Alan

( If replying by mail, please note that all "sardines" are canned.
There is also a password autoresponder but, unless this a very
old message, a "tuna" will swim right through. )

Re: procmail whitelist recipe problem

Alan Clifford wrote:
> On Fri, 5 Jan 2007 dpenney@gmail.com wrote:
>
> > So I'm a newbie with regards procmail but I'm having an odd issue
> > trying to get a whitelist to work. Here is my recipe
> >
> > * ? test (formail -zxFrom: | grep -i -f $HOME/whitelist)
> >
> > I have one address in the whitelist but everything is being logged as
> > whitelisted/matched with a return code of 0 being displayed in the log
> > message. When I run the test from the command line it returns a 1 for
> > everything that is not in the whitelist file. Why on earth is this
> > always returning true? Been baffling me for a couple of days as I've
> > tried many permutations of the test.
> >
>
> It is not a procmail issue but a grep issue. You may have only one
> address in the file but you also have a blank line. That is the cause of
> your problem.
>
>
> --
> Alan
>
No there is a single email address with no new lines; which is why when
I run it from the command line it works as expected. If there was a new
line in there then I'd get a return code of 0 from the cmd line as well.

Re: procmail whitelist recipe problem

Alan Clifford wrote:
> On Fri, 5 Jan 2007 dpenney@gmail.com wrote:
>
> > So I'm a newbie with regards procmail but I'm having an odd issue
> > trying to get a whitelist to work. Here is my recipe
> >
> > * ? test (formail -zxFrom: | grep -i -f $HOME/whitelist)
> >
> > I have one address in the whitelist but everything is being logged as
> > whitelisted/matched with a return code of 0 being displayed in the log
> > message. When I run the test from the command line it returns a 1 for
> > everything that is not in the whitelist file. Why on earth is this
> > always returning true? Been baffling me for a couple of days as I've
> > tried many permutations of the test.
> >
>
> It is not a procmail issue but a grep issue. You may have only one
> address in the file but you also have a blank line. That is the cause of
> your problem.
>
>
> --
> Alan
>
> ( If replying by mail, please note that all "sardines" are canned.
> There is also a password autoresponder but, unless this a very
> old message, a "tuna" will swim right through. )

Here is the result of a wc

wc whitelist
1 1 19 whitelist

and the file itself is essentially xxxxxx@xxxxxx.com

Re: procmail whitelist recipe problem

dpenney@gmail.com wrote:
> So I'm a newbie with regards procmail but I'm having an odd issue
> trying to get a whitelist to work. Here is my recipe
>
> :0 w
> * ? test (formail -zxFrom: | grep -i -f $HOME/whitelist)
> {
> LOG="WHITELISTED $? $FROM
> "
> :0
> !xxxxx@xxxxxx.com
> }
>
> I have one address in the whitelist but everything is being logged as
> whitelisted/matched with a return code of 0 being displayed in the log
> message. When I run the test from the command line it returns a 1 for
> everything that is not in the whitelist file. Why on earth is this
> always returning true? Been baffling me for a couple of days as I've
> tried many permutations of the test.
>
> Any help much appreciated.
>
> Damian
>

Take out the word "test" or use it to test for the presence of the
whitelist file thus:

:0 w
* ? test -f $HOME/whitelist && formail -zxFrom: | grep -i -f $HOME/whitelist
{
LOG="WHITELISTED $? $FROM
"
:0
!xxxxx@xxxxxx.com
}

--
Garen

Re: procmail whitelist recipe problem

Garen Erdoisa wrote:
> dpenney@gmail.com wrote:
>> So I'm a newbie with regards procmail but I'm having an odd issue
> [SNIP]
> :0
> !xxxxx@xxxxxx.com
> }
>

Just as an aside, if you are going to use an email address as an example
on USENET, you should use "example.com" or "example.org" as the domain
name. Those domain names are reserved for this purpose and are used in
documentation.

the one you used here, "xxxxxx.com" is a valid domain name. Some domain
names use catch all addresses, so doing tests with that sort of domain
name may end up with you bothering some hapless owner of the domain with
the catch all address. :)

--
Regards;
Garen

Re: procmail whitelist recipe problem

> Just as an aside, if you are going to use an email address as an example
> on USENET, you should use "example.com" or "example.org" as the domain
> name. Those domain names are reserved for this purpose and are used in
> documentation.
>
> the one you used here, "xxxxxx.com" is a valid domain name. Some domain
> names use catch all addresses, so doing tests with that sort of domain
> name may end up with you bothering some hapless owner of the domain with
> the catch all address. :)
>
> --
> Regards;
> Garen

Point taken, never even occurred to me, alas it still doesn't work...

procmail: Assigning "FROM= Damian Penney "
procmail: Executing " test -f $HOME/whitelist && formail -zxFrom: |
grep -i -f $HOME/whitelist"
procmail: Match on " test -f $HOME/whitelist && formail -zxFrom: | grep
-i -f $HOME/whitelist"
procmail: Assigning "LOG=WHITELISTED 0 Damian Penney

"
WHITELISTED 0 Damian Penney


:0 w
* ? test -f $HOME/whitelist && formail -zxFrom: | grep -i -f
$HOME/whitelist
{
LOG="WHITELISTED $? $FROM
"
:0
!pdo@example.com
}

whitelist file is one single line >> laurel@example.com

Now from the command line, running from the same spot my whitelist
lives

$ test -f whitelist && echo "Damian Penney " | grep
-i -f whitelist
$ echo $?
1

Baffling right?

Re: procmail whitelist recipe problem

On Mon, 8 Jan 2007 dpenney@gmail.com wrote:

>
> Alan Clifford wrote:
>> On Fri, 5 Jan 2007 dpenney@gmail.com wrote:
>>
>>> So I'm a newbie with regards procmail but I'm having an odd issue
>>> trying to get a whitelist to work. Here is my recipe
>>>
>>> * ? test (formail -zxFrom: | grep -i -f $HOME/whitelist)
>>>

The syntax I use is:

MARP_FROM=`formail -c -rtz -x To:`
# Is it not in the whitelist?
:0
* ! ? echo "${MARP_FROM}" | grep -F -isx -f "${MARP_WHITELIST}"
{

which has a "not" in the test and a -F in the grep but essentially the
same sort of thing.


--
Alan

( If replying by mail, please note that all "sardines" are canned.
There is also a password autoresponder but, unless this a very
old message, a "tuna" will swim right through. )

Re: procmail whitelist recipe problem

dpenney@gmail.com wrote:
>> Just as an aside, if you are going to use an email address as an example
>> on USENET, you should use "example.com" or "example.org" as the domain
>> name. Those domain names are reserved for this purpose and are used in
>> documentation.
>>
>> the one you used here, "xxxxxx.com" is a valid domain name. Some domain
>> names use catch all addresses, so doing tests with that sort of domain
>> name may end up with you bothering some hapless owner of the domain with
>> the catch all address. :)
>>
>> --
>> Regards;
>> Garen
>
> Point taken, never even occurred to me, alas it still doesn't work...
>
> procmail: Assigning "FROM= Damian Penney "

Curious, how are you generating the FROM= assignment above?

> procmail: Executing " test -f $HOME/whitelist && formail -zxFrom: |
> grep -i -f $HOME/whitelist"

That line looks correct.

> procmail: Match on " test -f $HOME/whitelist && formail -zxFrom: | grep
> -i -f $HOME/whitelist"

This looks correct if the contents of your whitelist actually matched
something in the From: header in whatever message you piped through to
procmail at the time.

Can you post the relevant headers in whatever test message you used
along with such examples of verbose procmail output so we have all the
information at hand. It's hard to analyze what is going on when I have
to guess at what might have been in the pipeline that procmail was
actually looking at.

> procmail: Assigning "LOG=WHITELISTED 0 Damian Penney
>
> "
> WHITELISTED 0 Damian Penney

Ok, this variable dumped the return code of the last command that was
executed. That indicated that the grep you did on whatever it was in the
From: header in your test message, actually matched with whatever it was
at the time in your whitelist file.

Then it wrote the contents of your "FROM" variable defined somewhere
else. The contents of that FROM variable has nothing to do with the
contents of the From: header in the pipeline unless you also show how
the variable obtained that information. What is a simple text
assignment? or did you run some embedded shell script to extract
information into it? or what?

In other words, in trying to analyze this issue, I have no way based on
what you posted of knowing that the contents of your FROM variable
matched the contents of the From: header in the test message.

>
>
> :0 w
> * ? test -f $HOME/whitelist && formail -zxFrom: | grep -i -f
> $HOME/whitelist
> {
> LOG="WHITELISTED $? $FROM
> "
> :0
> !pdo@example.com
> }
>

The recipe looks to me like it should work just fine as intended. Does it?

> whitelist file is one single line >> laurel@example.com
>
> Now from the command line, running from the same spot my whitelist
> lives
>
> $ test -f whitelist && echo "Damian Penney " | grep
> -i -f whitelist
> $ echo $?
> 1

The above line should have a return code of 1, since your file contains

laurel@example.com

which does not match with

Damian Penney

>
> Baffling right?
>

Well not so baffling.

Here is a set of examples that might illustrate what you want to do a
tad better. (Showing all work)


----/tmp/example/message.txt-----
To: nobody@example.com
From: Damian Penney
Subject: test message subject

Test message body
----End of file----

----/tmp/example/whitelist----
laurel@example.com
galoot@example.org
---End of file----

----/tmp/example/test.rc----
NL="
"

VERBOSE=yes
HOME='/tmp/example'
FROM=`formail -zxFrom:`

:0 w
* $ ? test -f $HOME/whitelist && formail -zxFrom: | grep -i -f
$HOME/whitelist
{ LOG="[$$]$_: Test matched: WHITELISTED $? ${FROM}${NL}" }

:0 E
{ LOG="[$$]$_: Test did not match: WHITELISTED $? ${FROM}${NL}" }

LOGABSTRACT=no
:0
/dev/null
----end of file----

procmail ./test.rc procmail: [12279] Wed Jan 10 04:52:11 2007
procmail: Assigning "HOME=/tmp/example"
procmail: Executing "formail,-zxFrom:"
procmail: Assigning "FROM=Damian Penney "
procmail: Executing " test -f /tmp/example/whitelist && formail -zxFrom:
| grep -i -f /tmp/example/whitelist"
procmail: Match on " test -f /tmp/example/whitelist && formail -zxFrom:
| grep -i -f /tmp/example/whitelist"
procmail: Assigning "LOG=[12279]./test.rc: Test matched: WHITELISTED 0
Damian Penney
"
[12279]./test.rc: Test matched: WHITELISTED 0 Damian Penney

procmail: Assigning "LOGABSTRACT=no"
procmail: Assigning "LASTFOLDER=/dev/null"
procmail: Opening "/dev/null"

Now removing the galoot@example.org from the whitelist file and
re-running the test
procmail ./test.rc procmail: [12288] Wed Jan 10 04:53:30 2007
procmail: Assigning "HOME=/tmp/example"
procmail: Executing "formail,-zxFrom:"
procmail: Assigning "FROM=Damian Penney "
procmail: Executing " test -f /tmp/example/whitelist && formail -zxFrom:
| grep -i -f /tmp/example/whitelist"
procmail: Non-zero exitcode (1) from " test -f /tmp/example/whitelist &&
formail -zxFrom: | grep -i -f /tmp/example/whitelist"
procmail: No match on " test -f /tmp/example/whitelist && formail
-zxFrom: | grep -i -f /tmp/example/whitelist"
procmail: Assigning "LOG=[12288]./test.rc: Test did not match:
WHITELISTED 1 Damian Penney
"
[12288]./test.rc: Test did not match: WHITELISTED 1 Damian Penney

procmail: Assigning "LOGABSTRACT=no"
procmail: Assigning "LASTFOLDER=/dev/null"
procmail: Opening "/dev/null"


I don't see a problem here. The recipe works as expected.

--
Garen

Re: procmail whitelist recipe problem

> I don't see a problem here. The recipe works as expected.

Thanks for all your help, getting closer.

When I run things locally with

$ procmail .procmailrc < gmail.txt

I get the expected response (gmail.txt is a msg just received from
gmail) however when it comes time to process an actual email I still
get my weird everything matches issue. I've asked the guys at my
hosting service to take a look, wondering if the version of procmail
that actually runs when a msg is received is different from the one I
have access to locally or something along those lines.

I'll update if I learn anything new and thanks again, much appreciated.

Damian