Re: Remove Internal Hops from Header

Victor Sudakov unleashed the infinite monkeys on 29/03/2008 07:12 producing:
>
> Alas, discussed without any result.

My memory is that at least one response included the "You could use
MIMEDefang to do this" response, though without providing the details as
to how (probably because they too don't think it's a good idea).

If you were to follow this route it would involve creating a perl filter
to strip out any Received headers, or alter them to remove the data
you're worried about leaking.

--
Rob MacGregor (BOFH)

Rule 37: "There is no 'overkill'. There is only 'open fire'
and 'I need to reload.'"

Re: Remove Internal Hops from Header

Rob MacGregor wrote:
> >
> > Alas, discussed without any result.

> My memory is that at least one response included the "You could use
> MIMEDefang to do this" response, though without providing the details as
> to how (probably because they too don't think it's a good idea).

I too dislike the idea of using MIMEDefang or any other milter or
external process for the task. Just because it is an external process,
while for example exim does this within the MTA itself.

The same can be said about other tasks which sendmail cannot do
without an external helper, like SPF checking, call ahead, antivirus
checking etc.

I have been told several times that the sendmail.cf language is so
powerful you can do anything with it. Yet it cannot even do the
removal of headers.

--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Remove Internal Hops from Header

Victor Sudakov wrote:

> I too dislike the idea of using MIMEDefang or any other milter or
> external process for the task. Just because it is an external process,
> while for example exim does this within the MTA itself.

> The same can be said about other tasks which sendmail cannot do
> without an external helper, like SPF checking, call ahead, antivirus
> checking etc.

But that's good design. An MTA should deliver mail. It shouldn't be
cluttered with policy tools like content scanners, SPF checking, etc.
It should have hooks for those tools, to be sure, but shouldn't include
them in its own code.

> I have been told several times that the sendmail.cf language is so
> powerful you can do anything with it. Yet it cannot even do the
> removal of headers.

Removal of headers is considered bad because it hampers diagnosing routing
problems. It also gains you practically no security, so the Sendmail authors
did not provide for header removal. If you really need it, a simple milter
is the best way to do it.

Please be aware of this clause in RFC 2821:

3.8.2 Received Lines in Gatewaying

When forwarding a message into or out of the Internet environment, a
gateway MUST prepend a Received: line, but it MUST NOT alter in any
way a Received: line that is already in the header.

If you remove an existing Received: line, you may technically be violating
RFC 2821.

Regards,

David.

Re: Remove Internal Hops from Header

David F. Skoll wrote:

> > I too dislike the idea of using MIMEDefang or any other milter or
> > external process for the task. Just because it is an external process,
> > while for example exim does this within the MTA itself.

> > The same can be said about other tasks which sendmail cannot do
> > without an external helper, like SPF checking, call ahead, antivirus
> > checking etc.

> But that's good design. An MTA should deliver mail. It shouldn't be
> cluttered with policy tools like content scanners, SPF checking, etc.
> It should have hooks for those tools, to be sure, but shouldn't include
> them in its own code.

I think you misunderstood me. For example exim does not include
content scanners in its own code. However, using content scanners, SPF
checking etc in exim does not involve running a bunch of ancillary
processes, which is the case with sendmail.

BTW what is your opinion on DNSBL lookups? Should a well designed MTA
include them in its own code?

> > I have been told several times that the sendmail.cf language is so
> > powerful you can do anything with it. Yet it cannot even do the
> > removal of headers.

> Removal of headers is considered bad because it hampers diagnosing routing
> problems. It also gains you practically no security, so the
> Sendmail authors did not provide for header removal. If you really
> need it, a simple milter is the best way to do it.

Why doesn't this surprise me?

> Please be aware of this clause in RFC 2821:

> 3.8.2 Received Lines in Gatewaying

> When forwarding a message into or out of the Internet environment, a
> gateway MUST prepend a Received: line, but it MUST NOT alter in any
> way a Received: line that is already in the header.

This is a different issue which should be addressed to the OP.
However, I vaguely remember some firewall products which remove
certain fields from outbound E-mail and even HTTP requests. Perhaps it
makes sense under certain circumstances. And I cannot think of any
good use of the private Received: headers to the outsider.

--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Remove Internal Hops from Header

Victor Sudakov wrote:

> I think you misunderstood me. For example exim does not include
> content scanners in its own code. However, using content scanners, SPF
> checking etc in exim does not involve running a bunch of ancillary
> processes, which is the case with sendmail.

How does Exim do it? With shared library plugins? Or does it fork/exec
a subprogram? If the latter, that's vastly *less* efficient than a milter.

> BTW what is your opinion on DNSBL lookups? Should a well designed MTA
> include them in its own code?

No. (And indeed, Sendmail does not directly include DNSBL lookups...
they are implemented in its configuration language.)

[...]

> However, I vaguely remember some firewall products which remove
> certain fields from outbound E-mail and even HTTP requests. Perhaps it
> makes sense under certain circumstances. And I cannot think of any
> good use of the private Received: headers to the outsider.

Received: headers are likely of very little use to the outside, but may
be very useful for insiders who are trying to diagnose mail loop problems.
Since Received: headers are not a security risk (in spite of some "firewall
products" opinions) and may be useful for diagnosing mail problems, you
might as well leave them in.

Regards,

David.

Re: Remove Internal Hops from Header

David F. Skoll wrote:

> > I think you misunderstood me. For example exim does not include
> > content scanners in its own code. However, using content scanners, SPF
> > checking etc in exim does not involve running a bunch of ancillary
> > processes, which is the case with sendmail.

> How does Exim do it? With shared library plugins?

Yes, SPF check is implemented as a shared library. Some checks like
call ahead are internal. Antivirus and antispam interfaces work via a
Unix socket or TCP socket. What I mean is, with sendmail you must run
a) clamav-milter and b) clamd. With exim you run only clamd.

> Or does it fork/exec
> a subprogram? If the latter, that's vastly *less* efficient than a milter.

Of course, it does not exec any subprograms.

[dd]

> > However, I vaguely remember some firewall products which remove
> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
> > makes sense under certain circumstances. And I cannot think of any
> > good use of the private Received: headers to the outsider.

> Received: headers are likely of very little use to the outside, but may
> be very useful for insiders who are trying to diagnose mail loop problems.

I rather doubt the insiders will diagnose internal mail loop problems
from the outside. If there is a mail loop within the corporate
network, the mail is unlikely to leave it.

> Since Received: headers are not a security risk (in spite of some "firewall
> products" opinions)

I am not so sure. In fact they can reveal some private, internal
informaion.

--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Remove Internal Hops from Header

Victor Sudakov wrote:

> David F. Skoll wrote:
>> > [...]
>> > However, I vaguely remember some firewall products which remove
>> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
>> > makes sense under certain circumstances. And I cannot think of any
>> > good use of the private Received: headers to the outsider.
>
>> Received: headers are likely of very little use to the outside, but may
>> be very useful for insiders who are trying to diagnose mail loop problems.
>
> I rather doubt the insiders will diagnose internal mail loop problems
> from the outside. If there is a mail loop within the corporate
> network, the mail is unlikely to leave it.

The subject states "remove". Removing received: header in full *WILL*
lead to creation of possibility of *nasty* email loops going undetected.
Rewriting "internal" received: headers (removing "private" parts) avoids
*this* risk.
[ Sendmail/MTA counts number of received: headers to *detect* email loops ]

>> Since Received: headers are not a security risk (in spite of some
>> "firewall products" opinions)
>
> I am not so sure. In fact they can reveal some private, internal
> informaion.

Stripping "internal" info from received: headers is not the thing you
should *start* from on your path to improve security. Anyway it makes
*some* sense in "higher security" oriented setups. It makes a lot of
sense in *HIGH* security setups of some computer networks.
I would not like to suggest that sendmail is "unfit" for such setups.

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl sip:896530@fwd.pulver.com
Open-Sendmail: http://open-sendmail.sourceforge.net/
Likewise, the national appetizer, brine-cured herring with raw onions,
wins few friends, Germans excepted.
-- Darwin Porter "Scandinavia On $50 A Day"

Re: Remove Internal Hops from Header

"David F. Skoll" wrote:

> Victor Sudakov wrote:
>
>> I too dislike the idea of using MIMEDefang or any other milter or
>> external process for the task. Just because it is an external process,
>> while for example exim does this within the MTA itself.
>
>> The same can be said about other tasks which sendmail cannot do
>> without an external helper, like SPF checking, call ahead, antivirus
>> checking etc.
>
> But that's good design. An MTA should deliver mail. It shouldn't be
> cluttered with policy tools like content scanners, SPF checking, etc.
> It should have hooks for those tools, to be sure, but shouldn't include
> them in its own code.
>
>> I have been told several times that the sendmail.cf language is so
>> powerful you can do anything with it. Yet it cannot even do the
>> removal of headers.
>
> Removal of headers is considered bad because it hampers diagnosing routing
> problems. It also gains you practically no security, so the Sendmail authors
> did not provide for header removal. If you really need it, a simple milter
> is the best way to do it.
>
> Please be aware of this clause in RFC 2821:
>
> 3.8.2 Received Lines in Gatewaying
>
> When forwarding a message into or out of the Internet environment, a
> gateway MUST prepend a Received: line, but it MUST NOT alter in any
> way a Received: line that is already in the header.
>
> If you remove an existing Received: line, you may technically be violating
> RFC 2821.

*Rewriting* of Received: headers generated by servers under you control
operating on "internal" network:
a) breaks letter of the RFC
b) *IMHO* it does not break spirit of the RFC

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl sip:896530@fwd.pulver.com
Open-Sendmail: http://open-sendmail.sourceforge.net/
The knowledge that makes us cherish innocence makes innocence unattainable.
-- Irving Howe

Re: Remove Internal Hops from Header

Andrzej Adam Filip wrote:
> >> > However, I vaguely remember some firewall products which remove
> >> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
> >> > makes sense under certain circumstances. And I cannot think of any
> >> > good use of the private Received: headers to the outsider.
> >
> >> Received: headers are likely of very little use to the outside, but may
> >> be very useful for insiders who are trying to diagnose mail loop
> >> problems.
> >
> > I rather doubt the insiders will diagnose internal mail loop problems
> > from the outside. If there is a mail loop within the corporate
> > network, the mail is unlikely to leave it.

> The subject states "remove". Removing received: header in full *WILL*
> lead to creation of possibility of *nasty* email loops going undetected.

Provided the said headers are removed at the very edge of the network,
not before the message leaves the corporate network, I see no
possibility of loop creation. What is a possible loop scenario in this
setup?

> Rewriting "internal" received: headers (removing "private" parts) avoids
> *this* risk.

But this is equally wrong from the point of view of the RFC which
states: "it MUST NOT alter in any way a Received: line that is already
in the header".


--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Remove Internal Hops from Header

Victor Sudakov wrote:

> Andrzej Adam Filip wrote:
>> >> > However, I vaguely remember some firewall products which remove
>> >> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
>> >> > makes sense under certain circumstances. And I cannot think of any
>> >> > good use of the private Received: headers to the outsider.
>> >
>> >> Received: headers are likely of very little use to the outside, but may
>> >> be very useful for insiders who are trying to diagnose mail loop
>> >> problems.
>> >
>> > I rather doubt the insiders will diagnose internal mail loop problems
>> > from the outside. If there is a mail loop within the corporate
>> > network, the mail is unlikely to leave it.
>
>> The subject states "remove". Removing received: header in full *WILL*
>> lead to creation of possibility of *nasty* email loops going undetected.
>
> Provided the said headers are removed at the very edge of the network,
> not before the message leaves the corporate network, I see no
> possibility of loop creation. What is a possible loop scenario in this
> setup?

Have you ever seen two hops loops?
[ e.g. smart host sends back to sending host ]

>> Rewriting "internal" received: headers (removing "private" parts) avoids
>> *this* risk.
>
> But this is equally wrong from the point of view of the RFC which
> states: "it MUST NOT alter in any way a Received: line that is already
> in the header".

IMHO full removal does alter the header :-)

I can see no reason to break the above "MUST NOT" on nets/servers under
my control. I can see reasons why some people want to break in very
specific case when the header has been generated by "internal server"
also under their control.


--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl sip:896530@fwd.pulver.com
Open-Sendmail: http://open-sendmail.sourceforge.net/
"I shall expect a chemical cure for psychopathic behavior by 10 A.M. tomorrow,
or I'll have your guts for spaghetti."
-- a comic panel by Cotham

Re: Remove Internal Hops from Header

Andrzej Adam Filip wrote:

> >> >> > However, I vaguely remember some firewall products which remove
> >> >> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
> >> >> > makes sense under certain circumstances. And I cannot think of any
> >> >> > good use of the private Received: headers to the outsider.
> >> >
> >> >> Received: headers are likely of very little use to the outside, but may
> >> >> be very useful for insiders who are trying to diagnose mail loop
> >> >> problems.
> >> >
> >> > I rather doubt the insiders will diagnose internal mail loop problems
> >> > from the outside. If there is a mail loop within the corporate
> >> > network, the mail is unlikely to leave it.
> >
> >> The subject states "remove". Removing received: header in full *WILL*
> >> lead to creation of possibility of *nasty* email loops going undetected.
> >
> > Provided the said headers are removed at the very edge of the network,
> > not before the message leaves the corporate network, I see no
> > possibility of loop creation. What is a possible loop scenario in this
> > setup?

> Have you ever seen two hops loops?

I have not seen mail loops for a long time already. The last ones I
saw were with some complicated UUCP+SMTP setups.

> [ e.g. smart host sends back to sending host ]

What do you mean by "smart host" in this context? The corporate
mailhub which is supposed to perform headers removal before passing
mail into the Internet, or the ISP smarthost outside the corporate network?

> >> Rewriting "internal" received: headers (removing "private" parts) avoids
> >> *this* risk.
> >
> > But this is equally wrong from the point of view of the RFC which
> > states: "it MUST NOT alter in any way a Received: line that is already
> > in the header".

> IMHO full removal does alter the header :-)

> I can see no reason to break the above "MUST NOT" on nets/servers under
> my control. I can see reasons why some people want to break in very
> specific case when the header has been generated by "internal server"
> also under their control.

^^^^^^^^^^^^ Yes, I am talking exactly of this scenario.


--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Remove Internal Hops from Header

Victor Sudakov wrote:

> Andrzej Adam Filip wrote:
>
>> >> >> > However, I vaguely remember some firewall products which remove
>> >> >> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
>> >> >> > makes sense under certain circumstances. And I cannot think of any
>> >> >> > good use of the private Received: headers to the outsider.
>> >> >
>> >> >> Received: headers are likely of very little use to the outside, but may
>> >> >> be very useful for insiders who are trying to diagnose mail loop
>> >> >> problems.
>> >> >
>> >> > I rather doubt the insiders will diagnose internal mail loop problems
>> >> > from the outside. If there is a mail loop within the corporate
>> >> > network, the mail is unlikely to leave it.
>> >
>> >> The subject states "remove". Removing received: header in full *WILL*
>> >> lead to creation of possibility of *nasty* email loops going undetected.
>> >
>> > Provided the said headers are removed at the very edge of the network,
>> > not before the message leaves the corporate network, I see no
>> > possibility of loop creation. What is a possible loop scenario in this
>> > setup?
>
>> Have you ever seen two hops loops?
>
> I have not seen mail loops for a long time already. The last ones I
> saw were with some complicated UUCP+SMTP setups.
>
>> [ e.g. smart host sends back to sending host ]
>
> What do you mean by "smart host" in this context? The corporate
> mailhub which is supposed to perform headers removal before passing
> mail into the Internet, or the ISP smarthost outside the corporate
> network?

ISP smart hosts are most frequently part of such short loops but
"stripping mail server" we discuss *MAY* be a part of such loops too.
BTW There are also "single user ~/.forward loops".

>> >> Rewriting "internal" received: headers (removing "private" parts) avoids
>> >> *this* risk.
>> >
>> > But this is equally wrong from the point of view of the RFC which
>> > states: "it MUST NOT alter in any way a Received: line that is already
>> > in the header".
>
>> IMHO full removal does alter the header :-)
>
>> I can see no reason to break the above "MUST NOT" on nets/servers under
>> my control. I can see reasons why some people want to break in very
>> specific case when the header has been generated by "internal server"
>> also under their control.
>
> ^^^^^^^^^^^^ Yes, I am talking exactly of this scenario.

I know and remember *otherwise* you would get proper bashing as
"RFC 'MUST NOT' breaker" :-)

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl sip:896530@fwd.pulver.com
Open-Sendmail: http://open-sendmail.sourceforge.net/
Confessions may be good for the soul, but they are bad for the reputation.
-- Lord Thomas Dewar

Re: Remove Internal Hops from Header

Andrzej Adam Filip wrote:
> >
> >> >> >> > However, I vaguely remember some firewall products which remove
> >> >> >> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
> >> >> >> > makes sense under certain circumstances. And I cannot think of any
> >> >> >> > good use of the private Received: headers to the outsider.
> >> >> >
> >> >> >> Received: headers are likely of very little use to the outside, but may
> >> >> >> be very useful for insiders who are trying to diagnose mail loop
> >> >> >> problems.
> >> >> >
> >> >> > I rather doubt the insiders will diagnose internal mail loop problems
> >> >> > from the outside. If there is a mail loop within the corporate
> >> >> > network, the mail is unlikely to leave it.
> >> >
> >> >> The subject states "remove". Removing received: header in full *WILL*
> >> >> lead to creation of possibility of *nasty* email loops going undetected.
> >> >
> >> > Provided the said headers are removed at the very edge of the network,
> >> > not before the message leaves the corporate network, I see no
> >> > possibility of loop creation. What is a possible loop scenario in this
> >> > setup?
> >
> >> Have you ever seen two hops loops?
> >
> > I have not seen mail loops for a long time already. The last ones I
> > saw were with some complicated UUCP+SMTP setups.
> >
> >> [ e.g. smart host sends back to sending host ]
> >
> > What do you mean by "smart host" in this context? The corporate
> > mailhub which is supposed to perform headers removal before passing
> > mail into the Internet, or the ISP smarthost outside the corporate
> > network?

> ISP smart hosts are most frequently part of such short loops but
> "stripping mail server" we discuss *MAY* be a part of such loops too.

If the "stripping mail server" receives mail back from the ISP smart
host (in error), of course it should not strip the "Received" header
added by the ISP. So it will detect the loop.

> BTW There are also "single user ~/.forward loops".

You mean the mail would be forwarded out of the corporate network and
then back thereto? Otherwise such a loop would be detected before the
mail leaves the corporate network.

--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Remove Internal Hops from Header

Victor Sudakov wrote:

> Andrzej Adam Filip wrote:
>> >
>> >> >> >> > However, I vaguely remember some firewall products which remove
>> >> >> >> > certain fields from outbound E-mail and even HTTP requests. Perhaps it
>> >> >> >> > makes sense under certain circumstances. And I cannot think of any
>> >> >> >> > good use of the private Received: headers to the outsider.
>> >> >> >
>> >> >> >> Received: headers are likely of very little use to the outside, but may
>> >> >> >> be very useful for insiders who are trying to diagnose mail loop
>> >> >> >> problems.
>> >> >> >
>> >> >> > I rather doubt the insiders will diagnose internal mail loop problems
>> >> >> > from the outside. If there is a mail loop within the corporate
>> >> >> > network, the mail is unlikely to leave it.
>> >> >
>> >> >> The subject states "remove". Removing received: header in full *WILL*
>> >> >> lead to creation of possibility of *nasty* email loops going undetected.
>> >> >
>> >> > Provided the said headers are removed at the very edge of the network,
>> >> > not before the message leaves the corporate network, I see no
>> >> > possibility of loop creation. What is a possible loop scenario in this
>> >> > setup?
>> >
>> >> Have you ever seen two hops loops?
>> >
>> > I have not seen mail loops for a long time already. The last ones I
>> > saw were with some complicated UUCP+SMTP setups.
>> >
>> >> [ e.g. smart host sends back to sending host ]
>> >
>> > What do you mean by "smart host" in this context? The corporate
>> > mailhub which is supposed to perform headers removal before passing
>> > mail into the Internet, or the ISP smarthost outside the corporate
>> > network?
>
>> ISP smart hosts are most frequently part of such short loops but
>> "stripping mail server" we discuss *MAY* be a part of such loops too.
>
> If the "stripping mail server" receives mail back from the ISP smart
> host (in error), of course it should not strip the "Received" header
> added by the ISP. So it will detect the loop.
>
>> BTW There are also "single user ~/.forward loops".
>
> You mean the mail would be forwarded out of the corporate network and
> then back thereto? Otherwise such a loop would be detected before the
> mail leaves the corporate network.

*Traditional* loop detection merely counts number of "received: like"
headers - it does not check content of the headers.

The wording of the suggestion have not excluded possibility of removing
*all* Received: headers from messages going "outside". It would cause
removing Received: headers "added by ISP" in messages forwarded by the
"internal forward" going "outside".

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl
Open-Sendmail: http://open-sendmail.sourceforge.net/
Plastic... Aluminum... These are the inheritors of the Universe!
Flesh and Blood have had their day... and that day is past!
-- Green Lantern Comics