DevHeads.net

conversation with ... timed out while sending end of data -- message may be sent more than once

Today I found that some sites behind a PIX/ASA firewall with "smtp
protocol fixup" would not accept DKIM signed mails.

Solution:
=========

master.cf:
nodkim unix - - - - - smtp -o smtp_header_checks=pcre:/etc/postfix/no_dkim.pcre

main.cf:
transport_maps = cdb:/etc/postfix/transport

and in /etc/postfix/transport:
mrnaz.com nodkim:

/etc/postfix/no_dkim.pcre contains:
/^DKIM-Signature:/ IGNORE
# this strips a DKIM Signature

Comments

Re: conversation with ... timed out while sending end of data --

By Robert Schetterer at 06/14/2011 - 14:57

Am 14.06.2011 15:34, schrieb Ralf Hildebrandt:

Re: conversation with ... timed out while sending end of data --

By Noel Jones at 06/14/2011 - 10:12

On 6/14/2011 8:34 AM, Ralf Hildebrandt wrote:

I think I posted something almost exactly like this a while
ago (year+?). Anyway, I can confirm that I've had this same
problem and came up with the same workaround, still in place.

-- Noel Jones

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/14/2011 - 13:48

* Noel Jones < ... at megan dot vbhcs.org>:

Yeah. Maybe it would make a cool addition to smtp_pix_workarounds!

Re: conversation with ... timed out while sending end of data --

By Benny Pedersen at 06/14/2011 - 18:49

On Tue, 14 Jun 2011 19:48:54 +0200, Ralf Hildebrandt wrote:
or list bad domains as rfc-ignorant if there is a rfc for this

Re: conversation with ... timed out while sending end of data --

By Noel Jones at 06/14/2011 - 20:32

On 6/14/2011 5:49 PM, Benny Pedersen wrote:
No, there is no RFC that says "you must receive my properly
formatted email even if your software chokes on it".

I was thinking along the lines of a smtp_pix_workarounds
keyword like "removeDKIM" or, more general and more complex,
"removeheaders" with a matching smtp_pix_removeheaders list of
header names to remove.

the choices I see are

A) single-purpose workaround:
smtp_pix_workarounds = removeDKIM ...

B) general anti-choke workaround
smtp_pix_workarounds = removeheaders
smtp_pix_removeheaders = DKIM, X-foo, Bar

(proposed docs available if there is any interest)

C) use existing smtp_header_checks solution.

For me, the existing workaround (master.cf dumbpix transport
with -o smtp_header_checks) is sufficient. I currently have
only two domains on the dumbpix transport -- apparently
unrelated government agencies, a school system in one city,
police in another.

-- Noel Jones

Re: conversation with ... timed out while sending end of data --

By Benny Pedersen at 06/14/2011 - 20:42

extend to smtp_header_checks_maps, and then use any maps postfix
support

is smtp_header_checks already pr recipients server ?

Re: conversation with ... timed out while sending end of data --

By Noel Jones at 06/14/2011 - 21:05

On 6/14/2011 7:42 PM, Benny Pedersen wrote:
That's an interesting idea in itself, but in the scope of pix
workarounds it's not a huge improvement since it still
requires manual intervention per server/domain.

anyway, don't think it's possible. I think all possible
tables would need to be loaded before postfix knew which one
to use, or postfix would need to wastefully launch a new smtp
for each delivery.

No, currently either a global setting or custom transports
with -o smtp_header_checks option.

I was thinking a setting integrated with smtp_pix_workarounds
would be more automatic, with little maintenance once configured.

-- Noel Jones

Re: conversation with ... timed out while sending end of data --

By Benny Pedersen at 06/14/2011 - 21:23

fail2ban could be ones friend if postfix have this

fail2ban then just grep logs for outgoing mails that failed pr ip, and
add this header ignore pr cidr maps

as is now its not, but i think it could be solved :-)

okay

suggest pfsense is out of the question for hosts that runs cisco
hardware

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/15/2011 - 02:39

* Benny Pedersen < ... at junc dot org>:

Yeah, that's a great idea!

Re: conversation with ... timed out while sending end of data --

By Robert Schetterer at 06/15/2011 - 03:23

Am 15.06.2011 08:39, schrieb Ralf Hildebrandt:

Re: conversation with ... timed out while sending end of data --

By Benny Pedersen at 06/15/2011 - 03:16

On Wed, 15 Jun 2011 08:39:11 +0200, Ralf Hildebrandt wrote:
it is ?, oh thanks :-)

Re: conversation with ... timed out while sending end of data --

By Victor Duchovni at 06/14/2011 - 21:22

Given that the banner detection is incomplete (some pixen are not
obviously such) one still needs manual configuration for some cases,
so I am not convinced that any new feature is warranted, the receiving
systems need to be incented to fix their bug.

Re: conversation with ... timed out while sending end of data --

By Wietse Venema at 06/15/2011 - 06:57

Victor Duchovni:
If enough "big mailers" sign their email (gmail, yahoo, etc.)
then that will provide the incentive.

Wietse

Re: conversation with ... timed out while sending end of data --

By Noel Jones at 06/14/2011 - 23:42

On 6/14/2011 8:22 PM, Victor Duchovni wrote:
OTOH, the current pix detection and workarounds are not
useless, so extending/improving them is worth discussing --
even if not necessarily worth doing.

At this time I'm inclined to set this aside. The DKIM bug
doesn't seem to be widespread; there is no compelling case to
add a new workaround right now.

Maybe an example of the current smtp_header_checks workaround
(Ralf's was fine) could be added to the docs somewhere rather
than a feature change.

-- Noel Jones

Re: conversation with ... timed out while sending end of data --

By Mark Martinec at 06/15/2011 - 06:17

On Wednesday June 15 2011 05:42:36 Noel Jones wrote:
Indeed the situation has much improved in the past year or two.

Many sites have turned off smtp fixups or upgraded their ASA
firmware or both. It also helps to send mail to postmasters of
affected sites with a pointer to Ralf's web page and the Heise
article, and suggest turning off the (mis)feature.

Perhaps the incentive was when they started missing some of the
mail from gmail.com and the like.

Mark

Re: conversation with ... timed out while sending end of data --

By Wietse Venema at 06/14/2011 - 13:53

Ralf Hildebrandt:
How does an SMTP client recognize an ASA box before it breaks email?

Wietse

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/14/2011 - 14:05

* Wietse Venema < ... at porcupine dot org>:

Only from the /^[02 *]+$/ banner.

# telnet mx.interfree.it 25
Trying 213.158.72.46...
Connected to mx.interfree.it.
Escape character is '^]'.
220 ******************************************************************

# telnet mailamir.com 25
Trying 114.31.73.44...
Connected to mailamir.com.
Escape character is '^]'.
220 **************************

Re: conversation with ... timed out while sending end of data --

By Wietse Venema at 06/14/2011 - 14:18

Ralf Hildebrandt:
Hmm...

% telnet mailamir.com 25
Trying 114.31.73.44...
Connected to mailamir.com.
Escape character is '^]'.
220 **************************
help
502 5.5.2 Error: command not recognized

Wietse

Re: conversation with ... timed out while sending end of data --

By Wietse Venema at 06/14/2011 - 15:14

Wietse Venema:
FYI, this is how I quickly identify Postfix MTAs.

Wietse

Re: conversation with ... timed out while sending end of data --

By Mark Martinec at 06/14/2011 - 14:30

Ralf wrote:
But you already knew that! :)

ASA bug CSCsy28792 and a couple of related header-parsing bugs,
triggered by encountering a "content-type" or "content-transfer-encoding"
in a header field body of some unrelated header field, such as an 'h' tag
of a DKIM signature:

<a href="http://www.arschkrebs.de/postfix/postfix_cisco_pix_bugs.shtml" title="http://www.arschkrebs.de/postfix/postfix_cisco_pix_bugs.shtml">http://www.arschkrebs.de/postfix/postfix_cisco_pix_bugs.shtml</a>

Mark

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/14/2011 - 14:32

* Mark Martinec <Mark.Martinec+ ... at ijs dot si>:
Yes I know.

Back then I didn't know the workaround!

Re: conversation with ... timed out while sending end of data --

By Mark Martinec at 06/14/2011 - 14:41

I think the newer versions of ASA can be configured to let ESMTP pass through
without censoring the greeting, while still exhibiting one of the header
parsing bugs - which can lead to dropping the TCP session without
a RST (but with a message in the log ... which noone reads).

Mark

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/14/2011 - 14:48

* Mark Martinec <Mark.Martinec+ ... at ijs dot si>:

:(

Re: conversation with ... timed out while sending end of data --

By Robert Schetterer at 06/14/2011 - 15:00

Am 14.06.2011 20:48, schrieb Ralf Hildebrandt:

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/14/2011 - 15:23

* Robert Schetterer < ... at schetterer dot org>:

For that one would need large scale statistics.

Re: conversation with ... timed out while sending end of data --

By Victor Duchovni at 06/14/2011 - 14:29

A Postfix system with a PIX in front of it and STARTTLS censored as
"XXXXXXXA" (same length).

Connected to mailamir.com[114.31.73.44]:25
< 220 **************************

Re: conversation with ... timed out while sending end of data --

By Ralf Hildebrandt at 06/14/2011 - 14:31

* Victor Duchovni <Victor. ... at morganstanley dot com>:

Yes, thought so too.

Re: conversation with ... timed out while sending end of data --

By Victor Duchovni at 06/14/2011 - 13:53

I guess you'd like:

smtp_pix_header_checks = ...

this feature would be rather a large concession to a problem that needs
to be fixed at the receiving system...