DevHeads.net

Regexp Postfix query doesn't stop at the first matching rule

Hello,

I would be really thankful if someone could clarify it, please. It says the
following, "Postfix works as documented in regexp_table(5) and
pcre_table(5), i.e. each query stops at the first matching rule. Now the
following two rules are in conflict:

/^From:\s*assistant\@gmail\.com$/ REPLACE Subject: New Report.
/^Subject:\s*$/ REJECT Empty subjects fields are rejected.

I doesn't stop at the first matching rule where the subject is replaced.
Instead, it stops at the second matching rule where empty subject fields are
rejected. Thus, it causes trouble, instead of accepting mail from
<a href="mailto: ... at gmail dot com"> ... at gmail dot com</a> that is being sent with an empty subject field, it
rejects it as per the second rule. How do I make it stop at the first
matching rule? Would highly appreciate any pointers / suggestions /
assistance at all. Postfix 2.9.6. Many thanks!

Comments

Re: Regexp Postfix query doesn't stop at the first matching rule

By Noel Jones at 06/09/2019 - 00:07

On 6/8/2019 10:11 AM, Den1 wrote:
No, it replaces the From: header with a second Subject: header. The
original Subject: is untouched.

At that point, processing of the From: header is stopped and postfix
proceeds with the next header presented.

header_checks does not check headers added by postfix.

And REJECT always wins. Rule processing is stopped as soon as a
rule resolves to REJECT, and the mail is rejected at the earliest
opportunity.

What you appear to be trying - if the mail is From some particular
address, replace the Subject: - cannot be done with header_checks.

You can probably do this with MIMEDefang or another milter, but the
actual implementation is outside the scope of this list.

-- Noel Jones

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/09/2019 - 02:36

Noel Jones-2 wrote
Thank you so much! What a perfectly informative answer. It's utmost clear
now. It explains now why all my tests failed.

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/10/2019 - 06:51

Just one teeny tiny comment. Header_checks will trigger before procmail,
sieve and so on therefore I am pretty much out of luck. Empty subject fields
will be rejected before either procmail or sieve inserts / replaces the
subject field (header) in question. Are there any ways to run procmail
before header_checks or is my best bet to move it all to procmail? Thanks.

Re: Regexp Postfix query doesn't stop at the first matching rule

By Viktor Dukhovni at 06/10/2019 - 14:34

Postfix header checks are by default not enabled, so do not reject
empty subjects by default, and can't reject messages with a missing
subject. You don't have to reject empty subjects.

Only by somehow using procmail as content_filter, for which it is
probably not well suited. So basically, no...

You can use a Sieve milter, proxy filter, or defer the processing
to delivery, but then you've already accepted the message, and
should not reject it. You can tag it as spam, and have it delivered
to a suitable sub-folder, or just leave the MUA to deal with the tags.

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/10/2019 - 21:32

Thank you for replying, Victor. I've had empty subject fields rejected for years to avoid spam. I mean I had this very basic regexp rule set in my header checks. It looks like I may have to remove it now and to re-configure it a bit in combination with procmail, according to this new prefatory /requirement. I think it will be easier than to put a cart before the horse in my attempts to use procmail before header_checks. Thanks again.

Den

Postfix header checks are by default not enabled, so do not reject
empty subjects by default, and can't reject messages with a missing
subject. You don't have to reject empty subjects.

Only by somehow using procmail as content_filter, for which it is
probably not well suited. So basically, no...

You can use a Sieve milter, proxy filter, or defer the processing
to delivery, but then you've already accepted the message, and
should not reject it. You can tag it as spam, and have it delivered
to a suitable sub-folder, or just leave the MUA to deal with the tags.

Re: Regexp Postfix query doesn't stop at the first matching rule

By Wietse Venema at 06/08/2019 - 12:16

man header_checks:

DESCRIPTION

Each message header or message body line is compared against a list of
patterns. When a match is found the corresponding action is executed,
and the matching process is repeated for the next message header or
message body line.

Thus, it stops at the first match. If the Subject line matches first,
then that rule determines the result.

Wietse

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/08/2019 - 20:16

Thank you for replying but I still do not understand it. Are you saying that I should swap rules' places? I have already tried that before posting my quesion but it doesn't help. The matter is that these rules work OK one by one, but when put together the REPLACE rule is skipped wheather it comes before or after the other rule.

1. REPLACE doesn't work.
/^From:\s*assistant\@gmail\.com$/ REPLACE Subject: New Report.
/^Subject:\s*$/ REJECT Empty subjects fields are rejected.

2. REPLACE doesn't work either.
/^Subject:\s*$/ REJECT Empty subjects fields are rejected.
/^From:\s*assistant\@gmail\.com$/ REPLACE Subject: New Report.

Does it perhaps mean that each email' to, from, subject, and body fields are treated as just one single rule, and the query is stopped whenever all of these 4 per 1 message are matched?

Any more pointers, please? Many thanks!

Den

Den1:
man header_checks:

DESCRIPTION

Each message header or message body line is compared against a list of
patterns. When a match is found the corresponding action is executed,
and the matching process is repeated for the next message header or
message body line.

Thus, it stops at the first match. If the Subject line matches first,
then that rule determines the result.

Wietse

Re: Regexp Postfix query doesn't stop at the first matching rule

By Richard Damon at 06/08/2019 - 20:57

I think the sequence (and Wietse can correct me if I am wrong) is that
the first header in the message is taken, and searched to see if any
rule matches, if so, that rule is used. If that header doesn't match any
rule, the next header is gotten and checked against the rules, and so
on. The first header in the message that matches a rule is the one that
dominates.

The order the rules are written doesn't matter that much (unless two of
them might match the same header). It is the order the headers are in
the message (which you really can't control).

On 6/8/19 8:16 PM, Webmaster wrote:

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/08/2019 - 23:06

Thank you so much for your input, Richard. Appreciate.

Hmm.. Right, this is the way I understand it too. However, my first rule that matches the header (from field) is skipped for some reason. Then the second rule that matches the header (subject field) is applied, not skipped. Now these two rules work fine if they go one by one, but they do not work if they go together in combination (tested). I am lost..

It looks like REJECT triggers this query match and stops there accordingly. However, REPLACE doesn't trigger the query match and the search goes on until it "meets" REJECT. In my opinion that's exactly what's going on here in my case.. . Is that not right? Am I wrong?

I think the sequence (and Wietse can correct me if I am wrong) is that
the first header in the message is taken, and searched to see if any
rule matches, if so, that rule is used. If that header doesn't match any
rule, the next header is gotten and checked against the rules, and so
on. The first header in the message that matches a rule is the one that
dominates.

The order the rules are written doesn't matter that much (unless two of
them might match the same header). It is the order the headers are in
the message (which you really can't control).

On 6/8/19 8:16 PM, Webmaster wrote:

Re: Regexp Postfix query doesn't stop at the first matching rule

By Richard Damon at 06/08/2019 - 23:31

But if in the message Subject is before From, then the Subject will be
tested before the From header is tested.

The rules process the message, starting from the first header IN THE
MESSAGE, and that gets checked against the rules.
THEN the second header in the message is tested, then the third.
The order of the headers in the message determines the order the rules
are checked.

It does NOT happen that the first rule is tested, sees that it wants a
From header, and the message is searched for a matching From header. To
do that, you need them to be totally separate passes, not rules within a
single pass.

Your 'first' rule isn't 'skipped', it is that the Subject rule gets
matched first because that is the first header that is being found (from
that sender) that matches a rule, so you never got to the From header
line to test it.

On 6/8/19 11:06 PM, Webmaster wrote:

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/09/2019 - 00:42

OK. This is gmail.com and the sequence / order of its headers is always the same isn't it? Plus, that is always the same user, always the same email address, always the same "empty" subject field and always the same recipient all the time. Now according to your explanation could you advise me please, how do I set it to check the subject field first and to stop the query right there without letting it go through any further rules? Thanks.

P. S. Changing the rules order in header checks didn't help. The first header in gmail.com that I have a rule set for is the subject field.

But if in the message Subject is before From, then the Subject will be
tested before the From header is tested.

The rules process the message, starting from the first header IN THE
MESSAGE, and that gets checked against the rules.
THEN the second header in the message is tested, then the third.
The order of the headers in the message determines the order the rules
are checked.

It does NOT happen that the first rule is tested, sees that it wants a
From header, and the message is searched for a matching From header. To
do that, you need them to be totally separate passes, not rules within a
single pass.

Your 'first' rule isn't 'skipped', it is that the Subject rule gets
matched first because that is the first header that is being found (from
that sender) that matches a rule, so you never got to the From header
line to test it.

On 6/8/19 11:06 PM, Webmaster wrote:

Re: Regexp Postfix query doesn't stop at the first matching rule

By Viktor Dukhovni at 06/09/2019 - 00:56

The answer is simple. Postfix header checks DO NOT support any sort
of multi-header conditionals or logic that depends on the order in
which headers are checked against the rules. Each header is tested
*in isolation* and whatever order the headers happen to appear in
the message.

For anything fancier you need a content filter or milter (a Sieve
milter would do the job). What you're trying to do CANNOT be done
with the built-in content inspection in Postfix.

Re: Regexp Postfix query doesn't stop at the first matching rule

By Den1 at 06/09/2019 - 02:28

Viktor Dukhovni wrote
Thank you. Appreciate. I thought about trying it with Sieve or Procmail and
I guess I will do it there.