Issue Postfix: Silent drop of e-mails to non-existing local addresses

[B_P]

Dear all,

when using the PHP function mail() or /usr/sbin/sendmail to locally send an e-mail, I discovered the following issue:

Normally, I would assume to receive a bounce message when sending an e-mail to a non-existing address.
If I use a non-existing external e-mail address as a recipient, the sender receives a normal bounce message.
However, if the recipient address belongs to a domain that is managed locally on my server (by Plesk) but has an invalid (non-existing) local part, the e-mail just seems to be discarded without further notification.

The following steps need to be performed to test this issue.

local (virtual) domain: mydomain.com (no catch-all address!)
local existing user: demo@mydomain.com
invalid address / non-existing local user: fake@mydomain.com
distant (existing) domain: otherdomain.com
non-existing user on distant domain: invalid@otherdomain.cm

Now try to send three e-mails as root:
a) echo "Test mail" | /usr/sbin/sendmail invalid@otherdomain.com
b) echo "Test mail" | /usr/sbin/sendmail fake@mydomain.com
c) echo "Test mail" | /usr/sbin/sendmail demo@mydomain.com

In all three cases, sendmail returns with exit code 0. For case (c), the e-mail is delivered as usual. For case (a), I receive a bounce e-mail telling that the address does not exist. However, for case (b), I do not receive any feedback. What I would expect instead to receive a bounce message just as I do for the other two cases as well.

For case (b) please have a look at the corresponding log file:
Jan 4 01:23:02 srv plesk sendmail[854]: handlers_stderr: PASS
Jan 4 01:23:02 srv plesk sendmail[854]: PASS during call 'limit-out' handler
Jan 4 01:23:02 srv plesk sendmail[854]: handlers_stderr: SKIP
Jan 4 01:23:02 srv plesk sendmail[854]: SKIP during call 'check-quota' handler
Jan 4 01:23:02 srv postfix/pickup[629]: 2D3311AD804B5: uid=0 from=<root>
Jan 4 01:23:02 srv postfix/cleanup[861]: 2D3311AD804B5: message-id=<20160104032302.2D3311AD804B5@srv.mydomain.com>
Jan 4 04:23:02 srv postfix/qmgr[3925]: 2D3311AD804B5: from=<root@srv.mydomain.com>, size=367, nrcpt=1 (queue active)
Jan 4 04:23:02 srv postfix-local[864]: postfix-local: from=root@srv.mydomain.com, to=fake@mydomain.com, dirname=/var/qmail/mailnames
Jan 4 01:23:02 srv dk_check[865]: DK_STAT_NOSIG: No signature available in message
Jan 4 01:23:02 srv postfix-local[864]: cannot chdir to mailname dir fake: No such file or directory
Jan 4 01:23:02 srv postfix-local[864]: Unknown user: fake@mydomain.com
Jan 4 01:23:02 srv postfix/pipe[863]: 2D3311AD804B5: to=<fake@mydomain.com>, relay=plesk_virtual, delay=0.3, delays=0.2/0/0/0.09, dsn=2.0.0, status=sent (delivered via plesk_virtual service)
Jan 4 01:23:02 srv postfix/qmgr[3925]: 2D3311AD804B5: removed

The same applies when using the mail() function of PHP.

Does anybody have an idea how to fix this?

B_P

 

[trialotto]

@B_P

You stated

However, for case (b), I do not receive any feedback. What I would expect instead to receive a bounce message just as I do for the other two cases as well.

and I can respond that this is rather normal and logical behaviour: the mail address does not exists, no bouncing of mails here (remember, it is local-to-local mail "traffic").

Regards....

 

[B_P]

Hi trialotto,

Thanks for your reply!

and I can respond that this is rather normal and logical behaviour: the mail address does not exists, no bouncing of mails here (remember, it is local-to-local mail "traffic").

Regards....

Well, from my point of view, this behavior is neither normal nor logical. Using an ordinary mail (without SPAM filtering etc.) server, I expect the following behavior for all e-mails that the server processes. This is just the same behavior that I would expect from snail mail: Either to deliver mail, refuse to accept a mail document, or return it to the sender if delivery fails:
- Either: E-Mails are accepted and delivered locally or relayed to a remote location
- Or: E-Mails cannot be delivered / relayed for whatever reason (smtpd_*_restrictions, unknown recipients etc.). If possible, in such a situation Postfix should already reject the e-mail before it is stored on the server. For locally initiated e-mails (pickup, sendmail), this obviously does not work since pickup may not immediately process the e-mail and paste it into the Postfix loop. Thus, in cases where mails cannot be rejected, a bounce needs to be generated to inform the sender that the e-mail could not be delivered.
(Short summary: either mails are delivered or returned, there should not be a situation where mails are not delivered but where the sender is not notified. SPAM excluded...)


I just tested this scenario for two different configurations:
- If you use an ordinary postfix server with only local (non-virtual domains) and try case b) with a non-existing local domain, the sender will receive a bounce message!
- If you try to send it to a non-existing address fake@srv.mydomain.com (virtual domain mydomain.com managed with plesk, Postfix running at srv.mydomain.com), you will also receive a bounce message:

Final-Recipient: rfc822; fake@srv.mydomain.com
Action: failed
Status: 5.0.0
Diagnostic-Code: X-Postfix; User unknown in virtual alias table

Thus, my assumption is that the case b) should not happen either!

B_P

 

[trialotto]

@B_P,

In essence, what you are trying to say is that you

a) found a way to work-around normal behaviour for (local-to-local) mail traffic, (and)

b) want to have your mailboxes filled up, (and)

c) want to have your logs filled up with barely relevant content, (and)

d) want to conclude about specific behaviour for mail servers (in general), without having tested the qmail server, (and)

e) want to conclude that the (alleged) issue is more or less related to Plesk or packages provided with Plesk.

To be honest, it is making a fuss about nothing.

In fact, if any issue exists, this issue is related to the sendmail script and the normal functioning thereof.

After all, you are using sendmail as the starting point of your analysis and the results of that analysis as the fundament of your conclusions.

In short, I am not sure what your point exactly is, but it is primarily concerning the sendmail script.

Regards....

 

[B_P]

Hi again,

The problem actually arised while using a PHP script that used the mail() function (and, thus, sendmail) to distribute e-mails.
As there was a typo in the e-mail address, neither the e-mail was successfully sent nor did we receive a bounce message.
Thus, I was wondering why I did not receive any message about the failure.

From my point of view, it is not the best design if such e-mails are discarded without notice. Basically, the problem seems to occur for each script that sends e-mails locally (without SMTP connection).

Regarding your statement

e) want to conclude that the (alleged) issue is more or less related to Plesk or packages provided with Plesk.

I assume that it is a Plesk-related issue. Please have a look at http://www.postfix.org/VIRTUAL_README.html#in_virtual_other, especially the comments about Lines 4, 7-13:

Lines 4, 7-13: The virtual_mailbox_maps parameter specifies the lookup table with all valid recipient addresses. The lookup result value is ignored by Postfix. In the above example, info@example.com and sales@example.com are listed as valid addresses; other mail for example.com is rejected with "User unknown" by the Postfix SMTP server. It's left up to the non-Postfix delivery agent to reject non-existent recipients from local submission or from local alias expansion.

As a quintessence, it is the task of plesk_virtual (/usr/lib/plesk-9.0/postfix-local) to take care of rejecting invalid e-mail addresses!

 

[trialotto]

@B_P,

Again, this is not a Plesk related issue.

You are confusing the sendmail script for a normal mail server, such as Postfix or Qmail.

Sendmail is only a (partial) mail "client", able to send to mails, but not able or not really able to handle mail delivery failures.

Using a proper mail server (Postfix or Qmail) would result in the error notifications you would like to see (i.e. that you deem to be appropriate).

In a sense, your analysis is starting from the wrong entry point: using sendmail script to establish some results and drawing conclusions from those results.

Naturally, if all of the above (including my other posts) does not convince you, than it´s fine and I will rest my case (and future responses).

Regards....

 

[B_P]

Hi trialotto,

Again, this is not a Plesk related issue.

Sendmail is only a (partial) mail "client", able to send to mails, but not able or not really able to handle mail delivery failures.

Well, maybe I need to explain the situation again in order to convince you that it is in fact a Postfix/Plesk issue:

The sendmail script (http://www.postfix.org/sendmail.1.html) in this case was used for debugging purposes, as it is for instance recommended even on the Postfix website (http://www.postfix.org/DEBUG_README.html).
I used the sendmail script to locally feed e-mails into Postfix: sendmail uses postdrop to queue mails in the postdrop queue which is then processed by pickup and sent to the cleanup server component of Postfix (http://www.postfix.org/OVERVIEW.html). At the latest from this point on, it is normal Postfix business to handle e-mails (as you can also see from the logfile above). Now the e-mail follow the ordinary Postfix workflow.

At one step, qmgr needs to decide how to deliver the e-mail. This is the moment where the virtual_* settings and the corresponding maps/databases are normally read to decide how to deliver or whether to reject an e-mail. However, as shown in my previous post (and documented at http://www.postfix.org/VIRTUAL_README.html#in_virtual_other), the rejection is not done by virtual/Postfix if a non-postfix software is used for final delivery. With Plesk systems, this is the case (virtual_transport = plesk_virtual instead of the default virtual_transport = virtual). As a conclusion, the issue described in my initial post happens due to decisions made by plesk_virtual, and, thus, by a Plesk component.
q.e.d.

You are confusing the sendmail script for a normal mail server, such as Postfix or Qmail.

Using a proper mail server (Postfix or Qmail) would result in the error notifications you would like to see (i.e. that you deem to be appropriate).

Well, as outlined above, the described behavior is the result of the cooperation between Postfix and plesk-virtual. In the specific situation described above, plesk-virtual seems to fail with regard to rejecting e-mails.

BTW: This problem also occurs if you use the mail command to send an e-mail to a non-existing mailbox / user of a virtual domain:
echo "test" | mail fake@mydomain.com

In a sense, your analysis is starting from the wrong entry point: using sendmail script to establish some results and drawing conclusions from those results.

Naturally, if all of the above (including my other posts) does not convince you, than it´s fine and I will rest my case (and future responses).

Regards....

I hope that my explanation clarified the situation and made you understand that it is in fact an issue between Postfix and Plesk.

Regards,

B_P

 

[trialotto]

@B_P,

If you are citing some Postfix documentation, make sure that you read the relevant parts.

Any non-SMTP mail (such as those originating from the sendmail script) will be treated by the postfix cleanup server as "ESMTP from localhost with IP 127.0.0.1", since the problem with non-SMTP mail is that a SMTP session is present, which on it´s turn requires that some "simulation" occurs in order to keep postfix happy.

In short, there is no bug and there is no Plesk related issue, it is simply normal behaviour of Postfix that case b occurs (i.e. the first statement that I made).

However, there is this (Postfix) milter called psa-pc-remote, provided with Plesk.

In essence, this milter application is causing a reject of mails in specific cases: your case b is probably one of them, check that in the mail.err file.

Any rejected mail is the result of a properly working milter application, it does not concern a bug or a Plesk related issue.

It simply indicates that your postfix configuration is setup in a specific fashion, that does not allow the mail from case b to be received by the postfix server.

Note that the above is the result of a milter application, not allowing the "simulation" (of sessions, required for non-SMTP mails) to occur.

In short, if you would have send the mail from an actual SMTP server establishing SMTP connections (and not mail originating from the sendmail script), case b would not occur at all.

In conclusion, it would be strange to conclude from your test case b that some issue with and/or a bug in Plesk is present.

In order to end the discussion finally, if you want to have the milter out of the equation: comment out the line "non_smtpd_milters" in /etc/postfix/main.cf.

Regards...