[tlug] Email forwarding

["Stephen J. Turnbull" <stephenjturnbull@example.com>]

Josh Glover writes:

 > How in the world does email forwarding work?

If you can do it at all, the forwarding part is already working.
Email is *fundamentally* a store and forward mechanism, except that
frequently the contact to the next MTA in the chain is optimized to a
pipe, a special-purpose socket (the submission protocol and similar),
or an internal data structure.  The algorithm is very simple:

1. Store a STD 11 (aka RFC 822) formatted message and some metadata
   (MAIL FROM and RCPT TO) in a place where the MTA knows to find it,
   aka The Queue.  This might be done by a user agent or by a
   receiving MTA.
2. Wait for the MTA to find it in The Queue.
3. The MTA tests for mail it knows how to route, and if
    a. yes, it sends it to the destination (an A or MX DNS record)
    b. no, it sends it to a "smart host" that handles all other mail
    c. if you get here and there's no smart host, the mail fails and
       depending on how it failed, it keeps the file(s) in The Queue
       and tries again, or returns them to the previous MTA or user.
4. The MTA waits for 250 OK, and if it gets it, removes the message
   and metadata from The Queue.

The process repeats at each receiving MTA in turn until the message
finally gets delivered or fails in a way that can't be retried.
There's a reason this is called the *Simple* Mail Transport Protocol.

Simple as it is, I can think of a bunch of reasons why this would
fail, mostly having to do with "you're not authorized to do this."

 > I recently pointed my domain MX records away from a friend's
 > mailserver and onto my own ramshackle setup, [1] which forwards my
 > email to a Gmail address. However, in order to deliver mail to my
 > Gmail address, I have to change the sender address

What do you mean by "sender address"?  There are six independent
sender addresses, with no necessary relationship among them.  The four
important ones are

RFC 821 HELO: the domain of the sending MTA for this hop
RFC 821 MAIL FROM: the agent responsible for the message (usually the
    same as RFC 822 "From").  This is the so-called "envelope from"
    and it does not appear in the header of the message as originally
    sent.  It typically appears in the initial From_ line of a message
    stored in a Unix mbox and in some other formats, and some MTAs
    will stuff it into a header field on receipt.  Some systems force
    this to be an address of the system user that injects the message;
    others allow you to use "sendmail -f" or similar to set it.
RFC 822 Sender: the agent that most recently injected the message into
    the Internet mail system, and responsible for technical problems
    with the mail (corrupted attachments and the like).  Usually omitted.
RFC 822 From: the author, ie, the agent responsible for the content of
    the message.

(The other two are "Original-From" and "Original-Sender" for use when
a message leave the mail system, is processed by some other agent, and
reinjected, such as a forward.)

Which of those are you modifying?

 > to the target Gmail address (which makes sense from a spam
 > prevention point of view).

That's actually bizarre.  Mail *from* a local address that appears to
have originated outside the local administrative domain typically is
flagged as spam (or gets "spam points").

 > [1] AWS Simple Email Service writing to an S3 bucket and a Lambda
 > function that picks up the mail and sends it on using SMTP to
 > mail.google.com.

ISTR AWS doesn't allow you to run an MTA.  Does it?  I wouldn't be
surprised if this setup violates your ToS, it looks awfully close to
an open relay.

But rules be damned, what error or anomolous behavior do you get if
you don't change the "sender"?

Steve