Support STARTTLS (port 25)

dfrankow commented

2020-04-23 16:52:50 +02:00

(Migrated from github.com)

As documented in #18, my ISP blocks port 25. Similarly, Amazon also blocks port 25 by default.

When I try port smtp.gmail.com:587, it wants TLS:

09:36:19.037743 send: 'helo smtp.gmail.com\r\n'
09:36:19.089443 reply: b'250 smtp.gmail.com at your service\r\n'
09:36:19.089573 reply: retcode (250); Msg: b'smtp.gmail.com at your service'
09:36:19.089910 send: 'mail FROM:<valid.email@example.com>\r\n'
09:36:19.137085 reply: b'530 5.7.0 Must issue a STARTTLS command first. h19sm1787816qtk.78 - gsmtp\r\n'
...

It might be useful to support TLS if it were easy, though I don't know enough to say for sure.

As documented in #18, my ISP blocks port 25. Similarly, [Amazon also blocks port 25 by default](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/). When I try port smtp.gmail.com:587, it wants TLS: ``` 09:36:19.037743 send: 'helo smtp.gmail.com\r\n' 09:36:19.089443 reply: b'250 smtp.gmail.com at your service\r\n' 09:36:19.089573 reply: retcode (250); Msg: b'smtp.gmail.com at your service' 09:36:19.089910 send: 'mail FROM:<valid.email@example.com>\r\n' 09:36:19.137085 reply: b'530 5.7.0 Must issue a STARTTLS command first. h19sm1787816qtk.78 - gsmtp\r\n' ... ``` It might be useful to support TLS if it were easy, though I don't know enough to say for sure.

karolyi commented

2020-04-23 17:02:20 +02:00

(Migrated from github.com)

Port 587 is not an SMTP port, it's just SMTP-ish.

That said, TLS support might be worth it, but as the current state of internet, 25 should suffice. If your ISP blocks it, you might want to inquire as to why they do it.

[Port 587 is not an SMTP port, it's just SMTP-ish.](https://www.mailgun.com/blog/which-smtp-port-understanding-ports-25-465-587/) That said, TLS support might be worth it, but as the current state of internet, 25 should suffice. If your ISP blocks it, you might want to inquire as to why they do it.

karolyi commented

2020-04-23 17:10:20 +02:00

(Migrated from github.com)

Also note in the case of Amazon, throttle != block. Throttling it makes more sense to me, to avoid spam from EC2 instances.

Moreover, a proper authenticated sender can still use EC2 instances to send emails, see the same document you linked.

What are you trying to use this module for?

Also note in the case of Amazon, throttle != block. Throttling it makes more sense to me, to avoid spam from EC2 instances. Moreover, a proper authenticated sender can still use EC2 instances to send emails, see the same document you linked. What are you trying to use this module for?

dfrankow commented

2020-04-23 19:21:47 +02:00

(Migrated from github.com)

My ISP blocks it for the same reason as everyone else: "In order to help protect against spam".

In the case of Amazon, I did not realize it's a throttle, because the behavior I see looks like a block. By "proper authenticated sender", I think you mean I could file a support ticket and try to convince them.

An alternative to filing support tickets with Amazon is a library that works with port 587, which is why I filed this feature request. You can accept my request or not, as you choose.

I am trying to use this module to validate a few hundred email addresses in a research project. Emails validated thus far: zero. Also, I may have to repeat the process, so it seemed worth trying to get it to work systematically. However, I may have to try other routes, like getting unblocked friends to run my script or using a paid validation service.

My ISP blocks it for [the same reason as everyone else](https://usinternet.com/support/access-control-filters/): "In order to help protect against spam". In the case of Amazon, I did not realize it's a throttle, because the behavior I see looks like a block. By "proper authenticated sender", I think you mean I could file a support ticket and try to convince them. An alternative to filing support tickets with Amazon is a library that works with port 587, which is why I filed this feature request. You can accept my request or not, as you choose. I am trying to use this module to validate a few hundred email addresses in a research project. Emails validated thus far: zero. Also, I may have to repeat the process, so it seemed worth trying to get it to work systematically. However, I may have to try other routes, like getting unblocked friends to run my script or using a paid validation service.

karolyi commented

2020-04-23 19:53:31 +02:00

(Migrated from github.com)

Again, port 587 won't work for address validation, as it's for submission, not for inbound SMTP. It speaks SMTP but won't do validation normally, at least it's not its job. Hence SMTP-ish, but not SMTP. Most likely all responses will be denied with "relay access denied without authorization". With submission, you have to authorize yourself as a sender, that happens with TLS using SASL.

I use this project in a couple of my projects to check emails at registration time, so not hundreds at a time. Doing so can make you (or your server's IP) end up in various blocklists. I told this to other users as well: this is is a tool, but every tool can become a weapon if you use it the wrong way.

One more thing: SMTP servers that use blocklists (such as mine) will block your IP, even if you get unblocked by your ISP. ISP IP pools are in blocklists for the most part, exactly because why your ISP blocks port 25: blocking spam from botnets (malware infected PCs) used by spammers.

The takeaway here is, SSL support (STARTTLS on port 25 or SSL on port 465 with SMTP_SSL) could be added as functionality, but unencrypted port 25 should normally suffice.

Again, port 587 won't work for address validation, as it's for submission, not for inbound SMTP. It speaks SMTP but won't do validation normally, at least it's not its job. Hence SMTP-ish, but not SMTP. Most likely all responses will be denied with "relay access denied without authorization". With submission, you have to authorize yourself as a sender, that happens with TLS using [SASL](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer). I use this project in a couple of my projects to check emails at registration time, so not hundreds at a time. Doing so can make you (or your server's IP) end up in various blocklists. I told this to other users as well: this is is a tool, but every tool can become a weapon if you use it the wrong way. One more thing: SMTP servers that use blocklists (such as mine) will block your IP, even if you get unblocked by your ISP. ISP IP pools are in blocklists for the most part, exactly because why your ISP blocks port 25: blocking spam from botnets (malware infected PCs) used by spammers. The takeaway here is, SSL support (STARTTLS on port 25 or SSL on port 465 with SMTP_SSL) could be added as functionality, but unencrypted port 25 should normally suffice.

dfrankow commented

2020-04-23 22:50:58 +02:00

(Migrated from github.com)

Thanks for the information, very helpful. I leave it to you whether to resolve this or not.

hack3r-0m commented

2020-06-17 12:49:16 +02:00

(Migrated from github.com)

As per RFC 8314, implicit TLS is recommended rather than explicit TLS on 587. can I try to add support for port 465? and what challenges I might face for this? Thanks

karolyi commented

2020-06-17 14:19:07 +02:00

(Migrated from github.com)

Again, port 587 is submission, as in outbound emails. It's not used for checking email addresses that the host has. Port 465 is defined as smtps, but is used as an implicit TLS version of the port 586 with which you have to use STARTTLS.

Let's clarify this once and for all: none of these two ports are used for inbound emails, normally.

You can try to implement a STARTTLS with port 25. The smtplib library has a starttls method, that might be useful. but be prepared to fall back to cleartext if the remote server doesn't support it. Submit a PR and I'll look into it.

Again, port 587 is submission, as in _outbound_ emails. It's not used for checking email addresses that the host has. Port 465 is defined as `smtps`, but is used as an implicit TLS version of the port 586 with which you have to use STARTTLS. Let's clarify this once and for all: none of these two ports are used for *inbound* emails, normally. You can try to implement a STARTTLS with port 25. The smtplib library has a [`starttls`](https://docs.python.org/3/library/smtplib.html#smtplib.SMTP.starttls) method, that might be useful. but be prepared to fall back to cleartext if the remote server doesn't support it. Submit a PR and I'll look into it.

hack3r-0m commented

2020-07-09 19:28:06 +02:00

(Migrated from github.com)

@karolyi starttls should only be viable if valid MX records exist in order to check whether MX accepts mail or not right?

@karolyi `starttls` should only be viable if valid MX records exist in order to check whether MX accepts mail or not right?

karolyi commented

2020-07-10 15:08:26 +02:00

(Migrated from github.com)

no, not really.

any host that will be probed might or might not support STARTTLS. the protocol being negotiated is completely different from the DNS records resolving.

no, not really. any host that will be probed might or might not support STARTTLS. the protocol being negotiated is completely different from the DNS records resolving.

karolyi commented

2020-10-11 14:08:18 +02:00

(Migrated from github.com)

This should work now in 0.2.10, please update and let me know if it works for you.

This should work now in `0.2.10`, please update and let me know if it works for you.

stale[bot] commented

2020-10-25 14:23:54 +01:00

(Migrated from github.com)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Support STARTTLS (port 25) #19