Develop Reference

I would like to define a server using smtplib on Python but an error occurred

A program I wanted to create requires to login to a mail server.
I was able to login using smtplib on Gmail, but it didn't work on Yahoo mail.
SMTP Port 465
Server mail: smtp.mail.yahoo.co.jp
python code on IDLE
import smtplib
smtp_obg = smtplib.SMTP("smtp.mail.yahoo.co.jp", 465)
Error message
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
smtp_obg = smtplib.SMTP("smtp.mail.yahoo.co.jp", 465)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/smtplib.py", line 253, in __init__
(code, msg) = self.connect(host, port)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/smtplib.py", line 341, in connect
(code, msg) = self.getreply()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/smtplib.py", line 398, in getreply
raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed
I would appreciate if someone could help. Thank you in advance.

You must run login after creating smtp_obj (smtp_obg.login('username', 'password')). If you trying to login, please mention it.
You must activate SMTP feature at Yahoo's web page (Same as Gmail). Sorry for Japanese, but not sure if yahoo.co.jp is using somewhere outside Japan. :)
In English, this parameter should be called like: IMAP/POP/SMTP access and mail transfer
I checked my account. STMP access is disabled by default.

When you create a connection to your smtp server you need to know more about the server about what way you want to connect to it.
There are normal with and without SSL connection based on port number to your SMTP server.
I use this site https://settings.email/yahoo-co-jp-email-settings/ to get more info about your SMTP server you want to use from yahoo and it says the port 465 is an SSL connection so you need to connect with SSL connection else you can't sending out this way.
in smtplib in python you have smtplib.SMTP and smtplib.SMTP_SSL functions to work with, the first it's used when it's not SSL requirement, and the second its used when you need SSL like yahoo smtp.
Can you try out to use smtplib.SMTP_SSL function and see about its working for you?
import smtplib
smtp_obg = smtplib.SMTP_SSL("smtp.mail.yahoo.co.jp", 465)
Mabey its required a username and password to sending out, I did not have so much info about yahoo SMTP to can explain that part for you.

MQTT PAHO [CERTIFICATE_VERIFY_FAILED]

I have a problem with Python (I'm a Python noob and learning it).
I used the version 2.7.9 on a Debian 9 System. I installed paho and tinkerforge package in python.
I developed a script using the Paho MQTT client to connected my mosquitto broker. I want to use a crypted connection. My connection work fine when not encrypted but fails when encrypted. The connection work fine encrypted on openHAB (MQTT-Subscriber) and MQTTFX (MQTT-Subscriber and Producer)
I'm using these parameters for my script:
self.client = mqtt.Client()
self.client.tls_set("/home/pi/ca-cert.pem","/home/pi/IWILR1-1-cert.pem","/home/pi/IWILR1-1.pem",tls_version=ssl.PROTOCOL_TLSv1)
# disables peer verification
self.client.tls_insecure_set(False)
self.client.on_connect = self.mqtt_on_connect
self.client.on_disconnect = self.mqtt_on_disconnect
self.client.on_message = self.mqtt_on_message
self.device_proxies = {}
self.device_proxy_classes = {}
for subclass in DeviceProxy.subclasses():
self.device_proxy_classes[subclass.DEVICE_CLASS.DEVICE_IDENTIFIER] = subclass
def connect(self):
if self.broker_username is not None:
self.client.username_pw_set(self.broker_username, self.broker_password)
self.client.connect(self.broker_host, self.broker_port)
self.client.loop_start()
But now the problem is the error on Python.
sudo python /home/pi/brick-mqtt-proxy.py
Traceback (most recent call last):
File "/home/pi/brick-mqtt-proxy.py", line 1250, in <module>
proxy.connect()
File "/home/pi/brick-mqtt-proxy.py", line 1109, in connect
self.client.connect(self.broker_host, self.broker_port)
File "/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py", line 760, in connect
return self.reconnect()
File "/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py", line 919, in reconnect
sock.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
and on mosquitto these errors arrived.
1504896114: New connection from 143.93.197.20 on port 8883.
1504896114: OpenSSL Error: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca
1504896114: OpenSSL Error: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure
1504896114: Socket error on client <unknown>, disconnecting.
Mosquitto conf
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example
pid_file /var/run/mosquitto.pid
persistence true
persistence_location /var/lib/mosquitto/
log_type all
log_facility 5
log_timestamp true
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d
port 8883
cafile /etc/mosquitto/ca_certificates/ca-cert.pem
certfile /etc/mosquitto/certs/server-cert.pem
keyfile /etc/mosquitto/certs/server-key.pem
Only Server and Ca matched the broker hostname. Client use there own hostname for CN. I hope thats right?
I hope you can help me to fix my problem.
PS: I used a self-signed certificate! TLS Version 1.2

If you are using TLS v1.2 you need to modify the expression (2nd line: self.client.tls_set()) 'tls_version=ssl.PROTOCOL_TLSv1' to 'tls_version=ssl.PROTOCOL_TLSv1_2', not as expected to ...TLSv1.2. This worked for me.

try providing something like below. Default port for ssl is 8883. We can start multiple listeners. In this case non-ssl on 1883 and ssl on 8883.
port 1883
listener 8883

Scapy packet manipulation in standard socket

Im writing HTTP Proxy Server that will open socket with the browser and the request from the browser will go to my HTTP Proxy Server and my server will open socket with the server that the browser ask for and send him the request.
It will go like this:
Browser --request--> HTTP Proxy Server --request--> Web Server
Browser <--response-- HTTP Proxy Server <--response-- Web Server
Now I need those sockets will be clearly that I can use scapy to see the layers of each packet and manipulate it like I want to. (for security reasons e.g Block Phishing or something like that)
In this code I write simple socket with the browser just for testing and learning about browser behavior with HTTP Requests.
from scapy.all import *
import socket
socket_with_browser = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
print "Start"
socket_with_browser.bind(('127.0.0.1',8080))
socket_with_browser.listen(1)
conn , addr = socket_with_browser.accept()
stream_sock_browser = StreamSocket(conn)
r = stream_sock_browser.recv(4096)
r[TCP].show()
socket_with_browser.close()
I get the following Error:
Traceback (most recent call last):
File "<string>", line 254, in run_nodebug
File "C:\Python26\ProxyServer\module1.py", line 22, in <module>
r[TCP].show()
File "C:\Python26\Lib\site-packages\scapy\packet.py", line 817, in __getitem__
raise IndexError("Layer [%s] not found" % lname)
IndexError: Layer [TCP] not found
There is a way that I can get packet from socket and use it (get packet layers or something) with scapy? Maybe I declare the socket badly? By the way Im using Windows 7 and python 2.6

Six years late but your question might appear to others with similar issues.
The problem with your approach is that not every packet received has a TCP layer, so you should use the haslayer() method.
if r.haslayer(TCP): # yes, without quotation marks
r.show()

Error using httlib's HTTPSConnection with PKCS#12 certificate

I'm trying to use httplib's HTTPSConnection for client validation, using a PKCS #12 certificate. I know the certificate is good, as I can connect to the server using it in MSIE and Firefox.
Here's my connect function (the certificate includes the private key). I've pared it down to just the basics:
def connect(self, cert_file, host, usrname, passwd):
self.cert_file = cert_file
self.host = host
self.conn = httplib.HTTPSConnection(host=self.host, port=self.port, key_file=cert_file, cert_file=cert_file)
self.conn.putrequest('GET', 'pathnet/,DanaInfo=200.222.1.1+')
self.conn.endheaders()
retCreateCon = self.conn.getresponse()
if is_verbose:
print "Create HTTPS connection, " + retCreateCon.read()
(Note: No comments on the hard-coded path, please - I'm trying to get this to work first; I'll make it pretty afterwards. The hard-coded path is correct, as I connect to it in MSIE and Firefox. I changed the IP address for the post.)
When I try to run this using a PKCS#12 certificate (a .pfx file), I get back what appears to be an openSSL error. Here is the entire error traceback:
File "Usinghttplib_Test.py", line 175, in
t.connect(cert_file=opts["-keys"], host=host_name, usrname=opts["-username"], passwd=opts["-password"])
File "Usinghttplib_Test.py", line 40, in connect
self.conn.endheaders()
File "c:\python26\lib\httplib.py", line 904, in endheaders
self._send_output()
File "c:\python26\lib\httplib.py", line 776, in _send_output
self.send(msg)
File "c:\python26\lib\httplib.py", line 735, in send
self.connect()
File "c:\python26\lib\httplib.py", line 1112, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "c:\python26\lib\ssl.py", line 350, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "c:\python26\lib\ssl.py", line 113, in __init__
cert_reqs, ssl_version, ca_certs) ssl.SSLError: [Errno 336265225] _ssl.c:337: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
Notice, the openSSL error (the last entry in the list) notes "PEM lib", which I found odd, since I'm not trying to use a PEM certificate.
For kicks, I converted the PKCS#12 cert to a PEM cert, and ran the same code using that. In that case, I received no error, I was prompted to enter the PEM pass phrase, and the code did attempt to reach the server. (I received the response "The service is not
available. Please try again later.", but I believe that would be because the server does not accept the PEM cert. I can't connect in Firefox to the server using the PEM cert either.)
Is httplib's HTTPSConnection supposed to support PCKS#12 certificates? (That is, pfx files.) If so, why does it look like openSSL is trying to load it inside the PEM lib? Am I doing this all wrong?
Any advice is welcome.
EDIT: The certificate file contains both the certificate and the private key, which is why I'm providing the same file name for both the HTTPSConnection's key_file and cert_file parameters.

This is no surprise. The Python library reference docs are pretty clear about this. From http://docs.python.org/library/httplib.html:
class httplib.HTTPSConnection(host[, port[, key_file[, cert_file[, strict[, timeout[, source_address]]]]]])
A subclass of HTTPConnection that uses SSL for communication with secure servers. Default port is 443. key_file is the name of a PEM formatted file that contains your private key. cert_file is a PEM formatted certificate chain file.

On the openSSL mailing list, I chatted with Mounir Idrassi. He noted that openSSL does support PKCS#12 files, and - based on the error message I'm receiving - it appears that httplib is calling the wrong function to load the key.
In his words:
Concerning the error you are getting, it appears that the phython module you are using is calling SSL_CTX_use_PrivateKey_file by giving it the PKCS#12 file name. This is does not because SSL_CTX_use_PrivateKey_file only accepts two formats : SSL_FILETYPE_PEM and SSL_FILETYPE_ASN1.
(I'm giving httplib the PKCS#12 file name as key file, because this file format includes both the cert and the private key in the same file.)
In order to correct this, you have two solutions :
- Either feed the python module with the private key in a PEM file.
- Or modify the source code of this python module in order to use the PKCS#12 functions I mentioned above to extract the private key as an EVP_PKEY and then call SSL_use_PrivateKey instead of SSL_CTX_use_PrivateKey_file, along with SSL_use_certificate for setting the associated certificate.
(I tried the former and wasn't able to get it to work. Doesn't necessarily mean it won't work; only that I wasn't able to.)

I/O error(socket error): [Errno 111] Connection refused

I have a program that uses urllib to periodically fetch a url, and I see intermittent
errors like :
I/O error(socket error): [Errno 111] Connection refused.
It works 90% of the time, but the othe r10% it fails. If retry the fetch immediately after it fails, it succeeds. I'm unable to figure out why this is so. I tried to see if any ports are available, and they are. Any debugging ideas?
For additional info, the stack trace is:
File "/usr/lib/python2.6/urllib.py", line 203, in open
return getattr(self, name)(url)
File "/usr/lib/python2.6/urllib.py", line 342, in open_http
h.endheaders()
File "/usr/lib/python2.6/httplib.py", line 868, in endheaders
self._send_output()
File "/usr/lib/python2.6/httplib.py", line 740, in _send_output
self.send(msg)
File "/usr/lib/python2.6/httplib.py", line 699, in send
self.connect()
File "/usr/lib/python2.6/httplib.py", line 683, in connect
self.timeout)
File "/usr/lib/python2.6/socket.py", line 512, in create_connection
raise error, msg
Edit - A google search isn't very helpful, what I got out of it is that the server
I'm fetching from sometimes refuses connections, how can I verify its not a bug in my code
and this is indeed the case?

Use a packet sniffer like Wireshark to look at what happens. You need to see a SYN-flagged packet outgoing, a SYN+ACK-flagged incoming and then a ACK-flagged outgoing. After that, the port is considered open on the local side.
If you only see the first packet and the error message comes after several seconds of waiting, the other side is not answering at all (like in: unplugged cable, overloaded server, misguided packet was discarded) and your local network stack aborts the connection attempt. If you see RST packets, the host actually denies the connection. If you see "ICMP Port unreachable" or host unreachable packets, a firewall or the target host inform you of the port actually being closed.
Of course you cannot expect the service to be available at all times (consider all the points of failure in between you and the data), so you should try again later.

Getting an ECONNREFUSED errno means that your kernel was refused a connection at the other end, so if it's a bug, it's either in your kernel or in the other end.
What you can do is to trap the error in a very specific way and try again in a little while, since this seems to work:
# This is Python > 2.5 code
import errno, time
for attempt in range(MAXIMUM_NUMBER_OF_ATTEMPTS):
try:
# your urllib call here
except EnvironmentError as exc: # replace " as " with ", " for Python<2.6
if exc.errno == errno.ECONNREFUSED:
time.sleep(A_COUPLE_OF_SECONDS)
else:
raise # re-raise otherwise
else: # we tried, and we had no failure, so
break
else: # we never broke out of the for loop
raise RuntimeError("maximum number of unsuccessful attempts reached")
Replace the two all-caps constants with your favourite numbers.

I previously had this problem with my EC2 instance (I was serving couchdb to serve resources -- am considering Amazon's S3 for the future).
One thing to check (assuming Ec2) is that the couchdb port is added to your open ports within your security policy.
I specifically encountered
"[Errno 111] Connection refused"
over EC2 when the instance was stopped and started. The problem seems to be a pidfile race. The solution for me was killing couchdb (entirely and properly) via:
pkill -f couchdb
and then restarting with:
/etc/init.d/couchdb restart

I'm not exactly sure what's causing this. You can try looking in your socket.py (mine is a different version, so line numbers from the trace don't match, and I'm afraid some other details might not match as well).
Anyway, it seems like a good practice to put your url fetching code in a try: ... except: ... block, and handle this with a short pause and a retry. The URL you're trying to fetch may be down, or too loaded, and that's stuff you'll only be able to handle in with a retry anyway.

Its seems that server is not running properly so ensure that with terminal by
telnet ip port
example
telnet localhost 8069
It will return connected to localhost so it indicates that there is no problem with the connection
Else it will return Connection refused it indicates that there is problem with the connection