I am trying to check if a domain name has MX records resolved using dnspython module. I am getting the following error while connecting to the mx record server. Can anyone explain why I am facing this issue?
Traceback (most recent call last):
File "c:\Users\iamfa\OneDrive\Desktop\test\email_mx.py", line 26, in <module>
dns.resolver.resolve("cmrit.ac.in", 'MX')
File "c:\Users\iamfa\OneDrive\Desktop\test\env1\lib\site-packages\dns\resolver.py", line 1193, in resolve
return get_default_resolver().resolve(qname, rdtype, rdclass, tcp, source,
File "c:\Users\iamfa\OneDrive\Desktop\test\env1\lib\site-packages\dns\resolver.py", line 1066, in resolve
timeout = self._compute_timeout(start, lifetime,
File "c:\Users\iamfa\OneDrive\Desktop\test\env1\lib\site-packages\dns\resolver.py", line 879, in _compute_timeout
raise LifetimeTimeout(timeout=duration, errors=errors)
dns.resolver.LifetimeTimeout: The resolution lifetime expired after 5.001 seconds: Server 10.24.0.1 UDP port 53 answered The DNS operation timed out.; Server 198.51.100.1 UDP port 53 answered The DNS operation timed out.; Server 10.95.11.110 UDP port 53 answered The DNS operation timed out.
This is my code:
import dns.resolver
if dns.resolver.resolve("cmrit.ac.in", 'MX'):
print(True)
else:
print(False)
However it was working fine till yesterday but when I try to run the same code today I am facing this issue.
If the remote DNS server takes a long time to respond, or accepts the connection but does not respond at all, the only thing you can really do is move along. Perhaps try again later. You can catch the error with try/except:
import dns.resolver
try:
if dns.resolver.resolve("cmrit.ac.in", 'MX'):
print(True)
else:
print(False)
except dns.resolver.LifetimeError:
print("timed out, try again later maybe?")
If you want to apply a longer timeout, the resolve method accepts a lifetime keyword argument which is documented in the Resolver.resolve documentation.
The Resolver class (documented at the top of the same page) also has a timeout parameter you can tweak if you build your own resolver.
For production code, you should probably add the other possible errors to the except clause; the exemplary documentation shows you precisely which exceptions resolve can raise.
...
except (dns.resolver.LifetimeTimeout, dns.resolver.NXDOMAIN,
dns.resolver.YXDOMAIN, dns.resolver.NoAnswer,
dns.resolver.NoNameservers) as err:
print("halp, something went wrong:", err)
Probably there is a base exception class which all of these inherit from; I was too lazy to go back and check. Then you only have to list the base class in the except statement.
It's probably more useful to extract the actual MX record and display it, rather than just print True, but that's a separate topic.
Your error message indicates that you were able to connect to your own resolver at 10.24.0.1 but in the general case, this error could also happen if your network (firewall etc) prevents you from accessing DNS for some reason.
Related
I was using psycopg2 in python script to connect to Redshift database and occasionally I receive the error as below:
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
This error only happened once awhile and 90% of the time the script worked.
I tried to put it into a try and except block to catch the error, but it seems like the catching didn't work. For example, I try to capture the error so that it will automatically send me an email if this happens. However, the email was not sent when error happened. Below are my code for try except:
try:
conn2 = psycopg2.connect(host="localhost", port = '5439',
database="testing", user="admin", password="admin")
except psycopg2.Error as e:
print ("Unable to connect!")
print (e.pgerror)
print (e.diag.message_detail)
# Call check_row_count function to check today's number of rows and send
mail to notify issue
print("Trigger send mail now")
import status_mail
print (status_mail.redshift_failed(YtdDate))
sys.exit(1)
else:
print("RedShift Database Connected")
cur2 = conn2.cursor()
rowcount = cur2.rowcount
Errors I received in my log:
Traceback (most recent call last):
File "/home/ec2-user/dradis/dradisetl-daily.py", line 579, in
load_from_redshift_to_s3()
File "/home/ec2-user/dradis/dradisetl-daily.py", line 106, in load_from_redshift_to_s3
delimiter as ','; """.format(YtdDate, s3location))
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
So the question is, what causes this error and why isn't my try except block catching it?
From the docs:
exception psycopg2.OperationalError
Exception raised for errors that are related to the database’s
operation and not necessarily under the control of the programmer,
e.g. an unexpected disconnect occurs, the data source name is not
found, a transaction could not be processed, a memory allocation error
occurred during processing, etc.
This is an error which can be a result of many different things.
slow query
the process is running out of memory
other queries running causing tables to be locked indefinitely
running out of disk space
firewall
(You should definitely provide more information about these factors and more code.)
You were connected successfully but the OperationalError happened later.
Try to handle these disconnects in your script:
Put the command you want to execute into a try-catch block and try to reconnect if the connection is lost.
Recently encountered this error. The cause in my case was the network instability while working with database. If network will became down for enough time that the socket detect the timeout you will see this error. If down time is not so long you wont see any errors.
You may control timeouts of Keepalive and RTO features using this code sample
s = socket.fromfd(conn.fileno(), socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
s.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 6)
s.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 2)
s.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, 2)
s.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, 10000)
More information you can find in this post
If you attach the actual code that you are trying to except it would be helpful. In your attached stack trace its : " File "/home/ec2-user/dradis/dradisetl-daily.py", line 106"
Similar except code works fine for me. mind you, e.pgerror will be empty if the error occurred on the client side, such as the error in my example. the e.diag object will also be useless in this case.
try:
conn = psycopg2.connect('')
except psycopg2.Error as e:
print('Unable to connect!\n{0}'.format(e))
else:
print('Connected!')
Maybe it will be helpful for someone but I had such an error when I've tried to restore backup to the database which has not sufficient space for it.
I'm writing application in python using Asyncio for networking. I have code similar too:
try:
data = await self._reader.readexactly(10000)
# Code that uses data
except IncompleteReadError as e:
data = e.parial
# More code
When I try running this code, it never seems to actually run. If I set a breakpoint on the second line, the breakpoint will trip, but the rest of the function is ignored.
The closest thing I get to an error is this from the asyncio logger:
Traceback (most recent call last):
File "c:\python36\Lib\asyncio\selector_events.py", line 724, in _read_ready
data = self._sock.recv(self.max_size)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
Replacing the second line with data = await self._reader.read(10000) appears to solve this issue, but read() doesn't solve my issue, I need to use readexactly(). So why does readexactly() cause a socket error but read() not?
the only difference between the two is that "read" read up to n bytes while readexactly reads exactly n bytes and if reaches the end before n bytes is raises an IncompleteReadError , which may cause your socket to get the error you have pointed out.
When using Python to make a connection to ShareFile via implicit FTPS I get the following:
Traceback (most recent call last):
ftps.storbinary("STOR /file, open(file, "rb"), 1024)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ftplib.py", line 769, in storbinary
conn.unwrap()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 791, in unwrap
s = self._sslobj.shutdown()
SSLError: ('The read operation timed out',)
My tyFTP (required because implicit FTPS is not directly supported in ftplib) class comes from here: Python FTP implicit TLS connection issue. Here's the code:
ftps = tyFTP()
try:
ftps.connect(‘ftps.host.domain’, 990)
except:
traceback.print_exc()
traceback.print_stack()
ftps.login(‘uid', ‘pwd')
ftps.prot_p()
try:
ftps.storbinary("STOR /file", open(file, "rb"), 1024)
# i also tried non-binary, but that didn't work either
# ftps.storlines("STOR /file", open(file, "r"))
except:
traceback.print_exc()
traceback.print_stack()
This question has been asked previously, but the only solution provided is to hack the python code. Is that the best/only option?
ShareFile upload with Python 2.7.5 code timesout on FTPS STOR
ftplib - file creation very slow: SSLError: The read operation timed out
ftps.storlines socket.timeout despite file upload completing
There is also an old discussion about this issue on python.org: http://bugs.python.org/issue8108. The suggestion there is that this is an ambiguous situation that's difficult to fix (and maybe never was?)
Please note: I would have added comments to the existing questions, but my reputation was not high enough to comment (new stack exchange user).
sometimes the help you need is your own.
In order to fix this without directly modifying ftplib code (which requires jumping through hoops on a Mac because you cannot easily write/modify files in your /System/Library) I overrode the storbinary method in ftplib.FTP_TLS. That's essentially using this fix for supporting implicit FTPS:
Python FTP implicit TLS connection issue
and then adding these lines to the class tyFTP, and commenting out the conn.unwrap() call, and replacing it with 'pass':
def storbinary(self, cmd, fp, blocksize=8192, callback=None, rest=None):
self.voidcmd('TYPE I')
conn = self.transfercmd(cmd, rest)
try:
while 1:
buf = fp.read(blocksize)
if not buf: break
conn.sendall(buf)
if callback: callback(buf)
if isinstance(conn, ssl.SSLSocket):
pass
# conn.unwrap()
finally:
conn.close()
return self.voidresp()
My issue with implicit ftp over TLS has bothered me for six months. This week, I decided it was time to fix it. Finally, I combined the code from George Leslie-Waksman and gaFF at here, and Manager_of_it here, and it work like champ! Thank you, those three persons.
I have this simple minimal 'working' example below that opens a connection to google every two seconds. When I run this script when I have a working internet connection, I get the Success message, and when I then disconnect, I get the Fail message and when I reconnect again I get the Success again. So far, so good.
However, when I start the script when the internet is disconnected, I get the Fail messages, and when I connect later, I never get the Success message. I keep getting the error:
urlopen error [Errno -2] Name or service not known
What is going on?
import urllib2, time
while True:
try:
print('Trying')
response = urllib2.urlopen('http://www.google.com')
print('Success')
time.sleep(2)
except Exception, e:
print('Fail ' + str(e))
time.sleep(2)
This happens because the DNS name "www.google.com" cannot be resolved. If there is no internet connection the DNS server is probably not reachable to resolve this entry.
It seems I misread your question the first time. The behaviour you describe is, on Linux, a peculiarity of glibc. It only reads "/etc/resolv.conf" once, when loading. glibc can be forced to re-read "/etc/resolv.conf" via the res_init() function.
One solution would be to wrap the res_init() function and call it before calling getaddrinfo() (which is indirectly used by urllib2.urlopen().
You might try the following (still assuming you're using Linux):
import ctypes
libc = ctypes.cdll.LoadLibrary('libc.so.6')
res_init = libc.__res_init
# ...
res_init()
response = urllib2.urlopen('http://www.google.com')
This might of course be optimized by waiting until "/etc/resolv.conf" is modified before calling res_init().
Another solution would be to install e.g. nscd (name service cache daemon).
For me, it was a proxy problem.
Running the following before import urllib.request helped
import os
os.environ['http_proxy']=''
response = urllib.request.urlopen('http://www.google.com')
This question is related to How do we handle Python xmlrpclib Connection Refused?
When I try to use the following code, with my RPC server down, _get_rpc() returns False and I'm good to go. However if the server is running, it fails with unknown method. Is it trying to execute .connect() on the remote server? How can I get around this, when I needed to use .connect() to detect if the returned proxy worked (see related question)?
import xmlrpclib
import socket
def _get_rpc():
try:
a = xmlrpclib.ServerProxy('http://dd:LNXFhcZnYshy5mKyOFfy#127.0.0.1:9001')
a.connect() # Try to connect to the server
return a.supervisor
except socket.error:
return False
if not _get_rpc():
print "Failed to connect"
Here is the issue:
ahiscox#lenovo:~/code/dd$ python xmlrpctest2.py
Failed to connect
ahiscox#lenovo:~/code/dd$ supervisord -c ~/.supervisor # start up RPC server
ahiscox#lenovo:~/code/dd$ python xmlrpctest2.py
Traceback (most recent call last):
File "xmlrpctest2.py", line 13, in <module>
if not _get_rpc():
File "xmlrpctest2.py", line 7, in _get_rpc
a.connect() # Try to connect to the server
File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
File "/usr/lib/python2.6/xmlrpclib.py", line 1253, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.6/xmlrpclib.py", line 1392, in _parse_response
return u.close()
File "/usr/lib/python2.6/xmlrpclib.py", line 838, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: 'UNKNOWN_METHOD'>
Well i was just looking in to it ; my old method suck because xmlrpclib.ServerProxy try to connect to the XmlRPC server when you call a method, not before !!!
Try this instead :
import xmlrpclib
import socket
def _get_rpc():
a = xmlrpclib.ServerProxy('http://dd:LNXFhcZnYshy5mKyOFfy#127.0.0.1:9001')
try:
a._() # Call a fictive method.
except xmlrpclib.Fault:
# connected to the server and the method doesn't exist which is expected.
pass
except socket.error:
# Not connected ; socket error mean that the service is unreachable.
return False, None
# Just in case the method is registered in the XmlRPC server
return True, a
connected, server_proxy = _get_rpc():
if not connected
print "Failed to connect"
import sys
sys.exit(1)
To summarize this we have 3 cases :
XmlRPC server is up and in it we defined a method called _():
(EDIT : i did choose the name _ because it unlikely to have a method with this name, but this case still can happen)
In this case no exception will be catch and the code will execute
the return True
XmlRPC server is up and in it we don't have any method methoded call
_():
This time xmlrpclib.Fault will be raised and we will also pass to the return True
XmlRPC server is down:
Now the socket.error exception will be raised and that
when we call a._() so we should return False
I don't know if there is an easy way to do this and i will love to see it until then , hope this can fix thing this time :)
N.B: when you do if a: python will again search for a method __nonzero__() to test the boolean value of a and this will fail to.
N.B 2: Some xmlrpc service offer a rpc path specialized to do an authentication , in this path the service offer methods like login() ... , this kinds of method can replace the _() method in our case, so just calling login(), will be enough to know if the service is up or down (socket.error), and in the same time this login() method authenticate the user if the service is up .