So, I'm still at the noob level when it comes to python. I know... I know... there's probably a more efficient way to do what I'm trying but still learning and hopefully, I'll get better with practice.
For a training project, I'm writing a script to do various DNS operations against a domain. I found DNSPython and it seemed to be exactly what I needed to use and I thought I was done with it but when I tried it against a different domain it keeps failing at the zone transfer.
I've got two domains hardcoded right now for testing. The megacorpone domain iw was working as I expected however, now it's failing (with no code change) in order to get it to work I had to filter the first record '#' that was returned otherwise it failed as well.
However, the zonetransfer.me domain sometimes completes the script with error but fails errors sporadically as well, but it never displays the host records for some reason and I've not been able to figure out how to fix it yet, been banging my head against it for a while now.
The megacoprone run was working every time earlier now it's not working at all. The only thing I can think of so far is that it may be a timing issue.
Run with megacoprpone
Attempting zone transfers for megacorpone.com
Traceback (most recent call last):
File "/home/kali/Exercises/Module_7/dns-axfer.py", line 56, in zoneXFR
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
File "/usr/lib/python3/dist-packages/dns/zone.py", line 1106, in from_xfr
for r in xfr:
File "/usr/lib/python3/dist-packages/dns/query.py", line 627, in xfr
raise TransferError(rcode)
dns.query.TransferError: Zone transfer error: REFUSED
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kali/Exercises/Module_7/dns-axfer.py", line 73, in <module>
zoneXFR()
File "/home/kali/Exercises/Module_7/dns-axfer.py", line 66, in zoneXFR
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
UnboundLocalError: local variable 'zone' referenced before assignment
Run 1 with zonetransfer.me
Attempting zone transfers for zonetransfer.me
Results for nsztm1.digi.ninja.
Zone origin: zonetransfer.me
---------------------------------------------------------------------------
Results for nsztm1.digi.ninja.
Zone origin: zonetransfer.me
---------------------------------------------------------------------------
[*] Error: <class 'dns.resolver.NoAnswer'> The DNS response does not contain an answer to the question: _acme-challenge.zonetransfer.me. IN A
Results for nsztm2.digi.ninja.
Zone origin: zonetransfer.me
---------------------------------------------------------------------------
Results for nsztm2.digi.ninja.
Zone origin: zonetransfer.me
---------------------------------------------------------------------------
[*] Error: <class 'dns.resolver.NoAnswer'> The DNS response does not contain an answer to the question: _acme-challenge.zonetransfer.me. IN A
Run 2 with no code change (zonetransfer.me)
Attempting zone transfers for zonetransfer.me
Traceback (most recent call last):
File "/home/kali/Exercises/Module_7/dns-axfer.py", line 56, in zoneXFR
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
File "/usr/lib/python3/dist-packages/dns/zone.py", line 1106, in from_xfr
for r in xfr:
File "/usr/lib/python3/dist-packages/dns/query.py", line 596, in xfr
_net_write(s, tcpmsg, expiration)
File "/usr/lib/python3/dist-packages/dns/query.py", line 364, in _net_write
current += sock.send(data[current:])
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kali/Exercises/Module_7/dns-axfer.py", line 73, in <module>
zoneXFR()
File "/home/kali/Exercises/Module_7/dns-axfer.py", line 66, in zoneXFR
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
UnboundLocalError: local variable 'zone' referenced before assignment
My script: bash away... I can always take constructive criticism.
#!/usr/bin/python3
import sys, argparse
import dns.query
import dns.zone
import dns.resolver
from colorama import Fore, Style
bracket = f"{Fore.BLUE}[{Fore.GREEN}*{Fore.BLUE}]{Style.RESET_ALL} "
bracket_err = f"{Fore.BLUE}[{Fore.RED}*{Fore.BLUE}]{Style.RESET_ALL} "
'''
parser = argparse.ArgumentParser()
parser.add_argument('domain')
args = parser.parse_args()
'''
# domain = (sys.argv[1])
domain = 'megacorpone.com'
#domain = 'zonetransfer.me'
def line():
print ('-' * 75)
return None
def resolveDNS(system):
resolver = dns.resolver.Resolver()
results = resolver.query(system , "A")
return results
def getNS ():
name_servers = dns.resolver.query(domain, 'NS')
print ("\nThe name servers for " + domain + " are:")
line()
for system in name_servers:
A_records = resolveDNS(str(system))
for item in A_records:
answer = ','.join([str(item)])
print (bracket, "{:30}".format(str(system).rstrip('.')), "{:15}".format(answer))
return name_servers
def getMX():
mail_server = dns.resolver.query(domain, 'MX')
print("\nMail servers for", domain)
line()
for system in mail_server:
A_records = resolveDNS(str(system.exchange))
for item in A_records:
answer = ','.join([str(item)])
print(bracket, "{:30}".format(str(system.exchange).rstrip('.')), "{:15}".format(str(answer)), '\t', "{:5}".format("Preference:"), str(system.preference))
return None
def zoneXFR():
print ("\nAttempting zone transfers for", domain,)
for server in name_servers:
try:
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
line()
for host in zone:
if str(host) != '#':
A_records = resolveDNS(str(host) + "." + domain)
for item in A_records:
answer = ','.join([str(item)])
print(bracket, "{:30}".format(str(host) + "." + domain), answer)
except Exception as e:
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
line()
print (bracket_err, f"{Fore.RED}Error:{Style.RESET_ALL}", e.__class__, e)
name_servers = getNS()
getMX()
zoneXFR()
print("\n")
I see that you are trying well-known name servers that are specifically set up for testing. However, for the benefit of other readers, I will add a couple explanations.
As you are probably aware, most name servers will not allow zone transfers nowadays. That being said, it is possible that each of the name servers for a given domain name will behave differently (they could have different configurations and even be running different software).
In the case of megacorpone.com there are 3 name servers listed:
ns2.megacorpone.com.
ns3.megacorpone.com.
ns1.megacorpone.com.
ns2.megacorpone.com is the only one that did allow a zone transfer.
This message
dns.query.TransferError: Zone transfer error: REFUSED
means what it means: your query was refused. Probably you talked to the wrong name server.
Then you have another error which suggest a variable scoping issue:
UnboundLocalError: local variable 'zone' referenced before assignment
You are calling functions in this order:
name_servers = getNS()
getMX()
zoneXFR()
If name_servers fails, then the subsequent call to zoneXFR will fail too. Because this code:
for server in name_servers:
will try to iterate over an empty list.
Intermittent DNS resolution failures are common so a few checks are required here. At the very least, make sure that the list of NS is not empty.
Another issue: you start a for loop outside of the try block so your control structure is broken right in the middle:
for server in name_servers:
try:
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
line()
Do this instead:
try:
for server in name_servers:
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
...
I suspect that your script fails intermittently because the list of name servers is not always returned in the same order. If the first NS returned is ns1.megacorpone.com. or ns3.megacorpone.com. then the code crashes. If the scripts starts with ns2.megacorpone.com (the sole NS allowing zone transfers) then it seems to work OK.
When this code fails (AXFR denied):
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
then zone is not defined, and that's why you cannot print it in your exception block. Instead, show the domain name or some other variables that you know are defined and valid.
So if the AXFR is denied, your script should handle this exception dns.query.TransferError, and quietly move on to the next NS if any, until the list has been exhausted.
Another bit of advice: you try to resolve resources names that are different than '#'. Instead, look at the record type. You should only resolve CNAME, MX or NS. The other common types are TXT, A, AAAA, SOA. The rest are more exotic such as NAPTR, LOC or SRV. Nothing that should be resolved I think.
Fixed up your code, doesn't look great yet, but it works
#!/usr/bin/python3
# you might want to run python3 -m pip install dnspython before running this script
import sys
import dns.query
import dns.zone
import dns.resolver
# formatting setup
from colorama import Fore, Style
bracket = f"{Fore.BLUE}[{Fore.GREEN}*{Fore.BLUE}]{Style.RESET_ALL} "
bracket_err = f"{Fore.BLUE}[{Fore.RED}*{Fore.BLUE}]{Style.RESET_ALL} "
def drawLine():
print ('-' * 75)
# read arguments
try:
domain = (sys.argv[1])
except:
print("[!] USAGE: python3 zt.py DOMAIN_NAME")
sys.exit(0)
# DNS functions
def resolveDNS(name):
resolver = dns.resolver.Resolver()
results = resolver.query(name , "A")
return results
def getNS (domain):
mapping = {}
name_servers = dns.resolver.query(domain, 'NS')
print ("\nThe name servers for " + domain + " are:")
drawLine()
for name_server in name_servers:
A_records = resolveDNS(str(name_server))
for item in A_records:
answer = ','.join([str(item)])
mapping[str(name_server)] = answer
print (bracket, "{:30}".format(str(name_server).rstrip('.')), "{:15}".format(answer))
return mapping
def zoneXFR(server):
try:
zone = dns.zone.from_xfr(dns.query.xfr(str(server).rstrip('.'), domain))
except Exception as e:
print (bracket_err, f"{Fore.RED}Error:{Style.RESET_ALL}", e.__class__, e)
else:
print ("\nResults for",server, "\nZone origin:", str(zone.origin).rstrip('.'))
drawLine()
for host in zone:
if str(host) != '#':
A_records = resolveDNS(str(host) + "." + domain)
for item in A_records:
answer = ','.join([str(item)])
print(bracket, "{:30}".format(str(host) + "." + domain), answer)
drawLine()
name_servers = getNS(domain)
for server in name_servers:
print ("\nAttempting zone transfers for", server,name_servers[server])
zoneXFR(name_servers[server])
Related
I get this error:
Traceback (most recent call last):
File "C:\Users\website\Desktop\collector.py", line 10, in <module>
start_server(636)
File "C:\Users\website\AppData\Local\Programs\Python\Python310\lib\site-packages\pywebio\platform\tornado.py", line 360, in start_server
handler = webio_handler(applications, cdn, allowed_origins=allowed_origins, check_origin=check_origin,
File "C:\Users\website\AppData\Local\Programs\Python\Python310\lib\site-packages\pywebio\platform\tornado.py", line 236, in webio_handler
applications = make_applications(applications)
File "C:\Users\website\AppData\Local\Programs\Python\Python310\lib\site-packages\pywebio\platform\page.py", line 171, in make_applications
assert iscoroutinefunction(app) or isgeneratorfunction(app) or callable(app), \
AssertionError: Don't support application type:<class 'int'>
my code:
from time import sleep
from pywebio.input import *
from pywebio.output import *
from pywebio import start_server
import pandas
pandas.options.display.max_rows = 6000
pandas.set_option('display.float_format', lambda x: '%.0f' % x)
a = pandas.read_csv("https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/latest/owid-covid-latest.csv",
delimiter=",",
)
out = (a[["location", "new_cases", "total_cases","new_deaths","total_deaths"]].to_string().replace('\n','\n'+' '*12))
out1 = '{:<11} {}'.format('',out)
Titles1 = out1.replace("location", "Location")
Titles2 = Titles1.replace("new_cases", "New Cases")
Titles3 = Titles2.replace("total_deaths", "Total Deaths")
Titles4 = Titles3.replace("total_cases", "Total Cases")
Titles5 = Titles4.replace("new_deaths", "New deaths")
No0 = Titles5.replace(".0", "")
print(No0)
put_code(No0)
start_server(636)
what I do wrong? Please sorry if somebody asking this question but I can't find answer in Google.
Thanks for suggestions because I'm newbie on this forum :)
Thank you in advance!
Your error comes from this line:
start_server(636)
Note that from the documentation, start_server()'s first argument applications must be a list or a dict or a callable
pywebio.platform.tornado.start_server(applications, port=0, host='', debug=False, cdn=True, static_dir=None, remote_access=False, reconnect_timeout=0, allowed_origins=None, check_origin=None, auto_open_webbrowser=False, max_payload_size='200M', **tornado_app_settings)
applications (list/dict/callable)
PyWebIO application. Can be a task function, a list of functions, or a dictionary. Refer to Advanced topic: Multiple applications in start_server() for more information.
When the task function is a coroutine function, use Coroutine-based session implementation, otherwise, use thread-based session implementation.
The error is telling you that you gave an argument of an unsupported type (int) to the application argument.
AssertionError: Don't support application type:<class 'int'>
If you meant for 636 to be the port on which your server runs, it needs to be the second argument, or passed as a keyword argument.
I am pulling images from the Internet Archive as a test of some python code and I am incorporating the requests module. My code is as follows: (note, not the entire code, just the relevant section)
image_results = []
image_hashes = []
session = requests.Session()
for image in image_list:
if txtUrl not in image:
continue
try:
self.rslBox.AppendText("[v] Downloading %s" % image + "\n")
self.rslBox.Refresh()
response = session.get(image)
except:
self.rslBox.AppendText("[!] Failed to download: %s" % image + "\n")
self.rslBox.Refresh()
# continue
if "image" in response.headers['content-type']:
sha1 = hashlib.sha1(response.content).hexadigest()
if sha1 not in image_hashes:
image_hashes.append(sha1)
image_path = "WayBackImages/%s-%s" % (sha1.image.split("/")[-1])
with open(image_path, "wb") as fd:
fd.write(response.content)
self.rslBox.AppendText("[*] Saved %s" % images + "\n")
self.rslBox.Refresh()
info = pyexifinfo.get_json(image_path)
info[0]['ImageHash'] = sha1
image_results.append(info[0])
image_results_json = json.dumps(image_results)
data_frame = pandas.read_json(image_results_json)
csv = data_frame.to_csv('results.csv')
self.rslBox.AppendText("[*] Finished writing CSV to results.csv" + '\n')
self.rslBox.Refresh()
return
When I run my code, I get the following message:
Traceback (most recent call last):
File "C:\eclipse-workspace\test\tabbedPage.py", line 136, in OnSearch
if "image" in response.headers['content-type']:
NameError: name 'response' is not defined
But response is defined in the try statement - or so I would think. It only complains on the if "image" section - why??
I am new to python and I am using python3.6 and pydev with Eclipse.
Thanks!
Something inside your try failed. Your except caught it handle the error but since there is no raise in it, it continues execution, but response is not set.
It's because you are declaring response in the try block. If the exception gets thrown then response is not declared.
A work around for this would be putting the code that relies on response being declared into that try block.
I'm running a django rest framework server hosted on google cloud. Every hour or so I get a couple of these errors that I can't figure out:
Exception ignored in: .cb at 0x7f50f275ebf8>
Traceback (most recent call last): (no traceback provided)
File "", line 191, in cb
"KeyError: ('django.contrib.sessions.serializers',)
There's no traceback since the error is caught and ignored. I've followed the code down to the cpython library in this method:
def _get_module_lock(name):
"""Get or create the module lock for a given module name.
Should only be called with the import lock taken."""
lock = None
try:
lock = _module_locks[name]()
except KeyError:
pass
if lock is None:
if _thread is None:
lock = _DummyModuleLock(name)
else:
lock = _ModuleLock(name)
def cb(_):
del _module_locks[name]
_module_locks[name] = _weakref.ref(lock, cb)
return lock
Has anyone seen this error before? I can't find any pattern of when this error comes in and can't manually reproduce it with any certainty.
I wrote a small python script to do a bulk whois check of some domains using pythonwhois to do the checking.
The script reads domains from testdomains.txt and checks them one by one. Then it logs some information regarding the domain to results.txt
This is my script:
from time import sleep
import pythonwhois
def lookup(domain):
sleep(5)
response = pythonwhois.get_whois(domain)
ns = response['nameservers']
return ns
with open("testdomains.txt") as infile:
domainfile = open('results.txt','w')
for domain in infile:
ns = (lookup(domain))
domainfile.write(domain.rstrip() + ',' + ns+'\n')
domainfile.close()
My problem arises when a domain is not registered or when the whois server fails to reply for some reason. The script exits like this:
Traceback (most recent call last):
File "test8.py", line 17, in <module>
ns = lookup(domain)
File "test8.py", line 9, in lookup
ns = response['nameservers']
TypeError: 'NoneType' object has no attribute '__getitem__'
My question is, what can I do to avoid the entire script from exiting?
In case of an error, I would like the script to jump to the next domain and keep running and not exit.
Logging the error to results.txt would definitely be good too.
Thanks!
You want to make use of exception handling here with a try/except.
Read the documentation on exception handling here
Taking the snippet of code of interest, you wrap your call inside a try:
for domain in infile:
try:
ns = lookup(domain)
except TypeError as e:
# should probably use a logger here instead of print
print('domain not found: {}'.format(e))
print('Continuing...')
domainfile.write(domain.rstrip() + ',' + ns+'\n')
domainfile.close()
with open("testdomains.txt") as infile:
domainfile = open('results.txt','w')
for domain in infile:
try:
ns = (lookup(domain))
domainfile.write(domain.rstrip() + ',' + ns+'\n')\
except TypeError:
pass
domainfile.close()
There are two ways:
1.) Either you can remove the brittle code to make sure expection doesn't occur.
Example:
from time import sleep
import pythonwhois
def lookup(domain):
sleep(5)
response = pythonwhois.get_whois(domain)
ns = response.get('nameservers')
return ns
with open("testdomains.txt") as infile:
domainfile = open('results.txt','w')
for domain in infile:
ns = (lookup(domain))
if ns:
domainfile.write(domain.rstrip() + ',' + ns+'\n')
domainfile.close()
2.) Handle exception gracefully and let code to continue. As suggested above.
I have been trying to control a camera through a wsdl file using SUDS. I have got the code working but I want to place error handling into the script. I have tried different exceptions but am unable to get the script working. When I enter an invalid coordinate I get an error. The code I am using is below followed by the error I am recieving.
#!/home/build/Python-2.6.4/python
import suds
from suds.client import Client
####################################################################
#
# Python SUDS Script that controls movement of Camera
#
####################################################################
#
# Absolute Move Function
#
####################################################################
def absoluteMove():
# connects to WSDL file and stores location in variable 'client'
client = Client('http://file.wsdl')
# Create 'token' object to pass as an argument using the 'factory' namespace
token = client.factory.create('ns4:ReferenceToken')
print token
# Create 'dest' object to pass as an argument and values passed to this object
dest = client.factory.create('ns4:PTZVector')
dest.PanTilt._x=400
dest.PanTilt._y=0
dest.Zoom._x=1
print dest
# Create 'speed' object to pass as an argument and values passed to this object
speed = client.factory.create('ns4:PTZSpeed')
speed.PanTilt._x=0
speed.PanTilt._y=0
speed.Zoom._x=1
print speed
# 'AbsoluteMove' method invoked passing in the new values entered in the above objects
try:
result = client.service.AbsoluteMove(token, dest, speed)
except RuntimeError as detail:
print 'Handling run-time error:', detail
print "absoluteMove result ", result
result = absoluteMove()
The error is below:
No handlers could be found for logger "suds.client"
Traceback (most recent call last):
File "ptztest.py", line 48, in <module>
if __name__ == '__main__': result = absoluteMove()
File "ptztest.py", line 42, in absoluteMove
result = client.service.AbsoluteMove(token, dest, speed)
File "build/bdist.linux-i686/egg/suds/client.py", line 537, in __call__
File "build/bdist.linux-i686/egg/suds/client.py", line 597, in invoke
File "build/bdist.linux-i686/egg/suds/client.py", line 632, in send
File "build/bdist.linux-i686/egg/suds/client.py", line 683, in failed
File "build/bdist.linux-i686/egg/suds/bindings/binding.py", line 235, in get_fault
suds.WebFault: Server raised fault: 'Error setting requested pan'
I am not sure which exception I should be using here. Does anyone know how to catch this error. The x coordinate with the value 400 is in degree's that is why the error happens.
Thanks
Okay I have found the solution. In SUDS if you enter:
faults=False
into the client definition, this catches faults and gives the reason why the fault happened. The line should read:
client = Client('http://file.wsdl', faults=False)
The post that I have marked as the correct answer also is able to catch that a problem has happened.
Thanks all
If you handled all exceptions and errors in your code and your code is working fine but still you are getting below message with your correct output.
Msg : "No handlers could be found for logger suds.client "
Then a simple solution is to add this line
logging.getLogger('suds.client').setLevel(logging.CRITICAL)
in yourclient.py file just after all import statement.
If you want to catch that exception you should put
try:
result = client.service.AbsoluteMove(token, dest, speed)
except suds.WebFault as detail:
...
You need to catch suds.WebFault by the looks of that traceback. The error itself seems legitimate, IE, your requests are being executed correctly, but perhaps your parameters are wrong in the given context.
I believe you refer to a harmless diagnostic message in your comment. I could suppress messages from suds calling logging.error() by assigning logging.INFO to basicConfig and logging.CRITICAL to suds.client.
https://fedorahosted.org/suds/wiki/Documentation