I wrote a try except block that I now realize was a bad idea because it keep throwing 'blind' exceptions that are hard to debug. The problem is that I do not know how to go about writing it another way besides going through each of the methods that are called and manually reading all the exceptions and making a case for each.
How would you structure this code?
def get_wiktionary_audio(self):
'''function for adding audio path to a definition, this is meant to be run before trying to get a specific URL'''
#this path is where the audio will be saved, only added the kwarg for testing with a different path
path="study_audio/%s/words" % (self.word.language.name)
try:
wiktionary_url = "http://%s.wiktionary.org/wiki/FILE:en-us-%s.ogg" % (self.word.language.wiktionary_prefix, self.word.name)
wiktionary_page = urllib2.urlopen(wiktionary_url)
wiktionary_page = fromstring(wiktionary_page.read())
file_URL = wiktionary_page.xpath("//*[contains(concat(' ', #class, ' '), ' fullMedia ')]/a/#href")[0]
file_number = len(self.search_existing_audio())
relative_path = '%s/%s%s.ogg' % (path, self.word.name, file_number)
full_path = '%s/%s' % (settings.MEDIA_ROOT, relative_path)
os.popen("wget -q -O %s 'http:%s'" % (full_path, file_URL))
except:
return False
WordAudio.objects.create(word=self.word, audio=relative_path, source=wiktionary_url)
return True
Often, exceptions come with error strings which can be used to pinpoint the problem. You can access this value like so:
try:
# code block
except Exception as e:
print str(e)
You can also print what class of exception it is along with any error messages by using the repr method:
try:
# code block
except Exception as e:
print repr(e)
One way I like to go about it is configure Python logging and log the output. This gives you a lot of flexibility in what you do with the log output. The below example logs the exception traceback.
import traceback
import logging
logger = logging.getLogger(__name__)
try:
...
except Exception as e:
logger.exception(traceback.format_exc()) # the traceback
logger.exception(e) # just the exception message
First your code is un-pythonic. You are using 'self' for a function. "self" is usually reserved for a class. So in reading your code, it feels unnatural. Second, my style is to line up "=" signs for readability. My advice is to start over -- Use standard pythonic conventions. You can get this by going through python tutorials.
Throw exception early and often -ONLY when the code stops running. You could also move some of the naming outside the try/except block.
def get_wiktionary_audio(self):
'''function for adding audio path to a definition, this is meant to be run before trying to get a specific URL'''
#this path is where the audio will be saved, only added the kwarg for testing with a different path
path = "study_audio/%s/words" % (self.word.language.name)
try:
wiktionary_url = "http://%s.wiktionary.org/wiki/FILE:en-us-%s.ogg" % (self.word.language.wiktionary_prefix, self.word.name)
wiktionary_page = urllib2.urlopen(wiktionary_url)
wiktionary_page = fromstring(wiktionary_page.read())
file_URL = wiktionary_page.xpath("//*[contains(concat(' ', #class, ' '), ' fullMedia ')]/a/#href")[0]
file_number = len(self.search_existing_audio())
relative_path = '%s/%s%s.ogg' % (path, self.word.name, file_number)
full_path = '%s/%s' % (settings.MEDIA_ROOT, relative_path)
os.popen("wget -q -O %s 'http:%s'" % (full_path, file_URL))
except Exception as e : print e
WordAudio.objects.create(word=self.word, audio=relative_path, source=wiktionary_url)
return True
Related
I am somewhat new to programming. I did not go to school to learn about it, but do read a lot online to learn more about coding.
In general, if I know a concept then I can usually figure it out with google.
Obvious from the post, I like to play games and so I thought one day while configuring some settings for a mod, I realized that the information was extremely difficult to dig up. I thought that I could try to solve it by writing a python script (no real success writing scripts before this point).
The following is the general workflow that I wrote up before tackling it in python:
Replace base path up to 'Mods' with '/Game/Mods'
Find all paths to files in Mods folder
Rename all mod files ending in uasset. E.g. filname.uasset -> filename.filename
Parse Mod's name from binary file in Mod subdirectory, replace integer name with text name for all related paths. E.g. mod name 899987403 -> Primal_Fear_Bosses
Example before (ubuntu path):
/mnt/c/Program Files (x86)/Steam/steamapps/common/ARK/ShooterGame/Content/Mods/899987403/Vanilla/Projectiles/Manticore/Proj/PFBProjManticoreQuill.uasset
& after with script: /Game/Mods/Primal_Fear_Bosses/Vanilla/Projectiles/Manticore/Proj/PFBProjManticoreQuill.PFBProjManticoreQuill,
*Note: align the "/Mods/" section for both lines to see the key difference.
Anyways, I was familiar with a lot of concepts at that point, so I did a lot of googling and wrote the following script (first ever), "Generate_Resource_Pulling_List_for_S+.py":
import os
import sys
from fnmatch import fnmatch
import re
# define the directory that mods are installed in
# ubuntu path
# modDir = "C:/Program Files (x86)/Steam/steamapps/common/ARK/ShooterGame/Content/Mods"
# windows path
modDir = "C:\Program Files (x86)\Steam\steamapps\common\ARK\ShooterGame\Content\Mods\\"
stringPattern = r"\w+(.mod)"
regexPattern = re.compile(stringPattern)
modFolders = [mod for mod in os.listdir(modDir) if not regexPattern.match(mod)]
# loop through mod list and append directory to mod installation path
for items in modFolders:
modPath = "%s%s" % (modDir, items)
# parse mod name from meta file, store for later
modMetaFilePath = "%s\%s" % (modPath, "modmeta.info")
validPath = os.path.exists(modMetaFilePath)
if not validPath:
print('x is:', validPath)
pass
try:
modFile = open(modMetaFilePath, "rb")
binaryContent = modFile.read(-1)
modFile.close()
# if len == 0:
# print('length:', len(binaryContent))
# break
# print(type(binaryContent))
text = binaryContent.decode('UTF-8')
try:
parsedModName = text.split('Mods/')[1].split('/')[0]
if not parsedModName:
break
except ValueError as e:
pass
except Exception as e:
pass
for path, subdirs, files in os.walk(modPath):
# for i in range(len(subdirs)):
# print('Number of Sub Directories: ', len(subdirs))
# print('Current Directory Number in ', path, ': ', subdirs[i])
for name in files:
pattern = "*.uasset"
if fnmatch(name, pattern):
try:
path = os.path.normpath(path) + '\\'
if not path:
continue
try:
basePath = os.path.join('\\Game\\Mods\\', parsedModName)
except Exception as e:
pass
print('failed here:', str(e))
strippedBasePath = os.path.dirname(path).split(items)[1]
if not strippedBasePath:
print('failed at this point stripped path', strippedBasePath, '\n\t', 'path', path)
continue
revisedFileName = os.path.join(
os.path.basename(name).split('.')[0] + '.' + os.path.basename(name).split('.')[0] + ',')
finalPath = "%s%s\%s" % (basePath, strippedBasePath, revisedFileName)
flippedSlashFinalPath = finalPath.replace("\\", "/")
print(flippedSlashFinalPath)
with open("out.txt", "a") as external_file:
add_text = flippedSlashFinalPath
external_file.write(add_text)
external_file.close()
except Exception as e:
print('Something happened', str(e))
I initially installed an Ubuntu environment on Windows because I am unfamiliar with command line/bash/scripting in Windows as seen in the mod path (slashes are reversed and commands are different). I am much more familiar with MacOS and generally work in Linux environments and terminals.
I thought that this script would be usable by others with a bit of programming knowledge, but it is not the most user friendly.
In any case, this was my first attempt at writing something and trying to use good practices.
As another side project to practice more coding, I rewrote it and implemented more complex concepts.
Here is the refactored version:
import os
import sys
from fnmatch import fnmatch
import re
import winreg
stringPattern = r"\w+(.mod)"
regexPattern = re.compile(stringPattern)
class Mod(object):
def __init__(self):
pass
self.steam_path = None
self.mod_folders = []
self.mod_path = None
self.mod_path_list = []
self.mod_meta_path_list = []
self.full_mod_path = None
self.mod_meta = None
self.resource_path = None
#property
def GetSteamPath(self):
try:
steam_key = "SOFTWARE\WOW6432Node\Valve\Steam"
hkey = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, steam_key)
except ValueError as e:
hkey = None
print(e.sys.exc_info())
try:
steam_path = winreg.QueryValueEx(hkey, "InstallPath")
except ValueError as e:
steam_path = None
print(e.sys.exc_info())
winreg.CloseKey(hkey)
self.steam_path = steam_path[0]
return self.steam_path
def GetArkModPath(self):
mod_dir = self.GetSteamPath[:] + "\steamapps\common\ARK\ShooterGame\Content\Mods\\"
self.mod_folders = [mod for mod in os.listdir(mod_dir) if not regexPattern.match(mod)]
mod_path_list = []
count = 1
while count < len(self.mod_folders):
for mod in self.mod_folders:
self.full_mod_path = ("%s%s\\" % (mod_dir, mod))
mod_path_list.append(self.full_mod_path)
self.mod_path_list = mod_path_list
if not mod:
break
# else:
# print('Test1', format(mod_path_list))
return self.mod_path_list
count += 1
def GetModResourceList(self, mod_path_list):
mod_folder_index = 0
for mod_path in mod_path_list:
mod_meta_path = "%s%s" % (mod_path, "modmeta.info")
self.mod_meta_path_list.append(mod_meta_path)
validPath = os.path.exists(mod_meta_path)
if not validPath:
# print('No Mod Meta File found at: ', mod_meta_path)
continue
try:
mod_file = open(mod_meta_path, "rb")
binary_content = mod_file.read(-1)
mod_file.close()
text = binary_content.decode('UTF-8')
try:
parsed_mod_name = text.split('Mods/')[1].split('/')[0]
if not parsed_mod_name:
break
except ValueError as e:
pass
except Exception as e:
pass
for path, subdirs, files in os.walk(mod_path):
# for i in range(len(subdirs)):
# print('Number of Sub Directories: ', len(subdirs))
# print('Current Directory Number in ', path, ': ', subdirs[i])
for uasset_file in files:
pattern = "*.uasset"
if fnmatch(uasset_file, pattern):
try:
path = os.path.normpath(path) + '\\'
if not path:
continue
try:
base_path = os.path.join('\\Game\\Mods\\', parsed_mod_name)
except Exception as e:
pass
print('failed here:', str(e))
stripped_base_path = os.path.dirname(path).split(self.mod_folders[mod_folder_index])[1]
resource_name = os.path.join(
os.path.basename(uasset_file).split('.')[0] + '.' + os.path.basename(uasset_file).split('.')[0] + ',')
full_path = "%s%s\%s" % (base_path, stripped_base_path, resource_name)
resource_path = full_path.replace("\\", "/")
self.resource_path = resource_path
# to see what text is written to the file, uncomment print here
print(self.resource_path)
with open("test_out.txt", "a") as external_file:
add_text = self.resource_path
external_file.write(add_text)
external_file.close()
except Exception as e:
print('Error: ', str(e))
# return self.resource_path[]
mod_folder_index += 1
ark = Mod()
ark_mods = ark.GetArkModPath()
ark.GetModResourceList(ark_mods)
This revised one is much more user friendly and does not necessarily require input or modification of variables. I tried to avoid requiring arguments passed in a method call because I wanted to automate the script as necessary without any input from user.
If you have Ark installed with some mods, then you may be able to actually use or test the scripts. I'm curious if they work for others. FYI, not all paths are usable. It might be best to delete/remove some or just pick out the ones you want. Can't really figure out the best way to identify this.
I struggled with creating an instance of my class, what attributes to set, what properties to set, setting(?) values for attributes for my class, calling those stored values, and just about everything else you can imagine.
Some things are obvious to me, such as it is pointless to create classes and methods in a script that is not necessarily being re-used. It was still fun and the point was to learn and practice.
In any case, I am open to any feedback because I do not know what is common sense for everything or best practices that others have seen. As a side note, I do versioning with local repos. I have worked with git and bitbucket before.
Preferred topics if not sure:
defensive programming
proper exception handling
SOLID (I'm def lacking here)
Classes (I barely got mine working if its not apparrent)
Methods (seems straight forward to me - if code is going to be copied, put it in a method)
Preferred nomenclature for everything, classes, methods, variables (I googled it, but still don't like how it looks especially when they get long)
Readability (I understand that if code requires comments, its not good)
Cleanliness (I guess this is an art, but I see spaghetti when I look at what I wrote)
Links to resources are welcome! I really appreciate any feedback and look forward to all comments.
I am writing a python script function to backup Wordpress. As part of the script i wrote a function to fetch database details from the config.php file.
Working of my function
function takes Wordpress installation location as an argument and using regex to match db_user,db_host,db_user,db_password from that file, the function will exist if can not find "config.php". I am using sys.exit(1) to exit from the function is that the proper way to exit from a function? I am pasting my function code snippet.
def parsing_db_info(location):
config_path = os.path.normpath(location+'/config.php')
if os.path.exists(config_path):
try:
regex_db = r'define\(\s*?\'DB_NAME\'\s*?,\s*?\'(.*?)\'\s*?'.group(1)
regex_user = r'define\(\s*?\'DB_USER\'\s*?,\s*?\'(.*?)\'\s*?'.group(1)
regex_pass = r'define\(\s*?\'DB_PASSWORD\'\s*?,\s*?\'(.*?)\'\s*?'.group(1)
regex_host = r'define\(\s*?\'DB_HOST\'\s*?,\s*?\'(.*?)\'\s*?'.group(1)
db_name = re.match(regex_db,config_path).group(1)
db_user = re.match(regex_user,config_path).group(1)
db_pass = re.match(regex_pass,config_path).group(1)
db_host = re.match(regex_host,config_path).group(1)
return {'dbname':db_name , 'dbuser':db_user , 'dbpass':db_pass , 'dbhost':db_host}
except exception as ERROR:
print(ERROR)
sys.exit(1)
else:
print('Not Found:',config_path)
sys.exit(1)
AFTER EDITING
def parsing_db_info(location):
config_path = os.path.normpath(location+'/wp-config.php')
try:
with open(config_path) as fh:
content = fh.read()
regex_db = r'define\(\s*?\'DB_NAME\'\s*?,\s*?\'(.*?)\'\s*?'
regex_user = r'define\(\s*?\'DB_USER\'\s*?,\s*?\'(.*?)\'\s*?'
regex_pass = r'define\(\s*?\'DB_PASSWORD\'\s*?,\s*?\'(.*?)\'\s*?'
regex_host = r'define\(\s*?\'DB_HOST\'\s*?,\s*?\'(.*?)\'\s*?'
db_name = re.search(regex_db,content).group(1)
db_user = re.search(regex_user,content).group(1)
db_pass = re.search(regex_pass,content).group(1)
db_host = re.search(regex_host,content).group(1)
return {'dbname':db_name , 'dbuser':db_user , 'dbpass':db_pass , 'dbhost':db_host}
except FileNotFoundError:
print('File Not Found,',config_path)
sys.exit(1)
except PermissionError:
print('Unable To read Permission Denied,',config_path)
sys.exit(1)
except AttributeError:
print('Parsing Error wp-config seems to be corrupt,')
sys.exit(1)
To answer your question, you shouldn't normally use sys.exit inside a function like that. Rather, get it to raise an exception in the case where it fails. Preferably, it should be an exception detailing what went wrong, or you could just let the existing exceptions propagate.
The normal rule in Python is this: deal with exceptions at the place you know how to deal with them.
In your code, you catch an exception, and then don't know what to do, so call sys.exit. Instead of this, you should:
let an exception propagate up to a top-level function which can catch it, and then call sys.exit if appropriate
wrap the exception in something more specific, and re-raise, so that a higher level function will have a specific exception to catch. For example, your function might raise a custom ConfigFileNotFound exception or ConfigFileUnparseable exception.
Also, you have put except exception, you probably mean except Exception. However, this is extremely broad, and will mask other programming errors. Instead, catch the specific exception class you expect.
I want to generate a one-line string from an Exception which tells me what happened where (don't need a full backtrace). The following information would be nice:
filename / linenumber
exception type
exception description (what you get from str(e))
nice to have: function/method/class
Currently I do the following:
import os
...
try:
os.nonexisting()
except Exception as e:
t = e.__traceback__
tbf = e.__traceback__.tb_frame
print('%s:%d: %s in %s(): "%s" ' %
os.path.basename(tbf.f_code.co_filename),
t.tb_lineno,
e.__class__.__name__,
tbf.f_code.co_name, e))
which gives me:
foo.py:203: AttributeError in foo(): "'module' object has no attribute 'nonexisting'"
Is there a more elegant way to print out the details given in this example? I'm thinking about s.th. like
print(e.format('%f: %l: %t %F: "%w"'))
I'd like to avoid importing extra modules except there is one exactly for this purpose.
I think traceback.format_exception_only does exactly what you want.
try:
os.nonexisting()
except Exception as e:
print(traceback.format_exception_only(e.__class__, e))
I have function, which looks for special Element if project files:
def csproj_tag_finder(mod_proj_file):
"""Looking for 'BuildType' element in each module's csproj file passed in `mod_proj_file`
ard return it's value (CLOUD, MAIN, NGUI, NONE)"""
try:
tree = ET.ElementTree(file=mod_proj_file)
root = tree.getroot()
for element in root.iterfind('.//'):
if ('BuildType') in element.tag:
return element.text
except IOError as e:
# print 'WARNING: cant find file: %s' % e
If no file found - it prints 'WARNING: cant find file: %s' % e.
This function called from another one:
def parser(modename, mod_proj_file):
...
# module's tag's from project file in <BuildType> elements, looks like CLOUD
mod_tag_from_csproj = csproj_tag_finder(mod_proj_file)
if not mod_tag_from_csproj:
print('WARNING: module %s have not <BuildType> elements in file %s!' % (modename, mod_proj_file))
...
So - when file doesn't found - csproj_tag_finder()return None type, and print WARNING. Second function - parser() find empty mod_tag_from_csproj variable, and also print WARNING. This is harmless, so I want make csproj_tag_finder() raise special Exception, so parser() except it and pass == check, instead of print text.
I tried add something like:
...
except IOError as e:
# print 'WARNING: cant find file: %s' % e
raise Exception('NoFile')
to csproj_tag_finder() to catch it later in parser() - but it's interrupt next steps immediately.
P.S. Later if not mod_tag_from_csproj: will call another function to add new Element. This task can be solved with just return 'NoFile' and then catch with if/else - but it seems to me that raise will more correct way here. Or not?
raise interrupting the next steps immediately is exactly what it's supposed to do. In fact, that's the whole point of exceptions.
But then return also interrupts the next steps immediately, because returning early is also the whole point of return.
If you want to save an error until later, continue doing some other work, and then raise it at the end, you have to do that explicitly. For example:
def spam():
error = None
try:
do_some_stuff()
except IOError as e:
print 'WARNING: cant find file %s' % e
error = Exception('NoFile')
try:
do_some_more_stuff()
except OtherError as e:
print 'WARNING: cant frob the glotz %s' % e
error = Exception('NoGlotz')
# etc.
if error:
raise error
Now, as long as there's no unexpected exception that you forgot to handle, whatever failed last will be in error, and it'll be raised at the end.
As a side note, instead of raising Exception('NoFile'), then using == to test the exception string later, you probably want to create a NoFileException subclass; then you don't need to test it, you can just handle it with except NoFileException:. And that means you can carry some other useful information (the actual exception, the filename, etc.) in your exception without it getting in the way, too. If this sounds scary to implement, it's not. It's literally a one-liner:
class NoFileException(Exception): pass
if __name__=="__main__":
fname= raw_input("Please enter your file:")
mTrue=1
Salaries=''
Salarieslist={}
Employeesdept=''
Employeesdeptlist={}
try:
f1=open(fname)
except:
mTrue=0
print 'The %s does not exist!'%fname
if mTrue==1:
ss=[]
for x in f1.readlines():
if 'Salaries' in x:
Salaries=x.strip()
elif 'Employees' in x:
Employeesdept=x.strip()
f1.close()
if Salaries and Employeesdept:
Salaries=Salaries.split('-')[1].strip().split(' ')
for d in Salaries:
s=d.strip().split(':')
Salarieslist[s[0]]=s[1]
Employeesdept=Employeesdept.split('-')[1].strip().split(' ')
for d in Employeesdept:
s=d.strip().split(':')
Employeesdeptlist[s[0]]=s[1]
print "1) what is the average salary in the company: %s "%Salarieslist['Avg']
print "2) what are the maximum and minimum salaries in the company: maximum:%s,minimum:%s "%(Salarieslist['Max'],Salarieslist['Min'])
print "3) How many employees are there in each department :IT:%s, Development:%s, Administration:%s"%(
Employeesdeptlist['IT'],Employeesdeptlist['Development'],Employeesdeptlist['Administration'])
else:
print 'The %s data is err!'%fname
When I enter a filename, but it didn't continue, why? If I enter a file named company.txt, but it always show the file does not exist. why?
I can give you some hints which can help you to resolve problem better
Create a function and call it in main e.g.
if __name__=="__main__":
main()
Don't put whole block under if mTrue==1: instead just return from function on error e.g.
def main():
fname= raw_input("Please enter your file:")
try:
f1=open(fname)
except:
print 'The %s does not exist!'%fname
return
... # main code here
Never catch all exceptions , instead catch specific exception e.g. IOError
try:
f1 = open(fname):
except IOError,e:
print 'The %s does not exist!'%fname
otherwise catching all exception may catch syntax error or mis-spelled names etc
Print the exception you are getting, it may not always be file not found, may be you don't have read permission or something like that
and finally your problem could be just that, file may not exist, try to input full path
Your current working directory does not contain company.txt.
Either set your current working directory or use an absolute path.
You can change the working directory like so:
import os
os.chdir(new_path)
In addition to be more specific about which exceptions you want to catch you should considered capturing the exception object itself so you can print a string representation of it as part of your error message:
try:
f1 = open(fname, 'r')
except IOError, e:
print >> sys.stderr, "Some error occurred while trying to open %s" % fname
print >> sys.stderr, e
(You can also learn more about specific types of Exception objects and perhaps handle
some sorts of exceptions in your code. You can even capture Exceptions for your own inspection from within the interpreter so you can run dir() on them, and type() on each of the interesting attributes you find ... and so on.