UPDATE: I think I may have just realized what I need to figure out re: the below, which is the correct error type to specify with the except clause (new to this, obviously)
Starting in a specified root directory, my program iterates through subdirectories, then files within those, identifying valid (.csv) data files and then aggregating and performing calculations on the data.
In cases where the root directory happens to be empty, can someone suggest a clean/graceful way to simply exit the program at the start without further processing?
I tried to adapt a suggestion I found here, but it didn't work as I expected:
Exit gracefully if file doesn't exist
That person's code was:
def main():
try:
file = open('file.txt', 'r')
except IOError:
print('There was an error opening the file!')
return
I gave the above a try, and it works for the particular case above. However, when I tried to adapt it as follows, it 'broke' and I got an "Index out of range error", instead dropping down to the except code.
dir = os.listdir(masterDirPath)
def main():
try:
item = dir[0]
except IOError:
print('The data area is empty.')
return
(Also/instead, I very much welcome suggestions for some completely other approach to the task overall)
To exit your program immediately you should use either exit() or quit(). Instead of throwing an error you could use the fact that many objects in Python are truthy; an empty list is False, and a list with one or more elements is True.
import os
dir_contents = os.listdir('.')
if dir_contents:
do_stuff
else:
print('Directory was empty. Exiting')
exit()
If you prefer explicitness to implicitness you could also check the length of your list using len(dir_contents) before indexing into it.
You might also want to avoid using dir in Python as a variable name as it will shadow the builtin function dir().
An empty list has no elements so you should catch IndexError instead of IOError.
def main():
try:
item = dir[0]
except IndexError:
print('The data area is empty.')
return
Why are you using an exception?
if not dir:
print('The data area is empty')
exit(0)
item = dir[0]
#!/usr/bin/env python
import os
path = 'path-to-your-dir'
if os.listdir(path) == []:
exit(0)
else:
print "Your other code goes here."
These are the exit commands and its definitions
exit(0) means a clean exit without any errors / problems
exit(1) means there was some issue / error / problem and that is why the program is exiting.
os._exit() for child processes
quit() the SystemExit exception behind the scenes.
sys.exit() kill the interpreter
Related
How do I exit a script early, like the die() command in PHP?
import sys
sys.exit()
details from the sys module documentation:
sys.exit([arg])
Exit from Python. This is implemented by raising the
SystemExit exception, so cleanup actions specified by finally clauses
of try statements are honored, and it is possible to intercept the
exit attempt at an outer level.
The optional argument arg can be an integer giving the exit status
(defaulting to zero), or another type of object. If it is an integer,
zero is considered “successful termination” and any nonzero value is
considered “abnormal termination” by shells and the like. Most systems
require it to be in the range 0-127, and produce undefined results
otherwise. Some systems have a convention for assigning specific
meanings to specific exit codes, but these are generally
underdeveloped; Unix programs generally use 2 for command line syntax
errors and 1 for all other kind of errors. If another type of object
is passed, None is equivalent to passing zero, and any other object is
printed to stderr and results in an exit code of 1. In particular,
sys.exit("some error message") is a quick way to exit a program when
an error occurs.
Since exit() ultimately “only” raises an exception, it will only exit
the process when called from the main thread, and the exception is not
intercepted.
Note that this is the 'nice' way to exit. #glyphtwistedmatrix below points out that if you want a 'hard exit', you can use os._exit(*errorcode*), though it's likely os-specific to some extent (it might not take an errorcode under windows, for example), and it definitely is less friendly since it doesn't let the interpreter do any cleanup before the process dies. On the other hand, it does kill the entire process, including all running threads, while sys.exit() (as it says in the docs) only exits if called from the main thread, with no other threads running.
A simple way to terminate a Python script early is to use the built-in quit() function. There is no need to import any library, and it is efficient and simple.
Example:
#do stuff
if this == that:
quit()
Another way is:
raise SystemExit
You can also use simply exit().
Keep in mind that sys.exit(), exit(), quit(), and os._exit(0) kill the Python interpreter. Therefore, if it appears in a script called from another script by execfile(), it stops execution of both scripts.
See "Stop execution of a script called with execfile" to avoid this.
While you should generally prefer sys.exit because it is more "friendly" to other code, all it actually does is raise an exception.
If you are sure that you need to exit a process immediately, and you might be inside of some exception handler which would catch SystemExit, there is another function - os._exit - which terminates immediately at the C level and does not perform any of the normal tear-down of the interpreter; for example, hooks registered with the "atexit" module are not executed.
I've just found out that when writing a multithreadded app, raise SystemExit and sys.exit() both kills only the running thread. On the other hand, os._exit() exits the whole process. This was discussed in "Why does sys.exit() not exit when called inside a thread in Python?".
The example below has 2 threads. Kenny and Cartman. Cartman is supposed to live forever, but Kenny is called recursively and should die after 3 seconds. (recursive calling is not the best way, but I had other reasons)
If we also want Cartman to die when Kenny dies, Kenny should go away with os._exit, otherwise, only Kenny will die and Cartman will live forever.
import threading
import time
import sys
import os
def kenny(num=0):
if num > 3:
# print("Kenny dies now...")
# raise SystemExit #Kenny will die, but Cartman will live forever
# sys.exit(1) #Same as above
print("Kenny dies and also kills Cartman!")
os._exit(1)
while True:
print("Kenny lives: {0}".format(num))
time.sleep(1)
num += 1
kenny(num)
def cartman():
i = 0
while True:
print("Cartman lives: {0}".format(i))
i += 1
time.sleep(1)
if __name__ == '__main__':
daemon_kenny = threading.Thread(name='kenny', target=kenny)
daemon_cartman = threading.Thread(name='cartman', target=cartman)
daemon_kenny.setDaemon(True)
daemon_cartman.setDaemon(True)
daemon_kenny.start()
daemon_cartman.start()
daemon_kenny.join()
daemon_cartman.join()
from sys import exit
exit()
As a parameter you can pass an exit code, which will be returned to OS. Default is 0.
I'm a total novice but surely this is cleaner and more controlled
def main():
try:
Answer = 1/0
print Answer
except:
print 'Program terminated'
return
print 'You wont see this'
if __name__ == '__main__':
main()
...
Program terminated
than
import sys
def main():
try:
Answer = 1/0
print Answer
except:
print 'Program terminated'
sys.exit()
print 'You wont see this'
if __name__ == '__main__':
main()
...
Program terminated Traceback (most recent call last): File "Z:\Directory\testdieprogram.py", line 12, in
main() File "Z:\Directory\testdieprogram.py", line 8, in main
sys.exit() SystemExit
Edit
The point being that the program ends smoothly and peacefully, rather than "I'VE STOPPED !!!!"
Problem
In my practice, there was even a case when it was necessary to kill an entire multiprocessor application from one of those processes.
The following functions work well if your application uses the only main process. But no one of the following functions didn't work in my case as the application had many other alive processes.
quit()
exit(0)
os._exit(0)
sys.exit(0)
os.kill(os.getppid(), 9) - where os.getppid() is the pid of parent process
The last one killed the main process and itself but the rest processes were still alive.
Solution
I had to kill it by external command and finally found the solution using pkill.
import os
# This can be called even in process worker and will kill
# whole application included correlated processes as well
os.system(f"pkill -f {os.path.basename(__file__)}")
In Python 3.5, I tried to incorporate similar code without use of modules (e.g. sys, Biopy) other than what's built-in to stop the script and print an error message to my users. Here's my example:
## My example:
if "ATG" in my_DNA:
## <Do something & proceed...>
else:
print("Start codon is missing! Check your DNA sequence!")
exit() ## as most folks said above
Later on, I found it is more succinct to just throw an error:
## My example revised:
if "ATG" in my_DNA:
## <Do something & proceed...>
else:
raise ValueError("Start codon is missing! Check your DNA sequence!")
My two cents.
Python 3.8.1, Windows 10, 64-bit.
sys.exit() does not work directly for me.
I have several nexted loops.
First I declare a boolean variable, which I call immediateExit.
So, in the beginning of the program code I write:
immediateExit = False
Then, starting from the most inner (nested) loop exception, I write:
immediateExit = True
sys.exit('CSV file corrupted 0.')
Then I go into the immediate continuation of the outer loop, and before anything else being executed by the code, I write:
if immediateExit:
sys.exit('CSV file corrupted 1.')
Depending on the complexity, sometimes the above statement needs to be repeated also in except sections, etc.
if immediateExit:
sys.exit('CSV file corrupted 1.5.')
The custom message is for my personal debugging, as well, as the numbers are for the same purpose - to see where the script really exits.
'CSV file corrupted 1.5.'
In my particular case I am processing a CSV file, which I do not want the software to touch, if the software detects it is corrupted. Therefore for me it is very important to exit the whole Python script immediately after detecting the possible corruption.
And following the gradual sys.exit-ing from all the loops I manage to do it.
Full code: (some changes were needed because it is proprietory code for internal tasks):
immediateExit = False
start_date = '1994.01.01'
end_date = '1994.01.04'
resumedDate = end_date
end_date_in_working_days = False
while not end_date_in_working_days:
try:
end_day_position = working_days.index(end_date)
end_date_in_working_days = True
except ValueError: # try statement from end_date in workdays check
print(current_date_and_time())
end_date = input('>> {} is not in the list of working days. Change the date (YYYY.MM.DD): '.format(end_date))
print('New end date: ', end_date, '\n')
continue
csv_filename = 'test.csv'
csv_headers = 'date,rate,brand\n' # not real headers, this is just for example
try:
with open(csv_filename, 'r') as file:
print('***\nOld file {} found. Resuming the file by re-processing the last date lines.\nThey shall be deleted and re-processed.\n***\n'.format(csv_filename))
last_line = file.readlines()[-1]
start_date = last_line.split(',')[0] # assigning the start date to be the last like date.
resumedDate = start_date
if last_line == csv_headers:
pass
elif start_date not in working_days:
print('***\n\n{} file might be corrupted. Erase or edit the file to continue.\n***'.format(csv_filename))
immediateExit = True
sys.exit('CSV file corrupted 0.')
else:
start_date = last_line.split(',')[0] # assigning the start date to be the last like date.
print('\nLast date:', start_date)
file.seek(0) # setting the cursor at the beginnning of the file
lines = file.readlines() # reading the file contents into a list
count = 0 # nr. of lines with last date
for line in lines: #cycling through the lines of the file
if line.split(',')[0] == start_date: # cycle for counting the lines with last date in it.
count = count + 1
if immediateExit:
sys.exit('CSV file corrupted 1.')
for iter in range(count): # removing the lines with last date
lines.pop()
print('\n{} lines removed from date: {} in {} file'.format(count, start_date, csv_filename))
if immediateExit:
sys.exit('CSV file corrupted 1.2.')
with open(csv_filename, 'w') as file:
print('\nFile', csv_filename, 'open for writing')
file.writelines(lines)
print('\nRemoving', count, 'lines from', csv_filename)
fileExists = True
except:
if immediateExit:
sys.exit('CSV file corrupted 1.5.')
with open(csv_filename, 'w') as file:
file.write(csv_headers)
fileExists = False
if immediateExit:
sys.exit('CSV file corrupted 2.')
In Python 3.9, you can also use: raise SystemExit("Because I said so").
Just put at the end of your code quit() and that should close a python script.
use exit and quit in .py files
and sys.exit for exe files
I'm in trouble about how to end a 'try' loop, which is occurred since I have the 'try', here is the code:
import time
class exe_loc:
mem = ''
lib = ''
main = ''
def wizard():
while True:
try:
temp_me = input('Please specify the full directory of the memory, usually it will be a folder called "mem"> ' )
if temp_me is True:
exe_loc.mem = temp_me
time.sleep(1)
else:
print('Error value! Please run this configurator again!')
sys.exit()
temp_lib = input('Please specify the full directory of the library, usually it will be a folder called "lib"> ')
if temp_lib is True:
exe_loc.lib = temp_lib
time.sleep(1)
else:
print('Invalid value! Please run this configurator again!')
sys.exit()
temp_main = input('Please specify the full main executable directory, usually it will be app main directory> ')
if temp_main is True:
exe_loc.main = temp_main
time.sleep(1)
I tried end it by using break, pass, and I even leaves it empty what I get is Unexpected EOF while parsing, I searched online and they said it is caused when the code blocks were not completed. Please show me if any of my code is wrong, thanks.
Btw, I'm using python 3 and I don't know how to be more specific for this question, kindly ask me if you did not understand. Sorry for my poor english.
EDIT: Solved by removing the try because I'm not using it, but I still wanna know how to end a try loop properly, thanks.
Your problem isn't the break, it's the overall, high-level shape of your try clause.
A try requires either an except or a finally block. You have neither, which means your try clause is never actually complete. So python keeps looking for the next bit until it reaches EOF (End Of File), at which point it complains.
The python docs explain in more detail, but basically you need either:
try:
do_stuff_here()
finally:
do_cleanup_here() # always runs, even if the above raises an exception
or
try:
do_stuff_here()
except SomeException:
handle_exception_here() # if do_stuff_here raised a SomeException
(You can also have both the except and finally.) If you don't need either the cleanup or the exception handling, that's even easier: just get rid of the try altogether, and have the block go directly under that while True.
Finally, as a terminology thing: try is not a loop. A loop is a bit of code that gets executed multiple times -- it loops. The try gets executed once. It's a "clause," not a "loop."
You have to also 'catch' the exception with the except statement, otherwise the try has no use.
So if you do something like:
try:
# some code here
except Exception:
# What to do if specified error is encountered
This way if anywhere in your try block an exception is raised it will not break your code, but it will be catched by your except.
I have a Python script that needs to look for a certain file.
I could use os.path.isafile(), but I've heard that's bad Python, so I'm trying to catch the exception instead.
However, there's two locations I could possibly look for the file. I could use nested trys to handle this:
try:
keyfile = 'location1'
try_to_connect(keyfile)
except IOError:
try:
keyfile = 'location2'
try_to_connect(keyfile)
except:
logger.error('Keyfile not found at either location1 or location2')
Or I could just put a pass in the first except block, and then have another one just below:
try:
keyfile = 'location1'
try_to_connect(keyfile)
except IOError:
pass
try:
keyfile = 'location2'
try_to_connect(keyfile)
except:
logger.error('Keyfile not found at either location1 or location2')
However, is there a more Pythonic way to handle the above situation?
Cheers,
Victor
for location in locations:
try:
try_to_connect(location)
break
except IOError:
continue
else:
# this else is optional
# executes some code if none of the locations is valid
# for example raise an Error as suggested #eumiro
Also you can add an else clause to the for loop; that is some code is executed only if the loop terminates through exhaustion (none of the locations is valid).
I've read three beginner-level Python books, however, I still don't understand exceptions.
Could someone give me a high level explanation?
I guess I understand that exceptions are errors in code or process that cause the code to stop working.
In the old days, when people wrote in assembly language or C, every time you called a function that might fail, you had to check whether it succeeded. So you'd have code like this:
def countlines(path):
f = open(path, 'r')
if not f:
print("Couldn't open", path)
return None
total = 0
for line in f:
value, success = int(line)
if not success:
print(line, "is not an integer")
f.close()
return None
total += value
f.close()
return total
The idea behind exceptions is that you don't worry about those exceptional cases, you just write this:
def countlines(path):
total = 0
with open(path, 'r') as f:
for line in f:
total += int(line)
return total
If Python can't open the file, or turn the line into an integer, it will raise an exception, which will automatically close the file, exit your function, and quit your whole program, printing out useful debugging information.
In some cases, you want to handle an exception instead of letting it quit your program. For example, maybe you want to print the error message and then ask the user for a different filename:
while True:
path = input("Give me a path")
try:
print(countlines(path))
break
except Exception as e:
print("That one didn't work:", e)
Once you know the basic idea that exceptions are trying to accomplish, the tutorial has a lot of useful information.
If you want more background, Wikipedia can help (although the article isn't very useful until you understand the basic idea).
If you still don't understand, ask a more specific question.
The best place to start with that is Python's list of built-in exceptions, since most you'll see derive from that.
Keep in mind that anybody can throw any error they want over anything, and then catch it and dismiss it as well. Here's one quick snippet that uses exceptions for handling instead of if/else where __get_site_file() throws an exception if the file isn't found in any of a list of paths. Despite that particular exception, the code will still work. However, the code would throw an uncaught error that stops execution if the file exists but the permissions don't allow reading.
def __setup_site_conf(self):
# Look for a site.conf in the site folder
try:
path = self.__get_site_file('site.conf')
self.__site_conf = open(path).read()
except EnvironmentError:
self.__site_conf = self.__get_site_conf_from_template()
Python's documentation: http://docs.python.org/2/tutorial/errors.html
For a high-level explanation, say we want to divide varA / varB. We know that varB can't equal 0, but we might not want to perform the check every time we do the division:
if varB != 0:
varA / varB
We can use exceptions to try the block without performing the conditional first, and then handle the behavior of the program based on whether or not something went wrong in the try block. In the following code, if varB == 0, then 'oops' is printed to the console:
try:
varA / varB
except ZeroDivisionError:
print 'oops'
Here is a list of exceptions that can be used: http://docs.python.org/2/library/exceptions.html#exceptions.BaseException
However, if you know how it may fail, you can just open a python console and see what exception is raised:
>>> 1 / 0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero
Exceptions are unexpected events that occur during the execution of a program. An exception might result from a logical error or an unanticipated situation.
In Python, exceptions (also known as errors) are objects that are raised (or thrown) by code that encounters an unexpected circumstance.
The Python interpreter can also raise an exception should it encounter an unexpected condition, like running out of memory. A raised error may be caught by a surrounding context that “handles” the exception in an appropriate fashion.
If uncaught, an exception causes the interpreter to stop executing the program and to report an appropriate message to the console.
def sqrt(x):
if not isinstance(x, (int, float)):
raise TypeError( x must be numeric )
elif x < 0:
raise ValueError( x cannot be negative )
Exceptions are not necessarily errors. They are things that get raised when the code encounters something it doesn't (immediately) know how to deal with. This may be entirely acceptable, depending on how you make your code. For instance, let's say you ask a user to put in a number. You then try to take that text (string) and convert it to a number (int). If the user put in, let's say, "cat", however, this will raise an exception. You could have your code handle that exception, however, and rather than break, just give the user a small message asking him to try again, and please use a number. Look at this link to see what I'm talking about: http://www.tutorialspoint.com/python/python_exceptions.htm
Also, you usually handle exceptions with a try, except (or catch) block. Example:
try:
integer = int(raw_input("Please enter an integer: "))
except Exception, exc:
print "An error has occured."
print str(exc)
Hope it helps!
How could I go about doing something like this
Try to do something.
If it works, great, continue with normal flow.
If it fails run a function and try again.
If it once again fails throw an exception and stop code.
I believe I would have to make use of try but I haven't quite come around yet to how to use it in this particular example.
It doesn't sound like you want to do a nested try-catch at all. Exceptions as control flow are a gnarly anti-pattern, and where you can avoid it, you should.
In this scenario, avoidance is easy. In the method that you describe, you want to be sure that a file exists before you do some operations on it. You also have a method to "correct" the path should it not. If both attempts fail, then you want to bail out.
With that into account, we would want to use os.path.isfile for this.
from os.path import isfile
def something(filepath):
# Don't mutate the parameter if you can help it.
p = filepath
if not isfile(p):
p = correct_path(p)
if not isfile(p):
raise Error("Cannot find file {} after correction to {}, aborting.".format(filepath, p))
with open(p, 'r') as f:
# Rest of file operations here
Try a nested try catch:
try:
do_something() #if this works, will skip rest and continue
except:
do_fix_function() #on error, do extra function
try:
do_something() #try again
except:
throw error() #if it fails this time, stop and throw an error
Note that if your do_fix_function() can also fail, you might want to put it inside the second try statement instead
This works for an arbitrary number of tries; I set it to two since that's what you want.
tries = 2
while True:
try:
step1()
except CatchThisException:
tries -= 1
if tries: # if tries != 0
step3()
continue # not necessary, just for clarity
else:
raise # step 4
else:
break # step 2
You can use retrying package to solve your retry attempt. Just write a block of code that keeps repeating on failure, until max retries is hit
For example:
import random
from retrying import retry
#retry
def do_something_unreliable():
if random.randint(0, 10) > 1:
raise IOError("Broken sauce, everything is hosed!!!111one")
else:
return "Awesome sauce!"
print do_something_unreliable()