I have a function that reads a CSV, checks the values in the rows, and if everything is okay, it writes the rows to a new CSV file. I have a few validation functions I'm calling within my main function to check the value and format of the rows.
I'm trying to implement my main function in a way so that when I call the other validation functions and something doesn't check out, I skip writing that row entirely.
#main function
for row in reader:
try:
row['amnt'] = divisible_by_5(row['amnt'])
row['issue_d'] = date_to_iso(row['issue_d'])
writer.writerow(row)
except:
continue
#Validation function
def divisible_by_5(value):
try:
float_value = float(value)
if float_value % 5 == 0 and float_value != 0:
return float_value
else:
raise ValueError
except ValueError:
return None
At the moment, the writer is still writing rows that should be skipped. For example, if a number is not divisible by 5, instead of skipping that row, the writer is just writing ''.
So, how can I handle the exception (ValueError) raised in divisible_by_5 in my for loop so I don't write lines that raise an exception?
You can re-raise an exception caught in an exception handler and have another handler up the call stack handle it by using an empty raise statement inside the except block:
except ValueError:
raise # re raises the previous exception
This way, the outer handler in 'main' can catch it and continue.
Note: As is, your empty except: block in the for loop will catch everything. Specifying the exception expected (except ValueError in this case) is generally considered a good idea.
More simply, don't put the body of your validation function divisible_by_5 in a try statement. Python automatically searches for the nearest except block defined and uses it if the current exception matches the exceptions specified:
def raiser():
if False:
pass
else:
raise ValueError
try:
raiser()
except ValueError:
print "caught"
This will print caught.
Related
I have below script which generates report for large size data.
Due to large size data request call times out.
I have added exception to handle this situation which works fine to get keep script running.
Issue I am having is after exception it goes to next project and skips the projects where it timed out.
I want it to start from same project again.
How can I achieve this ?
if __name__ = ‘__main__’
for project in AllProjectData['value']:
try:
project_name = project['name']
** code to be executed
except:
requests.ConnectionError,
requests.exceptions.ReadTimeout,
requests.exceptions.Timeout,
requests.exceptions.ConnectTimeout
continue
You are catching exceptions in a very odd way. I've never seen it done like this. I believe this code is catching all exceptions. For example:
try:
1/0
except:
ZeroDivisionError
pass
Works fine, but so does (it should raise IndexError):
try:
a = []
print(a[1])
except:
ZeroDivisionError
pass
So you shouldn't write except statements this way. What you should have is something along the lines:
success = False
while not success:
try:
# Your execution code
except (requests.ConnectionError,
requests.exceptions.ReadTimeout,
requests.exceptions.Timeout,
requests.exceptions.ConnectTimeout):
continue
else:
success = True
Also you should try and not put so much code in your except statement as it is confusing as to what you're trying to catch and where. Also, you're completely missing some possibilities like a KeyError when there's no id field in project and others.
Try this -
def myfunc():
# Write the code given in question completely
# Then add this in your except statement
except:
requests.ConnectionError,
requests.exceptions.ReadTimeout,
requests.exceptions.Timeout,
requests.exceptions.ConnectTimeout
# Call your function so that it will jump back to the same function
myfunc()
# You don't need continue keyword because it jumps to the same function
The simplest way would be to use a while loop with a counter variable. Let me demonstrate:
i = 0
while i < len(AllProjectData['value']):
try:
project = AllProjectData['value'][i]
# code to be executed
i += 1 # increment counter on success
except:
requests.ConnectionError,
requests.exceptions.ReadTimeout,
requests.exceptions.Timeout,
requests.exceptions.ConnectTimeout
continue
This way, you will go to the next project only if work on the previous project was executed, as the loop variable is incremented only in the try block.
Note: I have assumed that your iterable is indexable. If it's not, just pass it inside the list() constructor.
Is it possible to tell if there was an exception once you're in the finally clause? Something like:
try:
funky code
finally:
if ???:
print('the funky code raised')
I'm looking to make something like this more DRY:
try:
funky code
except HandleThis:
# handle it
raised = True
except DontHandleThis:
raised = True
raise
else:
raised = False
finally:
logger.info('funky code raised %s', raised)
I don't like that it requires to catch an exception, which you don't intend to handle, just to set a flag.
Since some comments are asking for less "M" in the MCVE, here is some more background on the use-case. The actual problem is about escalation of logging levels.
The funky code is third party and can't be changed.
The failure exception and stack trace does not contain any useful diagnostic information, so using logger.exception in an except block is not helpful here.
If the funky code raised then some information which I need to see has already been logged, at level DEBUG. We do not and can not handle the error, but want to escalate the DEBUG logging because the information needed is in there.
The funky code does not raise, most of the time. I don't want to escalate logging levels for the general case, because it is too verbose.
Hence, the code runs under a log capture context (which sets up custom handlers to intercept log records) and some debug info gets re-logged retrospectively:
try:
with LogCapture() as log:
funky_code() # <-- third party badness
finally:
# log events are buffered in memory. if there was an exception,
# emit everything that was captured at a WARNING level
for record in log.captured:
if <there was an exception>:
log_fn = mylogger.warning
else:
log_fn = getattr(mylogger, record.levelname.lower())
log_fn(record.msg, record.args)
Using a contextmanager
You could use a custom contextmanager, for example:
class DidWeRaise:
__slots__ = ('exception_happened', ) # instances will take less memory
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
# If no exception happened the `exc_type` is None
self.exception_happened = exc_type is not None
And then use that inside the try:
try:
with DidWeRaise() as error_state:
# funky code
finally:
if error_state.exception_happened:
print('the funky code raised')
It's still an additional variable but it's probably a lot easier to reuse if you want to use it in multiple places. And you don't need to toggle it yourself.
Using a variable
In case you don't want the contextmanager I would reverse the logic of the trigger and toggle it only in case no exception has happened. That way you don't need an except case for exceptions that you don't want to handle. The most appropriate place would be the else clause that is entered in case the try didn't threw an exception:
exception_happened = True
try:
# funky code
except HandleThis:
# handle this kind of exception
else:
exception_happened = False
finally:
if exception_happened:
print('the funky code raised')
And as already pointed out instead of having a "toggle" variable you could replace it (in this case) with the desired logging function:
mylog = mylogger.WARNING
try:
with LogCapture() as log:
funky_code()
except HandleThis:
# handle this kind of exception
else:
# In case absolutely no exception was thrown in the try we can log on debug level
mylog = mylogger.DEBUG
finally:
for record in log.captured:
mylog(record.msg, record.args)
Of course it would also work if you put it at the end of your try (as other answers here suggested) but I prefer the else clause because it has more meaning ("that code is meant to be executed only if there was no exception in the try block") and may be easier to maintain in the long run. Although it's still more to maintain than the context manager because the variable is set and toggled in different places.
Using sys.exc_info (works only for unhandled exceptions)
The last approach I want to mention is probably not useful for you but maybe useful for future readers who only want to know if there's an unhandled exception (an exception that was not caught in any except block or has been raised inside an except block). In that case you can use sys.exc_info:
import sys
try:
# funky code
except HandleThis:
pass
finally:
if sys.exc_info()[0] is not None:
# only entered if there's an *unhandled* exception, e.g. NOT a HandleThis exception
print('funky code raised')
raised = True
try:
funky code
raised = False
except HandleThis:
# handle it
finally:
logger.info('funky code raised %s', raised)
Given the additional background information added to the question about selecting a log level, this seems very easily adapted to the intended use-case:
mylog = WARNING
try:
funky code
mylog = DEBUG
except HandleThis:
# handle it
finally:
mylog(...)
You can easily assign your caught exception to a variable and use it in the finally block, eg:
>>> x = 1
>>> error = None
>>> try:
... x.foo()
... except Exception as e:
... error = e
... finally:
... if error is not None:
... print(error)
...
'int' object has no attribute 'foo'
Okay, so what it sounds like you actually just want to either modify your existing context manager, or use a similar approach: logbook actually has something called a FingersCrossedHandler that would do exactly what you want. But you could do it yourself, like:
#contextmanager
def LogCapture():
# your existing buffer code here
level = logging.WARN
try:
yield
except UselessException:
level = logging.DEBUG
raise # Or don't, if you just want it to go away
finally:
# emit logs here
Original Response
You're thinking about this a bit sideways.
You do intend to handle the exception - you're handling it by setting a flag. Maybe you don't care about anything else (which seems like a bad idea), but if you care about doing something when an exception is raised, then you want to be explicit about it.
The fact that you're setting a variable, but you want the exception to continue on means that what you really want is to raise your own specific exception, from the exception that was raised:
class MyPkgException(Exception): pass
class MyError(PyPkgException): pass # If there's another exception type, you can also inherit from that
def do_the_badness():
try:
raise FileNotFoundError('Or some other code that raises an error')
except FileNotFoundError as e:
raise MyError('File was not found, doh!') from e
finally:
do_some_cleanup()
try:
do_the_badness()
except MyError as e:
print('The error? Yeah, it happened')
This solves:
Explicitly handling the exception(s) that you're looking to handle
Making the stack traces and original exceptions available
Allowing your code that's going to handle the original exception somewhere else to handle your exception that's thrown
Allowing some top-level exception handling code to just catch MyPkgException to catch all of your exceptions so it can log something and exit with a nice status instead of an ugly stack trace
If it was me, I'd do a little re-ordering of your code.
raised = False
try:
# funky code
except HandleThis:
# handle it
raised = True
except Exception as ex:
# Don't Handle This
raise ex
finally:
if raised:
logger.info('funky code was raised')
I've placed the raised boolean assignment outside of the try statement to ensure scope and made the final except statement a general exception handler for exceptions that you don't want to handle.
This style determines if your code failed. Another approach might me to determine when your code succeeds.
success = False
try:
# funky code
success = True
except HandleThis:
# handle it
pass
except Exception as ex:
# Don't Handle This
raise ex
finally:
if success:
logger.info('funky code was successful')
else:
logger.info('funky code was raised')
If exception happened --> Put this logic in the exception block(s).
If exception did not happen --> Put this logic in the try block after the point in code where the exception can occur.
Finally blocks should be reserved for "cleanup actions," according to the Python language reference. When finally is specified the interpreter proceeds in the except case as follows: Exception is saved, then the finally block is executed first, then lastly the Exception is raised.
Consider the following code:
def f(x):
if x < 10:
return Exception("error")
else:
raise Exception("error2")
if __name__ == "__main__":
try:
f(5) # f(20)
except Exception:
print str(Exception)
Is there any difference?
When should I use return Exception and When should I use raise?
raise and return are two inherently different keywords.
raise, commonly known as throw in other languages, produces an error in the current level of the call-stack. You can catch a raised error by covering the area where the error might be raised in a try and handling that error in an except.
try:
if something_bad:
raise generate_exception()
except CertainException, e:
do_something_to_handle_exception(e)
return on the other hand, returns a value to where the function was called from, so returning an exception usually is not the functionality you are looking for in a situation like this, since the exception itself is not the thing triggering the except it is instead the raiseing of the exception that triggers it.
Hi, I wrote a thesaurus function and i am trying to get it to raise an exception, this is a extract as an example where i would like it to raise an exception.
#Extract from thesaurus function
if words.count(value) > 1:
raise Exception("Word appears twice")
This is the last line of the function for now and i follow this up with a try statement to open the file containing the words.
try:
file = open("thesaurus.txt", "r")
words = file.readlines()
Thesaurus(words)
print("Successful")
except LookupError as exceptObj:
print("Error:", str(exceptObj))
All the statements that should throw an exception(i tested the code outside the function as a standalone code and it worked) but as a function, the exception does it raise plus it prints successful when it should not, any ideas?
The exception thrown by the Thesaurus method (Exception) is more general than the one that you are catching in the except block (LookupError). So it will not be handled.
Change
raise Exception("Word appears twice")
to
raise LookupError("Word appears twice")
You are throwing some type of exception and catching another type. You need to throw and catch the same type.
Either you change the line
raise Exception("Word appears twice")
to
raise LookupError("Word appears twice")
Or change the line
except LookupError as exceptObj
to
except Exception as exceptObj
I have function, which looks for special Element if project files:
def csproj_tag_finder(mod_proj_file):
"""Looking for 'BuildType' element in each module's csproj file passed in `mod_proj_file`
ard return it's value (CLOUD, MAIN, NGUI, NONE)"""
try:
tree = ET.ElementTree(file=mod_proj_file)
root = tree.getroot()
for element in root.iterfind('.//'):
if ('BuildType') in element.tag:
return element.text
except IOError as e:
# print 'WARNING: cant find file: %s' % e
If no file found - it prints 'WARNING: cant find file: %s' % e.
This function called from another one:
def parser(modename, mod_proj_file):
...
# module's tag's from project file in <BuildType> elements, looks like CLOUD
mod_tag_from_csproj = csproj_tag_finder(mod_proj_file)
if not mod_tag_from_csproj:
print('WARNING: module %s have not <BuildType> elements in file %s!' % (modename, mod_proj_file))
...
So - when file doesn't found - csproj_tag_finder()return None type, and print WARNING. Second function - parser() find empty mod_tag_from_csproj variable, and also print WARNING. This is harmless, so I want make csproj_tag_finder() raise special Exception, so parser() except it and pass == check, instead of print text.
I tried add something like:
...
except IOError as e:
# print 'WARNING: cant find file: %s' % e
raise Exception('NoFile')
to csproj_tag_finder() to catch it later in parser() - but it's interrupt next steps immediately.
P.S. Later if not mod_tag_from_csproj: will call another function to add new Element. This task can be solved with just return 'NoFile' and then catch with if/else - but it seems to me that raise will more correct way here. Or not?
raise interrupting the next steps immediately is exactly what it's supposed to do. In fact, that's the whole point of exceptions.
But then return also interrupts the next steps immediately, because returning early is also the whole point of return.
If you want to save an error until later, continue doing some other work, and then raise it at the end, you have to do that explicitly. For example:
def spam():
error = None
try:
do_some_stuff()
except IOError as e:
print 'WARNING: cant find file %s' % e
error = Exception('NoFile')
try:
do_some_more_stuff()
except OtherError as e:
print 'WARNING: cant frob the glotz %s' % e
error = Exception('NoGlotz')
# etc.
if error:
raise error
Now, as long as there's no unexpected exception that you forgot to handle, whatever failed last will be in error, and it'll be raised at the end.
As a side note, instead of raising Exception('NoFile'), then using == to test the exception string later, you probably want to create a NoFileException subclass; then you don't need to test it, you can just handle it with except NoFileException:. And that means you can carry some other useful information (the actual exception, the filename, etc.) in your exception without it getting in the way, too. If this sounds scary to implement, it's not. It's literally a one-liner:
class NoFileException(Exception): pass