I have a for loop that is iteration through a column and doing some xml processing. There are some errors that I am encountering, but rather than just pass them, I would like to save them into a seperate list as part of the output. This is what I have so far:
def outputfunc(column_name):
output = []
errors = []
for i in column_name:
try:
tree = etree.fromstring(i)
except:
errors.append(i) #basic logic here is that i append errors with i that raise error and then pass
pass
Elaborating on Carcigenicates comment, if you want to find all errors using the Exception base class, you can change your code to:
def outputfunc(column_name):
output = []
errors = []
for i in column_name:
try:
tree = etree.fromstring(i)
except Exception as e:
errors.append(e) #basic logic here is that i append errors with i that raise error and then pass
This sets e to the text of the error message. e is then appended to errors. Furthermore, I've removed a nonfunctional pass statement. However, you currently have no way to output errors, so I suggest using
print('\n'.join(map(str,errors)))
Within the scope of outputfunc. Furthermore, as Paul Cornelius suggests, returning your output and errors in tuple format would be useful. This would make your code
def outputfunc(column_name):
output = []
errors = []
for i in column_name:
try:
tree = etree.fromstring(i)
except Exception as e:
errors.append(e) #basic logic here is that i append errors with i that raise error and then pass
print('\n'.join(map(str,errors)))
return output, errors
References
2. Lexical analysis — Python 3.9.6 documentation
Built-in Exceptions — Python 3.9.6 documentation
Related
One of the challenges I'm having is getting a stack trace or examining variables which should be in scope when an error occurs. However I am not finding this is the case. For example if a piece of code fields, I'd like to be able to see where in the loop in fails. However despite using %debug, I can never get any values out.
For example my code:
if a[field].to_list()[0] == b[field].to_list()[0]:
result = True
fails, and I'd like to know what the value of field is. But I can never find a way to make this work using %debug.
Maybe this try and except block helps you
for field in fields:
try:
if a[field].to_list()[0] == b[field].to_list()[0]:
result = True
except Exception as e:
print(field)
print(e)
I'm pretty new to coding so I apologize for this being stupid question. I'm writing a spark function that takes in a file path and file type and creates a dataframe. If the input is invalid, I want to just print some sort of error message and return an empty dataframe. Would I use try except?
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except ____ as error:
print(error)
return "" #I want to return an empty RDD here, but I can't figure out how to make one
How do I know what goes in the ____? I tried org.apache.spark.SparkException because that's the error I get when I pass in a .csv file as a parquet and it breaks but that isn't working
Welcome to StackOverflow!
You can catch multiple exceptions in the try-except block; for instance:
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except (SparkException, TypeError) as error:
print(error)
return ""
You could replace or add errors to that tuple.
Using a Exception will potentially silence errors that are unrelated to your code (like a networking issue if name is an S3 path). That is probably something you want your program to not handle.
Use Exception if you don't know what exception it might be:
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except Exception as error:
print(error)
return ""
WARNING: This is not good practice as it could silence errors that would be useful during debugging and troubleshooting.
(Thanks to #RafaelBarros)
I wanted to just print exception type and message.
I initially tried:
try:
raise Exception("duh!!!")
except Exception as err:
print(err)
But this only printed exception message.
duh!!!
Then I went through the whole traceback doc and I felt traceback.format_exception_only() is the one I am looking for.
So i tried it as follows:
try:
raise Exception("duh!!!")
except:
etype, evalue, tb = sys.exc_info()
print(traceback.format_exception_only(etype, evalue)))
and it printed following:
['Exception: duh!!!\n']
which looked a bit unexpected to me. So I re-read the doc for this method. It says following:
Format the exception part of a traceback. The arguments are the exception type and value such as given by sys.last_type and sys.last_value. The return value is a list of strings, each ending in a newline. Normally, the list contains a single string; however, for SyntaxError exceptions, it contains several lines that (when printed) display detailed information about where the syntax error occurred. The message indicating which exception occurred is the always last string in the list.
So I understood that the doc says its a list, which is why there is a list in the output [...]. Also the doc says each line ends in newline, thats why there is \n in the output. But I dont get when there will be multiple lines in case of SyntaxError? I am not able to produce SyntaxError which will result in multiple lines in the return value of format_exception_only().
Also it suddenly clicked to me that I can simply do
try:
raise Exception("duh!!!")
except:
etype, evalue, tb = sys.exc_info()
print('{}: {}'.format(etype.__name__, evalue))
to get
Exception: duh!!!
But then how format_exception_only() adds more value to this?
Python is open source so you can view the implementation and decide what the extra value is for yourself:
if self.exc_type is None:
yield _format_final_exc_line(None, self._str)
return
stype = self.exc_type.__qualname__
smod = self.exc_type.__module__
if smod not in ("__main__", "builtins"):
stype = smod + '.' + stype
if not issubclass(self.exc_type, SyntaxError):
yield _format_final_exc_line(stype, self._str)
else:
yield from self._format_syntax_error(stype)
It's a fairly simple function so yes, per your question if you ignore the parts of the functionality you don't use it doesn't add much value to you.
Of course, you're now on the hook for maintaining your code if there's any Exception changes in the future, or if turns out the Exception raiser starts returning SyntaxErrors, or any other edge cases that come in the future.
Reduced maintenance and increased readability (because everyone knows what the library code does) are the two relatively universal advantages of using standard library code.
I need to catch when one of my dependencies throws a specific ValueError and deal with that a certain way, and otherwise re-raise the error.
I don't find any recent questions that deal with this in a way that's Python 3 compliant, and that deals with cases where the only thing distinguishing errors returned is the string message.
This post is probably the closest: Python: Catching specific exception
Something like this-- catch specific HTTP error in python --won't work because I'm not using a dependency that also supplies specific codes like an HTTP error would have.
Here's my attempt:
try:
spect, freq_bins, time_bins = spect_maker.make(syl_audio,
self.sampFreq)
except ValueError as err:
if str(err) == 'window is longer than input signal':
warnings.warn('Segment {0} in {1} with label {2} '
'not long enough for window function'
' set with current spect_params.\n'
'spect will be set to nan.')
spect, freq_bins, time_bins = (np.nan,
np.nan,
np.nan)
else:
raise
If it matters, the dependency is scipy and I need to catch when the spectrogram fails for a specific reason (the segment I'm taking a spectrogram of is shorter than the window function).
I realize my approach is fragile because it depends on the error string not changing, but the error string is the only thing that distinguishes it from other ValueErrors returned by the same function. So I plan to have a unit test to defend myself against that.
Ok, so based on other people's comments, I'm guessing it should be something like this:
# lower-level module
class CustomError(Exception):
pass
# in method
Class Thing:
def __init__(prop1):
self.prop1 = prop1
def method(self,element):
try:
dependency.function(element,self.prop1)
except ValueError as err:
if str(err) == 'specific ValueError':
raise CustomError
else:
raise # re-raise ValueError because string not recognized
# back in higher-level module
thing = lowerlevelmodule.Thing(prop1)
for element in list_of_stuff:
try:
output = thing.method(element)
except CustomError:
output = None
warnings.warn('set output to None for {} because CustomError'.
format(element))
I'm trying to use try/except to query BigQuery tables, sometimes the query may not be correct in which case pandas raises a GenericGBQException error.
My problem is I get name 'GenericGBQException' is not defined when trying to handle this error, example code below:
try:
df = pd.read_gbq(query, projID)
query_fail = 0
except GenericGBQException:
query_fail = 1
if query_fail == 1:
do some stuff
I can get by with catching all exceptions though obviously it's not ideal.
I suspect you want to catch pd.GenericGBQException. (Or perhaps gbq.GenericGBQException -- it depends on your imports. Are you importing the module that defines the exception you're trying to catch?)
Also, consider catching PandasError, the base class of all exceptions from the package: https://github.com/pydata/pandas/blob/master/pandas/io/gbq.py#L85