Develop Reference

Getting variables upon python Jupyter crash

One of the challenges I'm having is getting a stack trace or examining variables which should be in scope when an error occurs. However I am not finding this is the case. For example if a piece of code fields, I'd like to be able to see where in the loop in fails. However despite using %debug, I can never get any values out.
For example my code:
if a[field].to_list()[0] == b[field].to_list()[0]:
result = True
fails, and I'd like to know what the value of field is. But I can never find a way to make this work using %debug.

Maybe this try and except block helps you
for field in fields:
try:
if a[field].to_list()[0] == b[field].to_list()[0]:
result = True
except Exception as e:
print(field)
print(e)

How to write try except for loading data

I'm pretty new to coding so I apologize for this being stupid question. I'm writing a spark function that takes in a file path and file type and creates a dataframe. If the input is invalid, I want to just print some sort of error message and return an empty dataframe. Would I use try except?
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except ____ as error:
print(error)
return "" #I want to return an empty RDD here, but I can't figure out how to make one
How do I know what goes in the ____? I tried org.apache.spark.SparkException because that's the error I get when I pass in a .csv file as a parquet and it breaks but that isn't working

Welcome to StackOverflow!
You can catch multiple exceptions in the try-except block; for instance:
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except (SparkException, TypeError) as error:
print(error)
return ""
You could replace or add errors to that tuple.
Using a Exception will potentially silence errors that are unrelated to your code (like a networking issue if name is an S3 path). That is probably something you want your program to not handle.

Use Exception if you don't know what exception it might be:
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except Exception as error:
print(error)
return ""
WARNING: This is not good practice as it could silence errors that would be useful during debugging and troubleshooting.
(Thanks to #RafaelBarros)

catch specific error message in Python

I need to catch when one of my dependencies throws a specific ValueError and deal with that a certain way, and otherwise re-raise the error.
I don't find any recent questions that deal with this in a way that's Python 3 compliant, and that deals with cases where the only thing distinguishing errors returned is the string message.
This post is probably the closest: Python: Catching specific exception
Something like this-- catch specific HTTP error in python --won't work because I'm not using a dependency that also supplies specific codes like an HTTP error would have.
Here's my attempt:
try:
spect, freq_bins, time_bins = spect_maker.make(syl_audio,
self.sampFreq)
except ValueError as err:
if str(err) == 'window is longer than input signal':
warnings.warn('Segment {0} in {1} with label {2} '
'not long enough for window function'
' set with current spect_params.\n'
'spect will be set to nan.')
spect, freq_bins, time_bins = (np.nan,
np.nan,
np.nan)
else:
raise
If it matters, the dependency is scipy and I need to catch when the spectrogram fails for a specific reason (the segment I'm taking a spectrogram of is shorter than the window function).
I realize my approach is fragile because it depends on the error string not changing, but the error string is the only thing that distinguishes it from other ValueErrors returned by the same function. So I plan to have a unit test to defend myself against that.

Ok, so based on other people's comments, I'm guessing it should be something like this:
# lower-level module
class CustomError(Exception):
pass
# in method
Class Thing:
def __init__(prop1):
self.prop1 = prop1
def method(self,element):
try:
dependency.function(element,self.prop1)
except ValueError as err:
if str(err) == 'specific ValueError':
raise CustomError
else:
raise # re-raise ValueError because string not recognized
# back in higher-level module
thing = lowerlevelmodule.Thing(prop1)
for element in list_of_stuff:
try:
output = thing.method(element)
except CustomError:
output = None
warnings.warn('set output to None for {} because CustomError'.
format(element))

Handling DisambiguationError?

I'm using the wikipedia library and I want to handle the DisambiguationError as an exception. My first try was
try:
wikipedia.page('equipment') # could be any ambiguous term
except DisambiguationError:
pass
During execution line 3 isn't reached. A more general question is: how can I find the error type for a library-specific class like this?

Here's a working example:
import wikipedia
try:
wikipedia.page('equipment')
except wikipedia.exceptions.DisambiguationError as e:
print("Error: {0}".format(e))
Regarding to your more general question how can I find the error type for a library-specific class like this?, my trick is actually quite simple, I tend to capture Exception and then just printing the __class__, that way I'll know what specific Exception I need to capture.
One example of figuring out which specific exception to capture here:
try:
0/0
except Exception as e:
print("Exception.__class__: {0}".format(e.__class__))
This would print Exception.__class__: <type 'exceptions.ZeroDivisionError'>, so I knew exceptions.ZeroDivisionError would be the exact Exception to deal with instead of something more generic

Pandas GenericGBQException

I'm trying to use try/except to query BigQuery tables, sometimes the query may not be correct in which case pandas raises a GenericGBQException error.
My problem is I get name 'GenericGBQException' is not defined when trying to handle this error, example code below:
try:
df = pd.read_gbq(query, projID)
query_fail = 0
except GenericGBQException:
query_fail = 1
if query_fail == 1:
do some stuff
I can get by with catching all exceptions though obviously it's not ideal.

I suspect you want to catch pd.GenericGBQException. (Or perhaps gbq.GenericGBQException -- it depends on your imports. Are you importing the module that defines the exception you're trying to catch?)
Also, consider catching PandasError, the base class of all exceptions from the package: https://github.com/pydata/pandas/blob/master/pandas/io/gbq.py#L85