I would like to use the numpy.savetxt function, however from the documentation there doesn't seem to be a way to have a flag indicating if the file has been saved returned.
Is there any other way to ensure that the document was saved before continuing?
My problem is that when I save my document, the next line opens that document and I get some problems. I used a for loop to open the document several times and compared the results. The first time it opened it was ok. After that the values are incorrect and the same.
Inside a for-loop
savetxt('forest_submitfile.csv', end_matrix , delimiter=',', fmt='%s,%s,%s',
header='EventId,RankOrder,Class', comments = '')
print('Saving for Submit in CSV SUCCESS')
is_file_ok = False
while not is_file_ok:
if os.path.isfile("forest_submitfile.csv") and os.access("forest_submitfile.csv", os.R_OK):
break
print('Calculate AMS Metric Score')
AMS_metric("solutionFile.csv", "forest_submitfile.csv")
You can use a couple functions from os to check for you.
isfile checks for the existence of a file.
R_OK checks that it is in a readable state, which implies numpy is done writing to it.
yourFile = "C:\folder\folder\file.txt"
import os
if os.path.isfile(yourFile) and os.access(yourFile, os.R_OK):
# if you got into this check, your file is good to go!
According to the source, it can raise a ValueError or an AttributeError on failure. So, maybe catch those:
try:
np.savetxt('file', dataStructure)
except ValueError, e:
print('Save failed! {}'.format(str(e))
raise SystemError
except AttributeError, e:
print('Save failed! {}'.format(str(e))
raise SystemError
Hope this helps...
Related
One of the challenges I'm having is getting a stack trace or examining variables which should be in scope when an error occurs. However I am not finding this is the case. For example if a piece of code fields, I'd like to be able to see where in the loop in fails. However despite using %debug, I can never get any values out.
For example my code:
if a[field].to_list()[0] == b[field].to_list()[0]:
result = True
fails, and I'd like to know what the value of field is. But I can never find a way to make this work using %debug.
Maybe this try and except block helps you
for field in fields:
try:
if a[field].to_list()[0] == b[field].to_list()[0]:
result = True
except Exception as e:
print(field)
print(e)
I'm pretty new to coding so I apologize for this being stupid question. I'm writing a spark function that takes in a file path and file type and creates a dataframe. If the input is invalid, I want to just print some sort of error message and return an empty dataframe. Would I use try except?
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except ____ as error:
print(error)
return "" #I want to return an empty RDD here, but I can't figure out how to make one
How do I know what goes in the ____? I tried org.apache.spark.SparkException because that's the error I get when I pass in a .csv file as a parquet and it breaks but that isn't working
Welcome to StackOverflow!
You can catch multiple exceptions in the try-except block; for instance:
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except (SparkException, TypeError) as error:
print(error)
return ""
You could replace or add errors to that tuple.
Using a Exception will potentially silence errors that are unrelated to your code (like a networking issue if name is an S3 path). That is probably something you want your program to not handle.
Use Exception if you don't know what exception it might be:
def rdf(name, type):
try:
df=spark.read.format(type).load(name)
return df
except Exception as error:
print(error)
return ""
WARNING: This is not good practice as it could silence errors that would be useful during debugging and troubleshooting.
(Thanks to #RafaelBarros)
I am trying to create a machine learning program. So far, I have stored each of the 'learned' meanings in a text file in a list of all things that are attribute to that meaning. The text file is read by a python file that generates class objects that attribute the data in the text file to them. Then, in a master python file, the main prompt is where I am having trouble.
else:
try:
for obj in gc.get_objects():
try:
if isinstance(obj, LearnedClasses.learned):
if str(user.lower()) in obj.keys:
exec(obj.exstring)
chat()
break
except:
raise Exception
except Exception:
user = user.split()
for x in user:
learnlist.append(x)
learnch = random.choice(learnlist)
learnp = input("What does '{}' mean?".format(learnch))
learn(learnch, learnp)
chat()
This code is what follows the basic 'built-in' responses that I made. This is what happens after it can not find any keywords in the built-in section. I am using GC to collect all of the class objects that were generated from the text file. Then, if the prompt matches any keywords with any of its 'learned' keywords, I intend for it to respond with the responses set for that class. However, I can not get it to move on to the if ALL ELSE fails part, which starts at except Exception. How could I arrange this so that it can do what I described? Thank you.
It's a bit hard to follow this code; a couple of suggestions for improvement:
raise Exception has a typo: it should be raise Exception()
There's not much point having an except block which just raises an exception with no additional information; omit it?
The garbage collector is not a good place to store information that you will, in fact, need later; can you change the code so that the learned classes are stored in a list in a variable, which is returned or passed around somehow?
As a general rule, you should never touch the garbage collector unless you're running out of memory, and then only to figure out how to help it discard things.
As others have suggested in the comments, rather than try/except, use something like a flag variable to keep track of whether you've already answered, or the for/break/else construct:
for lc in learned_class_list:
if str(user.lower()) in lc.keys:
exec(lc.exstring)
chat()
break
else:
user = user.split()
for x in user:
learnlist.append(x)
learnch = random.choice(learnlist)
learnp = input("What does '{}' mean?".format(learnch))
learn(learnch, learnp)
I wanted to just print exception type and message.
I initially tried:
try:
raise Exception("duh!!!")
except Exception as err:
print(err)
But this only printed exception message.
duh!!!
Then I went through the whole traceback doc and I felt traceback.format_exception_only() is the one I am looking for.
So i tried it as follows:
try:
raise Exception("duh!!!")
except:
etype, evalue, tb = sys.exc_info()
print(traceback.format_exception_only(etype, evalue)))
and it printed following:
['Exception: duh!!!\n']
which looked a bit unexpected to me. So I re-read the doc for this method. It says following:
Format the exception part of a traceback. The arguments are the exception type and value such as given by sys.last_type and sys.last_value. The return value is a list of strings, each ending in a newline. Normally, the list contains a single string; however, for SyntaxError exceptions, it contains several lines that (when printed) display detailed information about where the syntax error occurred. The message indicating which exception occurred is the always last string in the list.
So I understood that the doc says its a list, which is why there is a list in the output [...]. Also the doc says each line ends in newline, thats why there is \n in the output. But I dont get when there will be multiple lines in case of SyntaxError? I am not able to produce SyntaxError which will result in multiple lines in the return value of format_exception_only().
Also it suddenly clicked to me that I can simply do
try:
raise Exception("duh!!!")
except:
etype, evalue, tb = sys.exc_info()
print('{}: {}'.format(etype.__name__, evalue))
to get
Exception: duh!!!
But then how format_exception_only() adds more value to this?
Python is open source so you can view the implementation and decide what the extra value is for yourself:
if self.exc_type is None:
yield _format_final_exc_line(None, self._str)
return
stype = self.exc_type.__qualname__
smod = self.exc_type.__module__
if smod not in ("__main__", "builtins"):
stype = smod + '.' + stype
if not issubclass(self.exc_type, SyntaxError):
yield _format_final_exc_line(stype, self._str)
else:
yield from self._format_syntax_error(stype)
It's a fairly simple function so yes, per your question if you ignore the parts of the functionality you don't use it doesn't add much value to you.
Of course, you're now on the hook for maintaining your code if there's any Exception changes in the future, or if turns out the Exception raiser starts returning SyntaxErrors, or any other edge cases that come in the future.
Reduced maintenance and increased readability (because everyone knows what the library code does) are the two relatively universal advantages of using standard library code.
I need to catch when one of my dependencies throws a specific ValueError and deal with that a certain way, and otherwise re-raise the error.
I don't find any recent questions that deal with this in a way that's Python 3 compliant, and that deals with cases where the only thing distinguishing errors returned is the string message.
This post is probably the closest: Python: Catching specific exception
Something like this-- catch specific HTTP error in python --won't work because I'm not using a dependency that also supplies specific codes like an HTTP error would have.
Here's my attempt:
try:
spect, freq_bins, time_bins = spect_maker.make(syl_audio,
self.sampFreq)
except ValueError as err:
if str(err) == 'window is longer than input signal':
warnings.warn('Segment {0} in {1} with label {2} '
'not long enough for window function'
' set with current spect_params.\n'
'spect will be set to nan.')
spect, freq_bins, time_bins = (np.nan,
np.nan,
np.nan)
else:
raise
If it matters, the dependency is scipy and I need to catch when the spectrogram fails for a specific reason (the segment I'm taking a spectrogram of is shorter than the window function).
I realize my approach is fragile because it depends on the error string not changing, but the error string is the only thing that distinguishes it from other ValueErrors returned by the same function. So I plan to have a unit test to defend myself against that.
Ok, so based on other people's comments, I'm guessing it should be something like this:
# lower-level module
class CustomError(Exception):
pass
# in method
Class Thing:
def __init__(prop1):
self.prop1 = prop1
def method(self,element):
try:
dependency.function(element,self.prop1)
except ValueError as err:
if str(err) == 'specific ValueError':
raise CustomError
else:
raise # re-raise ValueError because string not recognized
# back in higher-level module
thing = lowerlevelmodule.Thing(prop1)
for element in list_of_stuff:
try:
output = thing.method(element)
except CustomError:
output = None
warnings.warn('set output to None for {} because CustomError'.
format(element))