Okay lets say I have a list, and I want to check if that list exists within another list. I can do that doing this:
all(value in some_map for value in required_values)
Which works fine, but lets say I want to the raise an exception when a required value is missing, with the value that it is missing. How can I do that using list comprehension?
I'm more or less curious, all signs seem to point to no.
EDIT Argh I meant this:
for value in required_values:
if value not in some_map:
raise somecustomException(value)
Looking at those I cant see how I can find the value where the error occurred
lets say i want to the raise an exception when a required value is missing, with the value that it is missing. How can i do that using list comprehension?
List comprehensions are a syntactically concise way to create a list based on some existing list—they're not a general-purpose way of writing any for-loop in a single line. In this example, you're not actually creating a list, so it doesn't make any sense to use a list comprehension.
You can't use raise in a list comprehension. You can check for yourself by looking at the grammar in the Python Language Reference.
You can however, invoke a function which raises an exception for you.
If you don't want to consider duplicates and the values are hashable, use sets. They're easier, faster, and can extract "all" elements missing in a single operation:
required_values = set('abc') # store this as a set from the beginning
values = set('ab')
missing = required_values - values
if missing:
raise SomeException('The values %r are not in %r' %
(missing, required_values))
Another (ugly) possibility would be the error_on_false function:
def error_on_false(value)
if value:
return value
else:
raise Exception('Wrong value: %r' % value)
if all(error_on_false(value in some_map) for value in required_values):
continue_code()
do_something('...')
That's ugly. I'd use the set instead.
I was wondering about tonight. My use case is iterating over a list of objects and raising an error when the object was not of a specific type. My solution is to use a generator.
def iter_my_class(my_class_list):
for c in my_class_list:
if not isinstance(c, MyClass):
raise ValueError('Expected MyClass')
yield c
Then used as
classes = [c for c in iter_my_class(my_class_list)]
I wrote this on my phone. If that runs without errors you all owe me a beer.
You can certainly hack something together, but it is not particularly readable.
(_ for _ in ()) defines a generator, from which you can use the throw method to raise any exception you want.
all((_ for _ in ()).throw(somecustomException(value)) for value in required_values if value not in some_map)
That said, readability aside, it doesn't make sense to use a list comprehension unless you're actually going to use the list. This might make more sense as something like:
map_values=[some_map[value] if value in some_map else (_ for _ in ()).throw(somecustomException(value)) for value in required_values]
But even then it probably makes more sense to handle the exception outside the loop. If you want to raise a custom exception for some reason you can just catch the KeyError and raise your own exception.
try:
found_values=[some_map[value] for value in required_values]
except KeyError as e:
raise somecustomException(e.args[0])
While I think using sets (like nosklo's example) is better, you could do something simple like this:
def has_required(some_map, value):
if not value in some_map:
raise RequiredException('Missing required value: %s' % value)
all(has_required(some_map, value) for value in required_values)
Related
def store(self) -> list:
result = []
for url in self.urls():
if url.should_store():
stored_url = self.func_that_can_throw_errors(url)
if stored_url: result.append(stored_url)
return result
Preface: not actual method names. Silly names chosen to emphasize
During the loop errors may occur. In that case, I desire the intermediate result to be returned by store() and still raise the original exception for handling in a different place.
Doing something like
try:
<accumulating results ... might break>
except Exception:
return result
raise
sadly doesn't do the trick, since trivially the raise stmt won't be reached (and thus an empty list get's returned).
Do you guys have recommendations on how not to lose the intermediate result?
Thanks a lot in advance - Cheers!
It is not possible as you imagine it. You can't raise an exception and return a value.
So I think what you are asking for is a work around. There, I see two possibilities:
return a Flag/Exception along the actual return value:
Return flag:
except Exception:
return result, False
where False is the Flag telling that something went wrong
Return Exception:
except Exception as e:
return result, e
Since it appears, that store is a method of some class, you could raise the exception and retrieve the intermediary result with a second call like so:
def store(self):
result = []
try:
# something
except Exception:
self.intermediary_result = result
raise
def retrieve_intermediary(self):
return self.intermediary_result
The best answer I can come up with given my limited knowledge of Python would be to always return a pair, where the first part of the pair is the result and the second part of the pair is an optional exception value.
def store(self) -> list:
'''
TODO: Insert documentation here.
If an error occurs during the operation, a partial list of results along with
the exception value will be returned.
:return A tuple of [list of results, exception]. The exception part may be None.
'''
result = []
for url in self.urls():
if url.should_store():
try:
stored_url = self.func_that_can_throw_errors(url)
except Exception as e:
return result, e
if stored_url: result.append(stored_url)
return result, None
That said, as you have mentioned, if you have this call multiple places in your code, you would have to be careful to change it in all relevant places as well as possibly change the handling. Type checking might be helpful there, though I only have very limited knowledge of Python's type hints.
Meanwhile I had the idea to just use an accumulator which appears to be the 'quickest' fix for now with the least amount of changes in the project where store() is called.
The (intermediate) result is not needed everywhere (let's say it's optional). So...
I'd like to share that with you:
def store(self, result_accu=None) -> list:
if result_accu is None:
result_accu = []
for url in self.urls():
if url.should_store():
stored_url = self.func(url)
if stored_url: result_accu.append(stored_url)
return result_accu
Still returning a list but alongside the intermediate result is accessible by reference on the accu list.
Making the parameter optional enables to leave most statements in project as they are since the result is not needed everywhere.
store() is rather some kind of a command where the most work on data integrity is done within already. The result is nice-to-have for now.
But you guys also enabled me to notice that there's work to do in ordner to process the intermediate result anyway. Thanks! #attalos #MelvinWM
I'm trying use ABC and...
Checking type of single parameter is quite straithforward:
def spam_method(param):
if not isinstance(param, SpamInterface):
raise TypeError
It looks good. In first line of method definition is mentioned what type I require. But what about passing lists? I'm doing it in this way:
def many_spams(list_param):
if list_param and not isinstance(list_param[0], SpamInterface):
raise TypeError
But I'm not exactly satisfied with that. Any more elegant way? How would you do it?
I assume that you want to check that every object in list_param are of type SpamInterface. If that is the case, there are 2 different ways to check. First, the lazy way in which we use the all() function:
def many_spams(list_param):
if not all(isinstance(p, SpamInterface) for p in list_param):
raise TypeError('Not all objects are of type SpamInterface')
This method is short, but it will not tell you which element in the list failed the test. The second method offers a little more details:
def many_spams(list_param):
for index, param in enumerate(list_param):
if not isinstance(param, SpamInterface):
raise TypeError('Parameter at index {} is not of type SpamInterface: {}'.format(index, param))
This method will throw an exception at the first element that is not of type SpamInterface.
I am developing this Python project where I encounter a situation many times and I wondered if there is a better way.
There is a list of class instances. Some part of lists are empty(filled with None).
Here is an example list.
ins_list = [ins_1, ins_2, None, ins_3, None]
I have to do some confirmations throughout the program flow. There are points where I need the control an attribute of these instances. But only indexes are given for choosing an instance from the list and it may be one of the empty elements. Which would give an error when the attribute is called. Here is an example program flow.
ind = 2
if ins_list[ind].some_attribute == "thing":
# This would give error when empty element is selected.
I deal with this by using,
if ins_list[ind]:
if ins_list[ind].some_attribute == "thing":
# This works
I am okay with using this. However the program is a long one, I apply this hundreds of times. Is there an easier, better way of doing this, it means I am producing reduntant code and increasing indentation level for no reason. I wish to know if there is such a solution.
Use a boolean operator and.
if ins_list[ind] and ins_list[ind].some_attribute == "thing":
# Code
As coder proposed, you can remove None from your list, or use dictionaries instead, to avoid to have to create an entry for each index.
I want to propose another way: you can create a dummyclass and replace None by it. This way there will be no error if you set an attribute:
class dummy:
def __nonzero__(self):
return False
def __setattr__(self, k, v):
return
mydummy = dummy()
mylist = [ins_1, ins_2, mydummy, ins_3, mydummy]
nothing will be stored to the dummyinstances when setting an attribute
edit:
If the content of the original list cannot be chosen, then this class could help:
class PickyList(list):
def __init__(self, iterable, dummyval):
self.dummy = dummyval
return super(PickyList, self).__init__(iterable)
def __getitem__(self, k):
v = super(PickyList, self).__getitem__(k)
return (self.dummy if v is None else v)
mylist = PickyList(ins_list, mydummy)
There are these two options:
Using a dictionary:
Another way would be to use a dictionary instead. So you could create your dictionary once the list is filled up with elements. The dictionary's keys would be the values of your list and as values you could use the attributes of the elements that are not None and "No_attr" for those that are None. (Note: Have in mind that python dictionaries don't support duplicate keys and that's why I propose below to store as keys your list indexes else you will have to find a way to make keys be different)
For example for a list like:
l = [item1,item2,None,item4]
You could create a dictionary:
d = {item1:"thing1", item2:"thing2", None:"No_attr", item3:"thing3"}
So in this way every time you would need to make a check, you wouldn't have to check two conditions, but you could check only the value, such as:
if d.values()[your_index]=="thing":
The only cons of this method is that standard python dictionaries are inherently unordered, which makes accessing dictionary values by index a bit dangerous sometimes - you have to be careful not to change the form-arrangement of the dictionary.
Now, if you want to make sure that the index stays stable, then you would have to store it some way, for example select as keys of your dictionary the indexes, as you will have already stored the attributes of the items - But that is something that you will have to decide and depends strongly on the architecture of your project.
Using a list:
In using lists way I don't think there is a way to avoid your if statement - and is not bad actually. Maybe use an and operator as it is mentioned already in another answer but I don't think that makes any difference anyway.
Also, if you want to use your first approach:
if ins_list[ind].some_attribute == "thing":
You could try using and exception catcher like this:
try:
if ins_list[ind].some_attribute == "thing":
#do something
except:
#an error occured
pass
In this case I would use an try-except statement because of EAFP (easier to ask for forgivness than permission). It won't shorten yout code but it's a more Pythonic way to code when checking for valid attributes. This way you won't break against DRY (Don't Repat Yourself) either.
try:
if ins_list[ind].some_attribute == "thing":
# do_something()
except AttributeError:
# do_something_else()
I frequently encounter a problem where I need to apply a function to a large iterator of data, but that function sometimes raises a known error that I want to ignore. Unfortunately, neither list compressions nor the map function has a good way to handle errors.
What is the best way to skip/deal with errors quickly in python?
For example, say I have a list of data and a function, the function raises a ValueError whenever the data is a str. I want it to skip these values. One way to do this would be:
result = []
for n in data:
try: result.append(function(n))
except ValueError: pass
You could also do the same thing without the error checking like:
result = [function(n) for n in data]
or
result = list(map(function, data))
I want an c-compiled approach to accomplishing the above. Something in the spirit of
result = list(map(function, data, skip_errors=True))
The feature of default=value would also be useful, so that raised errors create a default value.
I'm thinking this might be something I need to write a Cython extension for.
Note: one solution would be for me to write the catch function I wrote in this answer in c or cython. Then I could use it in list compressions and get the performance boost I want.
Why not just wrap your function in an error handler?
def spam(n):
try:
return function(n)
except ValueError:
pass
result = [spam(n) for n in data]
you can then add anything you want to the error handling (note that in this version it returns None, so you probably want to either filter the resulting list or return a default value). The same goes for using map.
If function returns a two value list or tuple on success or False on failure, how can I best unpack the return list into two variables while also checking for False?
def get_key_value():
if (cond != True):
return False
return [val1, val2]
# Call it
# How can I also check for False while unpacking?
key, value = get_key_value()
Coverting #Felix Kling's great comment into an answer.
If not being able to find a (key, value) pair indicates some kind of system failure, it would be better to throw an exception. If your failure doesn't really fall into any of the standard exceptions, you should build a new exception type of your own.
The cond != True is better written as not cond. Also it's better to not create a list if it's not necessary.
class DataNotFound(Exception): pass
def get_key_value():
if not cond:
raise DataNotFound("Couldn't find it!")
return val1, val2
try:
key,value = get_key_value()
except DataNotFound:
#handle the failure somehow
key, value = 'ERROR', 'ERROR'
This falls under the "Easier to Ask for Forgiveness than Permission" policy of Python. I avoid catching TypeError in your function, in case there's some other unforeseen problem.
data = get_key_value()
try:
key, value = data
except TypeError:
#handle the failure somehow
key, value = 'ERROR', 'ERROR'
I don't think there is an idiomatic way to do this -- not least because a function that behaves that way is itself unidiomatic. If you have to do it, I suggest you simply make use of the fact that your 2-element list or tuple is a "truthy" rather than a "falsy" value (this isn't Python terminology but it's useful):
pair_or_false = get_key_value()
if pair:
key,value = val
else:
# handle failure in whatever way
The obvious alternative is to treat the not-found case as an exception:
try:
key,value = get_key_value()
except TypeError:
# deal with not-found case
but if there's any possibility at all that something other than the unsuccessful unpacking could raise a TypeError then you run the risk of masking a genuine error that way.
You're running into problems because you're mixing return types. Just because you can doesn't mean you should.
Although I agree with the others here that an exception is one appropriate way to go, it may depend on whether you expect to find a valid key & value most of the time. If so, use an exception (something like KeyError) to indicate that the function failed. But if you expect it to fail at a high rate, you may not want the exception overhead. In that case, return something like [None, None] from get_key_value and then your calling code would look like:
key, value = get_key_value()
if key:
# take action
else:
# handle the error appropriately