If function returns a two value list or tuple on success or False on failure, how can I best unpack the return list into two variables while also checking for False?
def get_key_value():
if (cond != True):
return False
return [val1, val2]
# Call it
# How can I also check for False while unpacking?
key, value = get_key_value()
Coverting #Felix Kling's great comment into an answer.
If not being able to find a (key, value) pair indicates some kind of system failure, it would be better to throw an exception. If your failure doesn't really fall into any of the standard exceptions, you should build a new exception type of your own.
The cond != True is better written as not cond. Also it's better to not create a list if it's not necessary.
class DataNotFound(Exception): pass
def get_key_value():
if not cond:
raise DataNotFound("Couldn't find it!")
return val1, val2
try:
key,value = get_key_value()
except DataNotFound:
#handle the failure somehow
key, value = 'ERROR', 'ERROR'
This falls under the "Easier to Ask for Forgiveness than Permission" policy of Python. I avoid catching TypeError in your function, in case there's some other unforeseen problem.
data = get_key_value()
try:
key, value = data
except TypeError:
#handle the failure somehow
key, value = 'ERROR', 'ERROR'
I don't think there is an idiomatic way to do this -- not least because a function that behaves that way is itself unidiomatic. If you have to do it, I suggest you simply make use of the fact that your 2-element list or tuple is a "truthy" rather than a "falsy" value (this isn't Python terminology but it's useful):
pair_or_false = get_key_value()
if pair:
key,value = val
else:
# handle failure in whatever way
The obvious alternative is to treat the not-found case as an exception:
try:
key,value = get_key_value()
except TypeError:
# deal with not-found case
but if there's any possibility at all that something other than the unsuccessful unpacking could raise a TypeError then you run the risk of masking a genuine error that way.
You're running into problems because you're mixing return types. Just because you can doesn't mean you should.
Although I agree with the others here that an exception is one appropriate way to go, it may depend on whether you expect to find a valid key & value most of the time. If so, use an exception (something like KeyError) to indicate that the function failed. But if you expect it to fail at a high rate, you may not want the exception overhead. In that case, return something like [None, None] from get_key_value and then your calling code would look like:
key, value = get_key_value()
if key:
# take action
else:
# handle the error appropriately
Related
def store(self) -> list:
result = []
for url in self.urls():
if url.should_store():
stored_url = self.func_that_can_throw_errors(url)
if stored_url: result.append(stored_url)
return result
Preface: not actual method names. Silly names chosen to emphasize
During the loop errors may occur. In that case, I desire the intermediate result to be returned by store() and still raise the original exception for handling in a different place.
Doing something like
try:
<accumulating results ... might break>
except Exception:
return result
raise
sadly doesn't do the trick, since trivially the raise stmt won't be reached (and thus an empty list get's returned).
Do you guys have recommendations on how not to lose the intermediate result?
Thanks a lot in advance - Cheers!
It is not possible as you imagine it. You can't raise an exception and return a value.
So I think what you are asking for is a work around. There, I see two possibilities:
return a Flag/Exception along the actual return value:
Return flag:
except Exception:
return result, False
where False is the Flag telling that something went wrong
Return Exception:
except Exception as e:
return result, e
Since it appears, that store is a method of some class, you could raise the exception and retrieve the intermediary result with a second call like so:
def store(self):
result = []
try:
# something
except Exception:
self.intermediary_result = result
raise
def retrieve_intermediary(self):
return self.intermediary_result
The best answer I can come up with given my limited knowledge of Python would be to always return a pair, where the first part of the pair is the result and the second part of the pair is an optional exception value.
def store(self) -> list:
'''
TODO: Insert documentation here.
If an error occurs during the operation, a partial list of results along with
the exception value will be returned.
:return A tuple of [list of results, exception]. The exception part may be None.
'''
result = []
for url in self.urls():
if url.should_store():
try:
stored_url = self.func_that_can_throw_errors(url)
except Exception as e:
return result, e
if stored_url: result.append(stored_url)
return result, None
That said, as you have mentioned, if you have this call multiple places in your code, you would have to be careful to change it in all relevant places as well as possibly change the handling. Type checking might be helpful there, though I only have very limited knowledge of Python's type hints.
Meanwhile I had the idea to just use an accumulator which appears to be the 'quickest' fix for now with the least amount of changes in the project where store() is called.
The (intermediate) result is not needed everywhere (let's say it's optional). So...
I'd like to share that with you:
def store(self, result_accu=None) -> list:
if result_accu is None:
result_accu = []
for url in self.urls():
if url.should_store():
stored_url = self.func(url)
if stored_url: result_accu.append(stored_url)
return result_accu
Still returning a list but alongside the intermediate result is accessible by reference on the accu list.
Making the parameter optional enables to leave most statements in project as they are since the result is not needed everywhere.
store() is rather some kind of a command where the most work on data integrity is done within already. The result is nice-to-have for now.
But you guys also enabled me to notice that there's work to do in ordner to process the intermediate result anyway. Thanks! #attalos #MelvinWM
I am receiving from an API a dictionary, but the problem is that sometimes I get {value: test} and others {key: test}. I am using a try/except block to take the one set:
try:
var = received_dict['value']
except KeyError:
var = received_dict['key']
Is there a better way to do that in Python 3?
received_dict.get('value', received_dict.get('key', False))
This will still throw an error if neither key is present which is almost certainly the desired behavior.
received_dict.get('value', received_dict['key'])
However, python generally uses the: Easier to ask for forgiveness than permission mentality. If one of those occurs more frequently than the other how you're currently doing it may be best. If it really is 50/50 however using a get might be a cleaner way to go.
https://docs.python.org/3.4/glossary.html (See EAFP)
To be honest your approach seems good. If you want a bit more control over the different cases you can do this:
vars = [received_dict[key] for key in ["value","key"] if key in received_dict.keys()]
if not vars:
raise KeyError("No 'value' or 'key' keys exist in dictionary")
elif len(vars)==2:
raise KeyError("Both 'value' and 'key' keys exist in dictionary")
else:
var=vars[0]
The only advantage of this approach is that it throws an error also when both keys exist. If you don't care about the "both exist" error case and you don't mind a different type of error being raised when the "none exist" error case happens, then you can do the code below to have a one liner:
var = [received_dict[key] for key in ["value","key"] if key in received_dict.keys()][0]
I have problems to apply clean code principles to this code section.
In clean code principles is written, that a function should do one thing and do it well.
But in this case I do not know how to refractor this function according to those principles without decreasing performance.
for netobject in netObjectList:
for key, value in netobject.getObjectParams().items():
if value.getValue() == '' and key is not 'comment':
Errorhandler().error_on_pos(value.getRow(), value.getCol(),
str(key) + ' is missing')
else:
#do syntax check on values in the dictionary
NValidator.checkSyntaxOfValue(key, value)
A netobject is dictionary containing all its parameters.
So I know that this function does more than one thing. And I think it is quite hard to unittest this function.
But if I proof syntax and missing parameters separatley, I have to iterate twice over all network objects bundled in that neObjectList. But otherwise it is no clean code to check syntax and missing parameters in one function.
I very often have these inner conflicts, when I am writing code.
Do you have some tips or suggestions to fix these problems?
If you need more description for that code section, please let me know.
Errorhandler is just a way to print out my errors, if an missing parameter is found.
The function checkSyntaxOfValue(key, value) calls suiteable check methods depending on the key of the bundled parameters. I think there would be also a better solution to handle syntax checks. But I do not know.
As far as I'm concerned, your code IS "doing one thing": validating your netobjects values. The fact that the validation is done in two steps (first checking if there's a value where expected, then checking the value's 'syntax') is an implementation detail.
Now what can be questionned is whether checking for empty values is distinct from checking the value's syntax - ie if an empty value is not a valid syntax for a given key, shouldn't the check be a responsability of NValidator.checkSyntaxOfValue ?
Also if for whatever reason you really do have to keep the tests (empty value / value syntax) distinct, you should probably encapsulate the emptyness test in a distinct function so you have the same level of abstraction for both tests, ie:
def checkForNotEmptyValue(key, value):
if key == 'comment':
# we don't care about empty values here
return True
if value.getValue() == '':
Errorhandler().error_on_pos(
value.getRow(),
value.getCol(),
'{} is missing'.format(key)
)
return False
return True
and then
for netobject in netObjectList:
for key, value in netobject.getObjectParams().items():
if checkForNotEmptyValue(key, value):
NValidator.checkSyntaxOfValue(key, value)
NB: this is just an example based on your code snippet and question - I don't have enough context to tell whether it's the "best" solution here.
As a last word: don't get over the top with "best practices" either. Writing clean code IS important, but "clean" is a relative notion, and trying to strictly follow all "golden rules" is downright impossible (specially since no one really agrees on those rules xD).
Actually, the point of "golden rules" etc is to make you aware of what problem you might have if you don't follow the rule - once you know, it's your responsability to decide when the rule applies and when it's doesn't. It's like raising children: at first you just forbid them to cross the road alone because they are too young to really understand the dangers. Once they are mature enough to get the point, the rule becomes useless.
Let me comment on your code first:
for netobject in netObjectList:
for key, value in netobject.getObjectParams().items():
"""
Below you seem to be assuming that first item must have 'comment' as key?
What if the key 'comment' actually exists? If you iterate like this you will be
Throwing errors until you iterate through all the items (and either hit the required key or not)
"""
if value.getValue() == '' and key is not 'comment':
Errorhandler().error_on_pos(value.getRow(), value.getCol(), str(key) + ' is missing')
else:
NValidator.checkSyntaxOfValue(key, value)
So, I'd choose a different approach. If you definitely want to work with item having key 'comment'
(and assuming here that your keys are unique - ie no multiple 'comment' keys in a single netobject object) you could do this instead:
for netobject in netObjectList:
try:
# try to get the value directly
# (of course you could use .keys() instead of .items() to evaluate existing keys first)
value = netobject.getObjectParams()['comment']
NValidator.checkSyntaxOfValue('comment', value)
except:
# Due to KeyError we now know for sure that such key definitely doesn't exist
# So we can handle the Exception, or throw your own such as:
Errorhandler().error_on_pos(...)
This example will either throw you just one True (ie not false positive) error, or it will succeed with syntax check
EDIT:
Decided to illustrate with better example:
Here I'm first emulating your netobject
class netobject_class(object):
def __init__(self, datadict):
self.data = datadict
def getObjectParams(self):
return self.data
def __str__(self):
return "I'm a netobject_class instance (hash=" + str(self.__hash__()) + ") containing a data Dict"
Now we create the netObjectList which contains 3 sample netobject_class instances. Note that second Dict has no 'comment' key.
netObjectList = [
netobject_class({'comment': 'comment1'}),
netobject_class({'somekey':'somevalue','somekey2':'somevalue2'}),
netobject_class({'comment': 'comment2'})
]
Now we run the code:
print("-" * 40)
for netobject in netObjectList:
# print("Working with netobject: %s" % netobject)
try:
# try to get the value directly
# (of course you could use .keys() instead of .items() to evaluate existing keys first)
value = netobject.getObjectParams()['comment']
print("****** value='%s' # aka comment" % value)
#NValidator.checkSyntaxOfValue('comment', value)
except Exception as e:
# Due to KeyError we now know for sure that such key definitely doesn't exist
# So we can handle the Exception, or throw your own such as:
print('Error: key', str(e), 'does not exist! Skipping....')
print("-" * 40)
And this produces:
----------------------------------------
****** value='comment1' # aka comment
----------------------------------------
Error: key 'comment' does not exist! Skipping....
----------------------------------------
****** value='comment2' # aka comment
----------------------------------------
What is the pythonic way for checking a user provided vector index?
def get_value(vector, index):
try:
return vector[index]
except IndexError:
raise ValueError('bad index')
or
def get_value(vector, index):
if -1 < index < len(vector):
return vector[index]
else:
raise ValueError('bad index')
Trying first is more Pythonic. From the glossary:
EAFP
Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.
Although I wouldn't even put that around a try...except since you're just throwing an IndexError, which it already does.
I think its reasonable to just let the IndexError propagate up to the user. The user is passing you an index into something. If its out of range IndexError is reasonably thrown from your interface. So I would just do:
def get_value(vector, index):
""" gets value... throws IndexError() if index out of range"""
return vector[index]
This also lets you use valid negative indices in your interface.
One way is to let the IndexError be thrown -- it's an invalid request. Another is to do vector.get(index) which gives you a None type on failure (same as try: return vector[index] except: return None). It really depends on what you intend to do upon failure. If you want to request the user try again, then .get() or try except is probably best. Otherwise raising the exception up to the top level indicates the program can't handle the request.
Note that the "Pythonic" way would normally be a try except -- but I reiterate that it's much more dependent on your use case.
Okay lets say I have a list, and I want to check if that list exists within another list. I can do that doing this:
all(value in some_map for value in required_values)
Which works fine, but lets say I want to the raise an exception when a required value is missing, with the value that it is missing. How can I do that using list comprehension?
I'm more or less curious, all signs seem to point to no.
EDIT Argh I meant this:
for value in required_values:
if value not in some_map:
raise somecustomException(value)
Looking at those I cant see how I can find the value where the error occurred
lets say i want to the raise an exception when a required value is missing, with the value that it is missing. How can i do that using list comprehension?
List comprehensions are a syntactically concise way to create a list based on some existing list—they're not a general-purpose way of writing any for-loop in a single line. In this example, you're not actually creating a list, so it doesn't make any sense to use a list comprehension.
You can't use raise in a list comprehension. You can check for yourself by looking at the grammar in the Python Language Reference.
You can however, invoke a function which raises an exception for you.
If you don't want to consider duplicates and the values are hashable, use sets. They're easier, faster, and can extract "all" elements missing in a single operation:
required_values = set('abc') # store this as a set from the beginning
values = set('ab')
missing = required_values - values
if missing:
raise SomeException('The values %r are not in %r' %
(missing, required_values))
Another (ugly) possibility would be the error_on_false function:
def error_on_false(value)
if value:
return value
else:
raise Exception('Wrong value: %r' % value)
if all(error_on_false(value in some_map) for value in required_values):
continue_code()
do_something('...')
That's ugly. I'd use the set instead.
I was wondering about tonight. My use case is iterating over a list of objects and raising an error when the object was not of a specific type. My solution is to use a generator.
def iter_my_class(my_class_list):
for c in my_class_list:
if not isinstance(c, MyClass):
raise ValueError('Expected MyClass')
yield c
Then used as
classes = [c for c in iter_my_class(my_class_list)]
I wrote this on my phone. If that runs without errors you all owe me a beer.
You can certainly hack something together, but it is not particularly readable.
(_ for _ in ()) defines a generator, from which you can use the throw method to raise any exception you want.
all((_ for _ in ()).throw(somecustomException(value)) for value in required_values if value not in some_map)
That said, readability aside, it doesn't make sense to use a list comprehension unless you're actually going to use the list. This might make more sense as something like:
map_values=[some_map[value] if value in some_map else (_ for _ in ()).throw(somecustomException(value)) for value in required_values]
But even then it probably makes more sense to handle the exception outside the loop. If you want to raise a custom exception for some reason you can just catch the KeyError and raise your own exception.
try:
found_values=[some_map[value] for value in required_values]
except KeyError as e:
raise somecustomException(e.args[0])
While I think using sets (like nosklo's example) is better, you could do something simple like this:
def has_required(some_map, value):
if not value in some_map:
raise RequiredException('Missing required value: %s' % value)
all(has_required(some_map, value) for value in required_values)