I have a function called crawl which will return a link to a website. Then I do something like:
found.append(crawl()) (found is a list)
This works fine as long as crawl returns a valid link, but sometimes it does not return anything. So a value of None gets added to the list.
So my question is that, is it possible to return something from crawl that will not not add anything to the list?
In Python nothing is something: None. You have to use if-condition:
link = crawl()
if link is not None:
found.append(link)
or let crawl return a list, which can contain one or zero elements:
found.extend(crawl())
What you could do is to pass the list in to the crawl function and if there is anything to add append it otherwise not. Something like:
def crawl(found):
""" Add results to found list """
# do your stuff saving to result
if result is not None:
found.append(result)
# Call this as
crawl(found)
This is not possible directly.
You can test for a None being returned though, using the following code.
returned_link = crawl()
if returned_link is not None:
found.append(returned_link)
If a function returns it has to return an object, even if that object is None. However, there is another answer being overlooked, and that is to raise an exception rather than returning None.
As other people point out, one approach is to check if the returned object is None before appending it to the list:
link = crawl()
if link is not None:
found.append(link)
The other approach is to define some exception, perhaps WebsiteNotFoundError, and have crawl execute raise WebsiteNotFoundError instead of return None. Then you can write:
try:
found.append(crawl())
except WebsiteNotFoundError:
pass # or take appropriate action
The advantage of the exception handling approach is that it is generally faster than checking for None if returning None is a relatively rare occurrence compared to returning a valid link. Depending on the use, it may be more readable in the sense that the code naturally explains what is going wrong with the function.
Related
def store(self) -> list:
result = []
for url in self.urls():
if url.should_store():
stored_url = self.func_that_can_throw_errors(url)
if stored_url: result.append(stored_url)
return result
Preface: not actual method names. Silly names chosen to emphasize
During the loop errors may occur. In that case, I desire the intermediate result to be returned by store() and still raise the original exception for handling in a different place.
Doing something like
try:
<accumulating results ... might break>
except Exception:
return result
raise
sadly doesn't do the trick, since trivially the raise stmt won't be reached (and thus an empty list get's returned).
Do you guys have recommendations on how not to lose the intermediate result?
Thanks a lot in advance - Cheers!
It is not possible as you imagine it. You can't raise an exception and return a value.
So I think what you are asking for is a work around. There, I see two possibilities:
return a Flag/Exception along the actual return value:
Return flag:
except Exception:
return result, False
where False is the Flag telling that something went wrong
Return Exception:
except Exception as e:
return result, e
Since it appears, that store is a method of some class, you could raise the exception and retrieve the intermediary result with a second call like so:
def store(self):
result = []
try:
# something
except Exception:
self.intermediary_result = result
raise
def retrieve_intermediary(self):
return self.intermediary_result
The best answer I can come up with given my limited knowledge of Python would be to always return a pair, where the first part of the pair is the result and the second part of the pair is an optional exception value.
def store(self) -> list:
'''
TODO: Insert documentation here.
If an error occurs during the operation, a partial list of results along with
the exception value will be returned.
:return A tuple of [list of results, exception]. The exception part may be None.
'''
result = []
for url in self.urls():
if url.should_store():
try:
stored_url = self.func_that_can_throw_errors(url)
except Exception as e:
return result, e
if stored_url: result.append(stored_url)
return result, None
That said, as you have mentioned, if you have this call multiple places in your code, you would have to be careful to change it in all relevant places as well as possibly change the handling. Type checking might be helpful there, though I only have very limited knowledge of Python's type hints.
Meanwhile I had the idea to just use an accumulator which appears to be the 'quickest' fix for now with the least amount of changes in the project where store() is called.
The (intermediate) result is not needed everywhere (let's say it's optional). So...
I'd like to share that with you:
def store(self, result_accu=None) -> list:
if result_accu is None:
result_accu = []
for url in self.urls():
if url.should_store():
stored_url = self.func(url)
if stored_url: result_accu.append(stored_url)
return result_accu
Still returning a list but alongside the intermediate result is accessible by reference on the accu list.
Making the parameter optional enables to leave most statements in project as they are since the result is not needed everywhere.
store() is rather some kind of a command where the most work on data integrity is done within already. The result is nice-to-have for now.
But you guys also enabled me to notice that there's work to do in ordner to process the intermediate result anyway. Thanks! #attalos #MelvinWM
I'm currently working on function that gets some data from an API and outputs it to csv file. In this function there is a part where I filter the retrieved data before passing it on. Since there is a possibility that the request returns None I decided to explicitly check for that and implement a special behavior in this case. Please take a look at the following code snippet.
r = requests.post(self.settings['apiurl'],
headers=self.settings['header'],
json={'query': query_string, 'variables': vars})
jsd = json.loads(r.text)
if jsd is not None:
lists = jsd["data"]["User"]["stats"]["favouredGenres"]
newlist = [entry for entry in lists if entry["meanScore"] is not None]
if not newlist:
return None
else:
jsd["data"]["User"]["stats"]["favouredGenres"] = newlist
try:
jsd = jsd # json.loads(jsd)
except ValueError:
return None
else:
return jsd
else:
return None
The if jsd is not None part is the before mentioned check. If jsd is not None I filter again some parts out that I don't need and return a modified version of jsd.
The problem now is that I sporadically get the error message:
lists = jsd["data"]["User"]["stats"]["favouredGenres"]
TypeError: 'NoneType' object is not subscriptable
The first thing that really confuses me is that this error appears completely randomly. In one run it doesn't work on user_id=7 in the next one it doesn't work on user_id=8475 but works fine for user_id=7, etc. The second thing that confuses me is that it is even possible for the error to pop up since I explicitly check before accessing the variable if it is of type NoneType. Beside from these isolated cases where an error occurs the code produces exactly the results I expect....
I hope I provided you with everything necessary, if not, please let me know. Any kind of suggestions on how to approach this kind of problem are welcome.
Blindly sub-scripting a dictionary is usually fraught with this kind of problem. You should switch to using the get method on dictionaries. It can handle this case pretty nicely:
lists = (jsd.get("data", {}).get("User", {})
.get("stats", {}).get("favouredGenres", []))
Here we exploit the fact that get() can take a default argument, which it will use if the given key isn't found. In the first 3 cases, we provide an empty dictionary as the default, and we provide an empty list for the last case (since that's what I'm guessing you're expecting).
If you get into the habit of using this accessing method, you'll avoid these kinds of problems.
The Python Anti-Patterns book lists this as one to be aware of. The corollary is the setdefault() call when setting values in a dictionary.
Lets say I have a function myFunc defined as
def myFunc(value):
return value if isinstance(value, int) else None
Now wherever in my project I use myFunc the enclosing funciton should return automatically if the value returned from myFunc is None and should continue if some integer value is returned
For example:
def dumbFunc():
# some code
# goes here..
result = myFunc('foo')
# some code
# goes here..
This funciton should automatically behave like..
def dumbFunc():
# some code
# goes here..
result = myFunc('foo')
if not result:
return
# some code
# goes here..
PS - I don't know whether this thing even possible or not.
This is simply not possible.
Apart from exceptions, you cannot give a called function the ability to impact the control flow of the calling scope. So a function call foo() can never interrupt the control flow without throwing an exception. As a consumer of the function, you (the calling function) always have the responsibility yourself to handle such cases and decide about your own control flow.
And it is a very good idea to do it like that. Just the possibility that a function call might interrupt my control flow without having a possibility to react on it first sounds like a pure nightmare. Just alone for the ability to release and cleanup resources, it is very important that the control flow is not taken from me.
Exceptions are the notable exception from this, but of course this is a deeply rooted language feature which also still gives me the ability to act upon it (by catching exceptions, and even by having finally blocks to perform clean up tasks). Exceptions are deliberately not silent but very loud, so that interruptions from the deterministic control flow are clearly visible and have a minimum impact when properly handled.
But having a silent feature that does neither give any control nor feedback would be just a terrible idea.
If myFunc is used at 100 places in my project, everywhere I need to put an if condition after it.
If your code is like that that you could just return nothing from any function that calls myFunc without having to do anything, then either you are building an unrealistic fantasy project, or you simply are not aware of the implications this can have to the calling code of the functions that would be returned that way.
ok, I'll bite.
on the one hand, this isn't really possible. if you want to check something you have to have a line in your code that checks it.
there are a few ways you could achieve something like this, but i think you may have already found the best one.
you already have this function:
def myFunc(value):
return value if isinstance(value, int) else None
I would probably have done:
def myFunc(value):
return isinstance(value, int)
but either way you could use it:
def dumb_func():
value = do_something()
if myFunc(value):
return
do_more()
return value
alternately you could use try and except
I would raise a TypeError, seeing as that seems to be what you are checking:
def myFunc(value):
if not isinstance(value, int):
raise TypeError('myFunc found that {} is not an int'.format(value))
then you can use this as such
def dumb_func():
value = do_something()
try:
myFunc(value):
Except TypeError as e:
print e # some feedback that this has happened, but no error raised
return
do_more()
return value
for bonus points you could define a custom exception (which is safer because then when you catch that specific error you know it wasn't raised by anything else in your code, also if you did that you could be lazier eg:)
Class CustomTypeError(TypeError):
pass
def dumb_func():
try:
value = do_something()
myFunc(value):
do_more()
return value
Except CustomTypeError as e:
print e # some feedback that this has happened, but no error raised
return
but none of this gets around the fact that if you want to act based on the result of a test, you have to check that result.
Python has a ternary conditional operator, and the syntax you used is right, so this will work:
def myFunc(value):
return value if isinstance(value, int) else None
def dumbFunc():
print("Works?")
result = myFunc(5)
print(result)
dumbFunc()
Result:
Works?
5
I want the function to return automatically in that case
This is not possible. To do that, you have to check the return value of myFunc() and act upon it.
PS: You could do that with a goto statement, but Python, fortunately, doesn't support this functionality.
I have an object that may or may not exist. Due to this fact I have wrapped the statement in a try / except block.
try:
generic_type = ContentType.objects.get_for_model(myentity)
my_object = MyObject.objects.filter(content_type__pk=generic_type.id, object_id=myentity.id)[0]
except:
The reason I ask is because I have nothing to put in the Except condition in this situation, but django/python requires it.
Is this the proper way to handle this situation? If it is, what should I put after the Except?
You can use the pass statement anywhere a statement is required when you want to do nothing, although you should specify the actual exception to catch.
except WhateverExceptionGetsRaised:
pass
If you are retrieving an object based on "PK" then it is only going to return one object. There isn't a need for multiple filters. Instead of:
my_object = MyObject.objects.filter(content_type__pk=generic_type.id, object_id=myentity.id)[0]
You can use:
my_object = MyObject.objects.get(content_type__pk=generic_type.id, '')
You would use this if you want it to return nothing hence the double single quotes if it fails to get the first object specified. This may be a more concise answer than an try/except pattern. Also, using:
filter()[0] vs. get()
will both return one object, but filter()[0] returns a QuerySet object type, where as get() returns the object only, so this is another thing to consider.
Relevant code:
def return_t_done(queryList):
for item in list:
if item[0] == "t_done":
return int(item[1])
raise Error("Attempted to get t_done's value, yet no t_done exists")
So basically this code runs through a nested list in the form of
[[spam,eggs],[ham,yum]]
and looks for a field ([[field,value],[field,value]]) of 't_done' and, when it finds it, returns the value associate with that field. There should always be a t_done field, but in case there isn't (this is parsing automatically generated log files using an app I didn't write and can't access the source code for, so who knows what could go wrong) I would like to elegantly raise an exception in the most appropriate way possible.
That said, this seems like the most elegant way to me, but I'm not particularly versed in Python, and my research into the docs confused me a bit. So, should I wrap the for loop in a try, except clause? Should I write a class for this error? Should I raise an exception instead of an error? Is it okay as written?
Thanks!
The way you have it is fine. But one other possibility is to use the little-known else cause for the for loop. That is evaluated if and only if the for loop completes successfully.
for item in list:
if item[0] == "t_done":
return int(item[1])
else:
raise Error(...)
Note the indentation: the else lines up with the for, not the if.