Develop Reference

Checking URL Status without Throwing Error

I'm looking to check to see if 500+ strings in a given dataframe are URLs. I've seen that this can be done using the requests package but I've found that if I provide a URL, instead of receiving the error code 404, my program is crashing.
Because I'm looking to apply this function to a dataframe with many strings not being active URLs, the current function would not work for what I'm looking to accomplish.
I'm wondering if there is a way to adapt the coded below to actually return no (or anything else) in the case that the URL isn't real. For example, providing the url 'http://www.example.commmm' results in an error:
import requests
response = requests.get('http://www.example.com')
if response.status_code == 200:
print('Yes')
else:
print('No')
thanks in advance!

I would try and add a try/except to prevent your code from breaking
try:
print(x)
except:
print("An exception occurred")

variable structure in json data source

thanks for your time.
I have a dataframe in pyspark in Databricks that reads json. The data from the source does not always have the same structure, sometimes the 'emailAddress' field does not appear, causing me the error "org.apache.spark.sql.AnalysisException: cannot resolve ...".
I have tried to solve by applying a Try-Except function in this way:
try:
df_json = df_json.select("responseID", "surveyID", "surveyName","timestamp", "customVariables.Id_Cliente", "timestamp", "responseSet", "emailAddress")
except ValueError:
None
But it does not work for me, it returns the same error that I mentioned.
I am even trying to take another alternative but without results:
if 'Id_Cliente' in s_fields:
try:
df_json = df_json.select("responseID", "surveyID", "surveyName","timestamp", "customVariables.Id_Cliente", "timestamp", "responseSet", "emailAddress")
except ValueError:
df_json = df_json.select("responseID", "surveyID", "surveyName","timestamp", "customVariables.Id_Cliente", "timestamp", "responseSet")
Please help me with some idea to control this situation? I need to stop the execution of my notebook when it does not find the field in the structure, otherwise (it finds the emailAddress variable) to continue processing.
From already thank you very much.
Greetings.

You're catching ValueError while the exception is AnalysisException, that's why it doesn't work.
from pyspark.sql.utils import AnalysisException
try:
df.select('xyz')
except AnalysisException:
print(123)

Python try / except / else loop does both the try and else actions

I'm running a Python script to post a Tweet if the length is short enough with an exception for errors and an else statement for messages that are too long.
When I run this, it posts the Tweet and still gives the Tweet too long message. Any idea why that is happening and how to make it work as intended?
if len(tweet_text) <= (280-6):
try:
twitter = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)
twitter.update_status(status=tweet_text)
except TwythonError as error:
print(error)
else:
print("Tweet too Long. Please try again.")

The first string is checking the length of the tweet. Move the else four spaces back. Because try/except construction can be try/except/else construction

From the docs:
The try … except statement has an optional else clause, which, when present, must follow all except clauses. It is useful for code that must be executed if the try clause does not raise an exception. (emphasis added)
Spaces/Tabs matter in Python.
What your snippet says in common English
is "Try to post the tweet, unless there's an error, then print the error. If there's not an error, print 'Tweet too long. Please try again.'"
What you want is:
if len(tweet_text) <= (280-6):
try:
twitter = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)
twitter.update_status(status=tweet_text)
except TwythonError as error:
print(error)
else:
print("Tweet too Long. Please try again.")

Try and except whilst trying to writerow in Python

I have the following code that is throwing up an out of range error on the barcode looping section of the below code.
for each in data['articles']:
f.writerow([each['local']['name'],
each['information'][0]['barcodes'][0]['barcode']])
I wrote a try and except to catch and handle when a barcode is not present within the json I am parsing and this worked perfectly during testing using the print function however I have been having some trouble getting the try and except to work whilst trying to writerow to a csv file.
Does anyone have any suggestions or another method I could try to get this to work.
My try and accept which worked when testing using print was as follows:
for each in data['articles']:
print(each['local']['name'])
try:
print(each['information'][0]['barcodes'][0]['barcode'])
except:
"none"
Any help is much appreciated!

As komatiraju032 points out, one way of doing this is via get(), although if there are different elements of the dictionary that might have empty/incorrect values, it might get unwieldy to provide a default for each one. To do this via a try/except you might do:
for each in data['articles']:
row = [each['local']['name']]
try:
row.append(each['information'][0]['barcodes'][0]['barcode'])
except (IndexError, KeyError):
row.append("none")
f.writerow(row)
This will give you that "none" replacement value regardless of which of those lists/dicts is missing the requested index/key, since any of those lookups might raise but they'll all end up at the same except.

Use dict.get() method. It will return None if key not exist
res = each['information'][0]['barcodes'][0].get('barcode')

the urlib2.URLError and its reason in python

The title of the question may be a bit confusing but I don't really know how best to word it...
I've found the following chunk of code which downloads a web page from the web by making use of the urllib2 library.
import urllib2
def download(url):
try:
html = urllib2.urlopen(url).read()
except urllib2.URLError as e:
print 'Download error:', e.reason
html = None
return html
Now if it happens that e.code is 404 then e.reason is simply an empty string which means it bears absolutely no information on what triggered the error, thus I don't really understand the point of using e.reason here.
It seems like it would be more reasonable to print e instead but even if I change it to simply print e it will still yield something awkward: HTTP Error 404: and the colon is apparenty followed by an empty string...
So it appears to me that the abovementioned code is a little clumsy in terms of exception handling. Is it so?

It would seem that you could either use the error itself (print e) or the code and the reason (print "Download Error: ", e.code, e.reason) if you wanted to see the 404 code.