I've been trying to make multiple HTTP request using threading.
First, I have a method that calls to an API, and that method takes a dict as an argument and returns a JSON object. Additionally, it works perfectly fine when I run it without threading. However, here is the code and error I get when trying to use threading.
import sys
sys.path.append(r'C:/dict/path')
import apiModule
import threading
token = 'xxxx'
apiModule = apiModule.Module(token)
urls = [{'url': 'http://www.example.com'}, {'url': 'http://www.example.com/2'}]
data = []
for element in urls:
thread = threading.Thread(target=apiModule.method(),kwargs=element)
thread.start()
data.append(thread)
Traceback (most recent call last):
File "<pyshell#72>", line 2, in <module>
thread = threading.Thread(target=apiModule.method(),kwargs=element)
File "C:/dict/path", line 54, in method
return json.loads(r.content)
File "C:\Python27\lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Any help would be appreciated!
To expand on #falsetru's reply: According to the threading documentation:
target is the callable object to be invoked by the run() method
In your example you pass target=apiModule.method(), immediately invoking the method "method" on apiModule. This raises the error you're seeing. Currently the code is not even reaching the thread.start() call.
Related
I'm trying to take the JSON from a twitter get_user query and turn it into a Python object that I can extract data from (twitter handle, location, screen name, etc.)
Here is what I created. I am not sure why it doesn't work.
api = tweepy.API(auth,parser=tweepy.parsers.JSONParser())
user = api.search_users('google.com')
t_dict = json.loads(user)
pprint(t_dict)
Error:
Traceback (most recent call last):
File "Get_User_By_URL.py", line 23, in <module>
t_dict = json.loads(user)
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
api.search_users is already returning a python object. It isn't a json string that needs to be parsed. According to tweetpy documentation search_users actually returns a list of users. So the following is possible:
for user in api.search_users('google.com'):
print user.screen_name
I am trying to prints json in readable form. I already checked previous threads and tried out.
using
JSON.stringify(response)
gives error:
NameError: name 'JSON' is not defined
Using
response = json.loads(urllib.urlopen(url).read())
parsed = json.loads(response)
print json.dumps(parsed, indent=4, sort_keys=True)
gives error:
Traceback (most recent call last):
File "p6.py", line 15, in <module>
parsed = json.loads(response)
File "/usr/lib/python2.7/json/__init__.py", line 328, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
You get
NameError: name 'JSON' is not defined
because the first snippet is in JavaScript, not in Python.
As to the second snippet, you are calling json.loads() twice:
response = json.loads(urllib.urlopen(url).read()) # calling once
parsed = json.loads(response) # calling twice
Just call it once (and ensure that what you get from the HTTP server is actually JSON).
I'm new to python but would like to use urllib to download tweets, I'm following a tutorial instructions but get the same error every time, I print:
import urllib
import json
response = urllib.urlopen("https://twitter.com/search?q=Microsoft&src=tyah")
print json.load(response)
But everytime I get the error:
Traceback (most recent call last):
File "C:\Python27\print.py", line 4, in <module>
print json.load(response)
File "C:\Python27\Lib\json\__init__.py", line 278, in load
**kw)
File "C:\Python27\Lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\Lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\Lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
As noted in comments, the answer is: nothing is wrong with your code, per se.
The problem is that when json.load looks at response, it does not find JSON in there - it is finding HTML.
You need to pass a file-like object containing JSON into the json.load function, or it will raise the exception you see here.
To get JSON from Twitter, you need to call a URL that gives a JSON response. I can tell you now, that none of the Web interface URLs do this directly. You should use the Twitter API.
However, purely for sake of demonstration, if you deconstruct the page at the URL you are calling now, you will find that to load the tweet data, the page makes the following request:
https://twitter.com/i/search/timeline?q=Microsoft&src=tyah&composed_count=0&include_available_features=1&include_entities=1
And this URL does return JSON in response, which would work just fine with your current code.
Of course, I'm pretty sure doing so violates some sort of Twitter TOS, so if you do this there are all sorts of potential negative repercussions to consider. Plus it's just not good sportsmanship. :)
Why do I get
ValueError: No JSON object could be decoded
from this code:
import urllib.request,json
n = urllib.request.urlopen("http://graph.facebook.com/55")
d = json.loads(str(n.readall()))
The full error:
Traceback (most recent call last):
File "<pyshell#41>", line 1, in <module>
d= json.loads(str(n.readall()))
File "C:\Python33\lib\json\__init__.py", line 309, in loads
return _default_decoder.decode(s)
File "C:\Python33\lib\json\decoder.py", line 352, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python33\lib\json\decoder.py", line 370, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
The output of str(n.readall()):
'b\'{"id":"55","name":"Jillian Copeland","first_name":"Jillian","last_name":"Copeland","username":"JCoMD","gender":"female","locale":"en_US"}\''
Maybe the b is throwing it off?
If that is the issue, how do I convert the binary stream from the readall to a string and not have that b?
I am trying to learn a little python so please keep that in mind.
I am using Python 3.3 in Windows.
I believe that this is an exact duplicate of this question, but sadly there's no accepted answer.
On my end, this works:
import urllib.request,json
n = urllib.request.urlopen("http://graph.facebook.com/55")
d= json.loads(n.readall().decode('utf-8'))
So I'm working on a django project which uses a celery task queue to make HTTP requests.
In my celery task code I have:
json.loads('{"content-type": "application/json"}')
print test.headers
json.loads(test.headers)
Which results in:
[2012-07-19 17:02:38,536: WARNING/PoolWorker-4] '{"content-type": "application/json"}'
[2012-07-19 17:02:38,569: ERROR/MainProcess] Task core.tasks.test_run[f304bcdd-72b3-4dd5-9abb-927dc29e7f65] raised exception: ValueError('No JSON object could be decoded',)
Traceback (most recent call last):
File "/usr/local/bin/lib/python2.7/site-packages/celery/task/trace.py", line 212, in trace_task
R = retval = fun(*args, **kwargs)
File "/opt/ironman_deploy/Ironman/core/tasks.py", line 18, in test_run
json.loads(test.headers)
File "/usr/local/bin/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/local/bin/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/bin/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
No JSON object could be decoded: No JSON object could be decoded
I have literally no idea what's going on... clearly json can decode the string, because it doesn't fail 2 lines above, however when I pass the string in by reference it seems to choke.
Could anyone shed light on this for me?
test.headers could be a dict. If you print it, it will output something looking like JSON, yet test.headers might not be JSON at all, and decoding it would cause JSON to choke.
Whats is "test.headers" your snippet doesn't indicate this. If test.headers is being assigned to the result of the first json.loads call, then the second one will obviosuly fail since you aren't providing it a string. The second call should be json.dumps(test.headers)