I am working on a project where I have to scrape data from a website.I get 200 response while i run the code without json. But i am facing raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) while displaying the json response.
here is my code:
import requests
import json
PARAMS = {"customCrawlParam": True, "categoryId": -11, "pageNumber": 1, "pageSize": 60, "crawlerInfo": "0aoAfanUmycYY9dVkR6C_tawcS4PTcat2tyeioefYJvTz-K_x_HTVGslqTelRkU5hNZUrTG6C2zGn-abo8Er2xr4oH-1xPuM7nyLIAJYOdY9lYQ3tCpr2VOBopWuu2iTCrAWW-nJ2I2nqdWxLrtzvWtwtAJkQgWNR7v6DA_Xg_8_bEjpDUezOkiknnz-17XSLeTXUzaO5EaIYv1epxQef3rsGabBJyl4TsJZGfd2Rj70huRosXUJxmjSNTFaBDX3jQ1c1WgOkF0HXXEuwCmS7_VCoWo0tYqPiJhDmsZ6VQB476mxPUQVmXm6UzmdIZ6t16Ov7wmaTlY18KoO00V9jIZJs8wx_q5s5lampf7saWD5wsX0EfnWBkSo1vMacbCNkAVKrIMVViGjrtxXmstaGZ_uDoHuOeV4_RTWMX_F-NjlT3G0XRBPEJBiW_5D6U_76LcqHApmDJ130DHuynsOrYu7_k0IGoet8SAA6wAVElfMY8-Hjc3rZLr061S7SGxHC7y0uJAy3NMHr_RWBRu4lsWFzzy0ZLlUN6S6i7eWgP"}
r = requests.get(
"https://youhui.pinduoduo.com/search/landing?catId=-11", json=PARAMS)
print(r.status_code)
js1 = r.json()
# # data1 = json.dumps(js1, indent=4)
# #data2 = json.loads(r.decode("utf-8"))
print(js1)
here is the output i get:
200
Traceback (most recent call last):
File "e:\PROJECTS\first earn\test.py", line 13, in <module>
js1 = r.json()
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
In params I wrote True instead of JSON true . Is it causing the problem?
It's not working because the response is not a json. So you can't load it as a json. As simple as that.
Do r.text to see the response yourself.
Related
I am trying to pull data from fulcrum app using following code
import requests
url = "https://api.fulcrumapp.com/api/v2/query"
querystring = {"q":"SELECT * FROM \"Inspection Report\"","format":"json","headers":"true","metadata":"false","arrays":"false","page":"1","per_page":"200000"}
headers = {
"Accept": "application/json",
"X-ApiToken": "*************"
}
response = requests.request("GET", url, headers=headers, params=querystring)
#print(response.text)
item_dict = json.loads(response.text)
len(item_dict['rows'])
But I get following error
>>> item_dict = json.loads(response.text)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 80459 column 1 (char 200665199)
I am expecting to have more than 100K records in the response.text. But just to validate it I am trying to get a length here. If i run query with limit SELECT * FROM \"Inspection Report\" limit 3 it produces output without any error.
Update : Tried this to see what is in the data.
response.text[200665180:200665210]
'n_no":null},\n\n!!!!!!!!\n{"messa'
I've looked for similar questions, but I couldn't find anything what can help me. When i execute this code (which should work) I get "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"
TOKEN = '_EXAMPLE_TOKEN_'
def getPagedData(start,size):
url = 'http://EXAMPLE_URL'.format(start, size)
response = requests.get(url, headers={'token': TOKEN}, verify=False)
return response
def getDataInBatches():
start = 0
size = 1000
allData = []
data = getPagedData(start, size)
allData.extend(data.json()['data'])
total = data.json()['count']
for i in range(1, round(total/size) + 1):
print(allData[len(allData) - 1])
allData.extend(getPagedData(i * size, size).json()['data'])
return allData
print(getDataInBatches())
Here full traceback:
Traceback (most recent call last):
File "C:/Users/userxy/Documents/workProject/API/Get_Information.py", line 30, in <module>
print(getDataInBatches())
File "C:/Users/userxy/Documents/workProject/API/Get_Information.py", line 23, in getDataInBatches
allData.extend(data.json()['data'])
File "C:\Users\userxy\Anaconda3\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\userxy\Anaconda3\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\userxy\Anaconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\userxy\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
What can it be?
Make sure the response is in json format. The standard way to ensure the response is in json format is to add a headers={'Content-Type': 'application/json'}
But to be sure, check the documentation of an API you're accessing.
I am getting error, when I am trying to run my program with the API. Like that
Traceback (most recent call last):
File "/Users/hanson/Desktop/LEARNING/Python/model-test/simple-keras-rest-api-master/simple_request.py", line 17, in <module>
r = requests.post(KERAS_REST_API_URL, files=payload).json()
File "/Users/hanson/ICP6/lib/python3.6/site-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Here is my source code of the pyfile
import requests
# initialize the Keras REST API endpoint URL along with the input
# image path
KERAS_REST_API_URL = "http://localhost:5000/predict"
IMAGE_PATH = "/Users/hanson/Desktop/LEARNING/Python/model-test/simple-keras-rest-api-master/dog.jpg"
# load the input image and construct the payload for the request
image = open(IMAGE_PATH, "rb").read()
payload = {"image": image}
# submit the request
r = requests.post(KERAS_REST_API_URL, files=payload).json()
# ensure the request was sucessful
if r["success"]:
# loop over the predictions and display them
for (i, result) in enumerate(r["predictions"]):
print("{}. {}: {:.4f}".format(i + 1, result["label"],
result["probability"]))
# otherwise, the request failed
else:
print("Request failed")
The API as requested should be something like that
beagle: 0.9901
Walker_hound: 0.0024
pot: 0.0014
Brittany_spaniel: 0.0013
bluetick: 0.0011
I am uploading a file to a REST API with a Python 3 script via the windows command line. The filename is an argument passed in to the script. Everything works fine unless there is a space in the path name. i.e. c:\temp\myFolder\1.jpg works, but c:\temp\my Folder\1.jpg throws an error:
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The code up until the error is as follows:
def upload_photo(filename):
f = open(filename, "rb")
data = f.read()
data_md5 = hashlib.md5(data).hexdigest()
f.close()
r = requests.put('https://upload.mysite.com/{}'.format(filename), data=data)
response = json.loads(r.text)
I'm not sure how to fix it. Thanks for the help.
--Edit--
Full traceback
c:\>python test.py "c:/temp/my folder/1.jpg"
Traceback (most recent call last):
File "test.py", line 144, in <module>
print(test(sys.argv[1]))
File "test.py", line 132, in upload_photo
response = json.loads(r.text)
File "C:\Users\Default\AppData\Local\Programs\Python\Python37-32\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\Default\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Default\Default\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
You need to encode the link properly; escaping the invalid character(s). You can do that with urllib.
>>> import urllib
>>> filename = urllib.parse.quote('c:\temp\my Folder\1.jpg')
>>> 'https://upload.mysite.com/' + filename
'https://upload.mysite.com/c%3A%09emp%5Cmy%20Folder%01.jpg'
If I go for example from urlib2
import urllib.request
response = urllib.request.urlopen('http://python.org/')
html = response.read()
then it works fine.
I was suprised to see that my code
req = urllib.request.Request('https://www.tehnomanija.rs/it-shop/laptop-racunari')
response = urllib.request.urlopen(req)
the_page = response.read()
data = json.loads(the_page)
print (data)
produces error like this
data = json.loads(the_page)
File "/home/mm/anaconda3/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/home/mm/anaconda3/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/mm/anaconda3/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Except the other protocol I do not see why this would not work.
json.load() is for loading from file. You need json.loads():
data = json.loads(the_page)