I've looked for similar questions, but I couldn't find anything what can help me. When i execute this code (which should work) I get "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"
TOKEN = '_EXAMPLE_TOKEN_'
def getPagedData(start,size):
url = 'http://EXAMPLE_URL'.format(start, size)
response = requests.get(url, headers={'token': TOKEN}, verify=False)
return response
def getDataInBatches():
start = 0
size = 1000
allData = []
data = getPagedData(start, size)
allData.extend(data.json()['data'])
total = data.json()['count']
for i in range(1, round(total/size) + 1):
print(allData[len(allData) - 1])
allData.extend(getPagedData(i * size, size).json()['data'])
return allData
print(getDataInBatches())
Here full traceback:
Traceback (most recent call last):
File "C:/Users/userxy/Documents/workProject/API/Get_Information.py", line 30, in <module>
print(getDataInBatches())
File "C:/Users/userxy/Documents/workProject/API/Get_Information.py", line 23, in getDataInBatches
allData.extend(data.json()['data'])
File "C:\Users\userxy\Anaconda3\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\userxy\Anaconda3\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\userxy\Anaconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\userxy\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
What can it be?
Make sure the response is in json format. The standard way to ensure the response is in json format is to add a headers={'Content-Type': 'application/json'}
But to be sure, check the documentation of an API you're accessing.
Related
I am trying to pull data from fulcrum app using following code
import requests
url = "https://api.fulcrumapp.com/api/v2/query"
querystring = {"q":"SELECT * FROM \"Inspection Report\"","format":"json","headers":"true","metadata":"false","arrays":"false","page":"1","per_page":"200000"}
headers = {
"Accept": "application/json",
"X-ApiToken": "*************"
}
response = requests.request("GET", url, headers=headers, params=querystring)
#print(response.text)
item_dict = json.loads(response.text)
len(item_dict['rows'])
But I get following error
>>> item_dict = json.loads(response.text)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 80459 column 1 (char 200665199)
I am expecting to have more than 100K records in the response.text. But just to validate it I am trying to get a length here. If i run query with limit SELECT * FROM \"Inspection Report\" limit 3 it produces output without any error.
Update : Tried this to see what is in the data.
response.text[200665180:200665210]
'n_no":null},\n\n!!!!!!!!\n{"messa'
I am working on a project where I have to scrape data from a website.I get 200 response while i run the code without json. But i am facing raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) while displaying the json response.
here is my code:
import requests
import json
PARAMS = {"customCrawlParam": True, "categoryId": -11, "pageNumber": 1, "pageSize": 60, "crawlerInfo": "0aoAfanUmycYY9dVkR6C_tawcS4PTcat2tyeioefYJvTz-K_x_HTVGslqTelRkU5hNZUrTG6C2zGn-abo8Er2xr4oH-1xPuM7nyLIAJYOdY9lYQ3tCpr2VOBopWuu2iTCrAWW-nJ2I2nqdWxLrtzvWtwtAJkQgWNR7v6DA_Xg_8_bEjpDUezOkiknnz-17XSLeTXUzaO5EaIYv1epxQef3rsGabBJyl4TsJZGfd2Rj70huRosXUJxmjSNTFaBDX3jQ1c1WgOkF0HXXEuwCmS7_VCoWo0tYqPiJhDmsZ6VQB476mxPUQVmXm6UzmdIZ6t16Ov7wmaTlY18KoO00V9jIZJs8wx_q5s5lampf7saWD5wsX0EfnWBkSo1vMacbCNkAVKrIMVViGjrtxXmstaGZ_uDoHuOeV4_RTWMX_F-NjlT3G0XRBPEJBiW_5D6U_76LcqHApmDJ130DHuynsOrYu7_k0IGoet8SAA6wAVElfMY8-Hjc3rZLr061S7SGxHC7y0uJAy3NMHr_RWBRu4lsWFzzy0ZLlUN6S6i7eWgP"}
r = requests.get(
"https://youhui.pinduoduo.com/search/landing?catId=-11", json=PARAMS)
print(r.status_code)
js1 = r.json()
# # data1 = json.dumps(js1, indent=4)
# #data2 = json.loads(r.decode("utf-8"))
print(js1)
here is the output i get:
200
Traceback (most recent call last):
File "e:\PROJECTS\first earn\test.py", line 13, in <module>
js1 = r.json()
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
In params I wrote True instead of JSON true . Is it causing the problem?
It's not working because the response is not a json. So you can't load it as a json. As simple as that.
Do r.text to see the response yourself.
I tried many different ways to fix this error, but it failed.
Could you fix this error?
p.s.
formatted_url --> There's a problem that comma turns %2c when it calls request, and this way works.
https://opendart.fss.or.kr/api/fnlttMultiAcnt.json?crtfc_key=cd155118bef391723b04f8879878921476a97b9f&corp_code=01015160,00288495,00365590,01035678,00625942,00287812,00220622,00173351,00186559,00476036,00899556&bsns_year=2020&reprt_code=11013
request post --> params text is really long, and request get method occurs 414 error, so I changed it to post.
print(resp) :
<urllib3.response.HTTPResponse object at 0x7fb822c79ee0>
params.items() :
dict_items([('crtfc_key', 'cd155118bef391723b04f8879878921476a97b9f'), ('corp_code', '00260985,...........00173351,00186559,00476036,00899556'), ('bsns_year', '2020'), ('reprt_code', '11013')])
error message
enter image description here
def convertFnltt(url, items, item_names, params):
formatted_url = url + '?' + '&'.join(["{}={}".format(k,v) for k,v in params.items()])
print(formatted_url)
http = urllib3.PoolManager(ca_certs=certifi.where())
resp = http.request('POST',formatted_url)
print(resp)
print('1')
json_dict = json.load(resp) <--here!!
print('2')
data = []
#if self.get_status(json_dict['status']):
print(json_dict['status'])
if json_dict['status']=='000':
for line in json_dict['list']:
data.append([])
for itm in items:
if itm in line.keys():
data[-1].append(line[itm])
else:
data[-1].append('')
else:
return json_dict['status']
df = pd.DataFrame(data)
df.columns = item_names
return df #df.to_excel('data.xlsx')
1
Traceback (most recent call last):
File "main.py", line 106, in <module>
print(get_fnlttMultiAcnt(crtfc_key, get_corpcode(crtfc_key), bsns_year, reprt_code))
File "main.py", line 103, in get_fnlttMultiAcnt
return convertFnltt(url,items,item_names,params)
File "main.py", line 61, in convertFnltt
json_dict = json.load(resp)
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
It was the issue of the format of 'params'.
I used 'append' to make 'params', which caused a series of problems.
I changed 'append' to 'extend' , and it worked.
res = requests.get(url, params) <---'params format issue'
json_dict = json.loads(res.text)
data = []
for line in json_dict["list"]:
data.append([])
for itm in items:
if itm in line.keys():
data[-1].append(line[itm])
else:
data[-1].append('')
dfp = pd.DataFrame(data, columns = item_names)
I am getting error, when I am trying to run my program with the API. Like that
Traceback (most recent call last):
File "/Users/hanson/Desktop/LEARNING/Python/model-test/simple-keras-rest-api-master/simple_request.py", line 17, in <module>
r = requests.post(KERAS_REST_API_URL, files=payload).json()
File "/Users/hanson/ICP6/lib/python3.6/site-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Here is my source code of the pyfile
import requests
# initialize the Keras REST API endpoint URL along with the input
# image path
KERAS_REST_API_URL = "http://localhost:5000/predict"
IMAGE_PATH = "/Users/hanson/Desktop/LEARNING/Python/model-test/simple-keras-rest-api-master/dog.jpg"
# load the input image and construct the payload for the request
image = open(IMAGE_PATH, "rb").read()
payload = {"image": image}
# submit the request
r = requests.post(KERAS_REST_API_URL, files=payload).json()
# ensure the request was sucessful
if r["success"]:
# loop over the predictions and display them
for (i, result) in enumerate(r["predictions"]):
print("{}. {}: {:.4f}".format(i + 1, result["label"],
result["probability"]))
# otherwise, the request failed
else:
print("Request failed")
The API as requested should be something like that
beagle: 0.9901
Walker_hound: 0.0024
pot: 0.0014
Brittany_spaniel: 0.0013
bluetick: 0.0011
I am uploading a file to a REST API with a Python 3 script via the windows command line. The filename is an argument passed in to the script. Everything works fine unless there is a space in the path name. i.e. c:\temp\myFolder\1.jpg works, but c:\temp\my Folder\1.jpg throws an error:
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The code up until the error is as follows:
def upload_photo(filename):
f = open(filename, "rb")
data = f.read()
data_md5 = hashlib.md5(data).hexdigest()
f.close()
r = requests.put('https://upload.mysite.com/{}'.format(filename), data=data)
response = json.loads(r.text)
I'm not sure how to fix it. Thanks for the help.
--Edit--
Full traceback
c:\>python test.py "c:/temp/my folder/1.jpg"
Traceback (most recent call last):
File "test.py", line 144, in <module>
print(test(sys.argv[1]))
File "test.py", line 132, in upload_photo
response = json.loads(r.text)
File "C:\Users\Default\AppData\Local\Programs\Python\Python37-32\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\Default\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Default\Default\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
You need to encode the link properly; escaping the invalid character(s). You can do that with urllib.
>>> import urllib
>>> filename = urllib.parse.quote('c:\temp\my Folder\1.jpg')
>>> 'https://upload.mysite.com/' + filename
'https://upload.mysite.com/c%3A%09emp%5Cmy%20Folder%01.jpg'