I am trying to pull data from fulcrum app using following code
import requests
url = "https://api.fulcrumapp.com/api/v2/query"
querystring = {"q":"SELECT * FROM \"Inspection Report\"","format":"json","headers":"true","metadata":"false","arrays":"false","page":"1","per_page":"200000"}
headers = {
"Accept": "application/json",
"X-ApiToken": "*************"
}
response = requests.request("GET", url, headers=headers, params=querystring)
#print(response.text)
item_dict = json.loads(response.text)
len(item_dict['rows'])
But I get following error
>>> item_dict = json.loads(response.text)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 80459 column 1 (char 200665199)
I am expecting to have more than 100K records in the response.text. But just to validate it I am trying to get a length here. If i run query with limit SELECT * FROM \"Inspection Report\" limit 3 it produces output without any error.
Update : Tried this to see what is in the data.
response.text[200665180:200665210]
'n_no":null},\n\n!!!!!!!!\n{"messa'
Related
I'm using google api in python. My purpose is find and download any image similar with my image. This my error:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\models.py", line 971, in json
return complexjson.loads(self.text, **kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\Administrator\.vscode\tempCodeRunnerFile.python", line 27, in <module>
results = response.json()["items"]
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
What was wrong with my code? I noticed response was return <Response [400]>, i guess it is problem with my code. How to fix it? Any help is appreciated. Below is the code:
import requests
import urllib.request
# Replace YOUR_API_KEY with your actual API key
api_key = "YOUR_API_KEY"
# Load the image you want to search for
# image_data = open("path/to/image.jpg", "rb").read()
image_data = open("C:\\Users\\Administrator\\Downloads\\theme\\3f559715-f407-47de-8540-1cd9e4fdc56c.jpg", "rb").read()
# Send the request to the Google Search API's "similar images" feature
response = requests.post(
"https://www.googleapis.com/customsearch/v1/similar?",
params={
"key": api_key,
# "cx": "YOUR_CX_VALUE", # Replace YOUR_CX_VALUE with your actual CX value
"cx": "d47119ff66e004d6b",
},
headers={
"Content-Type": "image/jpeg",
},
data=image_data,
)
# print(response) -><Response [400]>
# Get the search results from the response
results = response.json()["items"]
# Iterate through the search results and download the images
for result in results:
image_url = result["link"]
image_filename = image_url.split("/")[-1]
urllib.request.urlretrieve(image_url, image_filename)
print(f"Downloaded image: {image_filename}")
I wrote the following code in PyCharm to login to a website automatically:
import requests
import json
def Login(username, password):
s = requests.Session()
payload = {
'username': username,
'password': password
}
res = s.post('https://domainNamePostUrl', json=payload)
s.headers.update({'authorization': json.loads(res.content)['token']})
print(res.content)
return s
Session = Login('realUserName', 'realPassword')
Session = Login('realUserName', 'realPassword') <-- This is line 17
When I run it, I get the following error:
/Users/PycharmProjects/pythonProject4/venv/bin/python /Users/PycharmProjects/pythonProject5/main.py
Traceback (most recent call last):
File "/Users/PycharmProjects/pythonProject5/main.py", line 17, in <module>
Session = Login('realUsername', 'realPassword')
File "/Users/PycharmProjects/pythonProject5/main.py", line 12, in Login
s.headers.update({'authorization': json.loads(res.content)['token']})
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Process finished with exit code 1
Can someone tell me where am I going wrong and where is the issue coming from?
I am working on a project where I have to scrape data from a website.I get 200 response while i run the code without json. But i am facing raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) while displaying the json response.
here is my code:
import requests
import json
PARAMS = {"customCrawlParam": True, "categoryId": -11, "pageNumber": 1, "pageSize": 60, "crawlerInfo": "0aoAfanUmycYY9dVkR6C_tawcS4PTcat2tyeioefYJvTz-K_x_HTVGslqTelRkU5hNZUrTG6C2zGn-abo8Er2xr4oH-1xPuM7nyLIAJYOdY9lYQ3tCpr2VOBopWuu2iTCrAWW-nJ2I2nqdWxLrtzvWtwtAJkQgWNR7v6DA_Xg_8_bEjpDUezOkiknnz-17XSLeTXUzaO5EaIYv1epxQef3rsGabBJyl4TsJZGfd2Rj70huRosXUJxmjSNTFaBDX3jQ1c1WgOkF0HXXEuwCmS7_VCoWo0tYqPiJhDmsZ6VQB476mxPUQVmXm6UzmdIZ6t16Ov7wmaTlY18KoO00V9jIZJs8wx_q5s5lampf7saWD5wsX0EfnWBkSo1vMacbCNkAVKrIMVViGjrtxXmstaGZ_uDoHuOeV4_RTWMX_F-NjlT3G0XRBPEJBiW_5D6U_76LcqHApmDJ130DHuynsOrYu7_k0IGoet8SAA6wAVElfMY8-Hjc3rZLr061S7SGxHC7y0uJAy3NMHr_RWBRu4lsWFzzy0ZLlUN6S6i7eWgP"}
r = requests.get(
"https://youhui.pinduoduo.com/search/landing?catId=-11", json=PARAMS)
print(r.status_code)
js1 = r.json()
# # data1 = json.dumps(js1, indent=4)
# #data2 = json.loads(r.decode("utf-8"))
print(js1)
here is the output i get:
200
Traceback (most recent call last):
File "e:\PROJECTS\first earn\test.py", line 13, in <module>
js1 = r.json()
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Asus\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
In params I wrote True instead of JSON true . Is it causing the problem?
It's not working because the response is not a json. So you can't load it as a json. As simple as that.
Do r.text to see the response yourself.
I've looked for similar questions, but I couldn't find anything what can help me. When i execute this code (which should work) I get "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)"
TOKEN = '_EXAMPLE_TOKEN_'
def getPagedData(start,size):
url = 'http://EXAMPLE_URL'.format(start, size)
response = requests.get(url, headers={'token': TOKEN}, verify=False)
return response
def getDataInBatches():
start = 0
size = 1000
allData = []
data = getPagedData(start, size)
allData.extend(data.json()['data'])
total = data.json()['count']
for i in range(1, round(total/size) + 1):
print(allData[len(allData) - 1])
allData.extend(getPagedData(i * size, size).json()['data'])
return allData
print(getDataInBatches())
Here full traceback:
Traceback (most recent call last):
File "C:/Users/userxy/Documents/workProject/API/Get_Information.py", line 30, in <module>
print(getDataInBatches())
File "C:/Users/userxy/Documents/workProject/API/Get_Information.py", line 23, in getDataInBatches
allData.extend(data.json()['data'])
File "C:\Users\userxy\Anaconda3\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\userxy\Anaconda3\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\userxy\Anaconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\userxy\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
What can it be?
Make sure the response is in json format. The standard way to ensure the response is in json format is to add a headers={'Content-Type': 'application/json'}
But to be sure, check the documentation of an API you're accessing.
If I go for example from urlib2
import urllib.request
response = urllib.request.urlopen('http://python.org/')
html = response.read()
then it works fine.
I was suprised to see that my code
req = urllib.request.Request('https://www.tehnomanija.rs/it-shop/laptop-racunari')
response = urllib.request.urlopen(req)
the_page = response.read()
data = json.loads(the_page)
print (data)
produces error like this
data = json.loads(the_page)
File "/home/mm/anaconda3/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/home/mm/anaconda3/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/mm/anaconda3/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Except the other protocol I do not see why this would not work.
json.load() is for loading from file. You need json.loads():
data = json.loads(the_page)