I have written a script which is hitting Azure DevOps API and get API response from it. The script is working file on local machine but when I am trying to run the same script from Azure pipeline it is giving me error.
Traceback (most recent call last):
File "/azp/$AZP_AGENT_NAME/3/s/terra/agent/agent_queue_status.py", line 19, in <module>
data = json.loads(r)
File "/_work/_tool/Python/3.9.0/x64/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/_work/_tool/Python/3.9.0/x64/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/_work/_tool/Python/3.9.0/x64/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 3 column 1 (char 4)
##[error]Bash exited with code '1'.
This the my script:
import base64
import requests
import json
AZURE_DEVOPS_PAT = 'my_pat'
authorization = str(base64.b64encode(bytes(':' + AZURE_DEVOPS_PAT, 'ascii')), 'ascii')
headers = {
'Accept': 'application/json',
'Authorization': 'Basic ' + authorization
}
url = 'https://dev.azure.com/{MY-ORG}/(MY-Project)/_settings/agentqueues?__rt=fps&__ver=2'
r = requests.get(url, headers=headers).text
data = json.loads(r)
print(data)
I tried everything. Like changing the way of getting response but no luck.
tried:-
r = requests.get(url, headers=headers).json
-------------------------------------------
r = requests.get(url, headers=headers)
data = r.json()
Please suggest something. The code is working fine on local machine but not with pipeline on Azure agent.
Related
When I call the json() method on request response I get an error.
Any suggestions to what could be wrong here?
My code:
import requests
import bs4
url = 'https://www.reddit.com/r/AskReddit/comments/l4styp/serious_what_is_the_the_scariest_thing_that_you/'
rsp = requests.get(url)
sc = rsp.json()
print(sc)
Output:
File "c:\VS_Code1\scrape.py", line 6, in <module>
sc = rsp.json()
File "C:\Users\User\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 5 (char 5)
Your rsp is actually returning <Response [200]> which is not a JSON. If you want to read the content of the response, you can simply do:
rsp.text
What you get from the URL you posted here is HTML, and not JSON.
This does not work because the page you are fetching does not return Json but HTML source code instead.
To fetch the content of the webpage you need to replace sc = rsp.json() with sc = rsp.text
If you need this data in Json, you can look into Reddit's API: https://www.reddit.com/dev/api
I figured out that if I want to get the JSON from the URL that I've inputted I have to add .json to the end of the URL, this is for some reason (to my knowledge) unique to Reddit and a few other sites that allow it.
Hello there,
I get the following error when running the following script in Python:
import requests
r = requests.get('https://www.instagram.com/p/CJDxE7Yp5Oj/?__a=1')
data = r.json()['graphql']['shortcode_media']
C:\ProgramData\Anaconda3\envs\test\python.exe C:/Users/Solba/PycharmProjects/test/main.py
Traceback (most recent call last):
File "C:/Users/Solba/PycharmProjects/test/main.py", line 4, in
data = r.json()
File "C:\ProgramData\Anaconda3\envs\test\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\ProgramData\Anaconda3\envs\test\lib\json_init_.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\ProgramData\Anaconda3\envs\test\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\ProgramData\Anaconda3\envs\test\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Process finished with exit code 1
Python version: 3.9
PyCharm version: 2020.3.1
Anaconda version: 1.10.0
Please help. Thank u.
r.json() expects a JSON string to be returned by the API. The API should explicitly say it is responding with JSON through response headers.
In this case, the URL you are requesting is either not responding with a proper JSON or not explicitly saying it is responding with a JSON.
You can first check the response sent by the URL by:
data = r.text
print(data)
If the response can be treated as a JSON string, then you can process it with:
import json
data = json.loads(r.text)
Note:
You can also check the content-type and Accept headers to ensure the request and response are in the required datatype
The reason is because the response is not returning JSON, but instead a whole HTML page. Try r.text instead of r.json()..., and then do whatever you want from there.
If you are not sure the type of content it returns:
h = requests.head('https://www.instagram.com/p/CJDxE7Yp5Oj/?__a=1')
header = h.headers
contentType = header.get('content-type')
print(contentType)
Based on your URL, it returns text/html.
Alternatively, you can try to add a User-Agent in your request - this is to emulate the request to make it look like it comes from a browser, and not a script.
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/46.0.2490.80'
}
r = requests.get('https://www.instagram.com/p/CJDxE7Yp5Oj/?__a=1', headers=headers)
data = r.json()
I am trying to get json object, and it tells me it's expecting a value even though i define the path to the json in r.json(). Also when i do r.headers[content-type] give me text/html;charset=ISO-8859-1 ... Thank you for your time everyone
import requests
import json
session = requests.Session()
username = "------"
password = "-------"
url_cookie = 'http://ludwig.podiumdata.com:----/podium/j_spring_security_check?j_username=--&j_password=----'
url_get = 'http://ludwig.corp.podiumdata.com:----/qdc/entity/v1/getEntities?type=EXTERNAL&count=2&sortAttr=name&sortDir=ASC'
r = requests.get(url_get, auth=(username,password), verify=False)
r.json()
r.headers['content-type']
Traceback (most recent call last):
File "<ipython-input-108-61f8159bb1b5>", line 10, in <module>
r.json()
File "//anaconda3/lib/python3.7/site-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "//anaconda3/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "//anaconda3/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "//anaconda3/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
JSONDecodeError: Expecting value
Your request is receiving the response as text/html, but you want to receive application/json.
You need to include {'Accept': 'application/json'} as a header in your request.
Example: requests.get(url_get, auth=(username,password), headers={'Accept': 'application/json'}, verify=False)
Also, it looks like you aren't using the Session you created on line 3, but that isn't what is causing this error.
I'm trying to get information from the Yahoo web site with the following code:
import json, requests
response = requests.get("https://finance.yahoo.com/quote/AAPL/options?p=AAPL")
output = json.loads(response.text)
print(output)
Yet i'm getting this error message:
File "C:/Users/ziggy/.spyder-py3/untitled4.py", line 4, in <module>
output = json.loads(response.text)
File "C:\Users\ziggy\Anaconda3\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\ziggy\Anaconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\ziggy\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
JSONDecodeError: Expecting value
Really unclear what you're trying to achieve here considering the response is an HTML page.
But, ignoring that, if we're just talking about getting rid of the error, serialize the response first with json.dumps:
import json, requests
response = requests.get("https://finance.yahoo.com/quote/AAPL/options?p=AAPL")
encoded_response = json.dumps(response.text)
output = json.loads(encoded_response)
print(output)
I'm trying to implement the Microsoft Bing Speech REST API with Python and I've found some code online.
https://www.taygan.co/blog/2018/02/09/getting-started-with-speech-to-text
import json
import requests
YOUR_API_KEY = 'ENTER_YOUR_KEY_HERE'
YOUR_AUDIO_FILE = 'ENTER_PATH_TO_YOUR_AUDIO_FILE_HERE'
MODE = 'interactive'
LANG = 'en-US'
FORMAT = 'simple'
def handler():
# 1. Get an Authorization Token
token = get_token()
# 2. Perform Speech Recognition
results = get_text(token, YOUR_AUDIO_FILE)
# 3. Print Results
print(results)
def get_token():
# Return an Authorization Token by making a HTTP POST request to Cognitive Services with a valid API key.
url = 'https://api.cognitive.microsoft.com/sts/v1.0/issueToken'
headers = {
'Ocp-Apim-Subscription-Key': YOUR_API_KEY
}
r = requests.post(url, headers=headers)
token = r.content
return(token)
def get_text(token, audio):
# Request that the Bing Speech API convert the audio to text
url = 'https://speech.platform.bing.com/speech/recognition/{0}/cognitiveservices/v1?language={1}&format={2}'.format(MODE, LANG, FORMAT)
headers = {
'Accept': 'application/json',
'Ocp-Apim-Subscription-Key': YOUR_API_KEY,
'Transfer-Encoding': 'chunked',
'Content-type': 'audio/wav; codec=audio/pcm; samplerate=16000',
'Authorization': 'Bearer {0}'.format(token)
}
r = requests.post(url, headers=headers, data=stream_audio_file(audio))
results = json.loads(r.content)
return results
def stream_audio_file(speech_file, chunk_size=1024):
# Chunk audio file
with open(speech_file, 'rb') as f:
while 1:
data = f.read(1024)
if not data:
break
yield data
if __name__ == '__main__':
handler()
I've followed the above code (and changed the key and audio file name) but keep getting this:
Traceback (most recent call last):
File "/Users/emi/Desktop/proj/taygan tutorial/handler.py", line 55, in <module>
handler()
File "/Users/emi/Desktop/proj/taygan tutorial/handler.py", line 15, in handler
results = get_text(token, YOUR_AUDIO_FILE)
File "/Users/emi/Desktop/proj/taygan tutorial/handler.py", line 40, in get_text
results = json.loads(r.content)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Complete newbie so really don't understand what's going on. Thanks!
NOTE: I've also tried the code from this
GitHub Repos
but I receive many errors with this one.