I am trying to simulate a web request using python library. This is the get request query created by the browser(I have replaced the actual url)
http://myurl.asp?treg=8338033&dob=14/09/2003&sid=0.3582164869592499
My code is here.
def individual(reg,dob):
session = Session()
myurl='http://myurl.asp'
# HEAD requests ask for *just* the headers, which is all you need to grab the
# session cookie
session.head('http://myurl')
response = session.get(
url=myurl,
data={
'treg': reg,
'dob': dob,
'sid':'0.4443253265244038'
},
headers={
'Referer': 'http://myurl.htm'
}
)
return response.text
It gives me invalid date response from server. But the same values sent through browser is successful. I have already tried yyyy-mm-dd format.
You are mixing up GET-style params with POST ones by using data instead of params in the request session.get call.
Python Request Post with param data
Related
I'm working on an API for my application to send a POST request to an external API. So for example, if my app hits endpoint /myapp/api I want it to call out to an external API and grab some data. In this case, I need this to be a POST request because in order to get the data that I want, I need to pass in some values.
I have made this POST call directly to the external API via Postman successfully many times, and I'm now trying to reproduce that in my python code.
In my views.py, I have this:
class MyAPPViews(APIView):
def post(self,request):
crt = 'path/to/crtfile'
key = 'path/to/keyfile'
#here I tried to reproduce the headers from the postman call
#as best as I could just to be extra specific
headers = {
'Content-Type': 'application/json',
'Accept': '/*/',
'Accept-Encoding': 'gzip,deflate,br',
'Connection': 'keep-alive'
}
payload = {
"unitList": [
"CFQU319303"
]
}
res = requests.post('external/api/url', headers=headers, data=payload, cert=(crt,key))
print('this is the true http resonse: ',res.status_code)
data = res.json()
return Response({"status": "success", "data": data})
in my urls.py I have this
path('externalAPI',MyAPPViews.as_view()),
Now, when I try to go into postman and hit http://127.0.0.1:8000/myapp/externalAPI
I get a 500 status code returned and the JSONDecodeError of Expecting value: line 1 column 1 (char 0). I understand that this means that basically the json response was empty and there was nothing for it to parse.
Is there anything blatantly wrong with my views.py code?
EDIT Within the res = requests.post('external/api/url', headers=headers, data=payload, cert=(crt,key)), I had to change data=payload to data=json.dumps(payload). I'm assuming it didn't like the way I formatted my payload initially. After that, I also deleted everything but the Content-Type: application/json out my headers dictionary.
I'd say, it's just a wrong URL in your call of the external API, as you are using some path without a domain.
So instead of
res = requests.post('external/api/url', headers=headers, data=payload, cert=(crt,key))
it should be sth. like
res = requests.post('https://SOMEDOMAIN.ORG/external/api/url', headers=headers, data=payload, cert=(crt,key))
Within the res = requests.post('external/api/url', headers=headers, data=payload, cert=(crt,key)), I had to change data=payload to data=json.dumps(payload). I'm assuming it didn't like the way I formatted my payload initially. After that, I also deleted everything but the Content-Type: application/json out my headers dictionary.
def queue_song(session_id):
song_uri='spotify:track:5RwV8BvLfX5injfqYodke9'
tokens = get_user_tokens(session_id)
headers = {'Content-Type': 'application/json',
'Authorization': "Bearer " + tokens.access_token,
}
url = BASE_URL +'player/queue'
data={
'uri':song_uri
}
response = requests.post(url,headers=headers,data=data).json()
print(response)
Output:
{'error': {'status': 400, 'message': 'Required parameter uri missing'}}
https://developer.spotify.com/documentation/web-api/reference/#/operations/add-to-queue
I dont thing there is any problem with auth tokens... coz 'GET' requests are working fine
By default, using data= in requests.post() sets the content type to application/x-www-form-urlencoded which makes the body a akin to a HTTP form request.
Spotify's API is JSON based, so your data needs to be a valid json data.
You can do it in 2 ways:
response = requests.post(url,headers=headers,data=json.dumps(data)).json()
Or, more simply:
response = requests.post(url,headers=headers,json=data).json()
and in this way you don't need to manually set the application/json header as well.
Edit:
After going through the API docs you linked, there's more wrong with the call you're making.
You're sending the parameters in data - which is the body of the request. But Spotify API specifies the parameters need to be put in the Query i.e. the query string of the URI. Which means your request should be:
response = requests.post(url,headers=headers,params=data).json() # set query string not body
I have simulated successfully API calls with my postman client.
Postman client automatically populates the required JSESSIONID, COOKIE, and CLIENT ID.
However, when I try to realize the same with python I get 401. It looks like the session variable of the request API does not hold the required information
I would like to replicate the entire step below with both Postman generated python script and my script
Postman script
In the first POST call, I use my secretKey to request the secret
import requests
url = "https://url.com/v1/session/auth/token?X-Requested-By=Maddy&Content-Type=application/x-www-form-urlencoded"
payload="secret_key=zxcvfc-103f-950d-856d-cxvfdgh&username=myuser001&access_level=FULL"
headers = {
'X-Requested-By': 'Maddy', #This is my header
'Content-Type': 'application/x-www-form-urlencoded', #My header
'Cookie': 'route=e7esaafb42ce234234242347482341f; clientId= zxcvfc-103f-950d-856d-cxvfdg; JSESSIONID=e34cc53d-e99c-4296-a4fe-0d70a246bd11' #Postman generated
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
I get a secret JHNXZasdasdasdasjdakdasdjdalsdjasdladasdlalsdjajsdjaskdjaksjdaskjdjaskdljasj very long
then the second GET call is done with the secret I have received.
import requests
url = "https://url.com/callosum/v1/session/login/token?username=myuser001&auth_token=JHNXZasdasdasdasjdakdasdjdalsdjasdladasdlalsdjajsdjaskdjaksjdaskjdjaskdljasj&redirect_url=https://url.com/callosum/v1/tspublic/v1/user/list"
payload={}
headers = {
'Cookie': 'route=e78as0dad9009q029420349249f; clientId=s0255-6598-5103-dec3-defererer090; JSESSIONID=dfgas8484-659595656-6526-5626-7589898ads'
}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
#The response contains JSON output with a list of users
Not the above header junk that contains id and session and cookie is generated by the postman
Now this is how my Python script looks like
url = "https://url.com/callosum/v1/session/auth/token?X-Requested-By=Maddy&Content-Type=application/x-www-form-urlencoded"
payload="secret_key=zxcvfc-103f-950d-856d-cxvfdgh&username=myuser001&access_level=FULL"
headers = {
'X-Requested-By': 'Maddy', #These one is mine
'Content-Type': 'application/x-www-form-urlencoded' #these one is mine
}
with requests.session() as s:
secret = s.post(url, data=payload, headers=headers,verify=False).text
#secret contain the lengthy secret for the get call stored in the variable secret
payload1 = {}
#The url2 contains login url and redirect url that contains the user API for get method
url2 = "https://url.com/callosum/v1/session/login/token?username=myuser001&auth_token={}&redirect_url=https://url.com/callosum/v1/tspublic/v1/user/list".format(secret)
r = s.get(url2,data=payload1,verify =False)
#Also tried without payload and with and without header results are the same
print(r.cookies)
print(r.text) #401 unauthroized
I get the secret but not the data. Do let me know if there is something that needs to be added.
Best Regards,
Gabby
I can't figure out how to call this api correctly using python urllib or requests.
Let me give you the code I have now:
import requests
url = "http://api.cortical.io:80/rest/expressions/similar_terms?retina_name=en_associative&start_index=0&max_results=1&sparsity=1.0&get_fingerprint=false"
params = {"positions":[0,6,7,29]}
headers = { "api-key" : key,
"Content-Type" : "application/json"}
# Make a get request with the parameters.
response = requests.get(url, params=params, headers=headers)
# Print the content of the response
print(response.content)
I've even added in the rest of the parameters to the params variable:
url = 'http://api.cortical.io:80/rest/expressions/similar_terms?'
params = {
"retina_name":"en_associative",
"start_index":0,
"max_results":1,
"sparsity":1.0,
"get_fingerprint":False,
"positions":[0,6,7,29]}
I get this message back:
An internal server error has been logged # Sun Apr 01 00:03:02 UTC
2018
So I'm not sure what I'm doing wrong. You can test out their api here, but even with testing I can't figure it out. If I go out to http://api.cortical.io/, click on the Expression tab, click on the POST /expressions/similar_terms option then paste {"positions":[0,6,7,29]} in the body textbox and hit the button, it'll give you a valid response, so nothing is wrong with their API.
I don't know what I'm doing wrong. can you help me?
The problem is that you're mixing query string parameters and post data in your params dictionary.
Instead, you should use the params parameter for your query string data, and the json parameter (since the content type is json) for your post body data.
When using the json parameter, the Content-Type header is set to 'application/json' by default. Also, when the response is json you can use the .json() method to get a dictionary.
An example,
import requests
url = 'http://api.cortical.io:80/rest/expressions/similar_terms?'
params = {
"retina_name":"en_associative",
"start_index":0,
"max_results":1,
"sparsity":1.0,
"get_fingerprint":False
}
data = {"positions":[0,6,7,29]}
r = requests.post(url, params=params, json=data)
print(r.status_code)
print(r.json())
200
[{'term': 'headphones', 'df': 8.991197733061748e-05, 'score': 4.0, 'pos_types': ['NOUN'], 'fingerprint': {'positions': []}}]
So, I can't speak to why there's a server error in a third-party API, but I followed your suggestion to try using the API UI directly, and noticed you're using a totally different endpoint than the one you're trying to call in your code. In your code you GET from http://api.cortical.io:80/rest/expressions/similar_terms but in the UI you POST to http://api.cortical.io/rest/expressions/similar_terms/bulk. It's apples and oranges.
Calling the endpoint you mention in the UI call works for me, using the following variation on your code, which requires using requests.post, and as was also pointed out by t.m. adam, the json parameter for the payload, which also needs to be wrapped in a list:
import requests
url = "http://api.cortical.io/rest/expressions/similar_terms/bulk?retina_name=en_associative&start_index=0&max_results=1&sparsity=1.0&get_fingerprint=false"
params = [{"positions":[0,6,7,29]}]
headers = { "api-key" : key,
"Content-Type" : "application/json"}
# Make a get request with the parameters.
response = requests.post(url, json=params, headers=headers)
# Print the content of the response
print(response.content)
Gives:
b'[[{"term":"headphones","df":8.991197733061748E-5,"score":4.0,"pos_types":["NOUN"],"fingerprint":{"positions":[]}}]]'
I'm trying to generate an http(s) request via Python Requests however I'm running into a problem:
No connection adapters were found for 'set(['http://www.example.com/target'])'
I'm attempting to make the connection like so:
url = ['http://www.example.com/target']
headers = {"Content-Type":"application/json"}
ver = False;
message = { bunch of json data }
curl_request(url,message,headers,ver)
With the curl_request API call being:
# API call to perform curl request and return resulting json status
def curl_request(url,message,headers,ver):
requests.post(url, data=json.dumps(message), verify=ver, headers=headers)
data = response.json()
return data
Use a string, not a set.
url = 'http://www.example.com/target'