I am trying to translate a specific curl method into Python's requests module to upload a file to to an api. My standard method that works for non-file requests looks like this:
import requests
requestObject = requests.Session()
standard_headers = {header1:headerValue1,header2:headerValue2}
payload = {key1:value1,key2:value2}
url = 'https://myUrl.com/apiCall'
requestObject.post(url,headers=standard_headers, json=payload)
This works for non-file requests that I need to make to the API. However for file uploads, the API documentation shows a method using curl:
curl -XPOST -H 'header1' -H 'header2 'https://myUrl.com/apiCall' \
-F 'attachment=#/path/to/my/file' \
-F '_json=<-;type=application/json' << _EOF_
{
"key1":"keyValue1",
"key2":"keyValue2"
}
_EOF_
I tested the curl command and it works successfully.
My question is how do I translate that curl method using the << _EOF_ method in Python requests. One idea I had was simply to use the 'files' option in the requests module:
requestObject = requests.Session()
standard_headers = {header1:headerValue1,header2:headerValue2}
payload = {key1:keyValue1,key2:keyValue2}
url = 'https://myUrl.com/apiCall'
file_to_upload = {'filename': open('/path/to/my/file', 'rb')}
requestObject.post(url,headers=standard_headers, files=file_to_upload, json=payload)
But that does not seem to work as the necessary json parameters (the values in payload) do not appear to get passed to the file upload
I also tried specifying the json parameters directly into the file_to_upload variable:
requestObject = requests.Session()
standard_headers = {header1:headerValue1,header2:headerValue2}
url = 'https://myUrl.com/apiCall'
file_to_upload = {'attachment': open('/path/to/my/file', 'rb'),'{"key1":"keyValue1","key2":"keyValue2"}'}
requestObject.post(url,headers=standard_headers, files=file_to_upload)
Similar result, it seems as though I am not passing the necessary json values correctly. I tried a few other ways but I am overlooking something. Any insight into how I should structure my request is appreciated.
Ok I managed to get it to work and posting for anyone who might need help in the future.
The trick was to include the _json key in the data field. My code ended up looking like so:
import requests
requestObject = requests.Session()
standard_headers = {header1:headerValue1,header2:headerValue2}
json_fields = json.dumps({
"key1": "key1Value",
"key2": "key2Value"
})
payload = {"_json":json_fields)
file = {"attachment": /path/to/my/file}
url = 'https://myUrl.com/apiCall'
requestObject.post(url,headers=standard_headers, files=file, data=payload)
Hope that helps some future person.
Related
Can you please help me convert the following CURL command into a command for Python's requests library?
curl -F pdb_file[pathvar]=#/path/myfile.pdb -X POST https://proteins.plus/api/pdb_files_rest -H "Accept: application/json"
I've tried many things, like this:
import requests
file = open("3w32.pdb", "rb")
url = "https://proteins.plus/api/pdb_files_rest"
response = requests.post(url, data=file)
response.json()
or this:
import requests
file = open("3w32.pdb", "rb")
url = "https://proteins.plus/api/pdb_files_rest"
response = requests.post(url, data={"pdb_file[pathvar]": file}, json={"Accept": "application/json"})
response.json()
but I can't make it work. I either get an error from the server, or a JSONDecodeError in Python.
Try this answer https://stackoverflow.com/a/22567429/4252013
Instead of using "data" argument try with "file"
Ok, I managed to solve it at last.
The problem was that the 'pdb_file[pathvar]' part in the curl command should've been added to the request, like this:
import requests
file = open("3w32.pdb", "rb")
url = "https://proteins.plus/api/pdb_files_rest"
response = requests.post(url, files={'pdb_file[pathvar]':file})
response.json()
Now it works!
The problem in your code is that you have only opened the file and passed the pointer. you can pass the file contents by having file.read() in place.
Please refer this block of code:
import requests
file = open("3w32.pdb", "rb")
url = "https://proteins.plus/api/pdb_files_rest"
response = requests.post(url, data=file.read())
response.json()
I am trying to send a turtle file via a Python script using REST api to the local repository but the System is returning the following error
MALFORMED DATA: Illegal subject value: "-"^^http://www.w3.org/2001/XMLSchema#integer [line 1]
400
The used code is as follows:
import requests
url = 'http://localhost:7200/repositories/metaphactory1/statements'
with open("graph-29.ttl", "rb") as ttl_file:
file_dict = {"graph-29.ttl" : ttl_file}
headers = {
"Content-type": "application/x-turtle;charset=UTF-8",
}
r = requests.post(url, files=file_dict, headers=headers)
print(r.text)
print(r.status_code)
The same file when tried with a Curl command is working fine:
curl -X POST -H "Content-Type: application/x-turtle" -T graph-29.ttl 'http://localhost:7200/repositories/metaphactory1/statements'
Any idea regarding this issue is welcome
I think your problem come from the way you pass your file to the post request.
You should try using the data parameter of the post request. For example:
import requests
url = 'http://localhost:7200/repositories/metaphactory1/statements'
file_path = '/path/to/your/file/graph-29.ttl'
graph_name = 'http://graph-29'
headers = {
'Content-type': 'application/x-turtle',
}
params = {'graph': graph_name} #optional
response = requests.post(url, headers=headers, params=params, data=open(file_path,'r', encoding='utf-8').read())
Using curl I get the hash
$ curl "http://127.0.0.1:8002/hash/" -X POST -d 'data={"key":"value"}'
{
"hash": "9724c1e20e6e3e4d7f57ed25f9d4efb006e508590d528c90da597f6a775c13e5"
}
Using Python requests return 400
import json
import requests
r = requests.post('http://127.0.0.1:8002/hash/', data = 'data={"key":"value"}', allow_redirects=False)
Try passing it in with just {"key":"value"} as your data.
So that would be:
r = requests.post('http://127.0.0.1:8002/hash/', data = '{"key":"value"}', allow_redirects=False)
requests can also accept a dictionary object or a list of tuples as it's data argument, which is probably easier and more intuitive than directly entering a string. Also note that you can work with a dictionary and turn it into a json string with json.dumps.
I am trying to use an API query in Python. From the command line I can use curl like so:
curl --header "Authorization:access_token myToken" https://website.example/id
This gives some JSON output. myToken is a hexadecimal variable that remains constant throughout.
I would like to make this call from python so that I can loop through different ids and analyze the output. Before authentication was needed I had done that with urllib2. I have also taken a look at the requests module but couldn't figure out how to authenticate with it.
The requests package has a very nice API for HTTP requests, adding a custom header works like this (source: official docs):
>>> import requests
>>> response = requests.get(
... 'https://website.example/id', headers={'Authorization': 'access_token myToken'})
If you don't want to use an external dependency, the same thing using urllib2 of the standard library looks like this (source: the missing manual):
>>> import urllib2
>>> response = urllib2.urlopen(
... urllib2.Request('https://website.example/id', headers={'Authorization': 'access_token myToken'})
I had the same problem when trying to use a token with Github.
The only syntax that has worked for me with Python 3 is:
import requests
myToken = '<token>'
myUrl = '<website>'
head = {'Authorization': 'token {}'.format(myToken)}
response = requests.get(myUrl, headers=head)
>>> import requests
>>> response = requests.get('https://website.com/id', headers={'Authorization': 'access_token myToken'})
If the above doesnt work , try this:
>>> import requests
>>> response = requests.get('https://api.buildkite.com/v2/organizations/orgName/pipelines/pipelineName/builds/1230', headers={ 'Authorization': 'Bearer <your_token>' })
>>> print response.json()
import requests
BASE_URL = 'http://localhost:8080/v3/getPlan'
token = "eyJhbGciOiJSUzI1NiIsImtpZCI6ImR"
headers = {'Authorization': "Bearer {}".format(token)}
auth_response = requests.get(BASE_URL, headers=headers)
print(auth_response.json())
Output :
{
"plans": [
{
"field": false,
"description": "plan 12",
"enabled": true
}
]
}
A lot of good answers already, but I didn't see this option yet:
If you're using requests, you could also specify a custom authentication class, similar to HTTPBasicAuth. For example:
from requests.auth import AuthBase
class TokenAuth(AuthBase):
def __init__(self, token, auth_scheme='Bearer'):
self.token = token
self.auth_scheme = auth_scheme
def __call__(self, request):
request.headers['Authorization'] = f'{self.auth_scheme} {self.token}'
return request
This could be used as follows (using the custom auth_scheme from the example):
response = requests.get(
url='https://example.com',
auth=TokenAuth(token='abcde', auth_scheme='access_token'),
)
This may look like a more complicated way to set the Request.headers attribute, but it can be advantageous if you want to support multiple types of authentication. Note this allows us to use the auth argument instead of the headers argument.
Have you tried the uncurl package (https://github.com/spulec/uncurl)? You can install it via pip, pip install uncurl. Your curl request returns:
>>> uncurl "curl --header \"Authorization:access_token myToken\" https://website.com/id"
requests.get("https://website.com/id",
headers={
"Authorization": "access_token myToken"
},
cookies={},
)
I'll add a bit hint: it seems what you pass as the key value of a header depends on your authorization type, in my case that was PRIVATE-TOKEN
header = {'PRIVATE-TOKEN': 'my_token'}
response = requests.get(myUrl, headers=header)
One of the option used in python to retrieve below:
import requests
token="abcd" < retrieved based>
headers = {'Authorization': "Bearer {}".format(token)}
response = requests.get(
'https://<url api>',
headers=headers,
verify="root ca certificate"
)
print(response.content)
If you get hostname mismatch error then additional SANs need to be configured in the server with the hostnames.
Hope this helps.
I am trying to use the toggl api.
I use Requests instead of Urllib2 for doing my GETs en POSTS. But i am stuck.
payload = {
"project":{
"name":"Another Project",
"billable":False,
"workspace":{
"Name":"jorrebor's workspace",
"id":213272
},
"automatically_calculate_estimated_workhours":False
}
}
url = "https://www.toggl.com/api/v6/projects.json"
r = requests.post(url, data=json.dumps(payload), auth=HTTPBasicAuth('j_xxxxx#gmail.com', 'mypassword'))
Authentication seems to be fine, but the payload format probably isn't.
a curl command with the same parameters:
curl -v -u heremytoken:api_token -H "Content-type: application/json" -d "{\"project\":{\"billable\":true,\"workspace\":{\"id\":213272},\"name\":\"Another project\",\"automatically_calculate_estimated_workhours\":false}}" -X POST https://www.toggl.com/api/v6/projects.json
does work fine.
What wrong with my payload? The response is get is:
["Name can't be blank","Workspace can't be blank"]
which leads me to conclude that the authentication works but toggl cannot read my json object.
Seems like you should try setting the header to a JSON application, rather than the default format, and send a JSON object instead of the Python dict. Check it out here:
payload = {"project":{"name":"Another Project",
"billable":False,
"workspace":{"Name":"jorrebor's workspace",
"id":213272},
"automatically_calculate_estimated_workhours":False
} }
parameters_json = json.dumps(payload)
headers = {'Content-Type': 'application/json')
r = client.post(url, data=parameters_json, headers=headers)
which should allow the site to read the json object just fine.