Develop Reference

Upload Turtle files by Post method in Python

I am trying to send a turtle file via a Python script using REST api to the local repository but the System is returning the following error
MALFORMED DATA: Illegal subject value: "-"^^http://www.w3.org/2001/XMLSchema#integer [line 1]
400
The used code is as follows:
import requests
url = 'http://localhost:7200/repositories/metaphactory1/statements'
with open("graph-29.ttl", "rb") as ttl_file:
file_dict = {"graph-29.ttl" : ttl_file}
headers = {
"Content-type": "application/x-turtle;charset=UTF-8",
}
r = requests.post(url, files=file_dict, headers=headers)
print(r.text)
print(r.status_code)
The same file when tried with a Curl command is working fine:
curl -X POST -H "Content-Type: application/x-turtle" -T graph-29.ttl 'http://localhost:7200/repositories/metaphactory1/statements'
Any idea regarding this issue is welcome

I think your problem come from the way you pass your file to the post request.
You should try using the data parameter of the post request. For example:
import requests
url = 'http://localhost:7200/repositories/metaphactory1/statements'
file_path = '/path/to/your/file/graph-29.ttl'
graph_name = 'http://graph-29'
headers = {
'Content-type': 'application/x-turtle',
}
params = {'graph': graph_name} #optional
response = requests.post(url, headers=headers, params=params, data=open(file_path,'r', encoding='utf-8').read())

How to get temporary AWS credentials via Cognito Identity Pool

I am looking to get temporary AWS credentials through a Cognito Identity Pool to send data to a Kinesis stream through API Gateway. One thing to note is that I am trying to do this without an AWS SDK because the language it will be written in eventually does not have a compatible SDK. I am testing the solution in Python and seem to be missing something as I keep getting a 400 error. Any ideas of what I am doing wrong here?
import requests
url = 'https://cognito-identity.us-east-1.amazonaws.com' #cognito regional endpoint
headers = {
'X-AMZ-TARGET': 'com.amazonaws.cognito.identity.model.AWSCognitoIdentityService.GetCredentialsForIdentity',
'X-AMZ-DATE': '20151020T232759Z'
}
body = {
'IdentityId': 'us-east-1:123456789' #identity id
}
response = requests.post(url = url, data = body, headers = headers)
print(response)

I was able to resolve the issue by adding a content type header, converting the single quotes in the body to double quotes, and wrapping the body dictionary in quotes as well.
import requests
url = 'https://cognito-identity.us-east-1.amazonaws.com' #cognito regional endpoint
headers = {
"CONTENT-TYPE": "application/x-amz-json-1.1",
"X-AMZ-TARGET": "com.amazonaws.cognito.identity.model.AWSCognitoIdentityService.GetCredentialsForIdentity",
"X-AMZ-DATE": "20151020T232759Z"
}
body = '''{
"IdentityId": "123456789"
}'''
response = requests.post(url = url, data = body, headers = headers)
print(response.text)

Http Basic Auth with RFC2045-MIME variant of Base64 in python

I am currently working with and API that requires "RFC2045-MIME variant of Base64, except not limited to 76 char/line" this seems to be different from the normal basic auth used in the requests library. Curious if anyone else has come across this and been able to solve it? I imagine I will have to write a function to do this encoding and build the header manually.

Yes, you can create a function as shown below where the headers for the request are created manually.
For this implementation, you will need these variables:
API_PUBLIC_KEY,
API_SECRET_KEY,
host_url,
endpoint (where the API is supposed to hit).
import base64
import requests
def create_request():
auth_header = base64.b64encode(bytes(f'{API_PUBLIC_KEY}:{API_SECRET_KEY}'.encode('ascii'))).decode('utf-8')
headers = {
'Host': host_url,
'Authorization': f'Basic {auth_header}',
'Content-Type': 'application/json'
}
api_endpoint = f'https://{host_url}/{endpoint}'
data = {}
response = requests.request("POST", api_endpoint, headers=headers, data=data)
print(response.text.encode('utf8'))

Python web scrape data request error

I'm trying to retrieve the response json data by a web site that I call.
The site is this:
WebSite DriveNow
On this page are shown on map some data. With browser debugger I can see the end point
end point
that sends response data json.
I have use this python to try scrape the json response data:
import requests
import json
headers = {
'Host': 'api2.drive-now.com',
'X-Api-Key': 'adf51226795afbc4e7575ccc124face7'
}
r = requests.get('https://api2.drive-now.com/cities/42756?expand=full', headers=headers)
json_obj = json.loads(r.content)
but I get this error:
hostname doesn't match either of 'activityharvester.com'
How I can retrieve this data?
Thanks
I have tried to call the endpoint that show json response using Postam, and passing into Header only Host and Api-Key. The result is the json that i want. But i i try the same call into python i recive the error hostname doesn't match either of 'activityharvester.com'

I don't understand your script, nor your question. Why two requests and three headers ? Did you mean something like this ?
import requests
import json
headers = {
'User-Agent': 'Mozilla/5.0',
'X-Api-Key':'adf51226795afbc4e7575ccc124face7',
}
res = requests.get('https://api2.drive-now.com/cities/4604?expand=full', headers=headers, allow_redirects=False)
print(res.status_code, res.reason)
json_obj = json.loads(res.content)
print(json_obj)

Python request with authentication (access_token)

I am trying to use an API query in Python. From the command line I can use curl like so:
curl --header "Authorization:access_token myToken" https://website.example/id
This gives some JSON output. myToken is a hexadecimal variable that remains constant throughout.
I would like to make this call from python so that I can loop through different ids and analyze the output. Before authentication was needed I had done that with urllib2. I have also taken a look at the requests module but couldn't figure out how to authenticate with it.

The requests package has a very nice API for HTTP requests, adding a custom header works like this (source: official docs):
>>> import requests
>>> response = requests.get(
... 'https://website.example/id', headers={'Authorization': 'access_token myToken'})
If you don't want to use an external dependency, the same thing using urllib2 of the standard library looks like this (source: the missing manual):
>>> import urllib2
>>> response = urllib2.urlopen(
... urllib2.Request('https://website.example/id', headers={'Authorization': 'access_token myToken'})

I had the same problem when trying to use a token with Github.
The only syntax that has worked for me with Python 3 is:
import requests
myToken = '<token>'
myUrl = '<website>'
head = {'Authorization': 'token {}'.format(myToken)}
response = requests.get(myUrl, headers=head)

>>> import requests
>>> response = requests.get('https://website.com/id', headers={'Authorization': 'access_token myToken'})
If the above doesnt work , try this:
>>> import requests
>>> response = requests.get('https://api.buildkite.com/v2/organizations/orgName/pipelines/pipelineName/builds/1230', headers={ 'Authorization': 'Bearer <your_token>' })
>>> print response.json()

import requests
BASE_URL = 'http://localhost:8080/v3/getPlan'
token = "eyJhbGciOiJSUzI1NiIsImtpZCI6ImR"
headers = {'Authorization': "Bearer {}".format(token)}
auth_response = requests.get(BASE_URL, headers=headers)
print(auth_response.json())
Output :
{
"plans": [
{
"field": false,
"description": "plan 12",
"enabled": true
}
]
}

A lot of good answers already, but I didn't see this option yet:
If you're using requests, you could also specify a custom authentication class, similar to HTTPBasicAuth. For example:
from requests.auth import AuthBase
class TokenAuth(AuthBase):
def __init__(self, token, auth_scheme='Bearer'):
self.token = token
self.auth_scheme = auth_scheme
def __call__(self, request):
request.headers['Authorization'] = f'{self.auth_scheme} {self.token}'
return request
This could be used as follows (using the custom auth_scheme from the example):
response = requests.get(
url='https://example.com',
auth=TokenAuth(token='abcde', auth_scheme='access_token'),
)
This may look like a more complicated way to set the Request.headers attribute, but it can be advantageous if you want to support multiple types of authentication. Note this allows us to use the auth argument instead of the headers argument.

Have you tried the uncurl package (https://github.com/spulec/uncurl)? You can install it via pip, pip install uncurl. Your curl request returns:
>>> uncurl "curl --header \"Authorization:access_token myToken\" https://website.com/id"
requests.get("https://website.com/id",
headers={
"Authorization": "access_token myToken"
},
cookies={},
)

I'll add a bit hint: it seems what you pass as the key value of a header depends on your authorization type, in my case that was PRIVATE-TOKEN
header = {'PRIVATE-TOKEN': 'my_token'}
response = requests.get(myUrl, headers=header)

One of the option used in python to retrieve below:
import requests
token="abcd" < retrieved based>
headers = {'Authorization': "Bearer {}".format(token)}
response = requests.get(
'https://<url api>',
headers=headers,
verify="root ca certificate"
)
print(response.content)
If you get hostname mismatch error then additional SANs need to be configured in the server with the hostnames.
Hope this helps.