I am trying to adapt my HTTP request from running it in Python to R. This is the post request in Python:
import requests
import json
r = requests.post("https://feed-dev.ihsmarkit.com/apikey",
data={'username': 'markit/resellers/API_OPS/accounts/demo.dv', 'password':
'Example#N6'})
print("POST /apikey", r.status_code, r.reason)
apikey = r.text
print(apikey)
I did some research and found the httr package in R is best for dealing with API related requests. However I tried to use the POST() function in a few attempts but got the same error 400 ("MISSING_PARAMETER": Parameter username not provided.") responses. Here are a few attempts I used:
#attempt 1
response <- POST(url = "https://feed-dev.ihsmarkit.com/apikey",
add_headers(.headers = c("Content-Type"="application/x-www-form-urlencoded")),
authenticate('markit/resellers/API_OPS/accounts/demo.dv', 'Example#N6')
)
#attempt 2
request_body <- data.frame(
username = 'markit/resellers/API_OPS/accounts/demo.dv',
password = 'Example#N6'
)
request_body_json <- toJSON(list(data = request_body), auto_unbox = TRUE)
POST(url = "https://feed-dev.ihsmarkit.com/apikey",
add_headers(.headers = c("Content-Type"="application/x-www-form-urlencoded","Accept"="application/json"),
body = request_body_json))
#attempt 3
result <- POST(url = "https://feed-dev.ihsmarkit.com/apikey",
add_headers(.headers = c("Content-Type"="application/x-www-form-urlencoded","Accept"="application/json")),
body = '{"data":{"username":"markit/resellers/API_OPS/accounts/demo.dv", "password":"Example#N6}}',
encode = 'raw')
Do you know how should I properly convert my request?
Use
response <- POST(url = "https://feed-dev.ihsmarkit.com/apikey",
encode = "form",
body= list(
username ='markit/resellers/API_OPS/accounts/demo.dv',
password = 'Example#N6')
)
Just pass your data as a list and the POST will take care of formatting it as form data when you choose encode="form". Your python code doesn't seem to use JSON at all. You just have literal dictionary values where you are storing your data. Only use authenticate() when the HTTP endpoint requires basic HTTP authentication. For endpoints that require a username/password in the body of the message, that's not how basic HTTP authentication works.
Related
I am trying to fetch data from API from a tool called Paylocity.
I have managed to generate the access token and trying to connect to their API as below
client_id = client_id
client_secret = client_secret
company_id = company_id
prod_auth_url = 'https://apisandbox.paylocity.com/IdentityServer/connect/token'
body_params = urllib.parse.urlencode({'grant_type': 'client_credentials','scope':'WebLinkAPI'})
# Requests can use auth= for basic authentication
auth_response = requests.post(prod_auth_url,auth=(client_id, client_secret), data=urllib.parse.urlencode(body_params))
response = json.loads(auth_response.content)
api_call_headers = {'Authorization': 'Bearer ' + response['access']}
This however returns an error
TypeError: not a valid non-string sequence or mapping object
Figured out by changing the ** auth_response** variable as below
auth_response = requests.post(prod_auth_url,auth=(client_id, client_secret), data=body_params)
response = json.loads(auth_response.text)
auth_response.content gives you a byte string. Which likely causes the error. Use auth_response.text.
Better even: You don't need json.loads() here.
Just do:
response = auth_response.json()
I am writing a Python3 script to trigger an Axis camera via their HTTP API. The string they need in the url to trigger the Digital Input (with authorization) is:
"http://{ip}/axis-cgi/param.cgi?action=update&IOPort.I0.Input.Trig={open/closed}"
If I put this in a browser it works AKA response 200 - OK. When I run this on my Linux machine using the urllib request.
#encode user,pass
values = { 'username': username,'password': password }
data = urllib.urlencode(values)
#Axis param.cgi action
action = "update"
trig = "closed"
cgi_args = {"action": action, "IOPort.I0.Input.Trig": trig}
cgi_data = urllib.urlencode(cgi_args)
url = "http://{ip}/axis-cgi/param.cgi?{data}".format(ip=ip, data=cgi_data)
req = urllib2.Request(url, data, headers={'Content-type': 'text/html'})
print(req.get_full_url())
response = urllib2.urlopen(req)
result = response.read()
print (result)
The output is:
http://192.168.50.191/axis-cgi/param.cgi?action=update&IOPort.I0.Input.Trig=closed
action must be specified
I know that I am authenticated otherwise I get Unauthorized response from server.
As you can see for debugging I print the req.get_full_url() to make sure I've built the url string correctly, however the server responds with action must be specified which is what I get in the browser when I just have http://192.168.50.191/axis-cgi/param.cgi? in the address bar. So the action portion of the URL appears to not be making it to the server.
I've tried:
Using %3F as the ? character, but I get 404 error
Embedding the data in the data parameter does not work either
Anything I am missing here?
used pycurl with digest authorization for a GET request with auth:
def configCurl():
username = "user"
password = "pass"
auth_mode = pycurl.HTTPAUTH_DIGEST
curl = pycurl.Curl()
curl.setopt(pycurl.HTTPAUTH, auth_mode)
curl.setopt(pycurl.USERPWD, "{}:{}".format(username, password))
curl.setopt(pycurl.WRITEFUNCTION, lambda x: None)
return curl
curl = configCurl()
curl.setopt(curl.URL, url)
curl.perform()
curl.close()
and thank you for you useful help already.
I am trying to make an API call using python. Sadly, the only documentation of the API is an implementation already existing in C#.
My problem is, that after i acquire an Azure AADTokenCredential object - i simply do not know how to use it in my HTTPS request.
def get_data_from_api(credentials):
serialNumber = "123456789"
fromDate = "01/10/2019 00:00:00" # DD/m/YYYY HH:MM:SS
untilDate = "09/10/2019 00:00:00" # DD/m/YYYY HH:MM:SS
PARAMS = {
serialNumber: serialNumber,
fromDate: fromDate,
untilDate: untilDate
}
url = "https://myapi.azurewebsites.net/api/sensordata/GetBySerialNumber"
r = requests.get(url = url, header={"Authorization": credentials}, params=PARAMS)
print(r)
#data = r.json()
return data
The credentials is an msrestazure.azure_active_directory.AADTokenCredentials retrieved using the adal package.
The above code results in an error as the header object can only be strings.
My question is - How do i pass the authorization object in the correct way?
The C# implementation looks like this:
// Make a request to get the token from AAD
AuthenticationResult result = Task.Run(async () => await authContext.AcquireTokenAsync(resource, cc)).Result;
// Get the auth header which includes the token from the result
string authHeader = result.CreateAuthorizationHeader();
// ...
// Prepare a HTTP request for getting data.
// First create a client
HttpClient client = new HttpClient();
// Create the actual request. It is a GET request so pass the arguments in the url is enough
HttpRequestMessage request = new HttpRequestMessage(
HttpMethod.Get, $"https://api.azurewebsites.net/api/sensordata/GetBySerialNumber?serialNumber={serialNumber}&from={fromDate}&until={untilDate}");
// Add the required authorization header that includes the token. Without it the request will fail as unauthorized
request.Headers.TryAddWithoutValidation("Authorization", authHeader);
// Prepare the response object
HttpResponseMessage response = Task.Run(async () => await client.SendAsync(request)).Result;
So yes! I finally solved it.
My problem was that i was passing on the ADAL object to the requests phase, however what I needed to do was pass on the actual token that is retrieved using: 'credentials = context.acquire_token_with_client_credentials(resource_uri,client_id,client_secret)'.
Here credentials is a dictionary and what the requests needs in the header for authentication was:
header = {
"Authorization": "Bearer "+credentials["accessToken"]
}
r = requests.get(url=url, headers=header, params=PARAMS)
passing this on to the requests.get method worked!
I am working with a cloud-based service that is exposed using OAuth REST API.
I have a working Python script, with which I am able to first post username, password and api token to get an authentication token. Then I use the api and authentication tokens to get data. Here is my Python script:
import requests
base_url = 'https://example.com'
user_name = 'john_doe'
password = 'pssw0rd'
api_token = '4r-eueyta-dh7cuq-26'
timeout_min = 1
headers = {'Accept': 'application/json'}
# get authentication token
auth_url = base_url + '/api/authenticate'
data = {'Username': user_name,
'Password': password,
'ApiToken': api_token,
'AuthorizationTokenTimeoutMinutes': timeout_min}
r = requests.post(auth_url, headers=headers, data=data)
# the entire string response is the auth token
auth_token = r.text
# get data
proj_url = base_url + '/api/project/active?' + \
'api-token={0}&authentication-token={1}'.format(api_token, auth_token)
r = requests.get(proj_url, headers=headers)
print(r.text)
This is working fine.
Next I want to use the same approach inside Microsoft Power BI to configure this data as a data source. I followed the approach outlined at http://angryanalyticsblog.azurewebsites.net/index.php/2016/05/16/api-strategies-with-power-bi/
As mentioned in the post, I set the variables
base_url
user_name
password
api_token
auth_token_timeout_min
as parameters and set their values. Then got the following power query script in advanced editor:
let
// get auth token
auth_url = #"base_url" & "/api/authenticate",
post_data = "Username=" & #"user_name" & "&Password=" & #"password" & "&ApiToken=" & #"api_token" & "&AuthorizationTokenTimeoutMinutes=" & #"auth_token_timeout_min",
AuthJsonQuery = Web.Contents(auth_url,
[Headers = [#"Accept"="application/json"],
Content = Text.ToBinary(post_data)]),
auth_token = Json.Document(AuthJsonQuery),
// get data
proj_url = #"base_url" & "/api/project/active?api-token=" & #"api_token" & "&authentication-token=" & auth_token,
GetJsonQuery = Web.Contents(proj_url,
[Headers = [#"Accept"="application/json"]]),
FormatAsJsonQuery = Json.Document(GetJsonQuery),
in
#"FormatAsJsonQuery"
This script gives the following error:
DataSource.Error: Web.Contents failed to get contents from 'https://example.com/api/authenticate' (415): Unsupported Media Type
Details:
DataSourceKind=Web
DataSourcePath=https://example.com/api/authenticate
Url=https://example.com/api/authenticate
The raw response from the first POST request is exactly the authentication token and I am missing something in correctly decoding it.
If I manually configure the authentication token directly in proj_url then I am able to get the data, so I am doing something wrong with getting the authentication token.
UPDATE:
My goal was to somehow make the data available inside Power BI. I took an alternative approach with using R as a data source. (You will need to install R and the httr package in R for this.) Posting that solution here:
library(httr)
base_url <- 'https://example.com'
user_name <- 'john_doe'
password <- 'pwws0rd'
api_token <- '4r-eueyta-dh7cuq-26'
timeout_min <- 1
# get authentication token
auth_url <- paste0(base_url, '/api/authenticate')
data <- list('Username' = user_name,
'Password' = password,
'ApiToken' = api_token,
'AuthorizationTokenTimeoutMinutes' = timeout_min)
resp <- POST(auth_url,
body = data,
encode = 'json')
auth_token <- content(resp, "parsed", "application/json")
# get data
proj_url <- paste0(base_url, '/api/project/active?',
'api-token=', api_token,
'&authentication-token=', auth_token)
resp <- GET(proj_url, encode = 'json')
# convert the response to data frame
results <- content(resp, "parsed", "application/json")
results <- lapply(results, function(x) {
x[sapply(x, is.null)] <- NA
unlist(x)})
projects <- do.call("rbind", results) # matrix
projects <- data.frame(projects, stringsAsFactors = FALSE)
projects is then available for importing into Power BI.
I know it is too late to answer, but I'm guessing it is because the header lacks "Content-Type=application/x-www-form-urlencoded".
Python requests library automatically adds this header part, while Power Query does not.
I'm trying to write a simple script to log into Wikipedia and perform some actions on my user page, using the Mediawiki api. However, I never seem to get past the first login request (from this page: https://en.wikipedia.org/wiki/Wikipedia:Creating_a_bot#Logging_in). I don't think the session cookie that I set is being sent. This is my code so far:
import Cookie, urllib, urllib2, xml.etree.ElementTree
url = 'https://en.wikipedia.org/w/api.php?action=login&format=xml'
username = 'user'
password = 'password'
user_data = [('lgname', username), ('lgpassword', password)]
#Login step 1
#Make the POST request
request = urllib2.Request(url)
data = urllib.urlencode(user_data)
login_raw_data1 = urllib2.urlopen(request, data).read()
#Parse the XML for the login information
login_data1 = xml.etree.ElementTree.fromstring(login_raw_data1)
login_tag = login_data1.find('login')
token = login_tag.attrib['token']
cookieprefix = login_tag.attrib['cookieprefix']
sessionid = login_tag.attrib['sessionid']
#Set the cookies
cookie = Cookie.SimpleCookie()
cookie[cookieprefix + '_session'] = sessionid
#Login step 2
request = urllib2.Request(url)
session_cookie_header = cookieprefix+'_session='+sessionid+'; path=/; domain=.wikipedia.org; HttpOnly'
request.add_header('Set-Cookie', session_cookie_header)
user_data.append(('lgtoken', token))
data = urllib.urlencode(user_data)
login_raw_data2 = urllib2.urlopen(request, data).read()
I think the problem is somewhere in the request.add_header('Set-Cookie', session_cookie_header) line, but I don't know for sure. How do I use these python libraries to send cookies in the header with every request (which is necessary for a lot of API functions).
The latest version of requests has support for sessions (as well as being really simple to use and generally great):
with requests.session() as s:
s.post(url, data=user_data)
r = s.get(url_2)