R convert python GET to R http GET - python

I'm trying to convert a working python program that retrieves information from a website with an api key to its R equivalent. Since I don't know much about httr or python it's a challenge. The python code is (somewhat abbreviated and with a dummy X-ApiKey)
url = 'https://api.clarivate.com/api/woslite'
query = 'ts=((land AND ocean AND climate AND change)) AND PY=2013-2019'
count = 100
firstRecord = 1
parameters = {'databaseId': 'WOK', 'usrQuery': query, 'count': count, 'firstRecord': firstRecord}
headers={'accept':'application/json','X-ApiKey':'********'}
response = requests.get(url,params=parameters, headers=headers)
My attempt at an R version is
library(httr)
wosliteKey <- Sys.getenv("wosliteKey")
firstRecord <- 1
count <- 100
url <- 'https://api.clarivate.com/api/woslite'
query <- 'ts=(land AND ocean AND climate AND change) AND PY=2013-2019'
r <- GET(url, query = list(api_key = wosliteKey, usrQuery = query, databaseId = 'WOK', count = count, firstRecord = firstRecord))
Running the above returns
Response [https://api.clarivate.com/api/woslite]
Date: 2019-05-09 22:50
Status: 401
Content-Type: application/json; charset=utf-8
Size: 41 B
status 401 means unauthorized access. The python code uses X-ApiKey rather than api_key. But I can't figure out a. what the difference is and b. how to put it into the query list.

With the help of the comments above I figured out how to make this work. The python version of httr GET looks like the following
response = requests.get(url,params=parameters, headers=headers)
For my problem, I have the following from the python program
parameters = {'databaseId': 'WOK', 'usrQuery': query, 'count': count, 'firstRecord': firstRecord}
headers={'accept':'application/json','X-ApiKey':'********'}
The equivalent with httr GET is
response <- httr::GET(url, httr::add_headers(accept = 'application/json', `X-APIKey` = wosliteKey), query = list(databaseId = 'WOK', usrQuery = query, count = count, firstRecord = firstRecord))
The python headers info is replaced by the add_headers function from httr. The python parameters info is added as a list in the query option.

I searched for a while, i found something looks working with your problem.
android package - R.attr nested class -- employing an api key

Related

Excel passing unrecognizable date format to python function in xlwings

I am trying to link a script to an excel file using xlwings that takes data from the sheet and queries skyscanners API to return flight prices. When I type dates in manually (yyyy-mm-dd), it works fine since it is a String. However, when I try to run the query by automatically loading the date from excel, it cannot work with excel's date format.
When I just return type(dept_date), I get the error: Unexpected Python Error: TypeError: must be a pywintypes time object (got type)"
When I try to convert dept_date to a string first and return that type, I get: Unexpected Python Error: TypeError: Objects of type 'type' can not be converted to a COM VARIANT.
For context, the API takes the yyyy-mm-dd format and when I print the querystring, excel automatically adds the "00:00:00" to the end of the date. I've tried subscripting [:9] to just pull the date but since it isn't a String, that doesn't work. I've tried casting to a string and doing that too with no luck. Any help is appreciated! I've included the whole function below just in case:
#xw.func
def airfare(trip_type, home_iata, dest_iata, dept_date, return_date=""):
""" Pass home IATA, destination IATA, departure/return dates (yyyy-mm-dd). Returns average of "best"
results. All params are strings. To account for last minute booking, if flight is < 500, result doubles.
If flight is 500+, it adds a 30% surcharge onto result. Trip type 1 = Round Trip; 0 = One-Way
"""
home_iata = str(home_iata)
dest_iata = str(dest_iata)
dept_date = str(dept_date)
# dept_date.strftime('%d-%m-%Y')
if trip_type == "0":
return_date = ""
url = "https://skyscanner44.p.rapidapi.com/search"
querystring = {"adults": "1",
"origin": home_iata,
"destination": dest_iata,
"departureDate": dept_date,
"returnDate": return_date,
"cabinClass": "economy",
"currency": "USD"}
headers = {
"X-RapidAPI-Key": "XXX",
"X-RapidAPI-Host": "skyscanner44.p.rapidapi.com"
}
# return querystring
# return type(dept_date)
response = requests.request("GET", url, verify=False, headers=headers, params=querystring).json()
best_flight = int(response["itineraries"]["buckets"][0]["items"][0]["price"]["raw"])
if best_flight < 500:
best_flight *= 2
else:
best_flight *= 1.3
return "$" + str(best_flight)

I have a problem in Trading with Kucoin API (Python)

I tried to do API trading in Kucoin. I developed a bot that is finding the trading opportunities well, while I encountered problems in making new orders. Pls check the code and help me to make it functional.
The code was edited in accordance with the comment of Lev Levitsky
is as follows:
import json
import urllib
import requests
import base64
import hmac
import hashlib
api_key = 'api_key'
api_secret = 'api_secret'
api_passphrase = 'api_passphrase'
base_uri = 'https://api-futures.kucoin.com'
endpoint = '/api/v1/orders?symbol=MATICUSDTM'
method = 'POST'
x= {}
x["symbol"] = "MATICUSDTM"
x["signal_type"] = "SHORT"
x["leverage"] = 5
x["exchange"] = "Kucoin"
x["entrance_price"] = 2.1000
x["trading_size"] = 150
x["tp1"] = 2.08
x["sl1"] = 2.12
all_futures_signals = list()
all_futures_signals.append(x)
def get_headers(method, endpoint, api_key, api_passphrase,body):
api_secret = ''
now = int(time.time() * 1000)
str_to_sign = str(now) + method + endpoint + str(body)
signature = base64.b64encode(hmac.new(api_secret.encode('utf-8'), str_to_sign.encode('utf-8'), hashlib.sha256).digest())
passphrase = base64.b64encode(hmac.new(api_secret.encode('utf-8'), api_passphrase.encode('utf-8'), hashlib.sha256).digest())
return {'KC-API-KEY': api_key,
'KC-API-KEY-VERSION': '2',
'KC-API-PASSPHRASE': passphrase,
'KC-API-SIGN': signature,
'KC-API-TIMESTAMP': str(now)}
body = {
"clientOid" : "",
"reduceOnly" : False, # A mark to reduce the position size only
"closeOrder" : False, # If closeOrder is set to TRUE, the system will close the position and the position size will become 0. Side, Size and Leverage fields can be left empty and the system will determine the side and size automatically.
"forceHold" : False, # The system will forcely freeze certain amount of funds for this order, including orders whose direction is opposite to the current positions. This feature is to ensure that the order won’t be canceled by the matching engine in such a circumstance that not enough funds are frozen for the order.
"hidden" : False, # A hidden order will enter but not display on the orderbook.
"iceberg" : False, # When placing an iceberg order, you need to set the visible size. The minimum visible size is 1/20 of the order size. The minimum visible size shall be greater than the minimum order size, or an error will occur.
"visibleSize" : 0, # When placing an iceberg order, you need to set the visible size. The minimum visible size is 1/20 of the order size. The minimum visible size shall be greater than the minimum order size, or an error will occur.
"leverage" : x["leverage"],
"postOnly" : False, # The post-only flag ensures that the trader always pays the maker fee and provides liquidity to the order book.
"price" : 2.1000, # The price specified must be a multiple number of the contract tickSize,
"remark" : "remark",
"side" : "buy",# sell/buy
"size" : x["trading_size"], # The size must be no less than the lotSize for the contract and no larger than the maxOrderQty.
"stop" : "", # down/up
"stopPrice" : "",
"stopPriceType": "", # TP/MP/IP: TP for trade price, MP for mark price, and IP for index price
"symbol" : x["symbol"],
"timeInForce" : "", # GTC/IOC: Good Till Canceled GTC and Immediate Or Cancel IOC.
"type" : "limit", # limit/market
}
headers = get_headers(method, endpoint, api_key, api_passphrase, body)
x["opening_response"] = requests.post( base_uri + endpoint, body, headers=headers).json()
print(x["opening_response"])
I receive this error: {'code': '400005', 'msg': 'Invalid KC-API-SIGN'}
All inputs are correct. I think there is a problem with the code.
Best Regards
Javad
I think the problem is your endpoint variable. I believe you should not add the symbol to endpoint, when you are trying to add a new order. Remove it from endpoint and pass the symbol in body object. The other thing is that I think you do not need to pass empty strings to optional fields in body object. I am not sure about this but I think you should remove them from body object. It is a signing problem, therefore you have to check 4 variables: timestamp, method, endpoint and body. I hope this works.
have you considered using ccxt? It makes dealing with lower level API stuff like this somewhat easier.
Given that it's singling out the signature as being invalid, but not saying any headers are missing, it could mean that the signature for str_to_sign variable is wrong?
Let's look at it:
api_secret = ''
now = int(time.time() * 1000)
str_to_sign = str(now) + method + endpoint + str(body)
From the looks of it, your api_secret is just an empty string, so the resulting signature won't be correct.
so, a couple of lines down when you make the signature:
signature = base64.b64encode(hmac.new(api_secret.encode('utf-8'), str_to_sign.encode('utf-8'), hashlib.sha256).digest())
Even though api_secret is given a value in the higher scope, it's overridden by this local variable which is an empty string.
If your api_secret were actually an empty string, then your code would produce the correct signature, but it's not so it isn't
So, If you give it as a param to your get_headers function ie.
def get_headers(method, endpoint, api_key, api_passphrase, api_secret, body):
And delete the first line api_secret = ''
Then maybe it will work?
If it doesn't work, then it's a different problem. (eg. if you had actually put your API key in there and just redacted it before posting). I don't know since I haven't tried running your code.
PS: To Edelweiss: Grüzi aus Berner Oberland!
According to a comment of edelweiss, an easier way is using the client of Kucoin API. It is here.
First, I installed the client using this code.
!pip install kucoin-futures-python
Then, I opened a position by this code:
from kucoin_futures.client import Trade
client = Trade(key='api_key', secret='api_secret', passphrase='api_passphrase', is_sandbox=False, url='')
order_id = client.create_limit_order(symbol, side, lever, size, price, clientOid='', **kwargs)

Python microsoft graph api

I am using microsoft graph api to pull my emails in python and return them as a json object. There is a limitation that it only returns 12 emails. The code is:
def get_calendar_events(token):
graph_client = OAuth2Session(token=token)
# Configure query parameters to
# modify the results
query_params = {
#'$select': 'subject,organizer,start,end,location',
#'$orderby': 'createdDateTime DESC'
'$select': 'sender, subject',
'$skip': 0,
'$count': 'true'
}
# Send GET to /me/events
events = graph_client.get('{0}/me/messages'.format(graph_url), params=query_params)
events = events.json()
# Return the JSON result
return events
The response I get are twelve emails with subject and sender, and total count of my email.
Now I want iterate over emails changing the skip in query_params to get the next 12. Any method of how to iterate it using loops or recursion.
I'm thinking something along the lines of this:
def get_calendar_events(token):
graph_client = OAuth2Session(token=token)
# Configure query parameters to
# modify the results
json_list = []
ct = 0
while True:
query_params = {
#'$select': 'subject,organizer,start,end,location',
#'$orderby': 'createdDateTime DESC'
'$select': 'sender, subject',
'$skip': ct,
'$count': 'true'
}
# Send GET to /me/events
events = graph_client.get('{0}/me/messages'.format(graph_url), params=query_params)
events = events.json()
json_list.append(events)
ct += 12
# Return the JSON result
return json_list
May require some tweaking but essentially you're adding 12 to the offset each time as long as it doesn't return an error. Then it appends the json to a list and returns that.
If you know how many emails you have, you could also batch it that way.

Problem with continuation token when querying from Cosmos DB

I'm facing a problem with continuation when querying items from CosmosDB.
I've already tried the following solution but with no success. I'm only able to query the first 10 results of a page even though I get a token that is not NULL.
The token has a size of 10733 bytes and looks like this.
{"token":"+RID:gtQwAJ9KbavOAAAAAAAAAA==#RT:1#TRC:10#FPP:AggAAAAAAAAAAGoAAAAAKAAAAAAAAAAAAADCBc6AEoAGgAqADoASgAaACoAOgBKABoAKgA6AE4AHgAuAD4ASgAeACoAPgBOAB4ALgA+AE4AHgAqAD4ASgAeAC4APgBOAB4ALgA+AE4AIgA2AEYAFgAmADYARgAaACYAPgBKABYAKgA6AE4AHgAuAD4ATgAeAC4APgBOAB4ALgA+AE4AIgAuAD4ATgAeAC4APgBOACIAMgA+AFIAIgAyAD4AUgAmADIAQgAWACIALgBCABIAIgAyAEIAEgAiADIAQgAOACYANgBKAB4AJgA6AEYAGgAqADoATgAeAC4APgB....etc...etc","range":{"min":"","max":"05C1BF3FB3CFC0"}}
Code looks like this. Function QueryDocuments did not work. Instead I had to use QueryItems.
options = {}
options['enableCrossPartitionQuery'] = True
options['maxItemCount'] = 10
q = client.QueryItems(collection_link, query, options)
results_1 = q._fetch_function(options)
#this is a string representing a JSON object
token = results_1[1]['x-ms-continuation']
data = list(q._fetch_function({'maxItemCount':10,'enableCrossPartitionQuery':True, 'continuation':token}))
Is there a solution to this? Thanks for your help.
Please use pydocumentdb package and refer to below sample code.
from pydocumentdb import document_client
endpoint = "https://***.documents.azure.com:443/";
primaryKey = "***";
client = document_client.DocumentClient(endpoint, {'masterKey': primaryKey})
collection_link = "dbs/db/colls/coll"
query = "select c.id from c"
query_with_optional_parameters = [];
q = client.QueryDocuments(collection_link, query, {'maxItemCount': 2})
results_1 = q._fetch_function({'maxItemCount': 2})
print(results_1)
token = results_1[1]['x-ms-continuation']
results_2 = q._fetch_function({'maxItemCount': 2, 'continuation': token})
print(results_2)
Output:

Django and JSON/AJAX testing

I tried looking around for an answer and gave it a great many tries, but there's something strange going on here. I got some functions in my view that operate on JSON data that comes in via AJAX. Currently I'm trying to do some unit testing on these.
In my test case I have:
kwargs = {'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest'}
url = '/<correct_url>/upload/'
data = {
"id" : p.id
}
c = Client()
response = c.delete(url, data, **kwargs)
content_unicode = response.content.decode('utf-8')
content = json.loads(content_unicode)
p.id is just an integer that comes from a model I'm using.
I then have a function that is being tested, parts of which looks like follows:
def delete_ajax(self, request, *args, **kwargs):
print (request.body)
body_unicode = request.body.decode('utf-8')
print (body_unicode)
body_json = json.loads(body_unicode)
The first print statement yields:
.....b"{'id': 1}"
The other one:
{'id': 1}
and finally I get an error for fourth line as follows:
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
What's going wrong here? I understand that correct JSON format should be {"id": 1} and that's what I'm sending from my test case. But somewhere along the way single-quotes are introduced into the mix causing me head ache.
Any thoughts?
You need to pass a json string to Client.delete(), not a Python dict:
kwargs = {'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest'}
url = '/<correct_url>/upload/'
data = json.dumps({
"id" : p.id
})
c = Client()
response = c.delete(url, data, **kwargs)
You should also set the content-type header to "application/json" and check the content-type header in your view but that's another topic.

Categories

Resources