I have a script that uses a dictionary stored in "my_dict" variable, and values of url, user, password, along with the "id" variable. The script then does an HTTP GET call to the url depending on headers passed. How do I create a Python Function which is equivalent to this? And how to the function later for another set of url, user, password etc?
import urllib, urllib2, base64, json
my_dict = {'server': {'user': 'patrick', 'url': 'http://192.168.0.1/tasks', 'password': 'secret'}}
id = "8d4lkf8kjhla8EnsdAjkjFjkdb6lklne"
for value in my_dict.keys():
url = my_dict[value]['url']
pass = my_dict[value]['password']
authKey = base64.b64encode("patrick:"+str(pass))
headers = {"login-session": id, "Content-Type": "application/json", "Authorization": "Basic " + authKey}
data = {"param": "value"}
request = urllib2.Request(url)
for key, value in headers.items():
request.add_header(key, value)
response = json.load(urllib2.urlopen(request))
print response
Your question has a lot of subquestions such as what do you want variable or constant? Do you know the syntax? How your function will be used? etc etc.
Therefore the best way for you to get an answer is learn some basics, like here (free) for the functions syntax in python. After that you may very well have the answers to your question.
A simple function like this will work if I understood you correctly,
def send_request(my_dict, id):
for value in my_dict.keys():
url = my_dict[value]['url']
pass = my_dict[value]['password']
authKey = base64.b64encode("patrick:"+str(pass))
headers = {"login-session": id, "Content-Type": "application/json", "Authorization": "Basic " + authKey}
data = {"param": "value"}
request = urllib2.Request(url)
for key, value in headers.items():
request.add_header(key, value)
response = json.load(urllib2.urlopen(request))
print response
Related
I have a UI end point that I am calling through a Python script. I want to pass variable in payload data but somehow Python is not reading it as a variable hence PUT request does not works, I need help in how to send this as a variable. In below code I want to pass tid value as a variable. tid is a json array with value tid = ["defaultd"]. I also tried to make it working by adding square brackets to the variable value.
def assign_tenant():
token = get_client_token()
user_id = get_user_id()
tid = get_tenants()
tid = f'[{tid}]'
url = mcs_url + "security/users/keycloak/" + user_id
headers = {
"Authorization": "Bearer " + token,
"Accept": "application/json",
"Content-Type": "application/json"
}
print('This is tenant id', tid)
data = '{"authenticationType":"LOCAL","email":"test#test.com","enabled":"true","firstName":"Super","lastName":"Admin","tenant":["tid"]}'
I am attempting to use the Woo exchange trading api. They provided a snippet of code which is an impressive Python class structure. Copied below.
My question is how can I use it?
I have tried to run:
get_orders(self, 'BTCUSDT')
which throws an error "NameError: name 'self' is not defined"
and
get_orders('BTCUSDT')
which throws "TypeError: get_orders() missing 1 required positional argument: 'symbol'"
Here is the code (class structure) the kind woo guys provided:
import requests
import datetime
import time
import hmac
import hashlib
from collections import OrderedDict
#Application ID 9d4d96f6-3d3b-4430-966d-8733aa3dc3bc
#API Key
api_key = 'my_api_key'
#API Secret
api_secret = 'my_api_secret'
class Client():
def __init__(self, api_key=None, api_secret=None):
self.api_key = api_key
self.api_secret = api_secret
self.base_api = "https://api.woo.network/v1/"
def get_signature(self, params, timestamp):
query_string = '&'.join(["{}={}".format(k, v) for k, v in params.items()]) + f"|{timestamp}"
signature = hmac.new(
self.api_secret.encode('utf-8'),
query_string.encode('utf-8'),
hashlib.sha256
).hexdigest()
return signature
def get_orders(self, symbol):
url = self.base_api + "orders/"
params = {
"symbol": 'BTCUSDT'
}
params = OrderedDict(sorted(params.items()))
timestamp = str(int(time.time() * 1000))
signature = self.get_signature(params, timestamp)
headers = {
'Content-Type': "application/x-www-form-urlencoded",
'x-api-key': self.api_key,
'x-api-signature': signature,
'x-api-timestamp': timestamp,
'cache-control': 'no-cache'
}
resp = requests.get(url=url, params=params, headers=headers).json()
So, to repeat and summarize, when I write my own code to use this class, how can I call the function get_orders() and, more generally, reference the elements in the class structure? Thanks, in advance, for help.
Looks like you've truncated the code because get_orders doesn't appear to return anything.
However, you would start by constructing an instance of Client like this:
client = Client(api_key, api_secret)
...then...
client.get_orders(None)
That may look a little strange but get_orders requires one parameter but it's never used. I don't think the implementation of get_orders is quite how it was intended to be because it will always use BTCUSDT
I'm using the requests module to collect some data from a website. This application runs once every day. The amount of rows of data I get changes every time, per request I can get a maximum 250 rows of data. If there is more then 250 rows of data the API gives me a follow uplink which can be used to get the rows 251 >- 500 etc.
Now I have a problem, sometimes the amount of data is < 250 rows, this means there is no followuplink to use and that's exactly where my program gives the following error:
KeyError: #odata.nextLink
This is a piece of the application:
proxies = {'https': 'proxy.***.***.com:8080'}
headers = {"grant_type": "password",
"username": "****",
"password": "****",
"persistent": "true",
"device": '{"DeviceUniqueId":"b680c452","Name":"Chrome","DeviceVersion":"36","PlatformType":"Browser"}'}
url1 = 'https://****-***.com/odata/Results'
params_1 = (
('$filter', mod_date),
('$count', 'true'),
('$select', 'Status'),
('$expand', 'Result($select=ResultId),Specification($select=Name), SpecificationItem($select=Name,MinimumValue, MaximumValue)\n\n'),)
response_1 = requests.get(url_1, headers=headers, proxies=proxies, params=params_1)
q_1 = response_1.json()
next_link_1 = q_1['#odata.nextLink']
q_1 = [tuple(q_1.values())]
while next_link_1:
new_response_1 = requests.get(next_link_1, headers=headers, proxies=proxies)
new_data_1 = new_response_1.json()
q_1.append(tuple(new_data_1.values()))
next_link_1 = new_data_1.get('#odata.nextLink', None)
Now I actually want Python to only read the variable next_link_1 if its available otherwise it should just ignore it and collect what is available...
You only want to enter the while loop when q_1 has the key '#odata.nextLink' Inside the while loop, this is already accomplished in the line next_link_1 = new_data_1.get('#odata.nextLink', None) You could use the same approach -- setting next_link_1 to None if there is no next link -- before the while loop:
next_link_1 = q_1.get('#odata.nextLink', None)
This can be simplified to
next_link_1 = q_1.get('#odata.nextLink')
as None is already the default default value of dict.get().
NB: The question title is wrong. The variable always exists, as you are setting it. Only the existence of the key #odata.nextLink is fragile. So, what you actually want to do is check the existence of a key in a dictionary. To understand what is going on, you should familiarize yourself with the dict.get() method.
There is also some obvious refactoring possible here, getting rid of the repetition of the first iteration, and moving it into the loop:
proxies = {'https': 'proxy.***.***.com:8080'}
headers = {
'grant_type': 'password',
'username': '****',
'password': '****',
'persistent': 'true',
'device': '{"DeviceUniqueId":"b680c452","Name":"Chrome","DeviceVersion":"36","PlatformType":"Browser"}'
}
params = (
('$filter', mod_date),
('$count', 'true'),
('$select', 'Status'),
('$expand', 'Result($select=ResultId),Specification($select=Name), SpecificationItem($select=Name,MinimumValue, MaximumValue)\n\n'),
)
url = 'https://****-***.com/odata/Results'
data = []
while url:
response = requests.get(
url,
headers=headers,
proxies=proxies,
params=params,
)
response_data = response.json()
data.append(tuple(response_data.values()))
url = response_data.get('#odata.nextLink')
params = tuple()
Use get in both places. Better yet, restructure your loop so that you only need one call.
proxies = {'https': 'proxy.***.***.com:8080'}
headers = {...}
url1 = 'https://****-***.com/odata/Results'
params = (...)
qs = []
next_link = url
get_args = {'headers': headers, 'proxies': proxies, 'params': params}
while True:
response = requests.get(next_link, **get_args)
q = response.json()
qs.append(tuple(q.values())
if (next_link := q.get('#odata.nextLink', None)) is None:
break
if 'params' in get_args:
del get_args['params'] # Only needed in the first iteration
(I'm not terribly excited about how we ensure params is used only on the first iteration, but I think it's better than duplicating the process of defining next_link before the loop starts. Maybe something like this would be an improvement?
get_args = {...} # As above
new_get_args = dict(headers=..., proxies=...) # Same, but without params
while True:
...
if (next_link := ...) is None:
break
get_args = new_get_arg
Repeated assignment to get_args is probably cheaper than repeatedly testing for and deleting the params key, at the cost of having a second dict in memory. You could even drop that after the first iteration by adding a second assignment new_get_args = get_args to the end of the loop, which would result in a pair of do-nothing assignments for later iterations.)
Im trying to write a REST Client for a closed API. I call a specific function twice but it only works the first time. It's the same command. If i run the script twice with batch It works. Im suspecting Requests keeps the connection alive or caches data. How do i "reset" Requests ?
base_headers = {"Connection":"keep-alive",
"User-Agent":user_agent,
"Accept-Encoding":"gzip",
"Host":"xxxxxxxxxxx",
"Content-Type":"application/json; charset=UTF-8"}
global auth
auth = 'xxxxxx'
def create_header(doAuth = True):
headers = base_headers
if doAuth:
headers['Authorization'] = auth
return headers
def do_get(url):
return requests.get(url, headers=create_header());
def do_put(url, payload):
if payload is None:
return requests.put(url, headers=create_header())
return requests.put(url, data=payload, headers=create_header())
def upvote(postid):
return do_put("xxxxxxxxxxxxx" % postid, None).content
def set_pos(longtitude, latitude, place, country="DE"):
payload = {'xxxxx':'xxxx'}
json_payload = json.dumps(payload)
return do_put("xxxxxxxxxxxxxxx",json_payload).content
def do_post(url, payload, doAuth=True):
return requests.post(url, data=payload, headers=create_header(doAuth=doAuth))
def new_acc(place, latitude, longtitude):
access = get_access_token(place, latitude, longtitude)
print access
global auth
auth = "Bearer %s" % access['access_token']
set_pos(longtitude, latitude, place) # Does a POST
for i in range(1,30):
posts = new_acc(uni['city'], uni['latitude'], uni['longtitude'])
upvote('xxxxxxxxxxxxx')
Basically the first upvote() goes through every time but every succeding does not.
I'm almost positive that keep-alive is not doing this. I would suggest writing some handlers to debug the response if you don't get the expected response code after the request.
Also, might I suggest a slightly different design that avoids global?
class RESTClient(object):
BASE_HEADERS = {"Connection":"keep-alive",
"Accept-Encoding":"gzip",
"Host":"xxxxxxxxxxx",
"Content-Type":"application/json; charset=UTF-8"}
def __init__(self, user_agent, auth = None):
self.headers = dict(self.BASE_HEADERS)
self.headers['User-Agent'] = user_agent
self.auth = auth
def create_header(self):
headers = dict(self.headers)
if auth:
headers['Authorization'] = auth
return headers
def do_get(self, url):
return requests.get(url, headers=self.create_header());
def do_put(self, url, payload = None):
if not payload:
return requests.put(url, headers=self.create_header())
return requests.put(url, data=payload, headers=self.create_header())
def upvote(self, postid):
return do_put("xxxxxxxxxxxxx" % postid).content
def set_pos(self, longtitude, latitude, place, country="DE"):
payload = {'xxxxx':'xxxx'}
json_payload = json.dumps(payload)
return do_put("xxxxxxxxxxxxxxx",json_payload).content
def do_post(self, url, payload, do_auth = None):
return requests.post(url, data=payload, headers=self.create_header())
def new_acc(self, place, latitude, longtitude):
access = get_access_token(place, latitude, longtitude)
print access
self.auth = "Bearer %s" % access['access_token']
set_pos(longtitude, latitude, place)
rc = RESTClient('Mozilla/5.0 ... Firefox/38.0', 'my-auth-token')
rc.upvote(post_id)
etc...
It could be that your use of globals is breaking things. Having not run your code, I can't check, but I would avoid using globals and favor tighter control of your state via objects.
EDIT:
Given that this answer was accepted by the OP, I am assuming the defect was caused by mutation of globals.
I have many functions that successfully POST a urlencoded body using python. However, I have a body that is a multidimensional dictionary. Using this dictionary, I only get a 400 (bad request). bodytmp (below) is an example of the body that works using Fiddler. (The actual url and body can not be supplied.) I have also included a recursive urlencode function that I found here and am using now but with no success.
Does anyone have experience with this type of POST request with a multidimensional dictionary for the urlencoded body?
Thank you. ( I have abbreviated the code to make it more readable, but this is the gist. )
from httplib2 import Http
import httplib
import urllib
def postAgentRegister():
h = Http(disable_ssl_certificate_validation=True)
headers={'Content-Type':'application/x-www-form-urlencoded'}
bodytmp={"Username": "username",
"Password": "password",
"AccountId": "accountid",
"PublicKey": {
"Value1a": "32ab45cd",
"value1b": "10001"
},
"snId": "SN1",
"IpAddresses": {
"Value2a": ["50.156.54.45"],
"Value2b": "null"
}
}
body=recursive_urlencode(bodytmp)
try:
response,content=h.request('https://server/endpoint','POST',headers=headers,body=body)
conn = httplib.HTTPConnection(testserver)
conn.request('POST', endpoint, body, headers)
r1 = conn.getresponse()
status = r1.status
reason = r1.reason
except httplib.HTTPException, e:
status= e.code
print 'status is: ', status
def recursive_urlencode(d):
def recursion(d, base=None):
pairs = []
for key, value in d.items():
if hasattr(value, 'values'):
pairs += recursion(value, key)
else:
new_pair = None
if base:
new_pair = "%s[%s]=%s" % (base, urllib.quote(unicode(key)), urllib.quote(unicode(value)))
else:
new_pair = "%s=%s" % (urllib.quote(unicode(key)), urllib.quote(unicode(value)))
pairs.append(new_pair)
return pairs
return '&'.join(recursion(d))
Might I suggest that you serialize your body to JSON and de-serialize when your server receives it? This way you just have to do a url string encode rather than using your own recursive url encode.