Is there a way to pass an HTTP verb (PATCH/POST) to a function and dynamically use that verb for Python requests?
For example, I want this function to take a 'verb' variable which is only called internally and will either = post/patch.
def dnsChange(self, zID, verb):
for record in config.NEW_DNS:
### LINE BELOW IS ALL THAT MATTERS TO THIS QUESTION
json = requests.verb(headers=self.auth, url=self.API + '/zones/' + str(zID) + '/dns_records', data={"type":record[0], "name":record[1], "content":record[2]})
key = record[0] + "record with host " + record[1]
result = json.loads(json.text)
self.apiSuccess(result,key,value)
I realize I cannot requests.'verb' as I have above, it's meant to illustrate the question. Is there a way to do this or something similar? I'd like to avoid an:
if verb == 'post':
json = requests.post(headers=self.auth, url=self.API + '/zones/' + str(zID) + '/dns_records', data={"type":record[0], "name":record[1], "content":record[2]}
else:
json = requests.patch(headers=self.auth, url=self.API + '/zones/' + str(zID) + '/dns_records', data={"type":record[0], "name":record[1], "content":record[2]}
Thanks guys!
Just use the requests.request() method. First argument is the HTTP verb that you want to use. requests.get(), requests.post(), etc. are just aliases to request('GET'), request('POST'): see the doc
Your code becomes:
verb = 'POST'
response = requests.request(
verb,
headers=self.auth,
url=self.API + '/zones/' + str(zID) + '/dns_records',
data={"type":record[0], "name":record[1], "content":record[2]}
)
With the request library, the requests.request method can be relied on directly (as Guillaume's answer suggested).
However, when encountering against libraries that don't have a generic method for methods that have similar calling signatures, getattr can be supplied with the name of the desired method as a string with a default value. Maybe like
action = getattr(requests, verb, None)
if action:
action(headers=self.auth, url=self.API + '/zones/' + str(zID) + '/dns_records', data={"type":record[0], "name":record[1], "content":record[2]})
else:
# handle invalid action as the default value was returned
For the default value it can be a proper action, or just leave it out and an exception will be raised; it's up to you how you want to handle it. I left it as None so you can deal with alternative case in the else section.
Related
Right now, I am using fine-uploader as my front-end and python-flask as my backend. What I need to do is that I need to send large files from the front end to my server. I can do this easily without concurrent chunking. But once I turn it on, the file gets corrupted. I think this is because when you concurrently chunk, I would guess part 7 could get added before part 5 gets added. Again, non-concurrent chunking is OK since it is sequential.
I need to know if there is some sort of temporary, global variable or way that I can store the chunk parts temporarily. I tried redis but unfortunately when I get the data from the redis, it shows a decoding error because probably redis tries to turn it into a string when I just need it to be bytes.
If all else fails, I'll just go to my last resort, which is to put the parts into their own little files then open them later on to combine them one by one.
Here is my code for your reference. It still has the redis methods on it.
def upload_temp_file_part(redis, request):
try:
# Remember the paramName was set to 'file', we can use that here to grab it
file = request.files['qqfile']
uuid = request.form['qquuid']
part = request.form['qqpartindex']
offset = request.form['qqpartbyteoffset']
key_content = 'file_content_' + uuid + part
key_offset = 'file_offset_' + uuid + part
value_content = file.stream.read()
value_offset = offset
logging.info("Setting part " + part + " of " + uuid)
redis.set(key_content, value_content)
redis.set(key_offset, value_offset)
except Exception as e:
logging.error(e)
def combine_temp_file_part(redis, request):
try:
uuid = request.form['qquuid']
total_parts = request.form['qqtotalparts']
save_path = os.path.join(os.getenv('UPLOAD_FOLDER_TEMP'), uuid)
with open(save_path, 'ab') as f:
for index in range(0, int(total_parts)):
key_content = 'file_content_' + uuid + str(index)
key_offset = 'file_offset_' + uuid + str(index)
logging.info("Get part " + str(index) + " of " + uuid)
value_content = redis.get(key_content)
value_offset = redis.get(key_offset)
if value_content is None or value_offset is None:
pass
# Throw Error
logging.info("Placing part " + str(index) + " of " + uuid)
f.seek(value_offset)
f.write(value_content)
redis.delete(value_offset)
redis.delete(value_content)
except Exception as e:
logging.error(e)
I do not know how exactly to ask this question, as I can't share API access. If someone could help with the correct way of asking, will appreciate it.
I have a code which works perfectly fine and executes what I need, BUT when I use this API request code inside the DEF function, it returns with nothing or error...
The error is simple that I cannot get the output, or return with 0 values
This is the code:
def sm_main_data():
#DATA PROCESSING - Impression|Engagements|VideoViews
urlMain = "https://api.simplymeasured.com/v1/analytics/" + key.accountId + "/posts/metrics?\
filter=analytics.timeseries_key.gte(" + config.start + ").lte(" + config.end + ")&\
filter=channel.eq(" + config.which_social_media + ")&\
metrics=analytics.engagement_total,analytics.video.views_count,analytics.impressions&\
dimensions=data_source_id,channel,analytics.timeseries_key.by(" + config.per_what + ")"
headers = {'content-type': 'application/json',
'Authorization': 'Bearer ' + key.token}
#Receive data from SM (main data) / modified it and save as JSON file
responseMain = requests.get(urlMain, headers=headers).json()
pprint.pprint(responseMain)
pass
sm_main_data()
I have tried to print variables inside def:
print(key.accountId)
print(config.start)
print(config.end)
print(config.which_social_media)
print(config.per_what)
Which all printed correctly.
Currently I'm lost... and cannot get an even theoretical idea about what could go wrong. Why this code does not work inside def function???
Edit 1.0
Error
{u'errors': [{u'detail': u'` filter` parameter is not supported.,` metrics` parameter is not supported.,` dimensions` parameter is not supported.',
u'status': u'422',
u'title': u'Unprocessable Entity'}]}
Edit 1.1
Code without def
#DATA PROCESSING - Impression|Engagements|VideoViews
urlMain = "https://api.simplymeasured.com/v1/analytics/" + key.accountId + "/posts/metrics?\
filter=analytics.timeseries_key.gte(" + config.start + ").lte(" + config.end + ")&\
filter=channel.eq(" + config.which_social_media + ")&\
metrics=analytics.engagement_total,analytics.video.views_count,analytics.impressions&\
dimensions=data_source_id,channel,analytics.timeseries_key.by(" + config.per_what + ")"
headers = {'content-type': 'application/json',
'Authorization': 'Bearer ' + key.token}
#Receive data from SM (main data) / modified it and save as JSON file
responseMain = requests.get(urlMain, headers=headers).json()
pprint.pprint(responseMain)
The error shows you are trying to send parameters with extra spaces before them to the server:
` filter` parameter is not supported.
` metrics` parameter is not supported.
Those extra spaces before the names are part of the parameter name, because you included those in your string:
def sm_main_data():
# ...
urlMain = "https://api.simplymeasured.com/v1/analytics/" + key.accountId + "/posts/metrics?\
filter=analytics.timeseries_key.gte(" + config.start + ").lte(" + config.end + ")&\
filter=channel.eq(" + config.which_social_media + ")&\
metrics=analytics.engagement_total,analytics.video.views_count,analytics.impressions&\
dimensions=data_source_id,channel,analytics.timeseries_key.by(" + config.per_what + ")"
# ^^^ those lines are indented but the whitespace is part of the string
You would get the same problem if you had indented the urlMain string definition for any other reason, like for a if statement or a try...except statement, not just a function. You'd have to not indent those parts that are inside a string literal.
Rather than use \ continuations in the string, you could use separate string literals to create one long string, or end the string literal with a closing " followed by a + and a " opening quote on the next line:
urlMain = (
"https://api.simplymeasured.com/v1/analytics/" +
key.accountId + "/posts/metrics?" +
"filter=analytics.timeseries_key.gte(" + config.start + ").lte(" + config.end + ")&" +
"filter=channel.eq(" + config.which_social_media + ")&" +
"metrics=analytics.engagement_total,analytics.video.views_count,analytics.impressions&" +
"dimensions=data_source_id,channel,analytics.timeseries_key.by(" + config.per_what + ")"
)
All those + concatenations are not very readable, you should really use string formatting to insert values into a string.
However, you do not need to build a string like that anyway, as requests can do this for you when you give it a dictionary as the params argument. Use lists to pass in multiple values for a given parameter name:
url = "https://api.simplymeasured.com/v1/analytics/{k.accountId}/posts/metrics".format(
k=key)
params = {
'filter': [ # a list for multiple entries: filter=...&filter=...
'analytics.timeseries_key.gte({c.start}).lte({c.end})'.format(c=config),
'channel.eq({c.which_social_media})'.format(c=config),
],
'metrics': (
'analytics.engagement_total,analytics.video.views_count,'
'analytics.impressions'),
'dimensions':
'data_source_id,channel,'
'analytics.timeseries_key.by({c.per_what})'.format(c=config),
}
headers = {'Authorization': 'Bearer {k.token}'.format(k=key)}
responseMain = requests.get(urlMain, params=params, headers=headers).json()
Here I used str.format() to insert values from the config and key objects; note that the placeholders pull out the attributes
Note: I removed the Content-Type header, as that header doesn't apply to a GET request (which doesn't have content, the request body is always empty).
I'm working on a Django application which retrieves some data from Twitter, and I need to work with requests manually since it's for an exam whose main focus is for us to learn how to work with network protocols such as HTTP. So please don't suggest me to use a wrapper.
My issue is that whenever I try to request the initial request token (with a POST request that is referenced in this doc and also here), I receive a 401: Authorization Required and I have no clue what could be causing it, since I'm including the Authorization header in exactly the format required by the docs I've linked. It may be that while working with urllib I don't manage to make the POST request as I wanted, or perhaps there's something wrongin how I made the signature/where I've placed it in the header, even though in my opinion it should be alright.
Here's my code to handle the request to "sign in with Twitter":
def sign(request):
url = 'https://api.twitter.com/oauth/request_token'
#I store consumer key, consumer secret and callback url in a txt
dir_path = path.dirname(path.realpath(__file__))
with open(path.join(dir_path, 'credentials.txt'), 'r') as TweetCred:
creds = TweetCred.read().split(linesep)
oauth_consumer_key = creds[0].split()[1]
oauth_callback = creds[1].split()[1]
consumer_secret = creds[2].split()[1]
oauth_nonce = ''.join(SystemRandom().choice(string.ascii_letters + string.digits) for _ in range(32))
#Desperate to understand what could be causing the error, I tried to use an integer timestamp sinceit was integer in the examples
oauth_timestamp = int(time.time())
parameter_string = ( 'oauth_nonce=' + urllib.parse.quote(oauth_nonce, safe='') +
'&oauth_callback=' + urllib.parse.quote(oauth_callback, safe='') +
'&oauth_signature_method=' + urllib.parse.quote('HMAC-SHA1', safe='') +
'&oauth_timestamp=' + urllib.parse.quote(str(oauth_timestamp), safe='') +
'&oauth_consumer_key=' + urllib.parse.quote(oauth_consumer_key, safe='') +
'&oauth_version=' + urllib.parse.quote('1.0', safe='')
)
sigbase = 'POST' + '&' + urllib.parse.quote(url, safe='') + '&' + urllib.parse.quote(parameter_string, safe='')
signkey = urllib.parse.quote(consumer_secret, safe='') + '&'
key = bytes(signkey, 'UTF-8')
base = bytes(sigbase, 'UTF-8')
digester = hmac.new(key, base, hashlib.sha1)
binsignature = digester.digest()
signature = urllib.parse.quote(base64.urlsafe_b64encode(binsignature), safe='')
oauth_header = ('Oauth ' + 'oauth_nonce="%s", ' % urllib.parse.quote(oauth_nonce, safe='') +
'oauth_callback="%s", ' % urllib.parse.quote(oauth_callback, safe='') +
'oauth_signature_method="%s", ' % urllib.parse.quote('HMAC-SHA1', safe='') +
'oauth_timestamp="%s", ' %urllib.parse.quote(str(oauth_timestamp), safe='') +
'oauth_consumer_key="%s", ' % urllib.parse.quote(oauth_consumer_key, safe='') +
'oauth_version="%s", ' % urllib.parse.quote('1.0', safe='') +
'oauth_signature="%s"' % signature
)
headers = {'Authorization': oauth_header}
#urllib wants a body to know that it's a POST, at least that's what the docs said, so I'm providing an empty one
values = {}
data = urllib.parse.urlencode(values).encode('ascii')
TokenRequest = urllib.request.Request(url, data, headers)
try:
print('opening request token url...')
with urllib.request.urlopen(TokenRequest) as response:
if response.getcode() != 200:
print('Response is %s' % response.getcode())
return HttpResponse('Error in getting the token from Twitter, please try again...')
body = loads(response.read())
if body['oauth_callback_confirmed'] != 'true':
print('oauth_callback not confirmed')
return HttpResponse('Error in getting the token from Twitter, please try again...')
oauth_token = body['oauth_token']
oauth_token_secret = body['oauth_token_secret']
except urllib.error.HTTPError as err:
#my program always ends up here, catches the exception and returns the line below
return HttpResponse('Error %s in getting the token from Twitter, please try again...' % err)
print('Successfully retrieved request token! Redirecting...')
loggee = User.objects.Create()
loggee.oauth_token = oauth_token
loggee.oauth_token_secret = oauth_token_secret
return HttpResponseRedirect('https://api.twitter.com/oauth/authenticate?oauth_token='+oauth_token)
Any help will be appreciated! I'm quite in a hurry and I really can't wrap my head around this!
Your signature base string needs to be sorted lexigraphically (which means alphabetically for all intents), before you calculate your signature (ref: Creating a signature)
Therefore your parameter string needs to be:
parameter_string = (
'oauth_callback=' + urllib.parse.quote(oauth_callback, safe='') +
'&oauth_consumer_key=' + urllib.parse.quote(oauth_consumer_key, safe='') +
'&oauth_nonce=' + urllib.parse.quote(oauth_nonce, safe='') +
'&oauth_signature_method=' + urllib.parse.quote('HMAC-SHA1', safe='') +
'&oauth_timestamp=' + urllib.parse.quote(str(oauth_timestamp), safe='') +
'&oauth_version=' + urllib.parse.quote('1.0', safe='')
)
I have below query stored in a variable I got and I need to fetch value of 'resource_status'.
I need 'UPDATE_IN_PROGRESS'
As requested, putting the code here. The variable evntsdata is storing the events list.
try:
evntsdata = str(hc.events.list(stack_name)[0]).split(" ") # this is the variable that is getting the JSON response (or so)
#print(evntsdata[715:733])
#event_handle = evntsdata[715:733]
if event_handle == 'UPDATE_IN_PROGRESS':
loopcontinue = True
while loopcontinue:
evntsdata = str(hc.events.list(stack_name)[0]).split(" ")
#event_handle = evntsdata[715:733]
if (event_handle == 'UPDATE_COMPLETE'):
loopcontinue = False
print(str(timestamp()) + " " + "Stack Update is Completed!" + ' - ' + evntsdata[-3] + ' = ' + evntsdata[-1])
else:
print(str(timestamp()) + " " + "Stack Update in Progress!" + ' - ' + evntsdata[-3] + ' = ' + evntsdata[-1])
time.sleep(10)
else:
print("No updates to perform")
exit(0)
except AttributeError as e:
print(str(timestamp()) + " " + "ERROR: Stack Update Failure")
raise
print(evntsdata) has below result
['<Event', "{'resource_name':", "'Stackstack1',", "'event_time':", "'2017-05-26T12:10:43',", "'links':", "[{'href':", "'x',", "'rel':", "'self'},", "{'href':", "'x',", "'rel':", "'resource'},", "{'href':", "'x',", "'rel':", "'stack'}],", "'logical_resource_id':", "'Stackstack1',", "'resource_status':", "'UPDATE_IN_PROGRESS',", "'resource_status_reason':", "'Stack", 'UPDATE', "started',", "'physical_resource_id':", "'xxx',", "'id':", "'xxx'}>"]
Do not serialize and parse objects when the data is in front of you. This is inefficient and hard to understand and maintain. The solution is quite trivial:
data = hc.events.list(stack_name)[0].to_dict()
event_handle = data['resource_status']
It's not JSON, it's a class that you've printed
class Event(base.Resource):
def __repr__(self):
return "<Event %s>" % self._info
Try poking around the source code to get access to the dictionary self._info, then access your fields according
For example,
event_info = hc.events.list(stack_name)[0]._info
event_handle = event_info['resource_status']
Though, there may be another way like calling to_dict() instead, since the underscore indicates a private variable
I have a python class working in zope 3 zcml kind of way, but i want to move the python into a standalone script that a could access via something along the lines of tal:content='context/get_tags'. This is the code as it stands:
class TagListView(BrowserView):
def getCategories(self):
categories = set()
for cat in self.portal_catalog.uniqueValuesFor('Subject'):
categories.add(cat.lower())
for cat in self.__mapping:
categories.add(cat.lower())
return tuple(sorted(categories))
def getSynonyms(self,category):
r = self.__mapping.get(category)
if r is None:
return ()
return r[0]
def __init__(self,context,request):
self.context = context
self.request = request
self.tool = self.context.portal_categories
def entries(self):
taglist = '(['
for category in self.tool.getCategories():
taglist = taglist + '\'' + category + '\','
for synonym in self.tool.getSynonyms(category):
if len(synonym) > 0:
taglist = taglist + '\'' + synonym + '\','
taglist = taglist + '])'
return taglist
Not great (as you may have guessed programmer isn't my job title) but it's what i have. How do I convert it to work as a standalone script?
You can access views from page templates with the ## syntax: context/##viewname:
tal:define="view context/##get_tags;
entries view/entries;"