Develop Reference

need help on using arguments out the function decorator in flask

I want to use parameters outside of the decarator, i tried using it but getting below error
I am using flask API endpoints to give arg parameters such as source, destination and prot and protcol and SDC , DDC
In my casse when i use /t2e/ get method i have to use four parameters but when i use /junos/ then need to use 6 parametes , i dont want to repeat 4 parameters such as source, destination and prot and protcol. hence i tried use them above the method as below
Non working and getting error as
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
from flask import Flask, request, jsonify
from main.hunter_function import t2e, junos,
method = Flask(__name__)
args = request.args
source = args.get('source')
destination = args.get('destination')
port = args.get('port')
protocol = args.get('protocol')
#method.route('/t2e/', methods=['GET']) # is a decorator used to match URLs to view functions in Flask apps
def method_t2e():
logger.info("method is fetching t2e output")
output = t2e(source, destination, port, protocol)
return output
#method.route('/junos/', methods=['GET']) # is a decorator used to match URLs to view functions in Flask apps
def method_junos():
sdc = args.get('sdc')
ddc = args.get("ddc")
logger.info("method is fetching Junos output from link")
output = jun(source, destination, port, protocol,sdc,ddc)
return output
if __name__ == '__main__':
method.run(ssl_context="adhoc",host='0.0.0.0', port=5002, debug=True)
Here is working code:
#method.route('/t2e/', methods=['GET']) # is a decorator used to match URLs to view functions in Flask apps
def method_t2e():
args = request.args
source = args.get('source')
destination = args.get('destination')
port = args.get('port')
protocol = args.get('protocol')
logger.info("method is fetching t2e output")
output = t2e(source, destination, port, protocol)
return output
#method.route('/junos/', methods=['GET']) # is a decorator used to match URLs to view functions in Flask apps
def method_junos():
args = request.args
source = args.get('source')
sdc = args.get('sdc')
destination = args.get('destination')
ddc = args.get("ddc")
port = args.get('port')
protocol = args.get('protocol')
logger.info("method is fetching Junos output from link")
output = jun(source, destination, port, protocol,sdc,ddc)
return output
I don't want to repeat the parameters more than once

You could create a function which returns those to reduce code duplication such as
def get_args(args):
return args.get('source'), args.get('destination'), args.get('port'), args.get('protocol')
#method.route('/t2e/', methods=['GET'])
def method_t2e():
args = request.args
source, destination, port, protocol = get_args(args)
The issue with what you're trying to do is the request variable doesn't exist outside the call to your methods so in order to get them you need to pass the specific request for the method somewhere

POST/PUT input to Rest API

I am launching REST API build in python.
I am inputting a list as input and getting the required data.
example:
your.api.com/birth?name=James&date=2015-02-01&name=Robert&date=2020-01-01
from flask import request
#app.route('/birth')
def birth():
names = request.form.getlist('name')
dates = request.form.getlist('date')
As the number of inputs I have is huge, the end point URL is becoming huge. Is there any way to do the same using PUT or POST where I dump a doc in some format (say json) as my input?

Sure, put the request fields in a JSON and go through the dictionary
#app.route('/birth', methods=['POST'])
def birth():
data = request.get_json()
names = data["names"]
# etc.

When the input grows it is suggested to use a POST call rather than GET call.
from flask import request
#app.route('/birth')
def birth():
names = request.form.getlist('name')
dates = request.form.getlist('date')
Convert /birth to the following
from flask import request
#app.route('/birth', methods=['POST'])
def birth():
input = request.get_json()
# <Do the processing>
And in the POST call from client-side use a JSON like the following
{
name: [<array of values>],
date: [<array of values>]
}

Slack Interactive Messages: POST request payload has an unexpected format

I'm getting a POST request inside a Flask app from Slack. The request is sent when a user presses on an interactive message button. According to Slack docs I must extract the body of the request to verify the signature.
My computed signature doesn't match the one sent by Slack, though.
In fact, the body of the request comes as some encoded string. The string is actually an encoded dictionary instead of a query str parameters, as expected.
Here's the beginning of my view:
#app.route('/register', methods=['POST'])
def register_visit():
data = request.get_data()
signature = request.headers.get('X-Slack-Signature', None)
timestamp = request.headers.get('X-Slack-Request-Timestamp', None)
signing_secret = b'aaaaaaaaaaaaaaaa'
# old message, ignore
if round(actual_time.time() - float(timestamp)) > 60 * 5:
return
concatenated = ("v0:%s:%s" % (timestamp, data)).encode('utf-8')
computed_signature = 'v0=' + hmac.new(signing_secret, msg=concatenated, digestmod=hashlib.sha256).hexdigest()
if hmac.compare_digest(computed_signature, signature):
...
I've tried to format the received data to make it look like:
token=fdjkgjl&user_id=1234... but I am not aware of all of the necessary parameters that have to be present in the data.
Any ideas are highly appreciated.
The body of the message is following - after being URL decoded (note I've modified possibly sensitive data):
b'payload={"type":"interactive_message","actions":
[{"name":"yes_button","type":"button","value":"236"}],"callback_id":"visit_button","team":{"id":"fffff","domain":"ffff"},"channel":{"id":"ffff","name":"directmessage"},"user":{"id":"ffffff","name":"fffft"},"action_ts":"1540403943.419120","message_ts":"1541403949.000100","attachment_id":"1","token":"8LpjBuv13J7xAjhl2lEajoBU","is_app_unfurl":false,"original_message":{"text":"Test","bot_id":"DDDDDDDDD","attachments":[{"callback_id":"visit_button","text":"Register","id":1,"color":"3AA3E3","actions":[{"id":"1","name":"yes_button","text":"Yes","type":"button","value":"236","style":""}],"fallback":"Register"}],"type":"message","subtype":"bot_message","ts":"1540413949.000100"},"response_url":"https://hooks.slack.com/actions/ffffff/ffffff/tXJjx1XInaUhrikj6oEzK08e","trigger_id":"464662548327.425084163429.dda35a299eedb940ab98dbb9386b56f0"}'

The reason you are getting the "garbled" data is that you are using request.get_data(). That method will return the raw data of a request, but not do any decoding for you.
Much more convenient is to use request.form.get('payload'), which will directly give you the JSON string of the request object. You can then convert that into a dict object with json.loads() to process it further in your app.
Note that the format you received is the correct format for interactive messages. You will not get a query string (e.g. "token=abc;user_id?def...") as you suggested (like for slash command requests). Interactive message request will always contain the request as JSON string in a payload form property. See here for reference.
Here is a simple working example, which will reply a greeting to the user that pressed the button. It will work directly with Slack, but I recommend using Postman to test it.
#app.py
from flask import Flask, request #import main Flask class and request object
import json
app = Flask(__name__) #create the Flask app
#app.route('/register', methods=['POST'])
def register_visit():
slack_req = json.loads(request.form.get('payload'))
response = '{"text": "Hi, <#' + slack_req["user"]["id"] + '>"}'
return response, 200, {'content-type': 'application/json'}
if __name__ == '__main__':
app.run(debug=True, port=5000) #run app in debug mode on port 5000

OK, the issue wasn't related to how Slack sends me the message. It was about misunderstanding which data comes as bytes and which data is unicode. The culprit was string formatting in my case - the line concatenated = ("v0:%s:%s" % (timestamp, data)).encode('utf-8') should have been concatenated = (b"v0:%b:%b" % (timestamp.encode("utf-8"), data)). Data is already bytes, timestamp meanwhile is unicode.
Cannot believe I've banged my head on this for hours -_-
#app.route('/register', methods=['POST'])
def register_visit():
data = request.get_data()
signature = request.headers.get('X-Slack-Signature', None)
timestamp = request.headers.get('X-Slack-Request-Timestamp', None)
signing_secret = b'aaaaaaaaaaaaaaaa'
# old message, ignore
if round(actual_time.time() - float(timestamp)) > 60 * 5:
return
concatenated = (b"v0:%b:%b" % (timestamp.encode("utf-8"), data))
computed_signature = 'v0=' + hmac.new(signing_secret, msg=concatenated,
digestmod=hashlib.sha256).hexdigest()
if hmac.compare_digest(computed_signature, signature):
...

This worked for me
from urllib import parse
parsed_text = parse.unquote('your bytes text here')

How to speed up JSON for a flask application?

I'm currently implementing a webapp in flask. It's an app that does a visualization of data gathered. Each page or section will always have a GET call and each call will return a JSON response which then will be processed into displayed data.
The current problem is that some calculation is needed before the function could return a JSON response. This causes some of the response to arrive slower than others and thus making the page loads a bit slow. How do I properly deal with this? I have read into caching in flask and wonder whether that is what the app need right now. I have also researched a bit into implementing a Redis-Queue. I'm not really sure which is the correct method.
Any help or insights would be appreciated. Thanks in advance

Here are some ideas:
If the source data that you use for your calculations is not likely to change often then you can run the calculations once and save the results. Then you can serve the results directly for as long as the source data remains the same.
You can save the results back to your database, or as you suggest, you can save them in a faster storage such as Redis. Based on your description I suspect the big performance gain will be in not doing calculations so often, the difference between storing in a regular database vs. Redis or similar is probably not significant in comparison.
If the data changes often then you will still need to do calculations frequently. For such a case an option that you have is to push the calculations to the client. Your Flask app can just return the source data in JSON format and then the browser can do the processing on the user's computer.
I hope this helps.

You can use
copy_current_request_context and Redis, Thread
It is helpful when you need long time to make JSON response.
The first request maybe slow, but next request will faster.
Example
from datetime import timedelta, datetime
from threading import Thread
from . import dbb, redis_client
from flask import Blueprint, request, jsonify, flash, after_this_request, copy_current_request_context, \
current_app, send_from_directory
from .models import Shop, Customers
def save_customer_json_to_redis(request):
response_json = {
"have_customer": False,
"status": False,
"anythingelse": None,
"message":"False, you have to check..."
}
#print(request.data)
headers = request.headers
Authorization = headers['Authorization']
token = Authorization.replace("Bearer", "")
phone = request.args.get('phone')
if phone is not None and phone != "":
print('token', token, "phone", phone)
now = datetime.utcnow() + timedelta(hours=7)
shop = Shop.query.filter(Shop.private_token == token, Shop.ended_date > now, Shop.shop_active == True).first()
customer = Customers.query.filter_by(shop_id=shop.id, phone=phone).first()
if customer:
redis_name = f'{shop.id}_api_v2_customer_phone_{phone}_customer_id_{customer.id}'
print(redis_name)
response_json["anythingelse"] = ...# do want you want, it need long time to do
response_json["status"] = True
response_json["message"] = "Successful"
redis_client.set(redis_name, json.dumps(response_json)) #Save JSON to Redis
#app.route('/api/v2/customer', methods=['GET'])
def api_customer():
#copy_current_request_context
def do_update_customer_to_redis():# this function to save JSON you want to response next time to Redis
save_customer_json_to_redis(request)
Thread(target=do_update_customer_to_redis).start()
response_json = {
"have_customer": False,
"status": False,
"anythingelse": {},
"message": "False, you have to check..."
}
#print(request.data)
headers = request.headers
Authorization = headers['Authorization']
token = Authorization.replace("Bearer", "")
phone = request.args.get('phone')
if phone is not None and phone != "":
print('token', token, "phone", phone)
now = datetime.utcnow() + timedelta(hours=7)
shop = Shop.query.filter(Shop.private_token == token, Shop.ended_date > now,Shop.shop_active == True).first()
customer = Customers.query.filter_by(shop_id=shop.id, phone=phone).first()
if customer:
redis_name = f'{shop.id}_api_v2_customer_phone_{phone}_customer_id_{customer.id}'
print(redis_name)
try:
response_json = json.loads(redis_client.get(redis_name)) # if have json from app
print("json.loads(redis_client.get(redis_name))")
except Exception as e:
print("json.loads(redis_client.get(redis_name))", e)
#do any thing you want to response json
response_json["anythingelse"] = ...# do want you want, it need long time to do
response_json["message"]= ...#do want you want
#redis_client.set(redis_name, json.dumps(response_json))
response_json["status"] = True
response_json["message"] = "Successful"
return jsonify(response_json)
In the init.py
from flask import Flask
from flask_cors import CORS
from flask_mail import Mail
from flask_sqlalchemy import SQLAlchemy
from redis import Redis
# init SQLAlchemy so we can use it later in our models
dbb = SQLAlchemy(session_options={"autoflush": False})
redis_client = Redis(
host='localhost',
port='6379',
password='your_redis_password'
)
def create_app():
app = Flask(__name__)
...........

Python Dynamically Generate HTML Page on GAE

On GAE, I need to make some REST calls to a PiCloud server, then create an output page based on PiCloud return values. However, it will take several minutes for PiCloud to process the model. Thus I am wondering if I could create a 'loading' page first, and after finishing calculation, present the real output page.
In detail, the questions is how do I keep checking the status of my REST service and then generate different HTML pages based upon.
I appreciate any suggestions and comments!
PS: jQuery BlockUI seems to be good example, but it requires to estimate the timeout duration, which I could not guess...
Function to Call REST Service:
def get_jid(pdf_t, pdf_nop, pdf_p):
response = urlfetch.fetch(url=url, payload=data, method=urlfetch.POST, headers=http_headers)
jid= json.loads(response.content)['jid']
output_st = "running"
while output_st!="done":
response_st = urlfetch.fetch(url='https://api.picloud.com/job/?jids=%s&field=status' %jid, headers=http_headers)
output_st = json.loads(response_st.content)['info']['%s' %jid]['status']
url_val = 'https://api.picloud.com/job/result/?jid='+str(jid)
response_val = urlfetch.fetch(url=url_val, method=urlfetch.GET, headers=http_headers)
output_val = json.loads(response_val.content)['result']
return(jid, output_st, output_val)
Generate HTML Page:
class pdfPage_loading(webapp.RequestHandler):
def post(self):
final_res=get_jid(pdf_t, pdf_nop, pdf_p)[2]
html = html + template.render(templatepath + 'popup_pdf_eco.html', {
'title':'Ubertool',
'model_page':'',
'model_attributes':'Please wait','text_paragraph':''})
self.response.out.write(html)
class pdfPage_done(webapp.RequestHandler):
def post(self):
final_res=get_jid(pdf_t, pdf_nop, pdf_p)[2]
html = html + template.render(templatepath + 'popup_pdf_eco.html', {
'title':'Ubertool',
'model_page':final_res,
'model_attributes':'Please download your PDF here','text_paragraph':''})
self.response.out.write(html)
app_loading = webapp.WSGIApplication([('/.*', pdfPage_loading)], debug=True)
app_done = webapp.WSGIApplication([('/.*', pdfPage_done)], debug=True)
def main():
##Here is the problematic part:
if get_jid(pdf_t, pdf_nop, pdf_p)!='done':
run_wsgi_app(app_pre)
else:
run_wsgi_app(app)
if __name__ == '__main__':
main()

First off, you do not need multiple WSGIApplication handlers. You can base requests off of URLs and GET/POST params.
I would use a mixture of tasks and the Channel API:
When a user visits the page:
Create a channel token for the user to use
Make the REST API calls
Kick off a task (described below) passing it the created token and the id of the PiCloud job
Show the "loading page" and have the user connect to the Channel API with the token created in step 1
In the task, have it check on the status of the PiCloud job. If the status is not done, have the task setup a new task to call itself again, and check on the status of the job. Once the job is complete, pass the data along through the Channel API to the user so that they can then load the contents of the page or redirect to a page that will have the contents ready for them.
Example status check code (this is using the example code from PiCloud, you can use your own requests as you have been to get the status through urlfetch - this is merely meant as an example so you can plug your code in to work as needed):
import cloud
from google.appengine.api import taskqueue
from google.appengine.api import channel
#<Your other hanlders here>
class CheckStatus(webapp.RequestHandler):
def post(self):
job_id = self.request.get('job_id')
token = self.request.get('token')
status = cloud.get('status')
if status != 'done':
taskqueue.add(url='/checkstatus', params={'job_id': job_id, 'token': token}, method="POST", countdown=3)
return
channel.send_message(token, '<some message here>')
app = webapp.WSGIApplication([('/done', pdfPage_done),
('/checkstatus', CheckStatus),
('/.*', pdfPage_loading)], debug=True)
def main():
run_wsgi_app(app)
if __name__ == '__main__':
main()