I have a python project with Flask.
I'm using SQL Alchemy (according to this page of the documentation : http://flask.pocoo.org/docs/0.10/patterns/sqlalche) to handle my database actions.
I'm using Flask.session to store user's information (authentication status, preferences, ...)
Default Flask's Session behaviour is to store sessions in user's cookie, and to sign this cookie with secret_key so users can't alter it, but they can read it.
I don't like that my users are able to "see" session's content. Does Flask offer a built-in way to store session's content in ORM (SQLAlchemy), or do I have to implement that myself ?
Thanks !
This was adapted from http://flask.pocoo.org/snippets/75/.
If you need to store a lot of session data it makes sense to move the data from the cookie to the server. In that case you might want to use redis as the storage backend for the actual session data.
The following code implements a session backend using redis. It allows you to either pass in a redis client or will connect to the redis instance on localhost. All the keys are prefixed with a specified prefix which defaults to session:.
import pickle
from datetime import timedelta
from uuid import uuid4
from redis import Redis
from werkzeug.datastructures import CallbackDict
from flask.sessions import SessionInterface, SessionMixin
class RedisSession(CallbackDict, SessionMixin):
def __init__(self, initial=None, sid=None, new=False):
def on_update(self):
self.modified = True
CallbackDict.__init__(self, initial, on_update)
self.sid = sid
self.new = new
self.modified = False
class RedisSessionInterface(SessionInterface):
serializer = pickle
session_class = RedisSession
def __init__(self, redis=None, prefix='session:'):
if redis is None:
redis = Redis()
self.redis = redis
self.prefix = prefix
def generate_sid(self):
return str(uuid4())
def get_redis_expiration_time(self, app, session):
if session.permanent:
return app.permanent_session_lifetime
return timedelta(days=1)
def open_session(self, app, request):
sid = request.cookies.get(app.session_cookie_name)
if not sid:
sid = self.generate_sid()
return self.session_class(sid=sid, new=True)
val = self.redis.get(self.prefix + sid)
if val is not None:
data = self.serializer.loads(val)
return self.session_class(data, sid=sid)
return self.session_class(sid=sid, new=True)
def save_session(self, app, session, response):
domain = self.get_cookie_domain(app)
if not session:
self.redis.delete(self.prefix + session.sid)
if session.modified:
response.delete_cookie(app.session_cookie_name,
domain=domain)
return
redis_exp = self.get_redis_expiration_time(app, session)
cookie_exp = self.get_expiration_time(app, session)
val = self.serializer.dumps(dict(session))
self.redis.setex(self.prefix + session.sid, val,
int(redis_exp.total_seconds()))
response.set_cookie(app.session_cookie_name, session.sid,
expires=cookie_exp, httponly=True,
domain=domain)
Here is how to enable it:
app = Flask(__name__)
app.session_interface = RedisSessionInterface()
If you get an attribute error that total_seconds is missing it means you're using a version of Python older than 2.7. In this case you can use this function as a replacement for the total_seconds method:
def total_seconds(td):
return td.days * 60 * 60 * 24 + td.seconds
Related
I want to test my tornado python application with pytest.
for that purpose, I want to have a mock db for the mongo and to use motor "fake" client to simulate the calls to the mongodb.
I found alot of solution for pymongo but not for motor.
any idea?
I do not clearly understand your problem — why not just have hard-coded JSON data?
If you just want to have a class that would mock the following:
from motor.motor_tornado import MotorClient
client = MotorClient(MONGODB_URL)
my_db = client.my_db
result = await my_db['my_collection'].insert_one(my_json_load)
So I recommend creating a Class:
Class Collection():
database = []
async def insert_one(self,data):
database.append(data)
data['_id'] = "5063114bd386d8fadbd6b004" ## You may make it random or consequent
...
## Also, you may save the 'database' list to the pickle on disk to preserve data between runs
return data
async def find_one(self, data):
## Search in the list
return data
async def delete_one(self, data_id):
delete_one.deleted_count = 1
return
## Then create a collection:
my_db = {}
my_db['my_collecton'] = Collection()
### The following is the part of 'views.py'
from tornado.web import RequestHandler, authenticated
from tornado.escape import xhtml_escape
class UserHandler(RequestHandler):
async def post(self, name):
getusername = xhtml_escape(self.get_argument("user_variable"))
my_json_load = {'username':getusername}
result = await my_db['my_collection'].insert_one(my_json_load)
...
return self.write(result)
If you would clarify your question, I will develop my answer.
I am using flask to create simple api. The api simply returns values from mongoDB. Everything works great if i do the connection within same function. I am not doing connection simply at start of file because i am using uwsgi and nginx server on ubuntu. If i do that then there will be a problem of fork.
However, I have to use this connection with other api so thought to make a seperate class for connection and each api will simply call it . I m using this functionality to make codes manageable. However when i try the these codes it always shows internal server error. I tried making this function static too , still the error exists.
Note - I have replaced mongodb address with xxx as i am using mongodbatlas account here
from flask import Flask
from flask import request, jsonify
from flask_pymongo import pymongo
from pymongo import MongoClient
from flask_restful import Resource, Api, reqparse
app = Flask(__name__)
api = Api(app)
#client = MongoClient("xxx")
#db = client.get_database('restdb')
#records = db.stars
class dbConnect():
def connect(self):
client = MongoClient("xxx")
db = client.get_database('restdb')
records = db.stars
return records
class Order(Resource):
def get(self):
#client = MongoClient("xxx")
#db = client.get_database('restdb')
#records = db.stars
#star = records
star = dbConnect.connect
output = []
for s in star.find():
output.append({'name' : s['name'], 'distance' : s['distance']})
return jsonify({'result' : output})
api.add_resource(Order, '/')
if __name__ == "__main__":
app.run(host='0.0.0.0')
ERROR {"message": "Internal Server Error"}
Preliminary investigation suggests that you haven't instantiated your dbConnect class. Also, you haven't called the method connect properly.
class Order(Resource):
def get(self):
db = dbConnect() # This was missing
star = db.connect() # This is how you make method call properly.
output = []
for s in star.find():
output.append({'name' : s['name'], 'distance' : s['distance']})
return jsonify({'result' : output})
Also class dbConnect() should be declared as class dbConnect:.
I'm working on building up an API endpoint w/ flask and flask_limiter, and I'm running into an issue. My goal is to rate limit on BOTH the client IP address, as well as an API key that the client provides (accessible through flask's "request" module), on a single endpoint.
The IP address limiter is important because I'd like to protect against brute force attacks, whereas the API key limiter is simply for business reasons. I've built up both rate limiters, and independently they work (i.e. I can limit on the IP address OR the provided API key), but I haven't been able to get both running on the endpoint at the same time. An example of what I've tried out so far:
from flask import Flask, request
from flask_limiter import Limiter
from flask_limiter.util import get_remote_address
app = Flask(__name__)
ip_limiter = Limiter(app, key_func=get_remote_address)
get_api_key = lambda : request.args.get('apikey')
api_key_limiter = Limiter(app, key_func=get_api_key)
#app.route('/theEndpoint')
#ip_limiter.limit('some_limit_here')
#api_key_limiter.limit('other_limit_here')
def theEndpointFunction(...):
.......
It would be nice to implement all rate limits in a single package (i.e. flask_limiter), but thus far the only way I can think to implement rate limits on multiple keys is to switch over to something like redis.
Any clues on a way to implement multi-key rate-limits with flask_limiter?
In case anyone is interested, I've decided to switch over to Redis to implement my dual-key rate limit. The solution goes something like this:
class RateLimit(object):
def __init__(self, key, max_requests, seconds):
self.reset = int(time.time()) + seconds
self.key = key
self.max_requests = max_requests
self.seconds = seconds
p = redis.pipeline()
p.incr(self.key)
p.expireat(self.key, self.reset)
self.current = min(p.execute()[0], max_requests)
remaining = property(lambda x: x.max_requests - x.current)
over_limit = property(lambda x: x.current >= x.max_requests)
def get_view_rate_limit():
return getattr(g, '_view_rate_limit', None)
def over_limit(limit):
#formatting a JSON response
response = {"result": "Max number of requests exceeded",
"status": False}
response = jsonify(response)
response.status_code = 400
return response
def ratelimit(max_requests, seconds, key_func, over_limit=over_limit):
def decorator(f):
def rate_limited(*args, **kwargs):
key = key_func()
rlimit = RateLimit(key, max_requests, seconds)
g._view_rate_limit = rlimit
if over_limit is not None and rlimit.over_limit:
return over_limit(rlimit)
return f(*args, **kwargs)
return update_wrapper(rate_limited, f)
return decorator
RATE_LIMIT_FUNCS = {'apikey': lambda: request.args.get('apikey'),
'ip': lambda: get_remote_address}
and then decorating the flask endpoint:
#app.route('/theEndpoint')
#ratelimit(max_requests=5, seconds=300, key_func=RATE_LIMIT_FUNCS['apikey'])
#ratelimit(max_requests=15, seconds=300, key_func=RATE_LIMIT_FUNCS['ip'])
def theEndpointFunction():
....
Flask limiter has since provided support for different kinds of storage:
Either through config/env values stored in RATELIMIT_STORAGE_URI or passed when initializing the Limiter object through the storage_uri kwarg. The value expected is the storage type and the Host to the storage system, if needed.
https://flask-limiter.readthedocs.io/en/stable/configuration.html#RATELIMIT_STORAGE_URI
As the documentation notes, it supports
In memory (by default) with memory://
Memcached: memcached://host:port
MongoDB: mongodb://host:port
Redis: redis://host:port
example
ip_limiter = Limiter(
app,
key_func=get_remote_address,
storage_uri='redis://127.1.0.1:6379'
)
I'm writing a simple internal REST API for our solution using Flask, serving JSON objects through get calls (including authentication). We have multiple backends to fetch data from. From what I understand these should be connected to in a function decorated with #app.before_request and assigned to the g global for use in the specific route being requested. It's not a pattern I'm used to.
Here is a toy example of what I'm doing:
#app.before_request
def before_request():
g.some_conn_a = create_connection('a')
g.some_conn_b = create_connection('b')
g.some_client = create_client()
#app.route('/get_some_data')
#requires_auth
def get_some_data():
# Fetch something from all connections in g
payload = ... # Construct payload using above connections
return jsonify(payload)
#app.route('/get_some_other_data')
#requires_auth
def get_some_other_data():
# Fetch something from maybe just g.some_conn_b
payload = ... # Construct payload using g.some_conn_b
return jsonify(payload)
This seems wasteful to me if the user makes a request for data residing in only one or two of these connections/clients, like in the get_some_other_data route example.
I'm considering just making the connections/clients in the route functions instead, or load it lazily. What's the "correct" way? I hope it isn't to make a new module, that seems extreme for what I'm doing.
Riffing on the Flask docs Database Connections example
you could modify get_db() to accept an argument for each of your multiple connections.
def get_db(conn):
"""Open specificied connection if none yet for the current app context. """
if conn == 'some_conn_a':
if not hasattr(g, 'some_conn_a'):
g.some_conn_a = create_connection('a')
db = g.some_conn_a
elif conn == 'some_conn_b':
if not hasattr(g, 'some_conn_b'):
g.some_conn_b = create_connection('b')
db = g.some_conn_b
elif conn == 'some_client':
if not hasattr(g, 'some_client'):
g.some_client = create_client()
db = g.some_client
else:
raise Exception("Unknown connection: %s" % conn)
return db
#app.teardown_appcontext
def close_db(error):
"""Closes the db connections. """
if hasattr(g, 'some_conn_a'):
g.some_conn_a.close()
if hasattr(g, 'some_conn_b'):
g.some_conn_b.close()
if hasattr(g, 'some_client'):
g.some_client.close()
Then you could query each connection as needed:
#app.route('/get_some_data')
def get_some_data():
data_a = get_db('some_conn_a').query().something()
data_b = get_db('some_conn_b').query().something()
data_c = get_db('some_client').query().something()
payload = {'a': data_a, 'b': data_b, 'c': data_c}
return jsonify(payload)
The get_db() pattern is preferred over the before_request pattern for lazy loading database connections. The docs examples for Flask 0.11 and up utilize the get_db() pattern to a larger extent.
I'm developing a twisted.web server - it consists of some resources that apart from rendering stuff use adbapi to fetch some data and write some data to postgresql database. I'm trying to figoure out how to write a trial unittest that would test resource rendering without using net (in other words: that would initialize a resource, produce it a dummy request etc.).
Lets assume the View resource is a simple leaf that in render_GET returns NOT_DONE_YET and tinkers with adbapi to produce simple text as a result. Now, I've written this useless code and I can't come up how to make it actually initialize the resource and produce some sensible response:
from twisted.trial import unittest
from myserv.views import View
from twisted.web.test.test_web import DummyRequest
class ExistingView(unittest.TestCase):
def test_rendering(self):
slug = "hello_world"
view = View(slug)
request = DummyRequest([''])
output = view.render_GET(request)
self.assertEqual(request.responseCode, 200)
The output is... 1. I've also tried such approach: output = request.render(view) but same output = 1. Why? I'd be very gratefull for some example how to write such unittest!
Here's a function that will render a request and convert the result into a Deferred that fires when rendering is complete:
def _render(resource, request):
result = resource.render(request)
if isinstance(result, str):
request.write(result)
request.finish()
return succeed(None)
elif result is server.NOT_DONE_YET:
if request.finished:
return succeed(None)
else:
return request.notifyFinish()
else:
raise ValueError("Unexpected return value: %r" % (result,))
It's actually used in Twisted Web's test suite, but it's private because it has no unit tests itself. ;)
You can use it to write a test like this:
def test_rendering(self):
slug = "hello_world"
view = View(slug)
request = DummyRequest([''])
d = _render(view, request)
def rendered(ignored):
self.assertEquals(request.responseCode, 200)
self.assertEquals("".join(request.written), "...")
...
d.addCallback(rendered)
return d
Here is a DummierRequest class that fixes almost all my problems. Only thing left is it does not set any response code! Why?
from twisted.web.test.test_web import DummyRequest
from twisted.web import server
from twisted.internet.defer import succeed
from twisted.internet import interfaces, reactor, protocol, address
from twisted.web.http_headers import _DictHeaders, Headers
class DummierRequest(DummyRequest):
def __init__(self, postpath, session=None):
DummyRequest.__init__(self, postpath, session)
self.notifications = []
self.received_cookies = {}
self.requestHeaders = Headers()
self.responseHeaders = Headers()
self.cookies = [] # outgoing cookies
def setHost(self, host, port, ssl=0):
self._forceSSL = ssl
self.requestHeaders.setRawHeaders("host", [host])
self.host = address.IPv4Address("TCP", host, port)
def addCookie(self, k, v, expires=None, domain=None, path=None, max_age=None, comment=None, secure=None):
"""
Set an outgoing HTTP cookie.
In general, you should consider using sessions instead of cookies, see
L{twisted.web.server.Request.getSession} and the
L{twisted.web.server.Session} class for details.
"""
cookie = '%s=%s' % (k, v)
if expires is not None:
cookie = cookie +"; Expires=%s" % expires
if domain is not None:
cookie = cookie +"; Domain=%s" % domain
if path is not None:
cookie = cookie +"; Path=%s" % path
if max_age is not None:
cookie = cookie +"; Max-Age=%s" % max_age
if comment is not None:
cookie = cookie +"; Comment=%s" % comment
if secure:
cookie = cookie +"; Secure"
self.cookies.append(cookie)
def getCookie(self, key):
"""
Get a cookie that was sent from the network.
"""
return self.received_cookies.get(key)
def getClientIP(self):
"""
Return the IPv4 address of the client which made this request, if there
is one, otherwise C{None}.
"""
return "192.168.1.199"