I am using GAE with python, and I am using many forms. Usually, my code looks something like this:
class Handler(BaseHandler):
#...
def post(self):
name = self.request.get("name")
last_name = self.request.get("last_name")
# More variables...
n = self.request.get("n")
#Do something with the variables, validations, etc.
#Add them to a dictionary
data = dict(name=name, last_name=last_name, n=n)
info = testdb.Test(**data)
info.put()
I have noticed lately that it gets too long when there are many inputs in the form (variables), so I thought maybe I could send a stringified JSON object (which can be treated as a python dictionary using json.loads). Right now it looks like this:
class Handler(BaseHandler):
#...
def post(self):
data = validate_dict(json.loads(self.request.body))
#Use a variable like this: data['last_name']
test = testdb.Test(**data)
test.put()
Which is a lot shorter. I am inclined to do things this way (and stop using self.request.get("something")), but I am worried I may be missing some disadvantage of doing this apart from the client needing javascript for it to even work. Is it OK to do this or is there something I should consider before rearranging my code?
There is absolutely nothing wrong with your short JSON-focused code variant (few web apps today bother supporting clients w/o Javascript anyway:-).
You'll just need to adapt the client-side code preparing that POST, from being just a traditional HTML form, to a JS-richer approach, of course. But, I'm pretty sure you're aware of that -- just spelling it out!-)
BTW, there is nothing here that's App Engine - specific: the same considerations would apply no matter how you chose to deploy your server.
Related
In my Django project, I have a situation where different views communicate via a request session data as follows:
def view_one(request):
...
request.session['my_session_key'] = data
...
def view_two(request):
...
data = request.session['my_session_key']
...
However, this has the following problems:
The key string my_session_key is not a constant so it will be hard to scale it up to other parts of the code if I start writing to and/or reading from it in other parts of my code.
As this system grows, it will become harder to identify which are the available keys being used and written in the session.
In Kotlin (which I'm more familiar with), one way to solve this would be using an extension function, like so:
var Request.my_session_key: Int
get() = this.session['my_session_key']
set(value) { this.session['my_session_key'] = value }
This way, I could now write my views as follows:
def view_one(request):
...
request.my_session_key = data
...
def view_two(request):
...
data = request.my_session_key
...
Is there any way to accomplish something similar in Python or with Django? Or, alternatively, taking a step back, what would be the best way to organize the data stored in a Django session across multiple views?
I'm trying to figure out if I'm following a user from which the streaming API just received a tweet. If I don't, then I want to follow him.
I've got something like:
def checkFollow(status):
relationship = api.lookup_friendships("Privacy_Watch_",status.user.id_str)
From there, how do I check if I follow this user already?
The lookup_friendships method will return everyone you follow each time you call it, in blocks of 100 users. Provided you follow a lot of people, that will be highly inefficient and consume a lot of requests.
You can use instead the show_friendship method, it will return a JSON containing information about your relationship with the id provided.
I cannot test it right now, but the following code should do what you want:
def checkFollow(status):
relation = api.show_friendship(source_screen_name=your_user_name, target_screen_name=status.user.id_str)
if relation.target.following: #I'm not sure if it should be "target" or "source" here
return True
return False
I am currently trying to write a little web-application using python, flask and sqlite and I'm not sure about how to handle the database-connections.
Basically I've been following the "official" Tutorial (and http://flask.pocoo.org/docs/patterns/sqlite3/#sqlite3 ), leaving me with code like this in my main Flask/App module (drastically shortened):
#vs_app.before_request
def before_request():
g.db = sqlite3.connect("somedb.db")
def execute_db(command):
return g.db.cursor().execute(command).fetchall()
#app.route("/givemeallusers")
def foo():
return execute_db("SELECT * FROM users")
So this creates a DB-Connection for each request that can be used by my application to execute sql-commands and it works fine for smaller applications.
But for a larger project I'd like to put my database-code (with some utility methods) and app-code in different modules, so I could (for example) write:
from my_database_module import user_auth #
#app.route("/login")
def foo():
if user_auth(request.form["name"], request.form["pw"]):
pass
But for that I would need access to g (or g.db) from my database-class and the only way I could get that to work was by "manually" handing over the db-connection to each method, leaving me with code like this:
#app.route("/login")
def foo():
if user_auth(g.db, request.form["name"], request.form["pw"]):
pass
and in my database-module
def user_auth(database, name, pw):
pass
I don't think thats the best approach, but other ideas I had (like importing the g object into my database-class) don't work. I also don't know whether my approach is safe with regards to concurrent db-access, so any help with this would be appreciated.
tl;dr How to split Flask-App and Database/Database-Utility the right way?
Just in case someone stumbles upon this question:
The correct answer is to not bother with the approaches described in the Question, and instead just start working with Flask-SQLAlchemy.
I've written a program in Python, which works with two distinct API to get the data from two different services (CKAN and MediaWiki).
In particular, there is a class Resource, which requests the data from the above mentioned services and process it.
At some point I've come to conclusion, that there is a need for tests for my app.
And the problem is that all examples I've found on web and in books do not deal with such cases.
For example, inside Resource class I've got a method:
def load_from_ckan(self):
"""
Get the resource
specified by self.id
from config.ckan_api_url
"""
data = json.dumps({'id': self.id})
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
url = config.ckan_api_url + '/action/resource_show'
r = requests.post(url, timeout=config.ckan_request_timeout, data=data, headers=headers)
assert r.ok, r
resource = json.loads(r.content)
resource = resource["result"]
for key in resource:
setattr(self, key, resource[key])
The load_from_ckan method get the data about resource from the CKAN API and assign it to the object. It is simple, but...
My question is: how to test the methods like this? OR What should I test here?
I thought about the possibility to pickle (save) results to HDD. Then I could load it in the test and compare with the object initialized with load_from_ckan(). But CKAN is community-driven platform and such behavior of such tests will be unpredictable.
If there exist any books on philosophy of automated tests (like what to test, what not to test, how to make tests meaningful etc.), please, give me a link to it.
With any testing, the key question really is - what could go wrong?
In your case, it looks like the three risks are:
The web API in question could stop working. But you check for this already, with assert r.ok.
You, or someone else, could make a mistaken change to the code in future (e.g. mistyping a variable) which breaks it.
The API could change, so that it no longer returns the fields or the format you need.
It feels like you could write a fairly simple test for the latter two, depending on what data from this API you actually rely on: for example, if you're expecting the JSON to have a field called "temperature" which is a floating-point Celsius number, you could write a test which calls your function, then checks that self.temperature is an instance of 'float' and is within a sensible range of values (-30 to 50?). That should leave you confident that both the API and your function are working as designed.
Typically if you want to test against some external service like this you will need to use a mock/dummy object to fake the api of the external service. This must be configurable at run-time either via the method's arguments or the class's constructor or another type of indirection. Another more complex option would be to monkey patch globals during testing, like "import requests; request.post = fake_post", but that can create more problems.
So for example your method could take an argument like so:
def load_from_ckan(self, post=requests.post):
# ...
r = post(url, timeout=config.ckan_request_timeout, data=data,
headers=headers)
# ...
Then during testing your would write your own post function that returned json results you'd see coming back from ckan. For example:
def mock_post(url, timeout=30, data='', headers=None):
# ... probably check input arguments
class DummyResponse:
pass
r = DummyResponse()
r.ok = True
r.content = json.dumps({'result': {'attr1': 1, 'attr2': 2}})
return r
Constructing the result in your test gives you a lot more flexibility than pickling results and returning them because you can fabricate error conditions or focus in on specific formats your code might not expect but you know could exist.
Overall you can see how complicated this could become so I would only start adding this sort of testing if you are experiencing repeated errors or other difficulties. This will just more code you have to maintain.
At this point, you can test that the response from CKAN is properly parsed. So you can pull the JSON from CKAN and ensure that it's returning data with the attributes you're interested in.
recently I wanted to return through a Django view a dynamically generated XML tree. The module I use for XML manipulation is the usual cElementTree.
I think I tackled what I wanted by doing the following:
def view1(request):
resp = HttpResponse(g())
return resp
def g():
root = Element("ist")
list_stamp = SubElement(root, "list_timestamp")
list_creation = str(datetime.now())
for i in range(1,1000000):
root.text = str(i)
yield cET.tostring(root)
Is something like this a good idea ? Do I miss something ?
About middlewares "breaking" streaming:
CommonMiddleware will try to consume the whole iterator if you set USE_ETAGS = True in settings. But in modern Django (1.1) there's a better way to do conditional get than CommonMiddleware + ConditionalGetMiddleware -- condition decorator. Use that and your streaming will stream okay :-)
Another thing that will try to consume the iterator is GzipMiddleware. If you want to use it you can avoid gzipping your streaming responses by turning it into a decorator and applying to individual views instead of globally.
Does it work? If it doesn't work, what error does it throw?
If you're building a full-blown API for a django site, take a look at django-piston. It takes care of a lot of the busywork related to that.
http://bitbucket.org/jespern/django-piston/wiki/Home
Yes, it's perfectly legitimate to return an iterator in an HttpResponse. As you've discovered, that allows you to stream content to the client.
Yes. That's THE WAY you do it on Django.