I have a routing table in apis.py using flask-restful like:
api.add_resource(sonarqube.SonarqubeHistory, '/sonarqube/<project_name>')
api.add_resource(project.ProjectFile, '/project/<sint:project_id>/file')
api.add_resource(redmine.RedmineFile, '/download', '/file/<int:file_id>')
The resource classes are in other modules like sonarqube.py, project.py, or redmine.py.
My question is: Is that possible to also move api.add_resource() into the other modules (For more structured code)? I tried simply move the statement and from apis import api, but since apis.py also do other works it will result circular import.
Is there an example for similar use cases?
Actually for better design, split the routing table into different module from apis.py, as apis.py is also doing other work. For example routingapis.py.
Import the new "routingapis" where ever required.
If still circular imports noticed and unavoidable even after splitting, then defer importing. Write a separate function for adding the resource
something like
def functionToAddResource():
from routingapis import api
api.add_resource(sonarqube.SonarqubeHistory, '/sonarqube/<project_name>')
In php files, this task may be a simple include of another file but I wondered if it was similar practice in Django? like described here: https://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs#include
Short answer: No, Python doesn't perform that way.
Here's a long one:
A little bit about difference between PHP and Python web development
First of all, include and require are parts of PHP language itself. It means that before processing source code PHP interpreter includes source code from files given as arguments to those functions and then makes interpreting. That's the inheritance of the main purpose of PHP design (HTML, or, generally, text preprocessing).
Web development in Python is a bit different. Language itself was designed as general purpose programming language, so it doesn't assume text-processing by default. Code is just executed. That's why web development is usually done using frameworks (or wsgi modules if you prefer doing everything from scratch.
All that above means that all text manipulations (all output you generate, you generally do manually by creating result text string (or strings list when used with buffering) and giving the output result yourself.
Template languages
That's kinda messy point from which template languages were born, becoming essential part of every (or almost every) web application written in Python (you can assume something simalar to Smarty in PHP world). There are a lot of them: jinja, jinja2, mako, chameleon ... and many others. All of them are different but serve the same purpose (easyfying text generation via providing templates with placeholders for data of basic representation logic).
And here comes the main feature of Python web frameworks: application logic and representation are strictly splitted.
Here we came to the substance of your question:
As all of those template libraries providing text templating facilities are different, there's no common way to make PHP-like includes that we can say they are general for Python. Each of those tools provide different ways and philosophy of handling, in particular, includes, mainly divided into two mainstream ways:
macros- and slots-based (Chameleon)
template inheritance-based (Jinja2, Django template language)
All you need is to take one that best fits your needs (whether you like/unlike their syntax, some of them are faster, some are more readable, and so on..) and work with it combining with other tools that provide logic for your application.
Getting closer: About Django
It's common to use include template tag when you want to include some re-used part of application (e.g. pagination widget), for which you've adapted your view context.
But when you have common header for all pages, or another common part, it's better to use inheritance, that's achieved in two steps:
Basic template is created (containing general header and footer)
Specific templates are created via template inheritance feature, in particular, extends template tag in Django
Python is not PHP!
Django doesn't allow you to write arbitrary scripts(code) in templates. You should do all the computations/business tasks in views.py (controller) and pass results into template. In template usually you iterate over your result set and display data. In django 1.4 a new assignment tag was added , so you can do
{% get_movie_tickets_for_theater movie theater as tickets %}
but your get_movie_tickets_for_theater will be defined in templatetags python file.
I've started looking in to unittest when using google app engine. And it seems to bit tricky from what I've read. Since you can't (and not suppose to) run your test's against the datastore.
I've written an abstract class to emulate a datastore model class. And it quite works pretty nice returning mockup data on get, all, fetch and so on (only tried on a small scale) returning dbModel like results.
The one thing I haven't found a solution I'm satisfied with is how to differentiate which model class to use. I want to use the mock-ups for unit tests and the actual db.Model for when webapp is running.
My current solution looks like this in my .py containing all db.Models:
if 'SERVER_SOFTWARE' in os.environ:
class dbTest(db.Model):
content = db.StringProperty()
comments = db.ListProperty(str)
else:
class dbTest(Abstract):
content = 'Test'
comments = ['test1', 'test2']
And it kinda feels like it could break any minute. Is this the way to go or could one combine these as one class and if the db.Model is invoked properly use that else the mockup?
Check out gaetestbed (docs). It stubs out the datastore (and all the other services like memcache) and makes testing from the command line very easy. It ensures a clean environment before every test runs.
I personally think it is nicer than the other solutions I have seen.
Instead of messing your models.py I would go with gaeunit.
I've used it with success in a couple of projects and the features I like are:
Just one file to add to your project (gaeunit.py) and you are almost done
Gaeunit isolates the test datastore from the development store (i.e. tests don't pollute your development db)
Since you can't (and not suppose to)
run your test's against the datastore.
This is not true. You can and should use the local datastore implementation as a test harness - there's no reason to waste your time creating mocks for every datastore behaviour. You can use a tool such as noseGAE or gaeunit, as suggested by other posters, but if you want to set it up yourself, see this snippet.
There's more than one problem you're trying to address here...
First, for running tests with GAE emulation you can take a look at gaeunit, which I like best. If you don't want to run them from the browser then you can take a look at noseGAE (part of nose). This should give you command-line testing.
Second, regarding your comment about about 'creating an overhead of dependencies' it sounds like you're searching for a good unit testing and mocking framework. These will let you mock out the database for tests which don't need to hit it. Try mox and mockito for python.
I have some custom logic that needs to be executed every single time a URL is reversed, even for third-party apps. My project is a multitenant web app, and the tenant is identified based on the URL. There isn't a single valid URL that doesn't include a tenant identifier.
I already have a wrapper function around reverse, but now I need a way to tell every installed app to use it. The wrapper around reverse uses a thread-local to inject the identifier into the resulting URL. I could write this function as a decorator on reverse, but I don't know where to do the actual decoration.
Moderately Firm Constraint: I'm already using 3 3rd-party apps, and I'll probably add more. A solution should not require me to modify the source code of all these third-party apps. I don't relish the idea of maintaining patches on top of multiple 3rd-party source trees if there is an easier way. I can make the documentation abundantly clear that reverse has been decorated.
The Original Question: Where could I make such a change that guarantees it would apply to every invocation of reverse?
Possible Alternate Question: What's a better way of making sure that every URL—including those generated by 3rd-party apps—gets the tenant identifier?
BTW, I'm open to a better way to handle any of this except the embedding of the tenant-id in the URL; that decision is pretty set in stone right now. Thanks.
Thanks.
only way so that django reverse is replaced by ur_reverse is
django.core.urlresolvers.reverse = ur_reverse
or if you like decorator syntactic sugar
django.core.urlresolvers.reverse = ur_reverse_decorator(django.core.urlresolvers.reverse )
which i would not advice(and many will shout), unless you are not willing to change every usage of reverse with ur_reverse
I am programming a website in which users will have a number of settings, such as their choice of colour scheme, etc. I'm happy to store these as plain text files, and security is not an issue.
The way I currently see it is: there is a dictionary, where all the keys are users and the values are dictionaries with the users' settings in them.
For example, userdb["bob"]["colour_scheme"] would have the value "blue".
What is the best way to store it on file? Pickling the dictionary?
Are there better ways of doing what I am trying to do?
I would use the ConfigParser module, which produces some pretty readable and user-editable output for your example:
[bob]
colour_scheme: blue
british: yes
[joe]
color_scheme: that's 'color', silly!
british: no
The following code would produce the config file above, and then print it out:
import sys
from ConfigParser import *
c = ConfigParser()
c.add_section("bob")
c.set("bob", "colour_scheme", "blue")
c.set("bob", "british", str(True))
c.add_section("joe")
c.set("joe", "color_scheme", "that's 'color', silly!")
c.set("joe", "british", str(False))
c.write(sys.stdout) # this outputs the configuration to stdout
# you could put a file-handle here instead
for section in c.sections(): # this is how you read the options back in
print section
for option in c.options(section):
print "\t", option, "=", c.get(section, option)
print c.get("bob", "british") # To access the "british" attribute for bob directly
Note that ConfigParser only supports strings, so you'll have to convert as I have above for the Booleans. See effbot for a good run-down of the basics.
Using cPickle on the dictionary would be my choice. Dictionaries are a natural fit for these kind of data, so given your requirements I see no reason not to use them. That, unless you are thinking about reading them from non-python applications, in which case you'd have to use a language neutral text format. And even here you could get away with the pickle plus an export tool.
I don't tackle the question which one is best. If you want to handle text-files, I'd consider ConfigParser -module. Another you could give a try would be simplejson or yaml. You could also consider a real db table.
For instance, you could have a table called userattrs, with three columns:
Int user_id
String attribute_name
String attribute_value
If there's only few, you could store them into cookies for quick retrieval.
Here's the simplest way. Use simple variables and import the settings file.
Call the file userprefs.py
# a user prefs file
color = 0x010203
font = "times new roman"
position = ( 12, 13 )
size = ( 640, 480 )
In your application, you need to be sure that you can import this file. You have many choices.
Using PYTHONPATH. Require PYTHONPATH be set to include the directory with the preferences files.
a. An explicit command-line parameter to name the file (not the best, but simple)
b. An environment variable to name the file.
Extending sys.path to include the user's home directory
Example
import sys
import os
sys.path.insert(0,os.path.expanduser("~"))
import userprefs
print userprefs.color
For a database-driven website, of course, your best option is a db table. I'm assuming that you are not doing the database thing.
If you don't care about human-readable formats, then pickle is a simple and straightforward way to go. I've also heard good reports about simplejson.
If human readability is important, two simple options present themselves:
Module: Just use a module. If all you need are a few globals and nothing fancy, then this is the way to go. If you really got desperate, you could define classes and class variables to emulate sections. The downside here: if the file will be hand-edited by a user, errors could be hard to catch and debug.
INI format: I've been using ConfigObj for this, with quite a bit of success. ConfigObj is essentially a replacement for ConfigParser, with support for nested sections and much more. Optionally, you can define expected types or values for a file and validate it, providing a safety net (and important error feedback) for users/administrators.
I would use shelve or an sqlite database if I would have to store these setting on the file system. Although, since you are building a website you probably use some kind of database so why not just use that?
The built-in sqlite3 module would probably be far simpler than most alternatives, and gets you ready to update to a full RDBMS should you ever want or need to.
If human readablity of configfiles matters an alternative might be the ConfigParser module which allows you to read and write .ini like files. But then you are restricted to one nesting level.
If you have a database, I might suggest storing the settings in the database. However, it sounds like ordinary files might suit your environment better.
You probably don't want to store all the users settings in the same file, because you might run into trouble with concurrent access to that one file. If you stored each user's settings as a dictionary in their own pickled file, then they would be able to act independently.
Pickling is a reasonable way to store such data, but unfortunately the pickle data format is notoriously not-human-readable. You might be better off storing it as repr(dictionary) which will be a more readable format. To reload the user settings, use eval(open("file").read()) or something like that.
Is there are particular reason you're not using the database for this? it seems the normal and natural thing to do - or store a pickle of the settings in the db keyed on user id or something.
You haven't described the usage patterns of the website, but just thinking of a general website - but I would think that keeping the settings in a database would cause much less disk I/O than using files.
OTOH, for settings that might be used by client-side code, storing them as javascript in a static file that can be cached would be handy - at the expense of having multiple places you might have settings. (I'd probably store those settings in the db, and rebuild the static files as necessary)
I agree with the reply about using Pickled Dictionary. Very simple and effective for storing simple data in a Dictionary structure.
If you don't care about being able to edit the file yourself, and want a quick way to persist python objects, go with pickle. If you do want the file to be readable by a human, or readable by some other app, use ConfigParser. If you need anything more complex, go with some sort of database, be it relational (sqlite), or object-oriented (axiom, zodb).