I'm building an application with pyramid and sqlalchemy. I'm trying to work as clean as possible, so right now I'm just writing models and try to do solid test cases for them. Ideally I'd expect them to accept a database configuration to work with (since different DB systems do differ; for the application I'll be working with mysql), but even a decent way of doing it with sqlite would help.
The page Adding Tests in the SQLAlchemy + URL Dispatch Wiki Tutorial somehow is not too helpful on that part, as it says
To test the model class Page we'll add a new PageModelTests class to our tests.py file that was generated as part of the alchemy scaffold.
but then it fails to mention said PageModelTests class ever again.
Does anyone know a helpful tutorial on that?
I just presented on testing SQLAlchemy models in Pyramid in my talk Building the App. Two approaches are presented, the "transactional" and the "mock" style. For a demonstration of these methods as well as a comparison of techniques, see the example app at https://bitbucket.org/zzzeek/pycon2014_atmcraft/:
fixtures
transactional testing examples
mock testing examples
Related
I'm coming from an iOS background to Django. As an app developer, I'm used to clearly defined MVC architecture. My storyboard contains my views. My view controllers contain my logic, and my model exists in an object-relational mapper (ORM) framework or database.
On the web, the separation of responsibility seems less clear cut. Sure, databases and ORMs exists. HTML seems to be my views. Django Models seem to be just that. But where is the controller?
Where does my business logic live?
As the Django FAQ itself points out, Django doesn't quite follow an MVC approach, at least not in a straightforward way. (They argue that Django itself is the controller, but that's not really how I think about MVC.)
The "controllers" in Django are basically what Django calls views. So you have your model classes, which are the M obviously. The templates/HTML are basically the V in MVC. Django views (either functions or classes) are effectively callbacks that run for a particular URL, and they tend to be where a lot of the logic is. So for example, you'll have a Django view called get_foo_bar that runs when someone makes a GET request to /foo/bar, and the Django view effectively becomes the C in MVC.
So long story short, your logic often goes in your Django views.
Django has controllers that are stored in views.py files, this naming leads to some confusion for newbies with some sort of MVC background, you can read about it here: https://docs.djangoproject.com/en/1.8/faq/general/#django-appears-to-be-a-mvc-framework-but-you-call-the-controller-the-view-and-the-view-the-template-how-come-you-don-t-use-the-standard-names
Every Django tutorial/book I've seen approaches Django projects from what I am going to characterize as an ad-hoc database design method. A project ends-up being a bunch of little apps, each with it's own models, view, etc.
I am trying to locate a resource that covers how to structure a Django project when you do, in fact, start with a traditional DB design process.
For example, let's say that this is the starting point:
And a document describing it:
LEAD Database Guide
How does one approach this so that the various Django apps access a single "global" (bad word) db model that covers the entire schema rather than a bunch of models spread across a bunch of apps?
Are there any resources (books, pdf's, tutorials) that cover this approach rather than the piece-meal approach most commonly seen?
A corollary to this question might be: Is there a, perhaps automated, way to go from an SQL (or MySQL Workbench) schema definition to the equivalent Django ORM?
You can use the inspectdb command to generate models.py from database:
http://docs.djangoproject.com/en/dev/ref/django-admin/#inspectdb
Then, you can treat the models as a single app, and add more features/tables into new apps in the future.
I have an existing python application that runs in a console. This application contain features that is currently using sqlite3 as its data storage.
I am now trying to make use of Django as the presentation and it seems that I would need some help from you guys on how to speed up my transition from console to a web enabled user interface.
I understand that web is stateless and the app will be mostly consumed several people.
Most of my data is in dict which I can be able to convert to a JSON. I am seeking for kind advice on what are the things that I need to understand or if any of you guys have encountered this kind of app that you will need to migrate to a web centric presentation.
It has mostly query based functions which have few argumenta/parameters passed but it seems that I will have problem with the database access in python due to that most of my db queries are raw sql.
Thank you guys in advance.
(if there is any existing projects that I can use as a pattern by all means please do help to share with me)
==========
Resolution
Added this into manage.py:
sys.path.insert(0, os.path.join(BASE_DIR, 'app', 'engine'))
Now I can call my 'engine' package classes/scripts.
This is a pretty open-ended question, but I'll address moving your database to Django's models and custom queries.
Data Layer
Before building your app, you can try and use the manage.py command inspectdb to describe your current schema as Django models. It won't write your model layer for you, but can help you get started.
As #joran-beasley mentioned, sqlite will not be suitable as a production database for a multi-client application. Postgres is widely-used in the Django community, but it is hardly your option that supports raw SQL.
It is possible to migrate some data in XML, YAML, or JSON and import them with django's de/serializers or via the loaddata manage.py command. Consider adding custom managers to your models that are the targets of 1-to-1, Foreign Key, or Many-to-Many fields. You can add a custom get_by_natural_key method that can simplify de/serializing data that contains foreign keys, etc.
Queries
Django's QuerySet api is a wonderful wrapper for wrap SQL, but it will have some limitations. Take some time to review it's capabilities and shortcomings to decide if it can help or hinder your needs.
If you want to continue making raw queries, Django can help you with that too as it offers helper queryset methods like raw or extra. You can avoid the model layer entirely using its django.db.connection.cursor class.
I can't speak to much else that will help you in your transition, but you may find that Django's class-based generic views like DetailView, ListView, UpdateView, or DeleteView may be helpful in setting up some of your more basic interfaces on the web.
EDIT: I'm new to this site but if you are going to down vote me, could you perhaps explain why? I've searched Google, this site and others but have not found anything that makes any sense and I thought this was a site to ask questions and get some help.
I've got a Custom PHP Forum that I am trying to convert to Python/Django as a learning experience and I'm having some problems. I've been reading up on Django and it is encouraged that our application is split into multiple apps. I went through the 6 part tutorial and many other parts of the documentation but I'm left with some questions.
Let's assume that I have about 30 tables.
Tables such as:
posts_index, posts, users, user_groups, user_activity, user_sessions, forums, payment_gateways, payment_logs, etc for a basic forum
I'm having issues structuring my models. With PHP all I needed was index.php, /admin/index.php, view_forum.php, view_thread.php and a few others, everything could pull directly from the database and I had no issues but now I have to deal with apps/modules.
I'm thinking I'd need to structure my apps in a manner similar to this:
/admin/ app
/forums/ app
/view_forum/ app
/view_thread/ app
/forums/view_forum/ app (instead of just /view_forum/, could be a sub app)
/forums/view_thread/ app (instead of /view_thread/, could be a sub app)
My problem and only question here is dealing with global state. For example Users/Group/Session/Logging/Permission information is going to need to be shared across multiple apps through importing in the other apps models file. To do this I need to reference their model information, what is the correct way to handle this?
Would either of these be acceptable?
Create a ton of different apps such as /users/ which would model my users_groups, users, user_sessions, another app for /posts/ that would include models for posts_index, forums, and so forth with these models existing but not actually being used publicly, they would be used in other apps only. They would be imported in areas such as the /view_forum/ app since when viewing a forum I might need to determine if the user is logged in, is a member of a particular group, etc and because of that would need access to a number of the hidden apps and hence would be imported from the hidden app.
What if I just had one single app, instead of it being an app it would just be my entire project. This sounds like the best solution to me but it seems to be suggested if we cannot summarize the entire application into a sentence it needs to be broken up. If I went with one single app being used as my entire project, my models file will have 30+ different models, is this acceptable? I assume not but figured I'd ask.
Do either of the above make any sense? If not what would you do fix it? I'll admit I'm lost so any feedback would mean a lot.
I'm new to Python/Django and am trying to figure things out. I hope I am clear on what I am trying to do. I'm more than welcome to any advice. I've been trying to play around with things but I figure it would be better to ask for advice from more experienced developers. I'm not a professional programmer and am still learning so please be nice :).
I have voted this up... I had similar questions when I first moved to Django (also coming from PHP)
Try not to think of apps in terms of db tables (or url paths), you want to create apps for independent pieces of functionality.
I would say most of your code will be in a single forum app with Forum and Post models in it and all your forum-related urls like /view_forum/ and /view_thread/. Note there's not really such thing as a sub-app in Django... these are different views which all belong to one related set of functionality in a single app.
The 'users' stuff... you normally want to hook in to the Django auth system (https://docs.djangoproject.com/en/dev/topics/auth/default/#user-objects) though if you are trying to keep the legacy database structure this may be harder... you may end up needing your own users app.
The payment gateways stuff sounds like another app again.
For the admin, you get this (almost) for free with Django:
https://docs.djangoproject.com/en/1.5/ref/contrib/admin/
You need an admin.py inside each app, where you register the models that you want to expose to the admin site.
You might find it's best to follow a tutorial and build a simple blog (or try and build a very simple minimal forum from scratch) to get a feel for Django before tackling your re-write. Because trying to exactly recreate the old functionality with legacy db tables will be a bit harder and may lead you into patterns which are not 'good Django'.
You might also find it useful to look at the source code of an existing Django forum app, such as this one:
https://bitbucket.org/slav0nic/djangobb/src/
They have just a single djangobb_forum app that does everything. I think this is entirely justified as a forum is a single complicated piece of functionality.
Note how their Profile model effectively extends the built-in Django auth.User model via a OneToOneField. It makes it easier to integrate with the built-in login and authentication system that way.
Start by integrating your legacy database and then build the Admin site. You'll see how the model information is available to the Admin app and every other app you write. The key is to import models in your apps.
Pretty new to this scene and trying to find some documentation to adopt best practices. We're building a fairly large content site which will consist of various media catalogs and I'm trying to find some comparable data / architectural models so that we can get a better idea of the approach we should use using a framework we've never made use of before. Any insight / help would be greatly appreciated!
"data / architectural models so that we can get a better idea of the approach we should use using a framework we've never made use of before"
Django imposes best practices on you. You don't have a lot of choices and can't make a lot of mistakes.
MVC (while a noble aspiration) is implemented as follows:
Data is defined in "models.py" files using the Django ORM models.
urls.py file maps URL to view function. Pick your URL's wisely.
View function does all processing, making use of models and methods in models
Presentation (via HTML templates) invoked by View function. Essentially no processing can be done in presentation, just lightweight iteration and decision-making
The model is defined for you. Just stick to what Django does naturally and you'll be happy.
Architecturally, you usually have a stack like this.
Apache does two things.
serves static content directly and immediately
hands dynamic URL to Django (via mod_python, mod_wsgi or mod_fastcgi). Django apps map URL to view functions (which access to database (via ORM/model) and display via templates.
Database used by Django view functions.
The architecture is well-defined for you. Just stick to what Django does naturally and you'll be happy.
Feel free to read the Django documentation. It's excellent; perhaps the best there is.
first, forget all MVC mantra. it's important to have a good layered structure, but MVC (as defined originally) isn't one, it was a modular structure, where each GUI module is split in these tree submodules. nothing to use on the web here.
in web development, it really pays to have a layered structure, where the most important layer is the storage/modelling one, which came to be called model layer. on top of that, you need a few other layers but they're really not anything like views and controllers in the GUI world.
the Django layers are roughly:
storage/modelling: models.py, obviously. try to put most of the 'working' concepts there. all the relationships, all the operations should be implemented here.
dispatching: mostly in urls.py. here you turn your URL scheme into code paths. think of it like a big switch() statement. try hard to have readable URLs, which map into user intentions. it will help a lot to add new functionality, or new ways to do the same things (like an AJAX UI later).
gathering: mostly the view functions, both yours and the prebuilt generic views. here you simply gather all the from the models to satisfy a user request. in surprisingly many cases, it just have to pick a single model instance, and everything else can be retrieved from relationships. for these URLs, a generic view is enough.
presentation: the templates. if the view gives you the data you need, it's simple enough to turn it into a webpage. it's here where you'll thank that the model classes have good accessors to get any kind of relevant data from any given instance.
To understand django fundementals and the django take on MVC, consult the following:
http://www.djangobook.com/
As a starting point to getting your hands dirty with ...
"...trying to find some comparable data / architectural models"
Here is a quick and dirty way to reverse engineer a database to get a models.py file,
which you can then inspect to see how django would handle it.
1.) get an er diagram that closely matches your target. For example something like this
http://www.databaseanswers.org/data_models/product_catalogs/index.htm
2.) create an sql script from the er diagram and create the database,
I suggest Postgre, as some MySQL
table type will not have forgien key constraints, but in a pinch MySQL or SQLITE
will do
3.) create and configure a django app to use that database. Then run:
python manage.py inspectdb
This will at least give you a models.py file which you can read to see how django attempts
to model it.
Note that the inspect command is intended to be a shortcut for dealing with legacy
database when developing in django, and as such is not perfect. Be sure to read the
following before attempting this:
http://docs.djangoproject.com/en/dev/ref/django-admin/#ref-django-admin