I keep hitting road blocks with Django and have read about Pylons. Pylons seemed to be exactly what I needed (greener grass), but then I realized that they have global variables all over the place and loads of black magic infused by dark spirits (spirits so dark that they even kill unicorns).
Is there anything out there that is enterprise worthy (ie, doesn't impose performance or scaling restrictions), stays the hell out of my way, but provides the basic request/response handling, sessions, SQLAlchemy (perhaps), and a way to plug in templates, etc? Is there any hope?
I've been trying to develop an SAAS in Django, which is a nightmare. They don't support multiple column primary keys, and there are a number of other problems with ModelForms, etc that you don't run into until you're developing a more complex application (especially with multitenancy. I don't use their auth system and don't need to as I built my own. I just need security (CSRF, XSS, SQL injection, etc).
The most hard-core low-level web-framework for python - Werkzeug - http://werkzeug.pocoo.org/
Flask: http://flask.pocoo.org/ It will look like an entry-level framework, but in fact it's extremely powerful. It's based on werkzeug and support Jinja2 out of the box. I'd go with this one. You can get easily integrated SQLAlchemy with extensions like flask-sqlalchemy and WTForms (similar API to django.forms) with flask-wtform. There are tons of useful other extensions for it, like extensions that add the ability to use mongodb and couchdb easily. What's most notable about flask extensions is they provide very consistent behavior and there is an actual approval process for them, as opposed to django reusable apps 95% of which are a mess.
Related
I'm running Django (1.5) on App Engine and I need to use some kind of key-value cache. I know App Engine's memcache API and also the Django's cache framework. I wonder which one should I use.
On one hand I would like my code to be as portable as possible for migrating it to another cloud platform. But on the other hand I would like to fully utilize the services offered by App Engine.
Is writing a custom cache backend for Django that will use the App Engine memcache is the best solution?
Tzach, I think you're already answering your question.
Putting your app in GAE and not using the services provided by Google it doesn't look to me as a wise decision, even more, when those features are key for performance at the same time free or very cheap.
On the other hand, the embedded default cache in Python is not guaranteed to give its best results under GAE, as GAE instances are not a normal server where you'd run your django instance, e.g. instances can be shutdown at any time.
These special characteristics found in Django are tuned in the django for GAE versions.
For that reason, and taking into account that using the GAE memcache is also straightforward, I'd recommend you using the easiest ones to add to your application.
And, if in the future, you move to another platform, there will be more things to change than the key-value cache.
My two cents on that is to focus firstly in getting the job done and secondly in optimizing the performance on GAE and only afterwards to start thinking on things to improve.
I am using MongoDB as my primary(and only till now) database and because of google and the links it provided me i am confused between Django or Pyramid.
I am comfortable with python but never done web development in python(i have done in PHP). Now because i will be using Mongo so i wont use Django ORM will that take away the ease of development people associate with Django?
I am new to Django(just a few hours) so i am not sure what parts of the framework the ORM affects.
Or should i go with the django fork django-norel with django mongodb engine (they are not actively maintained though) or should i use Pyramid because i plan to use jinja2 as my template layer so that makes 2 parts of django useless to me.
After removing these batteries from django does it still remain true that it a framework for people with deadlines?
Advice?
A year or two ago I was also deciding between django and pyramid w/ mongodb to build a high performance web application. I ultimately chose Pyramid :
Pros:
Pyramid is very light weight for a full stack framework. There is a minmal amount of 'magic' going on under the hood. I was able to wrap my head around all the pieces.
It is not as opinionated as Django. You can plug and play whatever templating engines or databases you need relatively easily.
I was impressed by the performance benchmarks between Pyramid over other full stack frameworks.
The Pylons/Pyramid culture of 100% code coverage and "only pay for what you eat" was appealing to my own development style.
Cons:
The community is growing, but still not as mature as Django
There's a lot of documentation, but it's not as robust as some of the Django docs out there
The lack of 'magic' and beginner friendly documentation make for a steeper learning curve.
As for using an ORM with MongoDB; I recommend you try building without one at first. After trying a few mongodb ORMs, I've ended up going back to plain pymongo with Colander or Validictory for validation. Pymongo is already very ORM-like, fast, and flexible. I felt the extra layers of abstraction took away from that flexibility, and caused me to write more code than was needed.
I'm going to suggest an alternative that has not been mentioned: Flask. Flask has a really great (albeit smaller than Django) community and there are a lot of extensions available for common web-app extensions, in the extensions directory.
There are several MongoDB extensions for Flask, which help to integrate MongoDB into the framework, but I would also highly recommend the mongoengine ORM. One of the people working on mongoengine has release an extension for Flask integration, Flask-MongoEngine.
Try it all (Django, Pyramid, Flask), read the docs, and you done.
If you will select Django, you will not use some parts of it (orm, formlib), and that is all.
ALso see this video:
https://www.10gen.com/presentations/mongosf-2011/mongodb-with-python-pylons-pyramid
I've been learning python for use in ArcGIS and some other non-web applications. However, now that I've taken on building a personal website I am interested in using it for web development (as it is the only scripting language I currently know).
I've noticed that there are a lot of these things called "frameworks", such as Django. From what I understand they are just a collection of packages to save you from re-inventing the wheel but I don't really know how they work.
Furthermore, I do not like GUIs, if I need a framework I would like to find one that could be used through a terminal, starts out simple and can be scaled for more complexity when I'm ready. Any advice or ideas on frameworks and why I would want to use one?
The Python web frameworks have nothing to do with GUIs, and can all be used via the terminal.
The benefits of a framework, as you say, are all to do with making your life easier by supplying the components you need to build a website: the main ones are database interaction through an ORM, a templating system, and URL routing. On top of that, the big frameworks also included optional extras like user authentication, administration interface, and so on.
Personally I like Django, but your mileage may vary: I would say, though, that whatever you do with Python and the web will require some sort of framework, even if it's one of the absolute minimal ones like Flask which basically do just the routing part. There's simply no point in writing all this stuff from scratch when it's been done for you.
I'd second the post above: Django is a great framework and will save you loads of time in the long run.
Pretty much every challenge you'll come across when writing a web application will already have been solved, e.g. How do I send emails? What about an admin interface to edit the data? User security?
In my view picking the best framework is all about the ecosystem around that framework. How well used is it? Is it discussed widely on the internet? Have others encountered, and solved, the problems I'm facing?
In terms of where you start, see the Django Tutorial here:
http://docs.djangoproject.com/en/1.2/intro/tutorial01/
If you think Django offers you too much, I'd recommend that you take a look at CherryPy just to compare the different, and much simpler, approach.
With Python, you've got lots of options. To start, I would recommend looking here -- it explains the basics and provides a fairly complete list of frameworks.
If you're looking for something that starts out simple but can also handle more complexity, then you should take a look at web2py. It requires no installation or configuration, has no dependencies, and includes a web server and a relational database. It also includes an optional web-based integrated development environment and admin interface, but you can work through the terminal instead if you prefer.
It's very easy to learn and was designed for ease of use, faster development, and security. You can get a lot done with very little code thanks to the included scaffolding app along with many sensible default behaviors. As things get more complex, web2py can handle it, as it is a well-integrated full-stack framework with lots of built-in functionality, including a database abstraction layer, form handling and validation, access control, web services, and easy Ajax integration.
Personnally, I don't use any framework, I write either from scratch on BaseHTTPServer, or using WSGI (with mod_wsgi).
It is a bit long to write the skeleton, but I think it is faster (I mean at runtime), there is less constraints, and there is lesser to learn.
I'm developing a web application and considering Django, Google App Engine, and several other options. I wondered what kind of "penalty" I will incur if I develop a complete Django application assuming it runs on a dedicated server, and then later want to migrate it to Google App Engine.
I have a basic understanding of Google's data store, so please assume I will choose a column based database for my "stand-alone" Django application rather than a relational database, so that the schema could remain mostly the same and will not be a major factor.
Also, please assume my application does not maintain a huge amount of data, so that migration of tens of gigabytes is not required. I'm mainly interested in the effects on the code and software architecture.
Thanks
Most (all?) of Django is available in GAE, so your main task is to avoid basing your designs around a reliance on anything from Django or the Python standard libraries which is not available on GAE.
You've identified the glaring difference, which is the database, so I'll assume you're on top of that. Another difference is the tie-in to Google Accounts and hence that if you want, you can do a fair amount of access control through the app.yaml file rather than in code. You don't have to use any of that, though, so if you don't envisage switching to Google Accounts when you switch to GAE, no problem.
I think the differences in the standard libraries can mostly be deduced from the fact that GAE has no I/O and no C-accelerated libraries unless explicitly stated, and my experience so far is that things I've expected to be there, have been there. I don't know Django and haven't used it on GAE (apart from templates), so I can't comment on that.
Personally I probably wouldn't target LAMP (where P = Django) with the intention of migrating to GAE later. I'd develop for both together, and try to ensure if possible that the differences are kept to the very top (configuration) and the very bottom (data model). The GAE version doesn't necessarily have to be perfect, as long as you know how to make it perfect should you need it.
It's not guaranteed that this is faster than writing and then porting, but my guess is it normally will be. The easiest way to spot any differences is to run the code, rather than relying on not missing anything in the GAE docs, so you'll likely save some mistakes that need to be unpicked. The Python SDK is a fairly good approximation to the real App Engine, so all or most of your tests can be run locally most of the time.
Of course if you eventually decide not to port then you've done unnecessary work, so you have to think about the probability of that happening, and whether you'd consider the GAE development to be a waste of your time if it's not needed.
Basically, you will change the data model base class and some APIs if you use them (PIL, urllib2, etc).
If your goal is app-engine, I would use the app engine helper http://code.google.com/appengine/articles/appengine_helper_for_django.html. It can run it on your server with a file based DB and then push it to app-engine with no changes.
It sounds like you have awareness of the major limitation in building/migrating your app -- that AppEngine doesn't support Django's ORM.
Keep in mind that this doesn't just affect the code you write yourself -- it also limits your ability to use a lot of existing Django code. That includes other applications (such as the built-in admin and auth apps) and ORM-based features such as generic views.
There are a few things that you can't do on the App Engine that you can do on your own server like uploading of files. On the App Engine you kinda have to upload it and store the datastore which can cause a few problems.
Other than that it should be fine from the Presentation part. There are a number of other little things that are better on your own dedicated server but I think eventually a lot of those things will be in the App Engine
I am thinking about using Google App Engine.It is going to be a huge website. In that case, what is your piece of advice using Google App Engine. I heard GAE has restrictions like we cannot store images or files more than 1MB limit(they are going to change this from what I read in the GAE roadmap),query is limited to 1000 results, and I am also going to se web2py with GAE. So I would like to know your comments.
Thanks
Having developed a smallish site with GAE, I have some thoughts
If you mean "huge" like "the next YouTube", then GAE might be a great fit, because of the previously mentioned scaling.
If you mean "huge" like "massively complex, with a whole slew of screens, models, and features", then GAE might not be a good fit. Things like unit testing are hard on GAE, and there's not a built-in structure for your app that you'd get with something like (famously) (Ruby on) Rails, or (Python powered) Turbogears.
ie: there is no staging environment: just your development copy of the system and production. This may or may not be a bad thing, depending on your situation.
Additionally, it depends on the other Python modules you intend to pull in: some Python modules just don't run on GAE (because you can't talk to hardware, or because there are just too many files in the package).
Hope this helps
using web2py on Google App Engine is a great strategy. It lets you get up and running fast, and if you do outgrow the restrictions of GAE then you can move your web2py application elsewhere.
However, keeping this portability means you should stay away from the advanced parts of GAE (Task Queues, Transactions, ListProperty, etc).
The AppEngine uses BigTable as it's datastore backend. Don't try to write a traditional relational-database driven application. BigTable is much more well suited for use as a highly-scalable key-value store. Avoid joins if at all possible.
I wouldn't worry about any of this. After having played with Google App Engine for a while now, I've found that it scales quite well for large data sets. If your data elements are large (i.e. photos), then you'll need to integrate with another service to handle them, but that's probably going to be true no matter what with data of that size. Also, I've found BigTable relatively easy to work with having come from a background entirely in relational databases. Finally, Django is a somewhat hidden, but awesome, "feature" of Google App Engine. If you've never used it, it's a really nice, elegant web framework that makes a lot of common tasks trivial (forms come to mind here).
Google has just released version 1.3.0 of the SDK with support with a new Blobstore API for storage of files up to 50MB. See the post "App Engine SDK 1.3.0 Released Including Support for Larger User Uploads".
What about Google Wave? It's being built on appengine, and once live, real-time translatable chat reaches the corporate sector... I could see it hitting top 1000th... But then again, that's an internal project that gets to do special stuff other appengine apps can't.... Like hanging threads; I think... And whatever else Wave has under the hood...
If you are planning on a 'huge' website, then don't use App Engine. Simple as that. The App Engine is not built to deliver the next top 1000th website.
Allow me to also ask what do you mean by 'huge', how many simultaneous users? Queries per second? DB load?