As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Looking to do a very small, quick 'n dirty side project. I like the fact that the Google App Engine is running on Python with Django built right in - gives me an excuse to try that platform... but my question is this:
Has anyone made use of the app engine for anything other than a toy problem? I see some good example apps out there, so I would assume this is good enough for the real deal, but wanted to get some feedback.
Any other success/failure notes would be great.
I have tried app engine for my small quake watch application
http://quakewatch.appspot.com/
My purpose was to see the capabilities of app engine, so here are the main points:
it doesn't come by default with Django, it has its own web framework which is pythonic has URL dispatcher like Django and it uses Django templates
So if you have Django exp. you will find it easy to use
But you can use any pure python framework and Django can be easily added see
http://code.google.com/appengine/articles/django.html
google-app-engine-django (http://code.google.com/p/google-app-engine-django/) project is excellent and works almost like working on a Django project
You can not execute any long running process on server, what you do is reply to request and which should be quick otherwise appengine will kill it
So if your app needs lots of backend processing appengine is not the best way
otherwise you will have to do processing on a server of your own
My quakewatch app has a subscription feature, it means I had to email latest quakes as they happend, but I can not run a background process in app engine to monitor new quakes
solution here is to use a third part service like pingablity.com which can connect to one of your page and which executes the subscription emailer
but here also you will have to take care that you don't spend much time here
or break task into several pieces
It provides Django like modeling capabilities but backend is totally different but for a new project it should not matter.
But overall I think it is excellent for creating apps which do not need lot of background processing.
Edit:
Now task queues can be used for running batch processing or scheduled tasks
Edit:
after working/creating a real application on GAE for a year, now my opnion is that unless you are making a application which needs to scale to million and million of users, don't use GAE. Maintaining and doing trivial tasks in GAE is a headache due to distributed nature, to avoid deadline exceeded errors, count entities or do complex queries requires complex code, so small complex application should stick to LAMP.
Edit:
Models should be specially designed considering all the transactions you wish to have in future, because entities only in same entity group can be used in a transaction and it makes the process of updating two different groups a nightmare e.g. transfer money from user1 to user2 in transaction is impossible unless they are in same entity group, but making them same entity group may not be best for frequent update purposes....
read this http://blog.notdot.net/2009/9/Distributed-Transactions-on-App-Engine
I am using GAE to host several high-traffic applications. Like on the order of 50-100 req/sec. It is great, I can't recommend it enough.
My previous experience with web development was with Ruby (Rails/Merb). Learning Python was easy. I didn't mess with Django or Pylons or any other framework, just started from the GAE examples and built what I needed out of the basic webapp libraries that are provided.
If you're used to the flexibility of SQL the datastore can take some getting used to. Nothing too traumatic! The biggest adjustment is moving away from JOINs. You have to shed the idea that normalizing is crucial.
Ben
One of the compelling reasons I have come across for using Google App Engine is its integration with Google Apps for your domain. Essentially it allows you to create custom, managed web applications that are restricted to the (controlled) logins of your domain.
Most of my experience with this code was building a simple time/task tracking application. The template engine was simple and yet made a multi-page application very approachable. The login/user awareness api is similarly useful. I was able to make a public page/private page paradigm without too much issue. (a user would log in to see the private pages. An anonymous user was only shown the public page.)
I was just getting into the datastore portion of the project when I got pulled away for "real work".
I was able to accomplish a lot (it still is not done yet) in a very little amount of time. Since I had never used Python before, this was particularly pleasant (both because it was a new language for me, and also because the development was still fast despite the new language). I ran into very little that led me to believe that I wouldn't be able to accomplish my task. Instead I have a fairly positive impression of the functionality and features.
That is my experience with it. Perhaps it doesn't represent more than an unfinished toy project, but it does represent an informed trial of the platform, and I hope that helps.
The "App Engine running Django" idea is a bit misleading. App Engine replaces the entire Django model layer so be prepared to spend some time getting acclimated with App Engine's datastore which requires a different way of modeling and thinking about data.
I used GAE to build http://www.muspy.com
It's a bit more than a toy project but not overly complex either. I still depend on a few issues to be addressed by Google, but overall developing the website was an enjoyable experience.
If you don't want to deal with hosting issues, server administration, etc, I can definitely recommend it. Especially if you already know Python and Django.
I think App Engine is pretty cool for small projects at this point. There's a lot to be said for never having to worry about hosting. The API also pushes you in the direction of building scalable apps, which is good practice.
app-engine-patch is a good layer between Django and App Engine, enabling the use of the auth app and more.
Google have promised an SLA and pricing model by the end of 2008.
Requests must complete in 10 seconds, sub-requests to web services required to complete in 5 seconds. This forces you to design a fast, lightweight application, off-loading serious processing to other platforms (e.g. a hosted service or an EC2 instance).
More languages are coming soon! Google won't say which though :-). My money's on Java next.
This question has been fully answered. Which is good.
But one thing perhaps is worth mentioning.
The google app engine has a plugin for the eclipse ide which is a joy to work with.
If you already do your development with eclipse you are going to be so happy about that.
To deploy on the google app engine's web site all I need to do is click one little button - with the airplane logo - super.
Take a look the the sql game, it is very stable and actually pushed traffic limits at one point so that it was getting throttled by Google. I have seen nothing but good news about App Engine, other than hosting you app on servers someone else controls completely.
I used GAE to build a simple application which accepts some parameters, formats and send email. It was extremely simple and fast. I also made some performance benchmarks on the GAE datastore and memcache services (http://dbaspects.blogspot.com/2010/01/memcache-vs-datastore-on-google-app.html ). It is not that fast. My opinion is that GAE is serious platform which enforce certain methodology. I think it will evolve to the truly scalable platform, where bad practices simply not allowed.
I used GAE for my flash gaming site, Bearded Games. GAE is a great platform. I used Django templates which are so much easier than the old days of PHP. It comes with a great admin panel, and gives you really good logs. The datastore is different than a database like MySQL, but it's much easier to work with. Building the site was easy and straightforward and they have lots of helpful advice on the site.
I used GAE and Django to build a Facebook application. I used http://code.google.com/p/app-engine-patch as my starting point as it has Django 1.1 support. I didn't try to use any of the manage.py commands because I assumed they wouldn't work, but I didn't even look into it. The application had three models and also used pyfacebook, but that was the extent of the complexity. I'm in the process of building a much more complicated application which I'm starting to blog about on http://brianyamabe.com.
Related
I'd like to ask you about your experiences in developing facebook applications in Python. Which of the popular web frameworks for this language you think best suits this purpose? I know "best" is a very subjective word, so I'm specifically interested in the following:
Most reusable libraries. For example one might want to automatically create accounts for new logged in facebook users, but at the same time provide an alternative username + password logging functionality. I need authentication to fit into this nicely.
Facebook applications tend to differ from CMS-like sites. They are action intensive. For more complicated use-cases, usually some kind of caching for the data fetched from Open Graph API is required in order to be able to perform some queries on local and facebook data at once (for example join some tables based on friendship relation).
I'd definitely prefer popular solutions. They just seem to be much more stable and better thought through. I've previously developed a facebook application in Grails and I as much as I liked the architecture and the general ideas, the amount of bugs and complication that I ran into was just a little bit too much. Also Groovy is still quite an exotic language to develop in, and this time I'm not going to work on my own.
I'm not new to Python, but definitely new to web development in Python. Though after the experience with Grails and all its twists and turns I doubt Python could really scare me.
I would almost undoubtedly go with Django as the easiest and most popular framework for developing any type of web applications, if there's a need for a full-stack framework.
Specifically, in regards to Django's app universe, it is plentiful with many active applications -- but that has its downfalls too. There's no standard application for any 'one' thing, but there are a few applications that will do basically 90% of all that's needed. Sometimes the code is poorly written, but most of the time, the apps work and do what they are needed to do, so there's almost no need for someone to dive right in to the code.
Narrowing down our options, I have had great luck with Omab's Django-Social-Auth, which was absolutely a snap to integrate. It required 3 variables in my settings.py and I was up and running.
The only issue might be if you do not want to use the django.contrib.auth.User model, but, if you are not thinking about using that, I would think about that decision twice :)
To narrow it down even further, pyfacebook is another option for integrating Facebook. It comes with a djangofb application so it's just drop, add to settings.py and all is well. It even comes with an example Django application as part of the distribution. I've had pretty good luck with this application, but, I still think Omab's much easier to integrate.
Finally, Facebook's own python-sdk is easy to integrate from a raw standpoint, where they just give you access to their APIs using a simple Python API. However, it seems to cater more to the AppEngine folks, so YMMV.
I've used Django for quite some time. As of late I use Jinja2 instead. No particular reason, but it's another option
If you do not want to start on Django now. Try learning Flask(which is comparatively a lot easier to begin than Django) and then start building app with Flask.
The app-engine-patch authors have officially marked this wonderful project as dead on their website. Over the last year a lot of people have asked what the best way to run Django on Google App Engine was, and time after time people have pointed to app-engine-patch being the way to go. Now that this project is dead, I would love to revisit this discussion and see what you all would recommend.
The old app-engine-patch website refers to a different project called djangoappengine that they are working on. It focuses on using Native Django on App Engine but by doing so appears to limit your ability to use all of App Engine's features (such as its built in User model). What do you all think about this as a replacement solution?
Does the Google App Engine Helper For Django project now look more inviting now that app-engine-patch has died?
Or is app-engine-patch still so great that you would recommend still using it despite it being dead?
I currently have a project in its middle stages relying on app-engine-patch and have loved using it so far, but am scared to continue using a project that is dead. I would really appreciate any comments you have about what you think the best long term solution is now. Thanks!
App engine patch is probably a safer bet for a given moment. Though not actively supported at the moment, it's still great, as it's been tested more thoroughly. If you're ready to take some risks - go and give the new djangoappengine+django-nonrel (native django support for non relational databases, primarily google datastore yest) option a try. That is the option we'll all use in a little while, so probably it's worth to try it out and not to look back. For instance, app engine patch only supports ~300 entities in the admin (i.e. doesn't take advantage of the 1.3.1 functionality), while with the djangoappengine+django-nonrel that is not an issues anymore. www.allbuttonspressed.com, a site of the guys behind app-engine-patch, is all built this new way, you can grab the source code of the site's skeleton # http://bitbucket.org/wkornewald/allbuttonspressed/.
good luck!
It looks like you can just import it using use_library...
http://code.google.com/appengine/docs/python/tools/libraries.html
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Currently I have a website on the Google App Engine written in Google's webapp framework. What I want to know is what are the benefits of converting my app to run with django? And what are the downsides? Also how did you guys code your GAE apps? Did you use webapp or django? Or did you go an entirely different route and use the Java api?
Thanks
With Django on GAE, you get all the benefits (and also the non-benefits) of GAE. The benefits are mainly automatic scalability and high reliability at a very low cost.
You may want to start by checking out the following:
Stack Overflow: Django on Google App Engine
Stack Overflow: Django and App Engine
Google Code: Running Django on Google App Engine
YouTube: Google I/O 2008 - Python, Django, and App Engine
Django on Google App Engine in 13 simple steps
Google - Why App Engine?
As it turns out, I asked this same question:
Why use Django on Google App Engine?
It mostly resolved to two items:
In functionality where webapp and Django overlap, Django tends to be a bit nicer and/or more flexible.
If you were to later want to move away from GAE, Django is a porting goal.
My question (in combination with the wealth of information Daniel Vassallo provided) should answer your question completely.
GAE is a great tool for new and small projects, that do not require a relational database. I use a range of web hosting solutions.
1) I built www.gaiagps.com on the App Engine, because it was just some brochureware, and a tiny key-value store for the blog part.
2) My colleague also built a web crawler on GAE, because it was just some simple Python scripts that collected web pages. That app actually sends the data over to EC2 though, where more work is done.
3) I host www.trailbehind.com on EC2 because it uses a geo-database (PostGIS) which you would basically have to implement yourself on App Engine.
4) I host TRAC and SVN on WebFaction, because it's off-the-shelf for any slice there.
If I need to do a site in a couple of days, I use GAE. If it's a large or existing project, or has a funky database, I use something else.
I've recently changed my development from webapp to django (using app-engine-patch b/c they have added more features). Check out app-engine-patch here:
http://code.google.com/p/app-engine-patch/
I really liked how simple webapp was to use and it had all the tools I needed for a while. However when it came to adding user accounts on to my site I found that I either had to roll my own system (with sessions etc. which can get complicated on GAE) or use google/gmail accounts with webapp.
I didn't want my users to have to sign up for google/gmail first before they signed up with me.
Therefore I'm trying out django with app-engine-patch because they have already implemented the user functionality from django and it seems to work just fine.
With Django vs. Webapp they both have similar performance, and both offer the automatic scalablity etc.
Django has some nifty form validation etc. but i see the user stuff as the biggest difference between the two.
Good Luck.
I did some django/GAE work a last summer (see Plunging into Web Development). The App Engine Patch is a must.
The only real problem I ran into was the 1M upload/download limit imposed by GAE. I could upload larger files by using HTTP directly to S3, but I couldn't retrieve them through GAE. I hope they've lifted this restriction since then.
If you are looking other GAE CMS frameworks besides Django, take a look at Vosao. It is a Java CMS framework which uses Velocity for templates. The framework is still in alpha and is under heavy development. The current version (0.2.3) is stable and highly functional.
You can see an example of a working Vosao site on my personal domain.
I prefer webapp. It scales better according to Google and seems to better integrated with the App Engine infrastructure. Plus it's more lightweight.
try kay-framework if you are looking for framework specifically designed for google app engine.
I'm developing a web application and considering Django, Google App Engine, and several other options. I wondered what kind of "penalty" I will incur if I develop a complete Django application assuming it runs on a dedicated server, and then later want to migrate it to Google App Engine.
I have a basic understanding of Google's data store, so please assume I will choose a column based database for my "stand-alone" Django application rather than a relational database, so that the schema could remain mostly the same and will not be a major factor.
Also, please assume my application does not maintain a huge amount of data, so that migration of tens of gigabytes is not required. I'm mainly interested in the effects on the code and software architecture.
Thanks
Most (all?) of Django is available in GAE, so your main task is to avoid basing your designs around a reliance on anything from Django or the Python standard libraries which is not available on GAE.
You've identified the glaring difference, which is the database, so I'll assume you're on top of that. Another difference is the tie-in to Google Accounts and hence that if you want, you can do a fair amount of access control through the app.yaml file rather than in code. You don't have to use any of that, though, so if you don't envisage switching to Google Accounts when you switch to GAE, no problem.
I think the differences in the standard libraries can mostly be deduced from the fact that GAE has no I/O and no C-accelerated libraries unless explicitly stated, and my experience so far is that things I've expected to be there, have been there. I don't know Django and haven't used it on GAE (apart from templates), so I can't comment on that.
Personally I probably wouldn't target LAMP (where P = Django) with the intention of migrating to GAE later. I'd develop for both together, and try to ensure if possible that the differences are kept to the very top (configuration) and the very bottom (data model). The GAE version doesn't necessarily have to be perfect, as long as you know how to make it perfect should you need it.
It's not guaranteed that this is faster than writing and then porting, but my guess is it normally will be. The easiest way to spot any differences is to run the code, rather than relying on not missing anything in the GAE docs, so you'll likely save some mistakes that need to be unpicked. The Python SDK is a fairly good approximation to the real App Engine, so all or most of your tests can be run locally most of the time.
Of course if you eventually decide not to port then you've done unnecessary work, so you have to think about the probability of that happening, and whether you'd consider the GAE development to be a waste of your time if it's not needed.
Basically, you will change the data model base class and some APIs if you use them (PIL, urllib2, etc).
If your goal is app-engine, I would use the app engine helper http://code.google.com/appengine/articles/appengine_helper_for_django.html. It can run it on your server with a file based DB and then push it to app-engine with no changes.
It sounds like you have awareness of the major limitation in building/migrating your app -- that AppEngine doesn't support Django's ORM.
Keep in mind that this doesn't just affect the code you write yourself -- it also limits your ability to use a lot of existing Django code. That includes other applications (such as the built-in admin and auth apps) and ORM-based features such as generic views.
There are a few things that you can't do on the App Engine that you can do on your own server like uploading of files. On the App Engine you kinda have to upload it and store the datastore which can cause a few problems.
Other than that it should be fine from the Presentation part. There are a number of other little things that are better on your own dedicated server but I think eventually a lot of those things will be in the App Engine
I am thinking about using Google App Engine.It is going to be a huge website. In that case, what is your piece of advice using Google App Engine. I heard GAE has restrictions like we cannot store images or files more than 1MB limit(they are going to change this from what I read in the GAE roadmap),query is limited to 1000 results, and I am also going to se web2py with GAE. So I would like to know your comments.
Thanks
Having developed a smallish site with GAE, I have some thoughts
If you mean "huge" like "the next YouTube", then GAE might be a great fit, because of the previously mentioned scaling.
If you mean "huge" like "massively complex, with a whole slew of screens, models, and features", then GAE might not be a good fit. Things like unit testing are hard on GAE, and there's not a built-in structure for your app that you'd get with something like (famously) (Ruby on) Rails, or (Python powered) Turbogears.
ie: there is no staging environment: just your development copy of the system and production. This may or may not be a bad thing, depending on your situation.
Additionally, it depends on the other Python modules you intend to pull in: some Python modules just don't run on GAE (because you can't talk to hardware, or because there are just too many files in the package).
Hope this helps
using web2py on Google App Engine is a great strategy. It lets you get up and running fast, and if you do outgrow the restrictions of GAE then you can move your web2py application elsewhere.
However, keeping this portability means you should stay away from the advanced parts of GAE (Task Queues, Transactions, ListProperty, etc).
The AppEngine uses BigTable as it's datastore backend. Don't try to write a traditional relational-database driven application. BigTable is much more well suited for use as a highly-scalable key-value store. Avoid joins if at all possible.
I wouldn't worry about any of this. After having played with Google App Engine for a while now, I've found that it scales quite well for large data sets. If your data elements are large (i.e. photos), then you'll need to integrate with another service to handle them, but that's probably going to be true no matter what with data of that size. Also, I've found BigTable relatively easy to work with having come from a background entirely in relational databases. Finally, Django is a somewhat hidden, but awesome, "feature" of Google App Engine. If you've never used it, it's a really nice, elegant web framework that makes a lot of common tasks trivial (forms come to mind here).
Google has just released version 1.3.0 of the SDK with support with a new Blobstore API for storage of files up to 50MB. See the post "App Engine SDK 1.3.0 Released Including Support for Larger User Uploads".
What about Google Wave? It's being built on appengine, and once live, real-time translatable chat reaches the corporate sector... I could see it hitting top 1000th... But then again, that's an internal project that gets to do special stuff other appengine apps can't.... Like hanging threads; I think... And whatever else Wave has under the hood...
If you are planning on a 'huge' website, then don't use App Engine. Simple as that. The App Engine is not built to deliver the next top 1000th website.
Allow me to also ask what do you mean by 'huge', how many simultaneous users? Queries per second? DB load?