I have a server with a database, the server will listen for http requests, and using JSONs for
data transferring.
Currently, what my server code(python) mainly do is read the JSON file, convert it to SQL and make some modifications to the database. And the function of the server, as I see, is only like a converter between JSON and SQL. Is this the right procedure that people usually do?
Then I come up with another idea, I can define a class for user information and every user's information in the database is an instance of that class. When I get the JSON file, first I put it into the instance, do some operation and then put it into the database. In my understanding, it adds a language layer between the http request and the database.
The question is, what do people usually do?
The answer is: people do usually that, what they need to do. The layer between database and client normally provides a higher level api, to make the request independent from the actual database. But how this higher level looks like depends on the application you have.
People usually make use of a Web framework, instead of implementing the basic machinery themselves as you are doing.
That is: Python i s a great language that easily allows one to translate "json into sql" with a small amount of code - and it is great for learning. If you are doing this for educational purposes, it is a nice project to continue fiddling with, and maybe you can have some nice ideas in this answer and in others.
But for "real world" usage, the apparent simple plan comes up with real world issues. Tens or even hundreds of them. How to proper separate html/css template from content from core logic, how to deal with many, many aspects of security, etc...
Them, the frameworks come into play: a web-framewrok project is a software project that had, over the years, and soemtimes hundreds of hours of work from several contributors, thought about, and addresses all of the issues a real web application can and will face.
So, it is ok if one want to to everything from scratch if he believes he can come up with a a framework taht has distinguished capabilities not available in any existing project. And it is ok to make small projects for learning purposes. It is not ok to try to come up with something from scratch for real server production, without having a deep knowledge of all the issues involved, and knowing well at least 3, 4 frameworks.
So, now, you've got the basic understanding of a way to get to a framework - it istime to learn some of the frameworks tehmselves. Try, for example, Bottle and Flask (microframeworks), and Django (a fully featured framework for web application development), maybe Tornado (an http server, but with enough of a web framework in it to be usable, and to be very instructive)- just reading the documentation on "how to get started" with these projects, to get to a "hello world" page will lead you to lots of concepts you probably had not thought about yet.
Related
I am new to learning Python for web development and have decided to go with Flask as my framework of choice as of now. I have chosen this primarily for its bare-bones approach on web development and I want to make as much of my web app custom and on my own as possible. I have made a very basic MVC framework in PHP and would like to make something like this in Python. I was researching some ways to interact with a database and add user login/register support and my original thoughts were that I wanted to do this all custom with my own methods and objects and what not. Similar to how I have done in PHP, so I can learn as much about developing my own back-end efficiently for long term production projects. In some research I found the basic objects in Flask such as Login Manager, Login-Form and Flask-Admin, etc. I don't like the idea of using these nicely packaged things that I have very little control over and have not dev'd custom.
Using that kind of stuff I also feel that I am not learning how all of that stuff works on the lowest level so I could not reproduce some of the benefits they are giving me later on in life when project circumstances change and maximum flexibility, customization, maintainability, and efficiency is needed. Obviously I am not in that position right now, but I DO know how to write this back end stuff in PHP and am just wondering if that means anything as far as managing sessions and Authentication on my own in Python/Flask, or if my thinking is totally out of ocontext and its not even close to managing the same stuff.
It is a really great idea to learn how authentication works at a low level -- it's particularly important as many people never learn this stuff, and it's quite interesting!
What I'd recommend you do is take a look at the official Flask tutorial (http://flask.pocoo.org/docs/0.10/tutorial/), as it covers a lot of this (working directly with sessions, etc.).
What I would not recommend, however, is using this sort of thing in production.
Using your own authentication code is almost never a good idea -- it's much better to rely on a well supported library that has been audited by other people for security issues.
In the Flask world you've got a couple choices:
Flask-Login: https://flask-login.readthedocs.org/en/latest/
Flask-Security: https://pythonhosted.org/Flask-Security/
Flask-Stormpath: http://flask-stormpath.readthedocs.org/en/latest/
Of those 3, I really like Flask-Stormpath -- but I'm super biased as I wrote it =)
Flask-Stormpath supports the widest array of customization / etc., and allows you to do whatever you want with it.
Hope that helps!
Flask official doc have some app examples. One of them is MiniTwit a micro Twitter clone. As it is a complete app, you could found much of what you're looking for. If you want to make something greater, try Full Stack Python Flask tutorial
I'd like to ask you about your experiences in developing facebook applications in Python. Which of the popular web frameworks for this language you think best suits this purpose? I know "best" is a very subjective word, so I'm specifically interested in the following:
Most reusable libraries. For example one might want to automatically create accounts for new logged in facebook users, but at the same time provide an alternative username + password logging functionality. I need authentication to fit into this nicely.
Facebook applications tend to differ from CMS-like sites. They are action intensive. For more complicated use-cases, usually some kind of caching for the data fetched from Open Graph API is required in order to be able to perform some queries on local and facebook data at once (for example join some tables based on friendship relation).
I'd definitely prefer popular solutions. They just seem to be much more stable and better thought through. I've previously developed a facebook application in Grails and I as much as I liked the architecture and the general ideas, the amount of bugs and complication that I ran into was just a little bit too much. Also Groovy is still quite an exotic language to develop in, and this time I'm not going to work on my own.
I'm not new to Python, but definitely new to web development in Python. Though after the experience with Grails and all its twists and turns I doubt Python could really scare me.
I would almost undoubtedly go with Django as the easiest and most popular framework for developing any type of web applications, if there's a need for a full-stack framework.
Specifically, in regards to Django's app universe, it is plentiful with many active applications -- but that has its downfalls too. There's no standard application for any 'one' thing, but there are a few applications that will do basically 90% of all that's needed. Sometimes the code is poorly written, but most of the time, the apps work and do what they are needed to do, so there's almost no need for someone to dive right in to the code.
Narrowing down our options, I have had great luck with Omab's Django-Social-Auth, which was absolutely a snap to integrate. It required 3 variables in my settings.py and I was up and running.
The only issue might be if you do not want to use the django.contrib.auth.User model, but, if you are not thinking about using that, I would think about that decision twice :)
To narrow it down even further, pyfacebook is another option for integrating Facebook. It comes with a djangofb application so it's just drop, add to settings.py and all is well. It even comes with an example Django application as part of the distribution. I've had pretty good luck with this application, but, I still think Omab's much easier to integrate.
Finally, Facebook's own python-sdk is easy to integrate from a raw standpoint, where they just give you access to their APIs using a simple Python API. However, it seems to cater more to the AppEngine folks, so YMMV.
I've used Django for quite some time. As of late I use Jinja2 instead. No particular reason, but it's another option
If you do not want to start on Django now. Try learning Flask(which is comparatively a lot easier to begin than Django) and then start building app with Flask.
I've been learning python for use in ArcGIS and some other non-web applications. However, now that I've taken on building a personal website I am interested in using it for web development (as it is the only scripting language I currently know).
I've noticed that there are a lot of these things called "frameworks", such as Django. From what I understand they are just a collection of packages to save you from re-inventing the wheel but I don't really know how they work.
Furthermore, I do not like GUIs, if I need a framework I would like to find one that could be used through a terminal, starts out simple and can be scaled for more complexity when I'm ready. Any advice or ideas on frameworks and why I would want to use one?
The Python web frameworks have nothing to do with GUIs, and can all be used via the terminal.
The benefits of a framework, as you say, are all to do with making your life easier by supplying the components you need to build a website: the main ones are database interaction through an ORM, a templating system, and URL routing. On top of that, the big frameworks also included optional extras like user authentication, administration interface, and so on.
Personally I like Django, but your mileage may vary: I would say, though, that whatever you do with Python and the web will require some sort of framework, even if it's one of the absolute minimal ones like Flask which basically do just the routing part. There's simply no point in writing all this stuff from scratch when it's been done for you.
I'd second the post above: Django is a great framework and will save you loads of time in the long run.
Pretty much every challenge you'll come across when writing a web application will already have been solved, e.g. How do I send emails? What about an admin interface to edit the data? User security?
In my view picking the best framework is all about the ecosystem around that framework. How well used is it? Is it discussed widely on the internet? Have others encountered, and solved, the problems I'm facing?
In terms of where you start, see the Django Tutorial here:
http://docs.djangoproject.com/en/1.2/intro/tutorial01/
If you think Django offers you too much, I'd recommend that you take a look at CherryPy just to compare the different, and much simpler, approach.
With Python, you've got lots of options. To start, I would recommend looking here -- it explains the basics and provides a fairly complete list of frameworks.
If you're looking for something that starts out simple but can also handle more complexity, then you should take a look at web2py. It requires no installation or configuration, has no dependencies, and includes a web server and a relational database. It also includes an optional web-based integrated development environment and admin interface, but you can work through the terminal instead if you prefer.
It's very easy to learn and was designed for ease of use, faster development, and security. You can get a lot done with very little code thanks to the included scaffolding app along with many sensible default behaviors. As things get more complex, web2py can handle it, as it is a well-integrated full-stack framework with lots of built-in functionality, including a database abstraction layer, form handling and validation, access control, web services, and easy Ajax integration.
Personnally, I don't use any framework, I write either from scratch on BaseHTTPServer, or using WSGI (with mod_wsgi).
It is a bit long to write the skeleton, but I think it is faster (I mean at runtime), there is less constraints, and there is lesser to learn.
This year me and a friend have to make a project for the final year of university. The plan is to make a proxy/sever that allows to store ontologies and RDF's, by this way this data is "chained" to a web, so you can make a request for that web and the proxy will send you the homepage with metadata.
We have been thinking to use python and rdflib, and for the web we don't know which framework is the best. We thought of django, but we think that is very big for our purpose, and we decided that webpy or web2py is a better option.
We don't have any python coding experience, this will be our very first time. We have been always programming in c++ and java.
So taking into account everything we've mentioned our question is, which would be the best web framework for our project? And will rdflib suit fine with this framework?
Thanks :)
I have developed several Web applications with Python framworks consuming RDF data. The choice always depends on the performance needed and the amount of data you'll have to handle.
If the number of triples you'll handle is in the magnitude of few thousands then you can easily put together a framework with RDFlib + Django. I have used this choice with toy applications but as soon as you have to deal with lots of data you'll realise that it simply doesn't scale. Not because of Django, the main problem is RDFlib's implementation of a triple store - it is not great.
If you're familiar with C/C++ I recommend you to have a look at Redland libraries. They are written in C and you have bindings for Python so you can still develop your Web layer with Django and pull out RDF data with Python. We do this quite a lot and it normally works. This option will scale a bit more but won't be great either.
In case your data grows to millions of triples then I recommend you to go for a Scalable Triple store. You can access them through SPARQL and HTTP. My choice is always 4store. Here you have a Python client to issue queries and assert/remove data 4store Python Client
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Looking to do a very small, quick 'n dirty side project. I like the fact that the Google App Engine is running on Python with Django built right in - gives me an excuse to try that platform... but my question is this:
Has anyone made use of the app engine for anything other than a toy problem? I see some good example apps out there, so I would assume this is good enough for the real deal, but wanted to get some feedback.
Any other success/failure notes would be great.
I have tried app engine for my small quake watch application
http://quakewatch.appspot.com/
My purpose was to see the capabilities of app engine, so here are the main points:
it doesn't come by default with Django, it has its own web framework which is pythonic has URL dispatcher like Django and it uses Django templates
So if you have Django exp. you will find it easy to use
But you can use any pure python framework and Django can be easily added see
http://code.google.com/appengine/articles/django.html
google-app-engine-django (http://code.google.com/p/google-app-engine-django/) project is excellent and works almost like working on a Django project
You can not execute any long running process on server, what you do is reply to request and which should be quick otherwise appengine will kill it
So if your app needs lots of backend processing appengine is not the best way
otherwise you will have to do processing on a server of your own
My quakewatch app has a subscription feature, it means I had to email latest quakes as they happend, but I can not run a background process in app engine to monitor new quakes
solution here is to use a third part service like pingablity.com which can connect to one of your page and which executes the subscription emailer
but here also you will have to take care that you don't spend much time here
or break task into several pieces
It provides Django like modeling capabilities but backend is totally different but for a new project it should not matter.
But overall I think it is excellent for creating apps which do not need lot of background processing.
Edit:
Now task queues can be used for running batch processing or scheduled tasks
Edit:
after working/creating a real application on GAE for a year, now my opnion is that unless you are making a application which needs to scale to million and million of users, don't use GAE. Maintaining and doing trivial tasks in GAE is a headache due to distributed nature, to avoid deadline exceeded errors, count entities or do complex queries requires complex code, so small complex application should stick to LAMP.
Edit:
Models should be specially designed considering all the transactions you wish to have in future, because entities only in same entity group can be used in a transaction and it makes the process of updating two different groups a nightmare e.g. transfer money from user1 to user2 in transaction is impossible unless they are in same entity group, but making them same entity group may not be best for frequent update purposes....
read this http://blog.notdot.net/2009/9/Distributed-Transactions-on-App-Engine
I am using GAE to host several high-traffic applications. Like on the order of 50-100 req/sec. It is great, I can't recommend it enough.
My previous experience with web development was with Ruby (Rails/Merb). Learning Python was easy. I didn't mess with Django or Pylons or any other framework, just started from the GAE examples and built what I needed out of the basic webapp libraries that are provided.
If you're used to the flexibility of SQL the datastore can take some getting used to. Nothing too traumatic! The biggest adjustment is moving away from JOINs. You have to shed the idea that normalizing is crucial.
Ben
One of the compelling reasons I have come across for using Google App Engine is its integration with Google Apps for your domain. Essentially it allows you to create custom, managed web applications that are restricted to the (controlled) logins of your domain.
Most of my experience with this code was building a simple time/task tracking application. The template engine was simple and yet made a multi-page application very approachable. The login/user awareness api is similarly useful. I was able to make a public page/private page paradigm without too much issue. (a user would log in to see the private pages. An anonymous user was only shown the public page.)
I was just getting into the datastore portion of the project when I got pulled away for "real work".
I was able to accomplish a lot (it still is not done yet) in a very little amount of time. Since I had never used Python before, this was particularly pleasant (both because it was a new language for me, and also because the development was still fast despite the new language). I ran into very little that led me to believe that I wouldn't be able to accomplish my task. Instead I have a fairly positive impression of the functionality and features.
That is my experience with it. Perhaps it doesn't represent more than an unfinished toy project, but it does represent an informed trial of the platform, and I hope that helps.
The "App Engine running Django" idea is a bit misleading. App Engine replaces the entire Django model layer so be prepared to spend some time getting acclimated with App Engine's datastore which requires a different way of modeling and thinking about data.
I used GAE to build http://www.muspy.com
It's a bit more than a toy project but not overly complex either. I still depend on a few issues to be addressed by Google, but overall developing the website was an enjoyable experience.
If you don't want to deal with hosting issues, server administration, etc, I can definitely recommend it. Especially if you already know Python and Django.
I think App Engine is pretty cool for small projects at this point. There's a lot to be said for never having to worry about hosting. The API also pushes you in the direction of building scalable apps, which is good practice.
app-engine-patch is a good layer between Django and App Engine, enabling the use of the auth app and more.
Google have promised an SLA and pricing model by the end of 2008.
Requests must complete in 10 seconds, sub-requests to web services required to complete in 5 seconds. This forces you to design a fast, lightweight application, off-loading serious processing to other platforms (e.g. a hosted service or an EC2 instance).
More languages are coming soon! Google won't say which though :-). My money's on Java next.
This question has been fully answered. Which is good.
But one thing perhaps is worth mentioning.
The google app engine has a plugin for the eclipse ide which is a joy to work with.
If you already do your development with eclipse you are going to be so happy about that.
To deploy on the google app engine's web site all I need to do is click one little button - with the airplane logo - super.
Take a look the the sql game, it is very stable and actually pushed traffic limits at one point so that it was getting throttled by Google. I have seen nothing but good news about App Engine, other than hosting you app on servers someone else controls completely.
I used GAE to build a simple application which accepts some parameters, formats and send email. It was extremely simple and fast. I also made some performance benchmarks on the GAE datastore and memcache services (http://dbaspects.blogspot.com/2010/01/memcache-vs-datastore-on-google-app.html ). It is not that fast. My opinion is that GAE is serious platform which enforce certain methodology. I think it will evolve to the truly scalable platform, where bad practices simply not allowed.
I used GAE for my flash gaming site, Bearded Games. GAE is a great platform. I used Django templates which are so much easier than the old days of PHP. It comes with a great admin panel, and gives you really good logs. The datastore is different than a database like MySQL, but it's much easier to work with. Building the site was easy and straightforward and they have lots of helpful advice on the site.
I used GAE and Django to build a Facebook application. I used http://code.google.com/p/app-engine-patch as my starting point as it has Django 1.1 support. I didn't try to use any of the manage.py commands because I assumed they wouldn't work, but I didn't even look into it. The application had three models and also used pyfacebook, but that was the extent of the complexity. I'm in the process of building a much more complicated application which I'm starting to blog about on http://brianyamabe.com.