I'm familiar with LAMP systems and have been programming mostly in PHP for the past 4 years. I'm learning Python and playing around with Nginx a little bit.
We're working on a project website which will handle a lot of http handle requests, stream videos(mostly from a provider like youtube or vimeo). My colleague has experience with OpenBSD and has insisted that we use it as an alternative to linux.
The reason that we want to use OpenBSD is that it's well known for
it's security.
The reason we chose Python is that it's fast.
The reason we want to use Nginx is that it's known to be able to
handle more http request when compared to Apache.
The reason we want to use NoSQL is that MySQL is known to have
problems in scalability when the databases grows.
We want the web pages to load as fast as possible (caching and cdn's will be used) using the minimum amount of hardware possible. That's why we want to use ONPN (OpenBSD,Nginx,Python,Nosql) instead of the traditional LAMP (Linux,Apache,Mysql,PHP).
We're not a very big company so we're using opensource technologies. Any suggestion is appreciated on how to use these software as a platform and giving hardware suggestions is also appreciated. Any criticism is also welcomed.
My advice - if you don't know how to use these technologies - don't do it. Few servers will cost you less than the time spent mastering technologies you don't know. If you want to try them out - do it. One by one, not everything at once. There is no magic solution on how to use them.
I agree with wdev, the time it takes to learn this is not worth the money you will save. First of all, MySQL databases are not hard to scale. WordPress utilizes MySQL databases, and some of the world's largest websites use MySQL (google for a list). I can also say the same of linux and PHP.
If you design your site using best practices (CSS sprites) Apache versus Nginx will not make a considerable difference in load times if you utilize a CDN and best practices (caching, gzip, etc).
I strongly urge you to reconsider your decisions. They seem very ill-advised.
Related
My company is highly dependent on Java and jsf; All projects since I was hired are implemented using them. But most of those projects are facing problems related to performance and availability. So am finally considering a shift to other technologies and I have tried to research in the net, and im about to decide to try python. But before i start i would like to hear ur answer that Python would solve me the performance problems we are facing.
To make things clear the performance problems we mostly face are related to glassfish server and page loading. We are currently using ice faces and have tried wood stock back then. Additionally I can't use .net for some policy related issues. And PHP is also out of question due to some security leaks experienced in earlier projects.
So am expecting to read the pros and cons related with performance and availability in trying to convince my boss and customers in to python.
I have some doubts that you will gain performance by using Django or a Python based solution. I don't know the Glassfish server nor how it scales up but unless badly designed I don't see why it should perform badly.
From the explanation of your performance issues, it doesn't seem to be a problem of language speed but instead, server configuration and availability.
Assuming that your Java code is reasonably optimal (i.e. efficient and acceptably fast), you won't solve the problem by using some Python solution. Instead you should invest some time into studying caching mechanisms and/or proxy solutions.
Depending on how your server is setup, an additional advice would be to let all the static content be served by a dedicated server such as Apache, nginx or similar and only leave the the dynamic content on to be interpreted by your glassfish server.
Since your projects are written in Java you are in theory using a language that can potentially be faster than Python, I don't see why a Python solution would perform better unless there is something wrong with the framework you are using.
If you want to talk about prototyping or faster development, then that's a different subject discussed multiple times on stackoverflow.
I was looking at a book on python network programming and i wanted to know what would be the benefits to learning python network programming comprehensively? This would be in the context of being able to develop some really cool, ground breaking web apps. I am a python newbe so all opinions woul be appreciated.
Kind Regards
4 Years later:
This was 4yrs ago, its crazy how much I've grown as a developer. Regarding how it has helped, I've developed an email application, a chat application using Objective C, python Twisted on the server side, it also helped with developing my apns push notification pipeline.
If you want to develop wed apps, than you should rather focus on web frameworks like Django or Pylons.
"Network programming" isn't about "cool web apps". It's more about creating servers, and creating clients that talk to servers. It's about sockets, tcp/ip communication, XMLRPC, http protocols and all the technologies that let two computers talk to each other.
If all you're interested in is web apps, learning network programming won't benefit you a whole lot.
"python network programming" isn't any special kind of network programming. It sounds like if you had a better grasp on network programming you would be able to see where python would fit in to your overall design. And instead of reading a generic book about it, you would dig through the python API's and go from there.
The cool thing about python is that it's a huge collection of libraries which are optimized to each task. At work we use python to do all our server-side heavy lifting. We then use jquery and the Objective-J based Cappuccino to present an interface to the user.
The network is and always will be the sexiest arena for a hacker. An attacker can do almost anything with simple network access, such as scan for hosts, inject packets, sniff data, remotely exploit hosts, and much more. But if you are an attacker who has worked your way into the deepest depths of an enterprise target, you may find yourself in a bit of a conundrum: you have no tools to execute network attacks. No netcat. No Wireshark. No compiler and no means to install one. However, you might be surprised to find that in many cases, you’ll find a Python install.
Hence, the one benefit of python network programming I see is that one has a chance of becoming a penetration tester or an Offensive security guy.
Edit (10-06-2020):
I have no knowledge of what I was thinking when I was writing this answer. It's clearly not helpful.
Regarding the question, making web apps is not equivalent to network programming. In network programming with python, you can start with lower level of the OSI model by using libraries like scapy. Here you can make raw packets and understand the various protocols with it. And then maybe move to application level with libraries like Scrapy which involves web scraping, making http requests, etc. But develping python web apps would use tools like flask, django, jinja, etc and the development of a web app can take the form totally different than that of a regular scripting tool.
Python's stronger web tool is definitely Django.
Said that the biggest benefit of learning it is to achieve a pretty robust backend on your web project.
I'm super new to programming and I've been using appengine to help me learn python and general coding. I'm getting better quickly and I'm loving it all the way :)
Appengine was awesome for allowing me to just dive into writing my app and getting something live that works (see http://www.7bks.com/). But I'm realising that the longer I continue to learn on appengine the more I'm constraining myself and locking myself into a single system.
I'd like to move to developing on full django (since django looks super cool!). What are my next steps? To give you a feel for my level of knowledge:
I'm not a unix user
I'm not familiar with command line controls (I still use appengine/python completely via the appengine SDK)
I've never programmed in anything other than python, anywhere other than appengine
I know the word SQL, but don't know what MySQL is really or how to use it.
So, specifically:
What are the skills I need to learn to get up and running with full django/python?
If I'm going to host somewhere else I suppose I'll need to learn some sysadmin type skills (maybe even unix?). Is there anywhere that offers easy hosting (like appengine) but that supports django?
I hear such great things about heroku I'm considering switching to RoR and going there
I appreciate that I'm likely not quite ready to move away from appengine just yet but I'm a fiercely passionate learner (http://www.7bks.com/blog/179001) and would love it if I knew all the steps I needed to learn so I could set about learning them. At the moment, I don't even know what the steps are I need to learn!
Thank you very much. Sorry this isn't a specific programming question but I've looked around and haven't found a good how-to for someone of my level of experience and I think others would appreciate a good roadmap for the things we need to learn to get up and running.
Thanks,
Tom
PS - if anyone is in London and fancies showing me the ropes in person that would be super awesome :)
First up, you can benefit by doing some RoR work by learning a new language. However, I don't know if that'll be entirely beneficial to you right now since you still are learning. I'd stick with Python and Django (or AppEngine) for the moment, until you can grasp some of the more advance concepts. Then, by all means, learning new languages will be fantastic.
As for moving to Django from AppEngine. There isn't a whole lot that's different. The way you define models is similar, but has different types for the definition. As you mentioned, hosting is another consideration.
There should be plenty of hosting options (mod_wsgi is what you're after) based on Apache. Django in particular has seen quite a bit of popularity, and hosting usually springs up for popular frameworks.
I don't think you'll need to know too much sysadmin stuff. This will all depend on the kind of hosting you can find. Same goes for the database. Hosting providers usually offer databases preconfigured so you shouldn't need to worry about that too much.
Django, along with many other frameworks, provide an ORM (Object Relational Mapper) which abstracts away having to write SQL, by calling methods on objects and accessing their properties. I'd advise learning a little bit of SQL to understand it at a bare minimum though.
The Django tutorial is excellent! If you decide to go the Django route, I'd highly recommend working through the entire thing. A development server comes bundled, so you can try out your work instantly without worrying about a provider. Once you have something you want to share with the world, you can worry about hosting then.
I started off using Windows for Django development and it was quite easy. The amount of command line work you need to do is minimal. Really. Not something you need to worry about, as the tutorial covers all 4 or so commands you need to know.
Django hosting provides links to hosting providers, though I'm not sure how up to date that list is.
Getting started in Django is pretty simple. Once you want to host it, there's a bit more work involved - but that can come later. The friction is minimal. Follow the tutorial, it will take you through running the server, setting up the database (a free one comes bundled), and coding your first app.
What makes you think you're being locked into a single platform? Did you know that Google's App Engine SDK is open source? Also, there are universities and other organizations who are working on building platforms that will use the App Engine SDK outside of the context of Google? Amazon EC2 is also capable of running App Engine's SDK in a limited capacity. I'd say lock-in is perhaps not the right word to use.
Additionally, I believe AppEngine is going to continue to improve as time goes on. Google is the leader of the Internet; they've done great things and will continue to do so. I believe that anyone who sticks with their platform as a service will experience these great benefits in the years to come.
If your reasoning for moving is purely academic, I'd suggest starting a new project. Moving off of AppEngine's SDK is similar to switching from one framework to another on an already-built application. Like with any framework or platform, there are dependencies that must be dealt with in order to successfully migrate the app from platform A to platform B.
django-nonrel makes it possible to run Django on Google AppEngine: http://www.allbuttonspressed.com/projects/django-nonrel
Beside that there exists a couple of cloud offers like djangy https://www.djangy.com/
With both options you can focus on Django and Python programming and don't have to care about the sysadmin stuff.
On the django homepage there is a very good tutorial to get started with django development: http://docs.djangoproject.com/en/1.2/intro/tutorial01/
What are the skills I need to learn to get up and running with full django/python?
The question can't easily be answered because you haven't described the app. You have to actually write down the technology stack -- in detail -- or you'll never know what skills you need.
The skill list mostly comes from your technology choices. So write down your technology choices. (That's part of configuration management, an important skill you'll need if you move away from GAE.)
Since you've chosen to talk about yourself and not your technology choices, I can only guess what technologies you're using and what skills you'll need.
Here's a common technology stack.
Technically, the OS doesn't matter. Most hosting environments use open source GNU/Linux because the licensing is inexpensive. You, too, can do this. You can start with VMWare and download a nice Linux distro. Or, you can buy a very cheap PC and install Linux directly from a DVD image that you can download and burn.
My company demands that I use the Windows PC they give me. So I develop in Windows and test in VMWare Linux (Fedora 14, actually)
To learn Linux, start with a download and install. Then find a tutorial. Then stop using Windows and learn by doing. Flipping back and forth between Windows and Linux is difficult. I can do it because I don't know Windows very well. I treat Windows as a hyper-complex IDE with all kinds of non-standard, non-POSIX quirks that I try to ignore.
RDBMS. Python comes with SQLite. For a lot of applications, it works fine. It works because web sites tend to be heavy on queries and light on updates/inserts, so SQLite works well. MySQL is nice. It's easy to install and runs on Windows as well as other OS's.
The good thing about Django is you need to know very little SQL. Very little.
However, you do need to know a tiny bit about the "Data Control Language" (Grant, Revoke and Create User) to work with MySQL. You won't create a lot of users. But you do need to create a few to get things running. Also, as your database matures, you'll often need to know a little bit about the "Data Definition Language" (Drop Table).
You will need to know how to backup and restore your database. That's absolutely critical.
So find Database Administrator tutorials to get started.
If your application really uses a lot of sophisticated data, you'll need to buy real books on database design so you can understand how the Django models really work. You don't need to become a SQL guru, but it does help to know what's really going on inside the database.
Application Server. We use Apache with the mod_wsgi module. There are numerous choices. Hosting services vary in what they require and what they permit. Some have Apache, mod_wsgi and Django pre-configured. Some don't. Some do not permit tinkering with the Apache configuration. Some do. You probably don't need to know much about this, because you can probably find a hosting service that will handle the details.
Apache tutorials are all over the place. mod_wsgi is very simple; once you understand how Apache works, mod_wsgi is obvious.
Since you have stuff working, presumably you know about HTML and CSS. Those are important skills, but you probably already have them.
Since you have stuff working, you also might know a lot about Configuration Management and how to control change. This isn't obvious and many people do it wrong. If you don't know about CM, you should find some books or articles on configuration management and change control.
Since you have stuff working, you also might know a lot about quality assurance, unit testing and related topics. If you don't have a complete suite of unit tests, you should probably start learning about unit testing before you start any serious coding for the next release of your product.
Bug Tracking, Problem Reporting, Feature Requests and other management skills are also essential. I can't tell if you have them or don't have them. Or what tools you're using for this. If you're working by yourself, you don't need a lot of formality. However, these are critical skills even if you're a one-person developer. Sticky-Notes on your workstation can work. What's important is the skills to manage bugs and features.
Hey Tom,
I suggest the reasonable evaluation you can make is carefully list the advantages and disadvantages of the choices.
The way I don't regret taking was physical rack server (2006-07), moved to virtual hosting (2008) and now moved to GAE (2009-current). Seeing the rate new features get added to GAE and the costeffectiveness are more reasons to stay. I agree more stuff django can are needed like in my case form preview and form validation with GAE are difficult or too difficult to set up.
I tried RoR and soon thought RoR requires more code to do the same GAE can with less code.
Also with GAE you have absolutely no hardware that can break. If you move to a rack server or a virtual hosting where there are places you can get 5 GB hosting for free but you don't have a plan when you run out of 5 GB and may need to migrate again which you don't want.
MySQL has been around for over 10 years and is quite a different kind of system. It's possible to save blobs in MySQL but don't you think the blobstore GAE has is much better?
If you choose to migrate to a solution with MySQL you can export you data from GAE and import it to MySQL with a tool such as approcket.
Kind regards/Niklas R
I know it's kinda subjective but, if you were to put yourself in my shoes which would you invest the time in learning?
I want to write a web app which deals securely with relatively modest amounts of peoples private data, a few thousand records of a few Kb each but stuff that needs to be kept safe, addresses, phone numbers etc. I've done several web projects in PHP/MYSQL and have decided, handy though it is I really don't like PHP and don't want to do another large project in it...
As such I figure I'd best learn something new and so I am considering 2 options (although I'll happily entertain others if you have suggestions). I'm having terrible trouble deciding though. They both look quite involved so rather than just jump in and potentially waste days getting up to speed enough on both of them to make an informed choice I thought I'd come here and canvas some opinion.
So the two options I'm considering are...
One of the PYTHON Web frameworks - TurboGears seems well regarded?
Advantage: Of all the languages I ever tried Python is by far and away my favorite. There's loads of frameworks to choose from and I have done quite a lot of non web python coding over the last few years.
Disadvantage: There's loads to choose from so it's hard to pick! Need to run single server process? or mod_python? which I don't like the sound of. What I do like is the notion of process separation and compartmentalization, i.e. if one users account is compromised it gives an attacker no leverage against the rest of the system. I'm not clear to what extent a python solution would handle that.
Writing it as a SEASIDE app Which I guess runs on a squeak app server?
Adv: From what I've heard it would permit good compartmentalization of users as each would have their own little private VM independent of all the systems other users which sounds wonderful from a security, scaling and redundancy standpoint.
Dis: I've not done any Smalltalk since Uni 15 years back and I never dug too deep into it then. I don't see much entry level help for seaside or that many projects using it. I suspect setting a server up to run it is hard for the same reason i.e. not because it's inherently hard but just cause there will be less help online and a presumption you are already rather au fait with Sqeak/Smalltalk.
So, what do people think? Would I be able to efficiently get the kind of strong separation and compartmentalization I'm after with a Python framework? Is Seaside as good as I think in terms of insulating users from each other? Might I be better off, security wise, sticking to the languages I'm most familiar with so I don't make any n00b mistakes or will Seaside be worth worth scaling the learning curve and prove more secure, comprehensible and maintainable in the long run? At the end of the day it's not a life or death decision and I can always bail if I start with one and then hate it so pls nobody get all holy language war and start flaming anyone! ;-)
Cheers for any replies this gets,
Roger :)
Disclaimer: I really don't like PHP, Python is nice, but doesn't come close to Smalltalk in my book. But I am a biased Smalltalker. Some answers about Seaside/Squeak:
Q: Which I guess runs on a squeak app server?
Seaside runs in several different Smalltalks (VW, Gemstone, Squeak etc). The term "app server" is not really used in Smalltalk country. :)
Q: From what I've heard it would permit good compartmentalization of users as each would have their own little private VM independent of all the systems other users which sounds wonderful from a security, scaling and redundancy standpoint.
Yes, each user has its own WASession and all UI components the user sees are instances living on the server side in that session. So sharing of state between sessions is something you must do explicitly, typically through a db.
Q: I've not done any Smalltalk since Uni 15 years back and I never dug too deep into it then. I don't see much entry level help for seaside or that many projects using it.
Smalltalk is easy to get going with and there is a whole free online book on Seaside.
Q: I suspect setting a server up to run it is hard for the same reason i.e. not because it's inherently hard but just cause there will be less help online and a presumption you are already rather au fait with Sqeak/Smalltalk.
No, not hard. :) In fact, quite trivial. Tons of help - Seaside ml, IRC on freenode, etc.
Q: Is Seaside as good as I think in terms of insulating users from each other?
I would say so.
Q: Might I be better off, security wise, sticking to the languages I'm most familiar with so I don't make any n00b mistakes or will Seaside be worth worth scaling the learning curve and prove more secure, comprehensible and maintainable in the long run?
The killer argument in favor of Seaside IMHO is the true component model. It really, really makes it wonderful for complex UIs and maintenance. If you are afraid of learning "something different" (but then you wouldn't even consider it in the first place I guess) then I would warn you. But if you are not afraid then you will probably love it.
Also - Squeak (or VW) is a truly awesome development environment - debugging live Seaside sessions, changing code in the debugger and resuming etc etc. It rocks.
Forget about mod_python, there is WSGI.
I'd recommend Django. It runs on any WSGI server, there are a lot to choose from. There is mod_wsgi for Apache, wsgiref - reference implementation included in Python and many more. Also Google App Engine is WSGI, and includes Django.
Django is very popular and it's community is rapidly growing.
I'd say take a look at Django. It's a Python framework with a ready-made authentication system that's independent of the hosting OS, which means that compromises are limited to the app that was compromised (barring some exploit against the web server hosting the Python process).
I've been getting into seaside myself but in many ways it is very hard to get started, which has nothing to do with the smalltalk which can be picked up extremely quickly. The challenge is that you are really protected from writing html directly.
I find in most frameworks when you get stuck on how to do something there is always a work around of solving it by using the template. You may later discover that this solution causes problems with clarity down the road and there is in fact a better solutions built into the framework but you were able to move on from that problem until you learned the right way to do it.
Seaside doesn't have templates so you don't get that crutch. No problems have permanently stumped me but some have taken me longer to solve than I would have liked. The flip side of this is you end up learning the seaside methodology much quicker because you can't cheat.
If you decide to go the seaside route don't be afraid to post to the seaside mailing list at squeakfoundation.org. I found it intimidating at first because you don't see a lot of beginner questions there due to the low traffic but people are willing to help beginners there.
Also there are a handful of seaside developers who monitor stackoverflow regularly. Good luck.
Have you taken a look at www.nagare.org ?
A framework particularly for web apps rather than web sites.
It is based around the Seaside concepts but you program in Python (nagare deploys a distribution of python called Stackless Python to get the continuations working).
Like Seaside it will auto generate HTML, but additionally can use templates as required.
It has been recently open sourced by http://www.net-ng.com/ who themselves have many years experience in delivering web apps/sites in quality web frameworks like zope and plone.
I am researching it myself at the moment to see if it fits my needs, so can't tell you what I think of it in the wild. If you take a look, please give your feedback.
While considering a Smalltalk web framework, look at Aida/Web as well. Aida has built-in security with user/group/role management and strong access control, which can help you a lot in your case. That way you can achieve safe enough separation of users at the user level in one image. But if you really want, you can separate them with running many images as well. But this brings increased maintenance and I'd think twice if it is worth.
I'm toying with Seaside myself and found this tutorial to be invaluable in gaining insight into the capabilities of the framework.
I think you've pretty much summed up the pros and cons. Seaside isn't that hard to set up (I've installed it twice for various projects) but using it will definitely affect how you work--in addition to re-learning the language you'll probably have to adjust lots of assumptions about your work flow.
It also depends on two other factors
If other people will eventually be maintaining it, you'll have better luck finding python programmers
If you are doing a highly stateful site, Seaside is going to beat the pants off any other framework I've seen.
There is now an online book on Seaside to complete the tutorial pointed out earlier.
What makes Python stand out for use in web development? What are some examples of highly successful uses of Python on the web?
Django is, IMHO, one of the major benefits of using Python. Model your domain, code your classes, and voila, your ORM is done, and you can focus on the UI. Add in the ease of templating with the built-in templating language (or one of many others you can use as well), and it becomes very easy to whip up effective web applications in no time. Throw in the built-in admin interface, and it's a no-brainer.
Certainly one successful use of Python on the web is Google App Engine. Site authors write code in (a slightly restricted subset of) Python, which is then executed by the App Engine servers in a distributed and scalable manner.
Quotes about Python:
"Python is fast enough for our site
and allows us to produce maintainable
features in record times, with a
minimum of developers," said Cuong Do,
Software Architect, YouTube.com.
YouTube uses a lot of Python and is probably the best example of a Python success story.
A great example of a Django success story is the Washington Post, who recently shared a big list of applications they have developed:
http://push.cx/2009/washington-post-update
www.lawrence.com and www.ljworld.com are two of the first sites to use Django (before it was even open source).
djangositeoftheweek.com has a bunch of good case studies.
www.everyblock.com is another great example.
Finally, http://www.djangosites.org/ links to nearly 2,000 other Django powered sites.
Short anwser: the diversity of tools readily available and freedom of choice.
This sounds like a simple question but which it really isn't. While Python is very good for web development and this has been shown by the, oh so famous, Google App Engine, Plone and Django. One has to point out that the development way in Python requires a lot more from the developer than PHP but it gives a lot more to the mix as well.
The entry level on actually producing something is higher. This is because there are bunch of different tools for doing web development with Python. Choosing the web development framework can be a hard decision for an inexperienced developer.
Having a lot of different tools is a two edged sword. To some extent it brings you the freedom of choice to pick the one you might want but then again how do you really know which one is good for what you're doing. This brings me to my point. Python stands out from the mass by not having a standard or de facto web development library. While this is pretty much against the principle of having only one simple way of doing on thing it also brings us a wide variety of different tools with different kind of design choices. At first this might feel very frustrating because it would be so much easier if somebody had made the choice for you but now that you're left to make the choice you actually might have to think about what you're doing and what would fit. ...or you might just end up picking one and blowing your head off after you've realized that you made the wrong choice. Anyway you end up, you've made the choice and no one else.
Furthermore,
Python is both strong in web and in data analytics and machine learning. For example scikit, sci-py and numpy are very strong. In some cases, it can be very interesting to have the both elements on the same server.
For example http://rankmytweet.com uses this a lot.
trac(bug tracker) and moinmoin(wiki) are too web based python tools that I find invaluable.
GNU Mailman is another project written in python that is widely successful.
As many have pointed out, Django is a great reason to use Python...so in order to figure out why Python is great for web development, the best bet is to look at why it is a good language to build a framework like Django.
IMHO Python combines the cleanest, or at least one of the cleanest, metaprogramming models of any language with a very pure object orientation. This not only makes it possible to write extremely general abstractions that are easy to use, but also allows the abstractions to combine relatively cleanly with others. This is harder to do in languages that take a code-generation based approach to metaprogramming (e.g. Ruby).
Dynamic languages are in general good for web apps because the speed of development. Python in particular has two advantages over most of them:
"batteries included" means lots of available libraries
Django. For me this is the only reason why i use Python instead of Lua (which i like a lot more).
Besides the frameworks...
Python's pervasive support for Unicode should make i18n much smoother.
A sane namespace system makes debugging much nicer, because it's typically easier to find where things are defined.
Python's inability to function as a standalone templating language should discourage the mixture of HTML with model code
Great standard library
Other examples of Python sites are Reddit and YouTube.