I was looking at a book on python network programming and i wanted to know what would be the benefits to learning python network programming comprehensively? This would be in the context of being able to develop some really cool, ground breaking web apps. I am a python newbe so all opinions woul be appreciated.
Kind Regards
4 Years later:
This was 4yrs ago, its crazy how much I've grown as a developer. Regarding how it has helped, I've developed an email application, a chat application using Objective C, python Twisted on the server side, it also helped with developing my apns push notification pipeline.
If you want to develop wed apps, than you should rather focus on web frameworks like Django or Pylons.
"Network programming" isn't about "cool web apps". It's more about creating servers, and creating clients that talk to servers. It's about sockets, tcp/ip communication, XMLRPC, http protocols and all the technologies that let two computers talk to each other.
If all you're interested in is web apps, learning network programming won't benefit you a whole lot.
"python network programming" isn't any special kind of network programming. It sounds like if you had a better grasp on network programming you would be able to see where python would fit in to your overall design. And instead of reading a generic book about it, you would dig through the python API's and go from there.
The cool thing about python is that it's a huge collection of libraries which are optimized to each task. At work we use python to do all our server-side heavy lifting. We then use jquery and the Objective-J based Cappuccino to present an interface to the user.
The network is and always will be the sexiest arena for a hacker. An attacker can do almost anything with simple network access, such as scan for hosts, inject packets, sniff data, remotely exploit hosts, and much more. But if you are an attacker who has worked your way into the deepest depths of an enterprise target, you may find yourself in a bit of a conundrum: you have no tools to execute network attacks. No netcat. No Wireshark. No compiler and no means to install one. However, you might be surprised to find that in many cases, you’ll find a Python install.
Hence, the one benefit of python network programming I see is that one has a chance of becoming a penetration tester or an Offensive security guy.
Edit (10-06-2020):
I have no knowledge of what I was thinking when I was writing this answer. It's clearly not helpful.
Regarding the question, making web apps is not equivalent to network programming. In network programming with python, you can start with lower level of the OSI model by using libraries like scapy. Here you can make raw packets and understand the various protocols with it. And then maybe move to application level with libraries like Scrapy which involves web scraping, making http requests, etc. But develping python web apps would use tools like flask, django, jinja, etc and the development of a web app can take the form totally different than that of a regular scripting tool.
Python's stronger web tool is definitely Django.
Said that the biggest benefit of learning it is to achieve a pretty robust backend on your web project.
Related
Well this is my last ditch effort of finding help/guidance. We've only just started learning Python at university and i really want to get a jump on everyone else and i've really been focusing on networking with Python. I've had a look at the twisted framework and it's massive ! even the krondo.com site is.
I've seen Concurrence for python on LinuxQuestions a couple of times, which seems easier (however it looks like Concurrence has been dis-continued?), and was wondering if im better off knuckling down and really getting into twisted or maybe trying something else.... Its not that im afraid of hard work, but i dont want to go down a path that when im finished, i find out that know one use's it out in the real world, or knows of it.
Im sorry its not a technical question, but this is really the only place where i know i can get proper guidance. If you think putting my head down and learning twisted will be worth my while, then ill do that. Thanks in advance for your time.
Honestly, I would take a step back on frameworks and work with the Python standard library on a more basic level. The python docs are a very good and authoritative source of information on Python. If you're looking into networking, here's a path I'd recommend looking over:
Sockets
Learn how to work with basic socket connections, making clients, and making servers.
SSL
A lot of web services use this to communicate data securely for APIs. It may be difficult to wrap your head around, but it's a highly valuable piece of knowledge to have.
Select
One way of handling multiple clients connecting to a service. This gets into the concurrency realm which is a fairly sought after skill.
Threads
Another way of handling multiple connections, networking tasks.
Protocols
Now that you've wrapped your head around basic networking, it's time to get into protocols. These are sending specifically formatted data over the network to a server that can handle such data. Yes, I'd recommend going through all the different protocols.
Data Parsing
Data that is sent and received during different protocol communication can be encoded in various ways to make it easier to deal with data.
Pack and Unpack
Sometimes network communication isn't done with a standard format. These functions are helpful in dealing with binary protocols if there's no library available.
Pickle
This is some advanced stuff and python specific. You can use it to send python objects over the network. Just be careful to do this on trusted networks only for security purposes.
With that in mind these are general recommendations. Check the docs out, write some code, and ask questions if you get stuck.
I'm familiar with LAMP systems and have been programming mostly in PHP for the past 4 years. I'm learning Python and playing around with Nginx a little bit.
We're working on a project website which will handle a lot of http handle requests, stream videos(mostly from a provider like youtube or vimeo). My colleague has experience with OpenBSD and has insisted that we use it as an alternative to linux.
The reason that we want to use OpenBSD is that it's well known for
it's security.
The reason we chose Python is that it's fast.
The reason we want to use Nginx is that it's known to be able to
handle more http request when compared to Apache.
The reason we want to use NoSQL is that MySQL is known to have
problems in scalability when the databases grows.
We want the web pages to load as fast as possible (caching and cdn's will be used) using the minimum amount of hardware possible. That's why we want to use ONPN (OpenBSD,Nginx,Python,Nosql) instead of the traditional LAMP (Linux,Apache,Mysql,PHP).
We're not a very big company so we're using opensource technologies. Any suggestion is appreciated on how to use these software as a platform and giving hardware suggestions is also appreciated. Any criticism is also welcomed.
My advice - if you don't know how to use these technologies - don't do it. Few servers will cost you less than the time spent mastering technologies you don't know. If you want to try them out - do it. One by one, not everything at once. There is no magic solution on how to use them.
I agree with wdev, the time it takes to learn this is not worth the money you will save. First of all, MySQL databases are not hard to scale. WordPress utilizes MySQL databases, and some of the world's largest websites use MySQL (google for a list). I can also say the same of linux and PHP.
If you design your site using best practices (CSS sprites) Apache versus Nginx will not make a considerable difference in load times if you utilize a CDN and best practices (caching, gzip, etc).
I strongly urge you to reconsider your decisions. They seem very ill-advised.
I have quite sometime now (a month or two) before I can actually start learning a web programming language; so I am still worrying about what language to learn.
Whatever programming language I choose, I'd also have to learn JavaScript alongside that language as it's the go to for client-side scripting.
I am looking at: (1) Python, Ruby, JavaScript (Node.js -- server-side & jQuery -- client-side); (2) DBMS - PostgreSQL (3) I am going to learn C/C++ anyway (but after I learn a high-level programming language among those mentioned above).
So, the question is, is it a brave move to choose Node.js/JavaScript for server-side programming? (brave in the sense, learning Python would be a lot easier and good, and you're only trying to be brave/foolish by choosing Node.js)
Points to consider:
I am sure Python, Ruby and all will have a great future. Will Node.js enjoy the same too? Do you think? (long-term plans)
Availability web frameworks: Is there a framework for Node.js that is as mature and complete as Django (or Pylons)?
Availability of support from SO community and the web: that is, can I be sure that I would get a few good replies from the SO community if something goes wrong and I need help? I mean, I am sure that i would, if it is Python, C/C++ etc.
is Node.js too slow to code than Python?
Why I am even thinking about Node.js?
I've read comments/answers on SO saying that Node.js is much much faster than other high-level languages like Python, Ruby or PHP. Sorry, but like I said, it's not my opinion [READ].
If I choose Node.js, I would only have to learn one programming language basically - - JavaScript. (I am not looking for a job, I want to develop a (large?) web application like a social network or photo-sharing service)
[I believe anyone can have these questions, so it's not entirely subjective]
1) I personally love (!) Ruby, I think it's the most beautiful language one can possibly code in, but I also love the speed of Node.js and JavaScript. Fortunately I can combine what I love about the two, the answer is CoffeeScript (it has a Ruby / Python style of programming, but it compiles to plain JavaScript). Note: you should learn JavaScript properly and only then pass to CoffeeScript.
2) I wouldn't use PostgreSQL, because I would like use something more "speedy" like MongoDB (JavaScript on the DB also - it's easier to learn MongoDB than CouchDB if you only know SQL) or CouchDB (which is a pretty amazing db, also in JavaScript - it supports synchronization by default). However if you want to use PostgreSQL there are drivers for it.
3) It's great that you are planning on learning C/C++, since every language is built on top of that. You could write C/C++ modules for Ruby, Node or Python.
Answering your (extra) 3 points:
1) There is a really good web framework called Express, which is highly inspired by Sinatra (Ruby). There are also other frameworks, but I believe Express is the best there is (you can even build your own framework on top of it etc, it's "flexible").
2) I am actually subscribed to the "node.js" tag on SO and I can say that if you post a valid question you'll get an answer from 1 minute to 24h. More than that, just join the IRC channels for Node ( #node.js #express #socket.io #nodejitsu ) and checkout the activity over there. I can assure you while I was learning, I didn't have any unanswered question (for real, the guys are extremely helpful).
3) No, that's not the case once you get used to it. The only thing you need to get used to when coding in Node.js (not only, Erlang, Twisted or EventMachine also) is the asynchronous way of programming. Also there are a ton of modules and helpers for almost everything (so that can speed up your coding too, you don't have to reinvent the wheel for everything - you can also use some JS code for the frontend).
I can tell you from my own experience that once you get used to the async style of programming (with callbacks etc) there's nothing that can slow you down anymore.
Extra
While Node.js is kind of young (2009), Github, Linkedin, Yahoo, 37signals, Learnboost, Trello are using it, so you won't be that brave :P
I'm making a simple web app where I have some simple python scripts that do the text crunchin g I need - but I'm not quite sure how to interface it with a client who'd only want to see some HTML forms.
There's so many different server side frameworks out there - but I don't think I need anything too heavy duty - just a mechanism to accept data from the forms that the user fills in, and feed it to my Python code and back.
Could someone suggest what tools I should look into and what design paradigms I should follow? Simple pointers to different references around the web with a single line about their significance would also help.
Best.
At the moment, the best lightweight and yet very powerful framework for python IMO is Flask. If you want form abstraction there is a WTFlask plugin for it which is WTForms adapted for flask - http://flask.pocoo.org/.
Web2py is also a very good framework for starters because it has helpers and wizards for creating/running an application, and also has something like an admin interface, offering an On-line IDE functionality. - http://www.web2py.com/
I proposed these two frameworks because you can start fast with them, they both have good documentation, both are powerful yet easy and they are pretty friendly for beginners.
Not normally supported with cheap hosting, but mod_python on apache might be right up your alley.
You could also have html talk to python through cgi (Which has been the way of doing it for years before php came along).
The standard Design pattern commonly seen in web apps is Model-View-Controller, so google around.
I've used Django (heavy and something of a non-trivial initial learning curve), and web.py (very small & simple), and over the past year, I've been pretty happy using tornado.
People can argue about the async details, but that's not even why I use it. It's pretty fast, it's super quick to get started with, it comes with examples that get you doing things quickly, and -- most important to me -- It's a web server and framework in one, which means you don't have to muck with mod_python vs mod_wsgi vs cgi vs fcgi vs whatever. Deployment is really easy.
I'm lucky enough to have full control over the architecture of my company's app, and I've decided to scrap our prototype written in Ruby/Rails and start afresh in Python. This is for a few reasons: I want to learn Python, I prefer the syntax and I've basically said "F**k it, let's do it."
So, baring in mind this is going to be a pretty intensive app, I'd like to hear your opinions on the following:
Generic web frameworks
ORM/Database Layer (perhaps to work with MongoDB)
RESTful API w/ oAuth/xAuth authentication
Testing/BDD support
Message queue (I'd like to keep this in Python if possible)
The API is going to need to interface with a Clojure app to handle some internal data stuff, and interface with the message queue, so if it's not Python it'd be great to have some libraries to it.
TDD/BDD is very important to me, so the more tested, the better!
It'll be really interesting to read your thoughts on this. Much appreciated.
My best,
Jamie
Frameworks
OK, so I'm a little biased here as I currently make extensive use of Django and organise the Django User Group in London so bear that in mind when reading the following.
Start with Django because it's a great gateway drug. Lots of documentation and literature, a very active community of people to talk to and lots of example code around the web.
That's a completely non-technical reason. Pylons is probably purer in terms of Python philosophy (being much more a collection of discrete bits and pieces) but lots of the technical stuff is personal preference, at least until you get into Python more. Compare the very active Django tag on Stack Overflow with that of pylons or turbogears though and I'd argue getting started is simply easier with Django irrespective of anything to do with code.
Personally I default to Django, but find that an increasing amount of time I actually opt for writing using simpler micro frameworks (think Sinatra rather than Rails). Lots of things to choose from (good list here, http://fewagainstmany.com/blog/python-micro-frameworks-are-all-the-rage). I tend to use MNML (because I wrote parts of it and it's tiny) but others are actively developed. I tend to do this for small, stupid web services which are then strung together with a Django project in the middle serving people.
Worth noting here is appengine. You have to work within it's limitations and it's not designed for everything but it's a great way to just play with Python and get something up and working quickly. It makes a great testbed for learning and experimentation.
Mongo/ORM
On the MongoDB front you'll likely want to look at the basic python mongo library ( http://api.mongodb.org/python/ ) first to see if it has everything you need. If you really do want something a little more ORM like then mongoengine (http://hmarr.com/mongoengine/) might be what you're looking for. A bunch of folks are also working on making Django specifically integrate more seamlessly with nosql backends. Some of that is for future Django releases, but django-norel ( http://www.allbuttonspressed.com/projects/django-nonrel) has code now.
For relational data SQLAlchemy ( http://www.sqlalchemy.org/) is good if you want something standalone. Django's ORM is also excellent if you're using Django.
API
The most official Oauth library is python-oauth2 ( http://github.com/simplegeo/python-oauth2), which handily has a Django example as part of it's docs.
Piston ( http://bitbucket.org/jespern/django-piston/wiki/Home) is a Django app which provides lots of tools for building APIs. It has the advantage of being pretty active and well maintained and in production all over the place. Other projects exist too, including Dagny ( http://zacharyvoase.github.com/dagny/) which is an early attempt to create something akin to RESTful resources in Rails.
In reality any Python framework (or even just raw WSGI code) should be reasonably good for this sort of task.
Testing
Python has unittest as part of it's standard library, and unittest2 is in python 2.7 (but backported to previous versions too http://pypi.python.org/pypi/unittest2/0.1.4). Some people also like Nose ( http://code.google.com/p/python-nose/), which is an alternative test runner with some additional features. Twill ( http://twill.idyll.org/) is also nice, it's a "a simple scripting language for Web browsing", so handy for some functional testing. Freshen ( http://github.com/rlisagor/freshen) is a port of cucumber to Python. I haven't yet gotten round to using this in anger, but a quick look now suggests it's much better than when I last looked.
I actually also use Ruby for high level testing of Python apps and apis because I love the combination of celerity and cucumber. But I'm weird and get funny looks from other Python people for this.
Message Queues
For a message queue, whatever language I'm using, I now always use RabbitMQ. I've had some success with stompserver in the past but Rabbit is awesome. Don't worry that it's not itself written in Python, neither is PostgresSQL, Nginx or MongoDB - all for good reason. What you care about are the libraries available. What you're looking for here is py-amqplib ( http://barryp.org/software/py-amqplib/) which is a low level library for talking amqp (the protocol for talking to rabbit as well as other message queues). I've also used Carrot ( http://github.com/ask/carrot/), which is easier to get started with and provides a nicer API. Think bunny in Ruby if you're familiar with that.
Environment
Whatever bits and pieces you decide to use from the Python ecosystem I'd recommend getting to who pip and virtualenv ( http://clemesha.org/blog/2009/jul/05/modern-python-hacker-tools-virtualenv-fabric-pip/ - note that fabric is also cool, but not essential and these docs are out of date on that tool). Think about using Ruby without gem, bundler or rvm and you'll be in the right direction.
Ok, you might be making a mistake, the same one I made when I started with python.
Before you decide on a thing like django, which is an excellent, yet atypical python web framework, spend an night cuddled up with:
This, is a good start. Make sure you do A little Werkzeug watching , Then check out
some classic WebOb. Maybe, if you feel the fire in the blood, and you might, wsgi is a bit flawed, but only to the gods, check out Flask
I'm not saying use it, Django is beautiful too, but if you don't know python, and you go through django, you run the risk of learning a framework.
WSGI is super straightforward. You'll find out about Paste, and Pastescript, and Pylons.
Then, make your decision. It'll be much easier learning stuff doing bare bones wsgi or Flask, stuff like variable assignment, using the interpreter, style concerns, testing, on 3 files for a couple of nights, instead of django. Take 2 nights. Then you'll see the great similarity between python web frameworks, instead of the differences. Hell, you might even roll with Flask.
Just some advice, I did the same thing with ruby, going in through Rails, and... well, strong words were said.
Language, then basic wsgi and testing, then pick your framework and roll
I'm new to python myself, and plan to get more in depth with it this year. I've had a few false starts at this, but always professional needs bring me back to PHP. The few times I've done some development, I've had really good experiences with web2py as a python framework. It's quite well done, and complete in features, while still being extremely lightweight. The database layer seems to be very flexible and mature.
As for TDD/BDD and the rest of your questions, I don't have any experience with python options, but would be interested to hear what others say.
I am using Twisted Framework based Nevow library for python based web app.
All your criteria fit into this single framework.