Develop Reference

Standardizing a Release/Tools group on a specific language

I'm part of a six-member build and release team for an embedded software company. We also support a lot of developer tools, such as Atlassian's Fisheye, Jira, etc., Perforce, Bugzilla, AnthillPro, and a couple of homebrew tools (like my Django release notes generator).
Most of the time, our team just writes little plugins for larger apps (ex: customize workflows in Anthill), long-term utility scripts (package up a release for QA), or things like Perforce triggers (don't let people check into a specific branch unless their change description includes a bug number; authenticate against Active Directory instead of Perforce's internal passwords). That's about the scale of our problems, although we sometimes tackle something slightly more sizable.
My boss, who is reasonably technical, has asked us to standardize on one or two languages so we can more easily substitute for each other. He's advocating bash scripts and Perl, due to their universality and simplicity. I can see his point--we mostly do "glue", so why not use "glue" languages rather than saddle ourselves with something designed for much larger projects? Since some of the tools we work with are Java-based, we do need to use something that speaks JVM sometimes. (The path of least resistance for these projects is BeanShell and Groovy.) I feel a tremendous itch toward language advocacy, but I'm trying to avoid saying "We should use Python 'cause I like it and Perl is gross."
Instead, I'm trying to come up with a good approach to defining our problem set: what problems do we solve with scripts? Would we benefit from a library of common functions by our team, or are most of our projects more isolated? What is it reasonable to expect my co-workers to learn? What languages give us the most ease of development and ease of modification?
Can you folks suggest some useful ways to approach this problem, both for my own thinking process and to help me facilitate some brainstorming among my coworkers?

Two points:
"Eww Perl gross" is somewhat of an urban legend. You can write great clean self-documenting code in Perl, and your can write write-only code in pretty much any language. It's a property of a developer, not a language.
Just because you're writing glue code, doesn't mean the code has to suck like some glue hacks tend to be.
From many threads comparing Perl vs Python on SO, it appears to me that Perl's CPAN is more expansive than Python's repository, but I have no experience with Python and can't substantiate with real comparison.
BUT, one thing I do know. After 5 seconds search, CPAN has a JIRA module. Whether that's a good factor for you or not, that's up to you.

Google standardized on Python for such tasks (and many more) a bit before I joined the company; as far as I know, huge advantages such as Python's great implementations on the JVM and .NET didn't even play a role in the decision -- it was all about readability. At the time (and to some extent, even now) the theory at Google was that every engineer must be able, at need, to tweak every part of the codebase -- including of course build scripts, spiders (which were in Python at the time), and so forth. Demanding of engineers already proficient in C++ and Java to learn many more "scripting" languages (Python, Perl, Bash, Awk, Sed, and so forth) was simply unconsciounable: one had to be selected. Given that constraint, Python was the clear choice (under other constraints, Perl might also have been -- but I can't see the inevitable mix of Bash, Awk and Sed ever competing on such grounds!_) -- and that's how I ended up working there, a bit later;-).
Given that the overall potential of Python vs Ruby vs Perl vs PHP vs Bash + Awk + Sed vs ... is roughly equal, picking one is clearly a winner -- and Python has clean readability, strong implementations on JVM and .NET as big vigorishes. Seriously, I can only think of Javascript (inevitable for client-side work, now rich with strong implementations such as V8) as a possible "competitor" (unfortunately, JS inevitably carries on a lot of baggage for backwards compatibility -- unless you can use a use strict;-like constraint to help on that, it must be an important downside).

I don't think anyone is going to be able to solve your problem on Stackoverflow. Your choice of tools, methods, and process are much more affected by social constraints, e.g. what your boss wants and what you want, then technical merits. That's not necessarily bad.
The short answer is "Use what is going to be most pleasing to the developers". If everyone likes Python more than Perl, for whatever reason, they are probably going to get more done in Python. If they like Ruby more than Python, it's the same thing.
Some things to evaluate as part of your selection:
What do the developers already know?
What are they most willing to learn?
How much weekly time can your team spend learning new things (e.g. lunch seminars, formal classes, etc)?
What do most people in the community use to work with the tool you need to support? For instance, Fisheye has a Java API, and some REST examples for Perl and Python. If you're writing Fisheye extensions, Java seems to be the win there. If you're merely accessing Fisheye data, any language can use the REST stuff.
What is most of your code base in already? What can you replace and what do you have to continue to support? I find that many companies can't answer this question because every developer seems to add two new technologies they don't tell anyone about. :)
Which platforms do you need to support? Some languages have platform specific issues, and I don't mean just Windows vs. Unix. Do you have legacy hardware you have to support? Does your tool work on that stuff?
How much of the stuff you produce can benefit other parts of the company? What are other teams using?
Do the people advocating one tool know it well enough to be its champion? I ask What are five things he hate about your favorite language? If people can't name five valid things that are wrong with their language or tool, they don't have enough experience with it.
The Longer Answer
People tend to try to reduce this to a technical argument because they are afraid to admit their biases or examine why they think what they think. Your boss might favor bash and Perl because that's what he did a lot of work in when he was getting started. You might like Python because you have a personal affinity for the way Python does things. I like Perl because I like its flexibility and DWIMmery. Like any social situation, different people are going to be attracted to different parts of different things. Just because you like chocolate doesn't make vanilla evil. I could give you lots of good arguments why Perl can be useful, but that doesn't mean that something else can't give you the same value.
What problems do we solve with scripts?
That one you have to answer for yourself. :)
Would we benefit from a library of common functions by our team, or are most of our projects more isolated?
This is most likely a good thing in Python, Perl, Ruby, Java, and almost any other language that you might choose. I think this part of your requirement is language agnostic. No matter which one you choose, you'll probably want to do this.
What is it reasonable to expect my co-workers to learn?
A good developer should be able to work with several different languages at least to an apprentice level. Those languages should include ones that have vastly different assumptions about how people express problems, say, for example, the set { Smalltalk Perl C Lisp Java }.
The best developers I've hired and worked with have always wanted to use the right tool for the job instead of making the job right for the tool. They might have their favorite language, but they didn't grouse (too much) about using a different tool when it made more sense.
Many "developers", however, seem to think that they are getting paid to play with their favorite tool. You need to convert them into people who think they have a toolbox to solve problems that create business value.
And remember, you never stop learning. As a developer you don't have to choose one language then defend it with your life, forsaking all others, in sickness and in health, and so on. Good developers are going to continue to track new technologies and evaluate them for usefulness for their tasks. Just because you choose one tool over another doesn't mean you stop paying attention.
No matter what you choose, someone is going to complain. Don't look for the solution that makes everyone happy. There isn't one, short of getting rid of the developers who aren't happy.
What languages give us the most ease of development and ease of modification?
A skilled practitioner in just about any language will think that his chosen language is the easiest to develop, modify, and maintain. Unskilled practitioners tend to blame the language and the tools for their problems. Some languages have steeper learning curves, and some have bigger payouts. A person's tolerance with immediate gratification is a big factor here.
That being said, different languages have developed different cultures and different toolsets. Perl people tend to like vi or emacs, Ruby people tend to like TextMate, Java people tend to like Eclipse or IntelliJ. That's not always true, but the culture that evolves around the tools are often more important than the technical details of the tool. If your developers like a particular type of tool, they are probably going to like the language that has a culture built around that sort of tool.
Some processes and tools take more time to get used to or require more education, but they can have larger advantages when used properly. Other tools get you started sooner but might not give you a path to bigger and greater things, such as cross-team development. The trick, however, is to not code to the tools so you aren't stuck in any particular toolset.

First, it's important to note that it is very hard to convince someone they're wrong.
He's advocating bash scripts and Perl,
due to their universality and
simplicity
Bash scripts are not simple. The bash programming model is really complex and unfriendly. if statements and expressions, in particular are horrifying.
Perl may or may not be simple.
Bash is universal. Perl, however, is exactly as universal as Python. Python is pre-installed in almost all Linux distributions. So that argument is specious.
The "universality" of bash, Perl and python is exactly the same. The "simplicity", however, is not the same. You won't find it easy to to "prove" or "convince" anyone of this once they've already pronounced Perl as simple.
The Situation.
If the boss is advocating Perl, and Perl is not the answer, you will find it is very hard to convince someone they're wrong, making this effort nearly impossible.
If the boss was just throwing out ideas, then this is just difficult.
Quick Hack.
An easy thing you can do is to attempt head-to-head comparisons of Python and Perl for some randomly-chosen jobs. You can then have a code walkthrough to demonstrate the relative opacity of Perl compared with the relative clarity of Python.
Even this is fraught with terrible dangers.
Some folks really think code golf is important. Python loses at code golf. Perl wins. There's nothing worse than "Angry Co-worker with Perl Bias" who will kill you with code-golf solutions that -- because they're smaller -- can baffle management into thinking that they're clearer or "better" on some arbitrary scale.
Some folks really think explicit is "wordy" and bad. Python often loses because the assumptions are stated as actual parameter values. Some folks can (and do) complain at having to actually write things down. Read Stack Overflow for all of the Python questions where someone wants to make the try: block go away in a puff of assumptions.
If you choose random problems, you may -- accidentally -- chose something for which there's an existing piece of Perl or Python that can be downloaded and installed. A language can win just through an accident of the draw. Rather than a more in-depth comparison of language features.
Best Bet
The best you can do is the following.
Identify what folks value. You can call these "non-functional" requirements. These are quality factors. What are the foundational, core principles? Open, Accessible, Transferrable Skills, Simplicity, Cleanliness, Honesty, Integrity, Thriftiness, Reverence, Patience, Hard Work, A Sense of Perspective, Reef the Main in Winds over 20 kn, etc. This is hard. No sympathy here.
Identify the technical use cases. These are "functional" requirements. Which bits of glue and integration there are? This is hard, also. Requirements erupt of out of the woodwork when you do this. Also, when you have a Perl bigot on the team, numerous non-functional requirements will pile into this area. Your manager -- who proposed Perl -- may be the Perl bigot, and the use cases may be difficult to collect in the presence of a Perl bigot.
Identify how (a) Perl + Bash vs. (b) Python vs. (c) Java fit this core values and the functional requirements. Note that using Python means you do not need to use Bash as much. My preference, BTW, is to pare Bash down to the rock-bottom minimum.
This is a big, difficult job. It's hard to short-cut. If you find that Perl is not the answer and the Perl bigot you need to convince is the manager who proposed Perl in the first place, you may find that convincing someone that they're wrong is very hard.
Edit. I am aware that I am forbidden from using the string "Perl Bigot" to describe the manager's potential level of bias toward Perl. I, however, insist on using "Perl Bigot" to describe the manager who proposed Perl. The question provides no information on which to change this. The worst case is that (a) the manager is the Perl Bigot and (b) Perl is not the answer.

Python progression path - From apprentice to guru

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I've been learning, working, and playing with Python for a year and a half now. As a biologist slowly making the turn to bio-informatics, this language has been at the very core of all the major contributions I have made in the lab. I more or less fell in love with the way Python permits me to express beautiful solutions and also with the semantics of the language that allows such a natural flow from thoughts to workable code.
What I would like to know is your answer to a kind of question I have seldom seen in this or other forums. This question seems central to me for anyone on the path to Python improvement but who wonders what his next steps should be.
Let me sum up what I do NOT want to ask first ;)
I don't want to know how to QUICKLY learn Python
Nor do I want to find out the best way to get acquainted with the language
Finally, I don't want to know a 'one trick that does it all' approach.
What I do want to know your opinion about, is:
What are the steps YOU would recommend to a Python journeyman, from apprenticeship to guru status (feel free to stop wherever your expertise dictates it), in order that one IMPROVES CONSTANTLY, becoming a better and better Python coder, one step at a time. Some of the people on SO almost seem worthy of worship for their Python prowess, please enlighten us :)
The kind of answers I would enjoy (but feel free to surprise the readership :P ), is formatted more or less like this:
Read this (eg: python tutorial), pay attention to that kind of details
Code for so manytime/problems/lines of code
Then, read this (eg: this or that book), but this time, pay attention to this
Tackle a few real-life problems
Then, proceed to reading Y.
Be sure to grasp these concepts
Code for X time
Come back to such and such basics or move further to...
(you get the point :)
I really care about knowing your opinion on what exactly one should pay attention to, at various stages, in order to progress CONSTANTLY (with due efforts, of course). If you come from a specific field of expertise, discuss the path you see as appropriate in this field.
EDIT: Thanks to your great input, I'm back on the Python improvement track! I really appreciate!

I thought the process of Python mastery went something like:
Discover list comprehensions
Discover generators
Incorporate map, reduce, filter, iter, range, xrange often into your code
Discover Decorators
Write recursive functions, a lot
Discover itertools and functools
Read Real World Haskell (read free online)
Rewrite all your old Python code with tons of higher order functions, recursion, and whatnot.
Annoy your cubicle mates every time they present you with a Python class. Claim it could be "better" implemented as a dictionary plus some functions. Embrace functional programming.
Rediscover the Strategy pattern and then all those things from imperative code you tried so hard to forget after Haskell.
Find a balance.

One good way to further your Python knowledge is to dig into the source code of the libraries, platforms, and frameworks you use already.
For example if you're building a site on Django, many questions that might stump you can be answered by looking at how Django implements the feature in question.
This way you'll continue to pick up new idioms, coding styles, and Python tricks. (Some will be good and some will be bad.)
And when you see something Pythony that you don't understand in the source, hop over to the #python IRC channel and you'll find plenty of "language lawyers" happy to explain.
An accumulation of these little clarifications over years leads to a much deeper understanding of the language and all of its ins and outs.

Understand (more deeply) Python's data types and their roles with regards to memory mgmt
As some of you in the community are aware, I teach Python courses, the most popular ones being the comprehensive Intro+Intermediate course as well as an "advanced" course which introduces a variety of areas of application development.
Quite often, I get asked a question quite similar to, "Should I take your intro or advanced course? I've already been programming Python for 1-2 years, and I think the intro one is too simple for me so I'd like to jump straight to the advanced... which course would you recommend?"
To answer their question, I probe to see how strong they are in this area -- not that it's really the best way to measure whether they're ready for any advanced course, but to see how well their basic knowledge is of Python's objects and memory model, which is a cause of many Python bugs written by those who are not only beginners but those who have gone beyond that.
To do this, I point them at this simple 2-part quiz question:
Many times, they are able to get the output, but the why is more difficult and much more important of an response... I would weigh the output as 20% of the answer while the "why" gets 80% credit. If they can't get the why, regardless how Python experience they have, I will always steer people to the comprehensive intro+intermediate course because I spend one lecture on objects and memory management to the point where you should be able to answer with the output and the why with sufficient confidence. (Just because you know Python's syntax after 1-2 years doesn't make you ready to move beyond a "beginner" label until you have a much better understanding as far as how Python works under the covers.)
A succeeding inquiry requiring a similar answer is even tougher, e.g.,
Example 3
x = ['foo', [1,2,3], 10.4]
y = list(x) # or x[:]
y[0] = 'fooooooo'
y[1][0] = 4
print x
print y
The next topics I recommend are to understanding reference counting well, learning what "interning" means (but not necessarily using it), learning about shallow and deep copies (as in Example 3 above), and finally, the interrelationships between the various types and constructs in the language, i.e. lists vs. tuples, dicts vs. sets, list comprehensions vs. generator expressions, iterators vs. generators, etc.; however all those other suggestions are another post for another time. Hope this helps in the meantime! :-)
ps. I agree with the other responses for getting more intimate with introspection as well as studying other projects' source code and add a strong "+1" to both suggestions!
pps. Great question BTW. I wish I was smart enough in the beginning to have asked something like this, but that was a long time ago, and now I'm trying to help others with my many years of full-time Python programming!!

Check out Peter Norvig's essay on becoming a master programmer in 10 years: http://norvig.com/21-days.html. I'd wager it holds true for any language.

Understand Introspection
write a dir() equivalent
write a type() equivalent
figure out how to "monkey-patch"
use the dis module to see how various language constructs work
Doing these things will
give you some good theoretical knowledge about how python is implemented
give you some good practical experience in lower-level programming
give you a good intuitive feel for python data structures

def apprentice():
read(diveintopython)
experiment(interpreter)
read(python_tutorial)
experiment(interpreter, modules/files)
watch(pycon)
def master():
refer(python-essential-reference)
refer(PEPs/language reference)
experiment()
read(good_python_code) # Eg. twisted, other libraries
write(basic_library) # reinvent wheel and compare to existing wheels
if have_interesting_ideas:
give_talk(pycon)
def guru():
pass # Not qualified to comment. Fix the GIL perhaps?

I'll give you the simplest and most effective piece of advice I think anybody could give you: code.
You can only be better at using a language (which implies understanding it) by coding. You have to actively enjoy coding, be inspired, ask questions, and find answers by yourself.
Got a an hour to spare? Write code that will reverse a string, and find out the most optimum solution. A free evening? Why not try some web-scraping. Read other peoples code. See how they do things. Ask yourself what you would do.
When I'm bored at my computer, I open my IDE and code-storm. I jot down ideas that sound interesting, and challenging. An URL shortener? Sure, I can do that. Oh, I learnt how to convert numbers from one base to another as a side effect!
This is valid whatever your skill level. You never stop learning. By actively coding in your spare time you will, with little additional effort, come to understand the language, and ultimately, become a guru. You will build up knowledge and reusable code and memorise idioms.

If you're in and using python for science (which it seems you are) part of that will be learning and understanding scientific libraries, for me these would be
numpy
scipy
matplotlib
mayavi/mlab
chaco
Cython
knowing how to use the right libraries and vectorize your code is essential for scientific computing.
I wanted to add that, handling large numeric datasets in common pythonic ways(object oriented approaches, lists, iterators) can be extremely inefficient. In scientific computing, it can be necessary to structure your code in ways that differ drastically from how most conventional python coders approach data.

Google just recently released an online Python class ("class" as in "a course of study").
http://code.google.com/edu/languages/google-python-class/
I know this doesn't answer your full question, but I think it's a great place to start!

Download Twisted and look at the source code. They employ some pretty advanced techniques.

Thoroughly Understand All Data Types and Structures
For every type and structure, write a series of demo programs that exercise every aspect of the type or data structure. If you do this, it might be worthwhile to blog notes on each one... it might be useful to lots of people!

I learned python first by myself over a summer just by doing the tutorial on the python site (sadly, I don't seem to be able to find that anymore, so I can't post a link).
Later, python was taught to me in one of my first year courses at university. In the summer that followed, I practiced with PythonChallenge and with problems from Google Code Jam.
Solving these problems help from an algorithmic perspective as well as from the perspective of learning what Python can do as well as how to manipulate it to get the fullest out of python.
For similar reasons, I have heard that code golf works as well, but i have never tried it for myself.

Learning algorithms/maths/file IO/Pythonic optimisation
This won't get you guru-hood but to start out, try working through the Project Euler problems
The first 50 or so shouldn't tax you if you have decent high-school mathematics and know how to Google. When you solve one you get into the forum where you can look through other people's solutions which will teach you even more. Be decent though and don't post up your solutions as the idea is to encourage people to work it out for themselves.
Forcing yourself to work in Python will be unforgiving if you use brute-force algorithms.
This will teach you how to lay out large datasets in memory and access them efficiently with the fast language features such as dictionaries.
From doing this myself I learnt:
File IO
Algorithms and techniques such as Dynamic Programming
Python data layout
Dictionaries/hashmaps
Lists
Tuples
Various combinations thereof, e.g. dictionaries to lists of tuples
Generators
Recursive functions
Developing Python libraries
Filesystem layout
Reloading them during an interpreter session
And also very importantly
When to give up and use C or C++!
All of this should be relevant to Bioinformatics
Admittedly I didn't learn about the OOP features of Python from that experience.

Have you seen the book "Bioinformatics Programming using Python"? Looks like you're an exact member of its focus group.

You already have a lot of reading material, but if you can handle more, I recommend you
learn about the evolution of python by reading the Python Enhancement Proposals, especially the "Finished" PEPs and the "Deferred, Abandoned, Withdrawn, and Rejected" PEPs.
By seeing how the language has changed, the decisions that were made and their rationales, you will absorb the philosophy of Python and understand how "idiomatic Python" comes about.
http://www.python.org/dev/peps/

Attempt http://challenge.greplin.com/ using Python

Teaching to someone else who is starting to learn Python is always a great way to get your ideas clear and sometimes, I usually get a lot of neat questions from students that have me to re-think conceptual things about Python.

Not precisely what you're asking for, but I think it's good advice.
Learn another language, doesn't matter too much which. Each language has it's own ideas and conventions that you can learn from. Learn about the differences in the languages and more importantly why they're different. Try a purely functional language like Haskell and see some of the benefits (and challenges) of functions free of side-effects. See how you can apply some of the things you learn from other languages to Python.

I recommend starting with something that forces you to explore the expressive power of the syntax. Python allows many different ways of writing the same functionality, but there is often a single most elegant and fastest approach. If you're used to the idioms of other languages, you might never otherwise find or accept these better ways. I spent a weekend trudging through the first 20 or so Project Euler problems and made a simple webapp with Django on Google App Engine. This will only take you from apprentice to novice, maybe, but you can then continue to making somewhat more advanced webapps and solve more advanced Project Euler problems. After a few months I went back and solved the first 20 PE problems from scratch in an hour instead of a weekend.

General questions regarding Python language

I'm a newbie to programming and I've decided to start with Python. Just curious though, is it enough/recommended to learn Python from online tutorials or from books? I want to go further than simple "Hello World!" programs. I'm not sure if books will actually teach you how to make more advanced programs.
One example is Exif-py. How do you even start programming a program like this? Do you just sit down and start writing the code, or do you have to search for APIs or anything? I went through the code briefly and it all looked alien to me.
I'm not really sure how to express my questions into words, so do check back because I'll edit my question if the right words come to me.

Well, I learnt all my Python from online sources (not just tutorials, but reference documentation, blog posts and other texts). It's certainly possible, although some people prefer the "guided" way a book teaches you, particularly people new to programming (at that point I had already been programming for years).
To create a program such as Exif.py, you would first have to know what you must do in broad terms BEFORE starting to program. You would study the EXIF format, then figure out how that is put into the image files, then you would have to formulate that in terms of the language you're using (in this case, Python). This usually requires that you're already familiar with it, otherwise it'll be a fairly slow process.
I'd suggest starting with simpler programs to begin with, or maybe follow a book such as Dive Into Python (free online), seeing as you're new to programming, and need to not only learn the language, but to think like a programmer.

The tutorials are there to teach you the language syntax and the standard library, not really on how to solve a particular programming problem. Think of what you learn there as your toolbox.
I think I understand what you are saying. You want to break above and beyond the simple applications and write your own stuff, right? Well, first you need to figure out what it is you want to make. Then comes the hard part; how are you going to make it? I'd suggest starting by trying to break it down into a number of small simple problems instead of tackling it as one large problem.
If you are trying to find examples of larger projects, I'd suggest looking at the Python Cheeseshop (package index) and download a few packages you've heard of to see how they did it. Also, people often post handy pieces of code on their personal blogs and that shows up on PlanetPython.

If stuff like recursion, modules, classes, iteration, exceptions, dictionaries, are indeed new to you, I suggest How to Think Like a Computer Scientist in Python (lovingly abbreviated as ThinkCSPy in the community ;-).
It is a bit outdated - for best results, use Python 2.6 or lower, but then again 3.0 is not that different.
EDIT: If ThinkCSPy has little to teach you, try Dive Into Python. It's advanced and pretty "real-world", but step-by-step.

If by 'newbie to programming' you mean that you just started it last week or something along the lines, then maybe you might want to give the MIT OpenCourseWare Introduction to Computer Science videos (and homework!) a bit of your time.
The opencourse covers the syntax of the Python language, some helpful hints and general do's and don'ts that apply to any programming paradigm.
Though, if by 'newbie to programming', that you already know the basic concepts you need to start writing basic programs (like adding, subtracting, multiplying, logical operations, functions etc).
If so, Project Euler is particularly good for exercising that cranial muscle in problem solving via programming, though most of the problems on the site require you at also have knowledge of mathematics (and if you don't, at least know how to google).
Just remember, if you get stuck on anything don't get put down! It's all part of learning! If you're really stuck, search SO!

There are a few very good online resources:
The Python Tutorial.
Dive Into Python. Python from novice to pro.
Code Like a Pythonista: Idiomatic Python
Another list of resources: Essential Python Reading List.
Most Python books I've seen so far are "not that good" - but that depends
on your background / prior knowledge about Python and programming.

You study the EXIF format, study the GIF/JPEG format, open the binary file, scan it and get the data out.

Dilemma: Should I learn Seaside or a Python framework?

I know it's kinda subjective but, if you were to put yourself in my shoes which would you invest the time in learning?
I want to write a web app which deals securely with relatively modest amounts of peoples private data, a few thousand records of a few Kb each but stuff that needs to be kept safe, addresses, phone numbers etc. I've done several web projects in PHP/MYSQL and have decided, handy though it is I really don't like PHP and don't want to do another large project in it...
As such I figure I'd best learn something new and so I am considering 2 options (although I'll happily entertain others if you have suggestions). I'm having terrible trouble deciding though. They both look quite involved so rather than just jump in and potentially waste days getting up to speed enough on both of them to make an informed choice I thought I'd come here and canvas some opinion.
So the two options I'm considering are...
One of the PYTHON Web frameworks - TurboGears seems well regarded?
Advantage: Of all the languages I ever tried Python is by far and away my favorite. There's loads of frameworks to choose from and I have done quite a lot of non web python coding over the last few years.
Disadvantage: There's loads to choose from so it's hard to pick! Need to run single server process? or mod_python? which I don't like the sound of. What I do like is the notion of process separation and compartmentalization, i.e. if one users account is compromised it gives an attacker no leverage against the rest of the system. I'm not clear to what extent a python solution would handle that.
Writing it as a SEASIDE app Which I guess runs on a squeak app server?
Adv: From what I've heard it would permit good compartmentalization of users as each would have their own little private VM independent of all the systems other users which sounds wonderful from a security, scaling and redundancy standpoint.
Dis: I've not done any Smalltalk since Uni 15 years back and I never dug too deep into it then. I don't see much entry level help for seaside or that many projects using it. I suspect setting a server up to run it is hard for the same reason i.e. not because it's inherently hard but just cause there will be less help online and a presumption you are already rather au fait with Sqeak/Smalltalk.
So, what do people think? Would I be able to efficiently get the kind of strong separation and compartmentalization I'm after with a Python framework? Is Seaside as good as I think in terms of insulating users from each other? Might I be better off, security wise, sticking to the languages I'm most familiar with so I don't make any n00b mistakes or will Seaside be worth worth scaling the learning curve and prove more secure, comprehensible and maintainable in the long run? At the end of the day it's not a life or death decision and I can always bail if I start with one and then hate it so pls nobody get all holy language war and start flaming anyone! ;-)
Cheers for any replies this gets,
Roger :)

Disclaimer: I really don't like PHP, Python is nice, but doesn't come close to Smalltalk in my book. But I am a biased Smalltalker. Some answers about Seaside/Squeak:
Q: Which I guess runs on a squeak app server?
Seaside runs in several different Smalltalks (VW, Gemstone, Squeak etc). The term "app server" is not really used in Smalltalk country. :)
Q: From what I've heard it would permit good compartmentalization of users as each would have their own little private VM independent of all the systems other users which sounds wonderful from a security, scaling and redundancy standpoint.
Yes, each user has its own WASession and all UI components the user sees are instances living on the server side in that session. So sharing of state between sessions is something you must do explicitly, typically through a db.
Q: I've not done any Smalltalk since Uni 15 years back and I never dug too deep into it then. I don't see much entry level help for seaside or that many projects using it.
Smalltalk is easy to get going with and there is a whole free online book on Seaside.
Q: I suspect setting a server up to run it is hard for the same reason i.e. not because it's inherently hard but just cause there will be less help online and a presumption you are already rather au fait with Sqeak/Smalltalk.
No, not hard. :) In fact, quite trivial. Tons of help - Seaside ml, IRC on freenode, etc.
Q: Is Seaside as good as I think in terms of insulating users from each other?
I would say so.
Q: Might I be better off, security wise, sticking to the languages I'm most familiar with so I don't make any n00b mistakes or will Seaside be worth worth scaling the learning curve and prove more secure, comprehensible and maintainable in the long run?
The killer argument in favor of Seaside IMHO is the true component model. It really, really makes it wonderful for complex UIs and maintenance. If you are afraid of learning "something different" (but then you wouldn't even consider it in the first place I guess) then I would warn you. But if you are not afraid then you will probably love it.
Also - Squeak (or VW) is a truly awesome development environment - debugging live Seaside sessions, changing code in the debugger and resuming etc etc. It rocks.

Forget about mod_python, there is WSGI.
I'd recommend Django. It runs on any WSGI server, there are a lot to choose from. There is mod_wsgi for Apache, wsgiref - reference implementation included in Python and many more. Also Google App Engine is WSGI, and includes Django.
Django is very popular and it's community is rapidly growing.

I'd say take a look at Django. It's a Python framework with a ready-made authentication system that's independent of the hosting OS, which means that compromises are limited to the app that was compromised (barring some exploit against the web server hosting the Python process).

I've been getting into seaside myself but in many ways it is very hard to get started, which has nothing to do with the smalltalk which can be picked up extremely quickly. The challenge is that you are really protected from writing html directly.
I find in most frameworks when you get stuck on how to do something there is always a work around of solving it by using the template. You may later discover that this solution causes problems with clarity down the road and there is in fact a better solutions built into the framework but you were able to move on from that problem until you learned the right way to do it.
Seaside doesn't have templates so you don't get that crutch. No problems have permanently stumped me but some have taken me longer to solve than I would have liked. The flip side of this is you end up learning the seaside methodology much quicker because you can't cheat.
If you decide to go the seaside route don't be afraid to post to the seaside mailing list at squeakfoundation.org. I found it intimidating at first because you don't see a lot of beginner questions there due to the low traffic but people are willing to help beginners there.
Also there are a handful of seaside developers who monitor stackoverflow regularly. Good luck.

Have you taken a look at www.nagare.org ?
A framework particularly for web apps rather than web sites.
It is based around the Seaside concepts but you program in Python (nagare deploys a distribution of python called Stackless Python to get the continuations working).
Like Seaside it will auto generate HTML, but additionally can use templates as required.
It has been recently open sourced by http://www.net-ng.com/ who themselves have many years experience in delivering web apps/sites in quality web frameworks like zope and plone.
I am researching it myself at the moment to see if it fits my needs, so can't tell you what I think of it in the wild. If you take a look, please give your feedback.

While considering a Smalltalk web framework, look at Aida/Web as well. Aida has built-in security with user/group/role management and strong access control, which can help you a lot in your case. That way you can achieve safe enough separation of users at the user level in one image. But if you really want, you can separate them with running many images as well. But this brings increased maintenance and I'd think twice if it is worth.

I'm toying with Seaside myself and found this tutorial to be invaluable in gaining insight into the capabilities of the framework.

I think you've pretty much summed up the pros and cons. Seaside isn't that hard to set up (I've installed it twice for various projects) but using it will definitely affect how you work--in addition to re-learning the language you'll probably have to adjust lots of assumptions about your work flow.
It also depends on two other factors
If other people will eventually be maintaining it, you'll have better luck finding python programmers
If you are doing a highly stateful site, Seaside is going to beat the pants off any other framework I've seen.

There is now an online book on Seaside to complete the tutorial pointed out earlier.

With Lua and Python embeddable, is there a place for Basic?

I started off programming in Basic on the ZX81, then BASICA, GW-BASIC, and QBasic. I moved on to C (Ah, Turbo C 3.1, I hardly knew ye...)
When I got started in microcontrollers I regressed with the BASIC Stamp from Parallax. However, BASIC is/was awesome because it was so easy to understand and so hard to make a mistake. I moved on to assembly and C eventually because I needed the additional power (speed, capacity, resources, etc.), but I know that if the bar was much higher many people would never get into programming microcontrollers.
I keep getting an itch to make my own on-chip BASIC interpretor, but I wonder if there's need for BASIC now that Lua and Python are easily embeddable, and just as approachable as BASIC.
What, if any, are the advantages BASIC has over other languages?
Why is it still around?
If I decide to make a general purpose microcontroller board with an interpreter, is there any reason to make a version of BASIC?
Plenty of other languages are considered dead, but BASIC just keeps hanging on.

[This may come off sounding more negative than it really is. I'm not saying Basic is the root of all evil, others have said that. I'm saying it's a legacy we can afford to leave behind.]
"because it was so easy to understand and so hard to make a mistake" That's certainly debatable. I've had some bad experiences with utterly opaque basic. Professional stuff -- commercial products -- perfectly awful code. Had to give up and decline the work.
"What, if any, are the advantages Basic has over other languages?" None, really.
"Why is it still around?" Two reasons: (1) Microsoft, (2) all the IT departments that started doing VB and now have millions of lines of VB legacy code.
"Plenty of other languages are considered dead..." Yep. Basic is there along side COBOL, PL/I and RPG as legacies that sometimes have more cost than value. But because of the "if it ain't broke don't fix it" policy of big IT, there they sit, sucking up resources who could easily replace it with something smaller, simpler and cheaper to maintain. Except it hasn't "failed" -- it's just disproportionately expensive.
30-year old COBOL is a horrible situation to rework. Starting in 2016 we'll be looking at 30-year old MS Basic that we just can't figure out, don't want to live without, and can't decide how to replace.
"but basic just keeps hanging on" It appears that some folks love Basic. Others see it as yet another poorly-designed language; it's advantages are being early to market and being backed by huge vendors (IBM, initially). Poorly-design, early-to-market only leaves us with a legacy that we'll be suffering with for decades.
I still have my 1965-edition Dartmouth Basic manual. I don't long for the good old days.

As an architecture, the main claim to fame of BASIC is that you could make BASIC interpreters very small - just a few KB. In the days of a DG Nova this was a win as you could use systems like Business BASIC to build a multiuser application on a machine with 64K of RAM (or even less).
BASIC (VB in particular) is a legacy system and has a large existing code-base. Arguably VB is really a language (some would say a thin wrapper over COM) that has a BASIC-like syntax. These days, I see little reason to keep the language around apart from people's familiarity with it and to maintain the existing code base. I certainly would not advocate new development in it (note that VB.Net is not really BASIC but just has a VB-like syntax. The type system is not broken in the way that VB's was.)
What is missing from the computing world is a relevant language that is easy to learn and tinker with and has mind-share in mainstream application development. I grew up in the days of 8-bit machines, and the entry barrier to programming on those systems was very low. The architecture of the machines was very simple, and you could learn to program and write more-or-less relevant applications on these machines very easily.
Modern architectures are much more complex and have a bigger hump to learn. You can see people pontificating on how kids can't learn to program as easily as they could back in the days of BASIC and 8-bit computers and I think that argument has some merit. There is something of a hole left that makes programming just that bit harder to get into. Toy languages are not much use here - for programming to be attractive it has to be possible to aspire to build something relevant with the language you are learning.
This leads to the problem of a language that is easy for kids to learn but still allows them to write relevant programmes (or even games) that they might actually want. It also has to be widely perceived as relevant.
The closest thing I can think of to this is Python. It's not the only example of a language of that type, but it is the one with the most mind-share - and (IMO) a perception of relevance is necessary to play in this niche. It's also one of the easiest languages to learn that I've experienced (of the 30 or so that I've used over the years).

Why not give Jumentum a try and see how it works for you?
http://jumentum.sourceforge.net/
it's an open source BASIC for micrcontrollers
The elua project is also lua for microcontrollers
http://elua.berlios.de/

BASIC persists, particularly in the STAMP implementation, because it is lower level than most other very-easy-to-learn programming languages. For most embedded BASIC implementations the BASIC instructions map directly to single or groups of machine instructions, with very little overhead. The same programs written in "higher level" languages like Lua or Python would run far slower on those same microcontrollers.
PS: BASIC variants like PBASIC have very little in common with, say, Visual BASIC, despite the naming similarity. They have diverged in very different ways.

Good question...
Basically (sic!), I have no answer. I would say just that Lua is very easy to learn, probably as easy as Basic (which was one of my first languages as well, I used dialects on lot of 8-bit computers...), but is more powerful (allowing OO or functional styles and even mixing them) and somehow stricter (no goto...).
I don't know well Python, but from what I have read, it is as easy, powerful and strict than Lua.
Beside, both are "standardized" de facto, ie. there are no dialects (beside the various versions), unlike Basic which has many variants.
Also both have carefully crafted VM, efficient, (mostly) bugless. Should you make your own interpretor, you should either take an existing VM and generate bytecode for it from Basic source, or make your own. Sure fun stuff, but time consuming and prone to bugs...
So, I would just let Basic have a nice retirement... :-P
PS.: Why it is hanging on? Perhaps Microsoft isn't foreign to that... (VB, VBA, VBScript...)
There are also lot of dialects around (RealBasic, DarkBasic, etc.), with some audience.

At the risk of sounding like two old-timers on rocking chairs, let me grumpily say that "Kids today don't appreciate BASIC" and then paradoxically say "They don't know how good they've got it."
BASICs greatest strength was always its comprehensibility. It was something that people could get. That was long ignored by academics and language developers.
When you talk about wanting to implement BASIC, I assume you're not talking about line-numbered BASIC, but a structured form. The problem with that is that as soon as you start moving into structured programming -- functions, 'why can't I just GOTO that spot?', etc. -- it really becomes unclear what advantages, if any, BASIC would have over, say, Python.
Additionally, one reason BASIC was "so easy to get right" was that in those days libraries weren't nearly as important as they are today. Libraries imply structured if not object-oriented programming, so again you're in a situation where a more modern dynamic scripting language "fits" the reality of what people do today better.
If the real question is "well, I want to implement an interpreter and so it comes down to return on investment," then it becomes a problem of an grammar that's actually easy to implement. I'd suggest that BASIC doesn't really have that many advantages in that regard either (unless you really do return to line numbers and a very limited grammar).
In short, I don't think you should invest your effort in a BASIC interpreter.

Well, these people seem to think that not only basic still has a place in the mobile space but also that they can make money off it:
http://www.nsbasic.com/symbian/

I started out on a ZX81 too. But as Tony Hoare said, programming in BASIC is like trying to do long division using roman numerals.
Plenty of other languages are
considered dead, but basic just keeps
hanging on.
Sadly yes. I blame Bill Gates for this...BASIC was on a stretcher with a priest saying the last rites for it, and then MS brought it back like Smallpox.

I used to program in BASIC in the QBasic days. QBASIC had subroutines, functions, structures (they used to be called types), and I guess that's it. Now, this seems limited compared to all the features that Python has - OO, lambdas, metaclasses, generators, list comprehensions, just to name a few off the top of my head. But that simplicity, I think, is a strength of BASIC. If you're looking at a simple embeddable language, I'd bet that QBasic will be faster and easier to understand. And a procedural langauge is probably more than sufficient for most embedding/scripting type of applications.
I'd say the most important reason BASIC is still around is Visual Basic. For a long time in the 90s, VB was the only way to write GUIs, COM and DB code for Windows without falling into one of the C++ Turing tarpits. [Maybe Delphi was a good option too, but unfortunately it never became as popular as VB]. I do think it is because of all this VB and VBA code that is still being used and maintained that BASIC still isn't dead.
That said, I'd say there's pretty a good rationale to write BASIC interpreter (maybe even compiler using LLVM or something similar) for BASIC today. You'll get a clean, simple easy to use and fast language if you implement something that resembles QBasic. You won't have to solve any language design issues and the best part is people will already know your language.