Python Language Nuances [duplicate] - python

This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Common Pitfalls in Python
I'm learning Python and I come from a diverse background of programming languages. In the last five years, I've written quite a bit of Java, C++, VB.Net, and PHP. As many of you might agree, once you learn one programming language, learning another is just a mater of learning the differences in syntax and best practices.
Coming down from PHP, I've become very accustomed to a lot of script-style language features. For instance, stuff like this tickles me insides:
# Retrieve the value from the cache; otherwise redownload.
if(!($value = $cache->get($key)))
# Redownload the value and store in the cache.
$cache->set($key, $value = redownload($key));
Python, though, doesn't consider assignment to be an expression. OTOH, it does support nice things like the in construct, which I find to be one of the greatest inventions of all time. x in y is so much nicer than !empty($y[$x]).
What other nuances, "missing" features, and performance bottlenecks should I watch out for? I'm hoping to make as seamless a transition into Python development as possible, and hope to learn some of the secrets that will help to smooth out development time and eliminate trial and error. Your insight is appreciated!

This one took me a few hours to figure out when I first encountered it in a real program:
A default argument to a function is a mutable, static value.
def foo(bar = []):
bar.append(1)
print(bar)
foo()
foo()
This will print
[1]
[1, 1]

For your example, the usual way would be something like this
try:
value = cache[key]
except KeyError:
value = cache[key] = redownload(key)

Threads do not do what you think they do, and probably shouldn't be used how you're used to using them. This is a huge gotcha for many people, especially for those used to Java where the custom is to subclass Thread implement the Runnable interface to do asynchronous work, and where there is language support for running threads in parallel (on machines with multiple CPU cores).
In general you probably don't want threads at all, but subprocesses. See my answer to the question "python threading and performace?".
(In more general, there might be a better way altogether.)

Exceptions are your friend.
In contrast with languages like C and PHP which use the return value to indicate errors, Python uses exceptions to interrupt the program instead of allowing the errors to cause further problems down the line.

Pythonic code is usually much faster than C-like code.
Something like:
new_list=[]
for i in xrange(len(old_list)):
new_list.append(some_function(old_list[i]))
is better written as:
new_list=[some_function(x) for x in old_list]
or
new_list=map(some_function,old_list)

Related

Are there technical reasons a Ruby DSL like RSpec couldn't be rewritten in Python?

The section below goes into more detail, but basically someone stated that the Ruby-written DSL RSpec couldn't be rewritten in Python. Is that true? If so, why?
I'm wanting to better understand the technical differences between Ruby and Python.
Update: Why am I asking this question?
The Running away from RSpec discussion has some statements about it being "impossible" to recreate RSpec in Python. I was trying to make the question a little broader in hopes of learning more of the technical differences between Ruby and Python. In hindsight, maybe I should have tightened the question's scope to just asking if it truly is impossible to recreate RSpec in Python, and if so why.
Below are just a few quotes from the Running away from RSpec discussion.
Initial Question
For the past few weeks I have been thinking a lot about RSpec and why there is no clear, definite answer when someone asks:
"I'm looking for a Python equivalent of RSpec. Where can I find such a
thing?"
Probably the most common (and understandable) answer is that Python syntax
wouldn't allow such a thing whereas in Ruby it is possible.
First Response to Initial Question
Not syntax exactly. Rspec monkeypatches every object inside of its
scope, inserting the methods "should" and "should_not". You can do
something in python, but you can't monkeypatch the built-in types.
Another Response
As you suggest, it's impossible. Mote and PySpec are just fancy ways
to name your tests: weak implementations of one tiny corner of RSpec.
Mote uses horrible settrace magic; PySpec adds a bunch of
domain-irrelevant noise. Neither even supports arbitrary context
strings. RSpec is more terse, more expressive, removes the noise, and
is an entirely reasonable thing to build in Ruby.
That last point is important: it's not just that RSpec is possible in
Ruby; it's actually idiomatic.
If I had to point out one great difficulty for creating a Python RSpec, it would be the lack of a good syntax in Python for creating anonymous functions (as in JavaScript) or blocks (as in Ruby). The only option for a Python programmer is to use lambdas, which is not an option at all because lambdas just accept one expression. The do ... end blocks used in RSpec would have to be written as a function before calling describe and it, as in the example below:
def should_do_stuff():
# ...
it("should do stuff", should_do_stuff)
Not so sexy, right?
There are some difficulties in creating the should methods, but I bet it would be a smaller problem. Actually, one does not even need to use such an unusual syntax—you could get similar results (maybe even better, depending on your taste) using the Jasmine syntax, which can be trivially implemented.
That said, I feel that Python syntax is more focused on efficiently representing the usual program components such as classes, functions, variables, etc. It is not well suited to be extended. I, for one, think that a good Python program is one where I can see objects, and functions, and variables, and I understand what each one of these elements do. Ruby programmers, OTOH, seem to seek for a more prose-like style, where a new language is defined for a new problem. It is a good way of doing things, too, but not a Pythonic way. Python is good to represent algorithms, not prose.
Sometimes it is a draconian limit. How could one use BDD for example? Well, the usual way of pushing these limits in Python is to effectively write your own DSL, but it should REALLY be another language. That is what Pyccuracy is, for example: another language for BDD. A more mainstream example is doctest. (Actually, if I would write some BDD Python library, I would write it based on doctest.) Another example of Python DSL is Twill. And yet another example is reStructuredText, used in Sphinx.
Summarizing: IMHO the hardest barrier to DSLs in Python is the lack of a flexible syntax for creating anonymous functions. And it is not a fault*: Python is not fond of having its syntax heavily explored anyway—it is considered to make code less clear in the Python universe. If you want a new syntax in Python you are well advised to write your own language, or at least it is the way I feel.
* Or maybe it is - I have to confess that I miss anonymous functions. However, I recognize that they would be hard to implement elegantly given the Python semantic indentation.
I set out on an attempt to implement something like rspec in Python.
I got this:
with It('should pass') as test:
test.should_be_equal(1, 1)
source: https://gist.github.com/2029866
(thoughts?)
EDIT: My answer to your question is that the lack of anonymous blocks prevents a Ruby DSL like RSpec from being rewritten in Python but you can get a close approximation using with statements.
One of Ruby's strengths is in the creation of DSLs. However the reasons given for it being difficult in python can be sidestepped. For example you can easily subclass the builtin types, e.g:
>>> class myint(int): pass
>>> i = myint(5)
>>> i
5
If I were going to create a DSL in python I'd use pyparsing or Parsley and something like the above behind the scenes, optimizing the syntax for the problem, not the implementation language.
By mixing Mamba and Expects, I think you can get very close to what RSpec is for Rails...
https://github.com/nestorsalceda/mamba
https://github.com/jaimegildesagredo/expects
Also, I think Specter should match your expectations with testing:
https://github.com/jmvrbanac/Specter
http://specter.readthedocs.io/en/latest/writing_tests/index.html
I think this is what you are looking for. Yes, we made the "impossible" in python
"sure" is an utility belt for expressive python tests, created by Gabriel Falcão

Will using less calls to functions increase speed of total execution? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Does creating separate functions instead of one big one slow processing time?
OOP Bloop boop!
Yes, OOP is great, keeps code clean and well organized. That's why I like to use it. I"m referring to to OOP very primitively specifically in using functions (defs).
Will taking out my function calls and sticking the content of the function straight into my algo increase speed of execution of the overall code?
Yes, I know I can run a test myself, but I'm choosing to ask it here in the forum of my fellow coder colleagues, because I know this is a question that floats around in many heads....
def myFunc(var):
return var*3243 #perhaps more complicated function code will make a difference?
i = 0
hungry = True
while hungry:
i = i + 1
x = myFunc(i)
if i > 50:
hungry = False
Write it correctly (i.e. with concerns properly separated into distinct functions and classes), then use PyPy to make it fast.
PyPy uses function inlining and assorted other tricks in its Just-In-Time compiler to speed up code execution without having to make it unmaintainable in the name of speed.
Note that this solution only works if you're using the Python 2.x series with any C extensions you use being compatible with cpyext.
Apparently there is very high function call overhead in Python.
http://wiki.python.org/moin/PythonSpeed/PerformanceTips#Data_Aggregation
Is it worth keeping in mind while choosing the best way to write a piece of code?
Sure.
Is the speed up worth making your coder harder to understand?
Probably not.
Python won't "inline" the function for you, so of course there is overhead. Usually the code in the function will take enough time that the call overhead is not significant.
Bear in mind that it's much easier to test,debug and profile programs that are broken up into functions.
If you really really need more performance it's usually a better idea to write the function in C or Cython than to eliminate the call overhead
It's also worth noting that the usual code set up like this
def main():
...
if __name__=="main":
main()
is faster than just running code at the top level as identifier lookups are faster in functions/methods than in the global namespace

Why is it not possible to create a practical Perl to Python source code converter?

It would be nice if there existed a program that automatically transforms Perl code to Python code, making the resultant Python program as readable and maintainable as the original one, let alone working the same way.
The most obvious solution would just invoke perl via Python utils:
#!/usr/bin/python
os.exec("tail -n -2 "+__file__+" | perl -")
...the rest of file is the original perl program...
However, the resultant code is hardly a Python code, it's essentially a Perl code. The potential converter should convert Perl constructs and idioms to easy-to-read Python code, it should retain variable and subroutine names (i.e. the result should not look obfuscated) and should not shatter the wrokflow too much.
Such a conversion is obviously very hard. The hardness of the conversion depends on the number of Perl features and syntactical constructs, which do not have easy-to-read, unobfuscated Python equivalents. I believe that the large amount of such features renders such automatic conversion impossible practically (while theoretical possibility exists).
So, could you please name Perl idioms and syntax features that can't be expressed in Python as concise as in the original Perl code?
Edit: some people linked Python-to-Perl conventers and deduced, on this basis, that it should be easy to write Perl-to-Python as well. However, I'm sure that converting to Python is in greater demand; still this converter is not yet written--while the reverse has already been! Which only makes my confidence in impossibility of writing a good converter to Python more solid.
Your best Perl to Python converter is probably 23 years old, just graduated university and is looking for a job.
Why Perl is not Python.
Perl has statements which Python more-or-less totally lacks. While you can probably contrive matching statements, the syntax will be so utterly unlike Perl as to make it difficult to call it a "translation". You'd really have to cook up some fancy Python stuff to make it as terse as the original Perl.
Perl has run-time semantics which are so unlike Python as to make translation very challenging. We'll look at just one example below.
Perl has data structures which are enough different from Python that translation is hard.
Perl threads don't share data by default. Only selected data elements can be shared. Python threads have more common "shared everything" data.
One example of #2 should be enough.
Perl:
do_something || die()
Where do_something is any statement of any kind.
To automagically translate this into Python you'd have to wrap every || die() statement in
try:
python_version_of_do_something
except OrdinaryStatementFailure, e:
die()
sys.exit()
Where the more common formulation
Perl
do_something
Would become this using simple -- unthinking -- translation of the source
try:
python_version_of_do_something
except OrdinaryStatementFailure, e:
pass
And, of course,
Perl
do_this || do_that || die()
Is even more complex to translate into Python.
And
Perl
do_this && do_that || die()
really push the envelope. My Perl is rusty, so I can't recall the precise semantics of this kind of thing. But you have to totally understand the semantics to work out a Pythonic implementation.
The Python examples are not good Python. To write good Python requires "thinking", something an automatic translated can't do.
And every Perl construct would have to be "wrapped" like that in order to get the original Perl semantics into a Pythonic form.
Now, do a similar analysis for every feature of Perl.
Just to expand on some of the other lists here, these are a few Perl constructs that are probably very clumsy in python (if possible).
dynamic scope (via the local keyword)
typeglob manipulation (multiple variables with the same name)
formats (they have a syntax all their own)
closures over mutable variables
pragmas
lvalue subroutines (mysub() = 5; type code)
source filters
context (list vs scalar, and the way that called code can inspect this with wantarray)
type coercion / dynamic typing
any program that uses string eval
The list goes on an on, and someone could try to create a mapping between all of the analogous constructs, but in the end it will be a failure for one simple reason.
Perl can not be statically parsed. The definitions in Perl code (particularly those in BEGIN blocks) change the way the compiler is going to interpret the remaining code. So for non-trivial programs, conversion from Perl => Python suffers from the halting problem.
There is no way to know exactly how all of the program will be compiled until the program has finished running, and it is theoretically possible to create a Perl program that will compile differently every time it is run. Meaning that one Perl program could map to an infinite number of Python programs, the correct of which is only know after running the original program in the perl interpreter.
It is not impossible, it would just take a lot of work.
By the way, there is Perthon, a Python-to-Perl translator. It just seems like nobody is willing to make one that goes the other way.
EDIT: I think I might I've found the reason why a Python to Perl translator is much easier to implement. It's because Python lets you fiddle with a script's AST. See parser module.
Perl can experimentally be built to collect additional information (for instance, comments) during compilation of perl code and even emit the results as XML. There doesn't appear to be any documentation of this outside the source, except for: http://search.cpan.org/perldoc/perl5100delta#MAD
This should be helpful in building a translator. I'd expect you to get 80% of the way there fairly easily, 95% with great difficulty, and never much better than that. There are too many things that don't map well.
Fundamentally, these are two different languages. Converting from one to another and have the result be mostly readable would mean that the software would have to be able to recognize and generate code idioms, and be able to do some static analysis.
The meaning of a program may be exactly defined by the language definition, but the programmer did not necessarily require all the details. A C programmer testing if the value a printf() returned is negative is checking for an error condition, and doesn't typically care about the exact value. if (printf("%s","...") < 0) exit(); can be translated into Perl as print "..." or die();. These statements may not mean exactly the same thing, but they'll typically be what the programmer means, and to create idiomatic C or Perl code from idiomatic Perl or C code the translator must take this into account.
Since different computer languages tend to have different slightly semantics for similar things, it's typically impossible to translate one language into another and come up with the exact same meaning in readable form. To create readable code, the translator needs to understand what the programmer was intending to do, and that's real difficult.
In addition, it would be easier to translate from Python to Perl rather than Perl to Python. Python is intended as a straightforward language with clear standard ways to do things, while Perl is an unduly complex language with the motto "There's More Than One Way To Do It." Translating a Python expression into one of the innumerable corresponding Perl expressions is easier than figuring out what the Perl programmer meant and expressing it in Python.
Python scope and namespace are different from Perl.
In Python, everything is an object. In Perl, everything under the hood seems to be a list/hash/scalar/reference/function. This induces different design approaches and idioms.
Perl has anonymous code blocks and can generate closures on the fly with some branches. I am pretty sure that is not a python feature.
I do think that a very smart chap could statically analyze the bulk of Perl and produce a program that takes small Perl programs and output Python programs that do the same job.
I am much more doubtful about the feasibility of large and/or gnarly Perl translation. Some of us write some really funky code at times.... :)
This is impossible just because you can't even properly parse perl code. See Perl Cannot Be Parsed: A Formal Proof for more details.
The B set of modules by Malcolm Beattie would be the only sane starting point for something like this, though I'm with other answers in that this would be a difficult problem to solve. In general, translating the sense of one high-level language into another high-level language requires a high-level translator, and, for the time being, that can mean only a human.
The difficulty of this problem, for any pair of languages, is due to fundamental differences in the nature of the languages in question, such as runtime semantics and common idioms, not to mention libraries.
The reason it is close to impossible to create a generic translator from one high-level language to another, is that the program only describe HOW and not WHY (this is the reason for comments in the source code).
In order to create a meaningful program in another highlevel language you (or the translator program) needs to know WHY to be able to create the best possible program. If you cannot do that, all you can do is essentially to create a Python interpreter for the compiled version of the Perl program.
In other words, to do this properly you need to go outside the box, and this is very hard for a computer.
NullUserException basically summed it up - it certainly can be done; it would just be an enormous amount of effort to do so. Some language conversion utilities I've seen compile to an intermediate language (such as .NET's CIL) and then decompile that to the desired language. I have not seen any for Perl to Python. You can, however, find a Python to Perl converter here, though that's likely of little use to you unless you're trying to create your own, in which case it may provide some helpful reference.
Edit: if you just need the exact functionality in a Python script, PyPerl may be of some use to you.
Try my version of the Pythonizer: http://github.com/snoopyjc/pythonizer - it does a decent job

haskell vs python typing

I am looking for example where things in python would be easier to program just because it is dynamically typed?
I want to compare it with Haskell type system because its static typing doesn't get in the way like c# or java. Can I program in Haskell as I can in python without static typing being a hindrance?
PS: I am a python user and have played around little bit with ML and Haskell.. ... I hope it is clear now..
Can I program in Haskell as I can in python without static typing being a hindrance
Yes.
To elaborate, I would say the main gotcha will be the use of existential types in Haskell for heterogeneous data structures (regular data structures holding lists of variously typed elements). This often catches OO people used to a top "Object" type. It often catches Lisp/Scheme programmers. But I'm not sure it will matter to a Pythonista.
Try to write some Haskell, and come back when you get a confusing type error.
You should think of static typing as a benefit -- it checks a lot of things for you, and the more you lean on it, the less things you have to test for. In addition, it enables the compiler to make your code much faster.
Well for one you can't create a list containing multiple types of values without wrappers (like to get a list that may contain a string or an int, you'd have to create a list of Either Int String and wrap each item in a Left or a Right).
You also can't define a function that may return multiple types of values (like if someCondition then 1 else "this won't compile"), again, without using wrappers.
Like Chris said, this is one objective question (what can a dynamically typed language do that a statically typed one can't?) and one subjective question (can I use Haskell without static typing being a hindrance). So you're going to get mostly subjective answers, because the first question is not as interesting.
For me, the biggest hindrance was Haskell's IO type, because I had to stop and think about what code does I/O and what code doesn't, and explicitly pass information between the two. Everything else was pretty easy. If you commonly write
if someCondition:
return 1
else:
return "other"
Then you're making your own problems, Python just doesn't stop you from doing it. Haskell will, and that's about the only difference. The only exception is that this is sort of common in Python:
if someErrorCondition:
return None
else:
return NewItem(Success)
You can't do that in Haskell because there is no common None object. But there are easy ways to work around it.
I did find the type errors confusing at first, but I learned to read them in about a week.
I want to echo Don's advice: just try writing some Haskell and come back when you get a confusing type error.

Python progression path - From apprentice to guru

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I've been learning, working, and playing with Python for a year and a half now. As a biologist slowly making the turn to bio-informatics, this language has been at the very core of all the major contributions I have made in the lab. I more or less fell in love with the way Python permits me to express beautiful solutions and also with the semantics of the language that allows such a natural flow from thoughts to workable code.
What I would like to know is your answer to a kind of question I have seldom seen in this or other forums. This question seems central to me for anyone on the path to Python improvement but who wonders what his next steps should be.
Let me sum up what I do NOT want to ask first ;)
I don't want to know how to QUICKLY learn Python
Nor do I want to find out the best way to get acquainted with the language
Finally, I don't want to know a 'one trick that does it all' approach.
What I do want to know your opinion about, is:
What are the steps YOU would recommend to a Python journeyman, from apprenticeship to guru status (feel free to stop wherever your expertise dictates it), in order that one IMPROVES CONSTANTLY, becoming a better and better Python coder, one step at a time. Some of the people on SO almost seem worthy of worship for their Python prowess, please enlighten us :)
The kind of answers I would enjoy (but feel free to surprise the readership :P ), is formatted more or less like this:
Read this (eg: python tutorial), pay attention to that kind of details
Code for so manytime/problems/lines of code
Then, read this (eg: this or that book), but this time, pay attention to this
Tackle a few real-life problems
Then, proceed to reading Y.
Be sure to grasp these concepts
Code for X time
Come back to such and such basics or move further to...
(you get the point :)
I really care about knowing your opinion on what exactly one should pay attention to, at various stages, in order to progress CONSTANTLY (with due efforts, of course). If you come from a specific field of expertise, discuss the path you see as appropriate in this field.
EDIT: Thanks to your great input, I'm back on the Python improvement track! I really appreciate!
I thought the process of Python mastery went something like:
Discover list comprehensions
Discover generators
Incorporate map, reduce, filter, iter, range, xrange often into your code
Discover Decorators
Write recursive functions, a lot
Discover itertools and functools
Read Real World Haskell (read free online)
Rewrite all your old Python code with tons of higher order functions, recursion, and whatnot.
Annoy your cubicle mates every time they present you with a Python class. Claim it could be "better" implemented as a dictionary plus some functions. Embrace functional programming.
Rediscover the Strategy pattern and then all those things from imperative code you tried so hard to forget after Haskell.
Find a balance.
One good way to further your Python knowledge is to dig into the source code of the libraries, platforms, and frameworks you use already.
For example if you're building a site on Django, many questions that might stump you can be answered by looking at how Django implements the feature in question.
This way you'll continue to pick up new idioms, coding styles, and Python tricks. (Some will be good and some will be bad.)
And when you see something Pythony that you don't understand in the source, hop over to the #python IRC channel and you'll find plenty of "language lawyers" happy to explain.
An accumulation of these little clarifications over years leads to a much deeper understanding of the language and all of its ins and outs.
Understand (more deeply) Python's data types and their roles with regards to memory mgmt
As some of you in the community are aware, I teach Python courses, the most popular ones being the comprehensive Intro+Intermediate course as well as an "advanced" course which introduces a variety of areas of application development.
Quite often, I get asked a question quite similar to, "Should I take your intro or advanced course? I've already been programming Python for 1-2 years, and I think the intro one is too simple for me so I'd like to jump straight to the advanced... which course would you recommend?"
To answer their question, I probe to see how strong they are in this area -- not that it's really the best way to measure whether they're ready for any advanced course, but to see how well their basic knowledge is of Python's objects and memory model, which is a cause of many Python bugs written by those who are not only beginners but those who have gone beyond that.
To do this, I point them at this simple 2-part quiz question:
Many times, they are able to get the output, but the why is more difficult and much more important of an response... I would weigh the output as 20% of the answer while the "why" gets 80% credit. If they can't get the why, regardless how Python experience they have, I will always steer people to the comprehensive intro+intermediate course because I spend one lecture on objects and memory management to the point where you should be able to answer with the output and the why with sufficient confidence. (Just because you know Python's syntax after 1-2 years doesn't make you ready to move beyond a "beginner" label until you have a much better understanding as far as how Python works under the covers.)
A succeeding inquiry requiring a similar answer is even tougher, e.g.,
Example 3
x = ['foo', [1,2,3], 10.4]
y = list(x) # or x[:]
y[0] = 'fooooooo'
y[1][0] = 4
print x
print y
The next topics I recommend are to understanding reference counting well, learning what "interning" means (but not necessarily using it), learning about shallow and deep copies (as in Example 3 above), and finally, the interrelationships between the various types and constructs in the language, i.e. lists vs. tuples, dicts vs. sets, list comprehensions vs. generator expressions, iterators vs. generators, etc.; however all those other suggestions are another post for another time. Hope this helps in the meantime! :-)
ps. I agree with the other responses for getting more intimate with introspection as well as studying other projects' source code and add a strong "+1" to both suggestions!
pps. Great question BTW. I wish I was smart enough in the beginning to have asked something like this, but that was a long time ago, and now I'm trying to help others with my many years of full-time Python programming!!
Check out Peter Norvig's essay on becoming a master programmer in 10 years: http://norvig.com/21-days.html. I'd wager it holds true for any language.
Understand Introspection
write a dir() equivalent
write a type() equivalent
figure out how to "monkey-patch"
use the dis module to see how various language constructs work
Doing these things will
give you some good theoretical knowledge about how python is implemented
give you some good practical experience in lower-level programming
give you a good intuitive feel for python data structures
def apprentice():
read(diveintopython)
experiment(interpreter)
read(python_tutorial)
experiment(interpreter, modules/files)
watch(pycon)
def master():
refer(python-essential-reference)
refer(PEPs/language reference)
experiment()
read(good_python_code) # Eg. twisted, other libraries
write(basic_library) # reinvent wheel and compare to existing wheels
if have_interesting_ideas:
give_talk(pycon)
def guru():
pass # Not qualified to comment. Fix the GIL perhaps?
I'll give you the simplest and most effective piece of advice I think anybody could give you: code.
You can only be better at using a language (which implies understanding it) by coding. You have to actively enjoy coding, be inspired, ask questions, and find answers by yourself.
Got a an hour to spare? Write code that will reverse a string, and find out the most optimum solution. A free evening? Why not try some web-scraping. Read other peoples code. See how they do things. Ask yourself what you would do.
When I'm bored at my computer, I open my IDE and code-storm. I jot down ideas that sound interesting, and challenging. An URL shortener? Sure, I can do that. Oh, I learnt how to convert numbers from one base to another as a side effect!
This is valid whatever your skill level. You never stop learning. By actively coding in your spare time you will, with little additional effort, come to understand the language, and ultimately, become a guru. You will build up knowledge and reusable code and memorise idioms.
If you're in and using python for science (which it seems you are) part of that will be learning and understanding scientific libraries, for me these would be
numpy
scipy
matplotlib
mayavi/mlab
chaco
Cython
knowing how to use the right libraries and vectorize your code is essential for scientific computing.
I wanted to add that, handling large numeric datasets in common pythonic ways(object oriented approaches, lists, iterators) can be extremely inefficient. In scientific computing, it can be necessary to structure your code in ways that differ drastically from how most conventional python coders approach data.
Google just recently released an online Python class ("class" as in "a course of study").
http://code.google.com/edu/languages/google-python-class/
I know this doesn't answer your full question, but I think it's a great place to start!
Download Twisted and look at the source code. They employ some pretty advanced techniques.
Thoroughly Understand All Data Types and Structures
For every type and structure, write a series of demo programs that exercise every aspect of the type or data structure. If you do this, it might be worthwhile to blog notes on each one... it might be useful to lots of people!
I learned python first by myself over a summer just by doing the tutorial on the python site (sadly, I don't seem to be able to find that anymore, so I can't post a link).
Later, python was taught to me in one of my first year courses at university. In the summer that followed, I practiced with PythonChallenge and with problems from Google Code Jam.
Solving these problems help from an algorithmic perspective as well as from the perspective of learning what Python can do as well as how to manipulate it to get the fullest out of python.
For similar reasons, I have heard that code golf works as well, but i have never tried it for myself.
Learning algorithms/maths/file IO/Pythonic optimisation
This won't get you guru-hood but to start out, try working through the Project Euler problems
The first 50 or so shouldn't tax you if you have decent high-school mathematics and know how to Google. When you solve one you get into the forum where you can look through other people's solutions which will teach you even more. Be decent though and don't post up your solutions as the idea is to encourage people to work it out for themselves.
Forcing yourself to work in Python will be unforgiving if you use brute-force algorithms.
This will teach you how to lay out large datasets in memory and access them efficiently with the fast language features such as dictionaries.
From doing this myself I learnt:
File IO
Algorithms and techniques such as Dynamic Programming
Python data layout
Dictionaries/hashmaps
Lists
Tuples
Various combinations thereof, e.g. dictionaries to lists of tuples
Generators
Recursive functions
Developing Python libraries
Filesystem layout
Reloading them during an interpreter session
And also very importantly
When to give up and use C or C++!
All of this should be relevant to Bioinformatics
Admittedly I didn't learn about the OOP features of Python from that experience.
Have you seen the book "Bioinformatics Programming using Python"? Looks like you're an exact member of its focus group.
You already have a lot of reading material, but if you can handle more, I recommend you
learn about the evolution of python by reading the Python Enhancement Proposals, especially the "Finished" PEPs and the "Deferred, Abandoned, Withdrawn, and Rejected" PEPs.
By seeing how the language has changed, the decisions that were made and their rationales, you will absorb the philosophy of Python and understand how "idiomatic Python" comes about.
http://www.python.org/dev/peps/
Attempt http://challenge.greplin.com/ using Python
Teaching to someone else who is starting to learn Python is always a great way to get your ideas clear and sometimes, I usually get a lot of neat questions from students that have me to re-think conceptual things about Python.
Not precisely what you're asking for, but I think it's good advice.
Learn another language, doesn't matter too much which. Each language has it's own ideas and conventions that you can learn from. Learn about the differences in the languages and more importantly why they're different. Try a purely functional language like Haskell and see some of the benefits (and challenges) of functions free of side-effects. See how you can apply some of the things you learn from other languages to Python.
I recommend starting with something that forces you to explore the expressive power of the syntax. Python allows many different ways of writing the same functionality, but there is often a single most elegant and fastest approach. If you're used to the idioms of other languages, you might never otherwise find or accept these better ways. I spent a weekend trudging through the first 20 or so Project Euler problems and made a simple webapp with Django on Google App Engine. This will only take you from apprentice to novice, maybe, but you can then continue to making somewhat more advanced webapps and solve more advanced Project Euler problems. After a few months I went back and solved the first 20 PE problems from scratch in an hour instead of a weekend.

Categories

Resources