Is there an exact replacement for the old functional.curry? - python

I am trying to run this snippet from http://www.ibm.com/developerworks/linux/library/l-prog3.html on a python 2.6 runtime.
from functional import *
taxcalc = lambda income,rate,deduct: (income-(deduct))*rate
taxCurry = curry(taxcalc)
taxCurry = taxCurry(50000)
taxCurry = taxCurry(0.30)
taxCurry = taxCurry(10000)
print "Curried taxes due =",taxCurry
print "Curried expression taxes due =", \
curry(taxcalc)(50000)(0.30)(10000)
Ok, so I understand from http://www.python.org/dev/peps/pep-0309/ that functional is renamed to functools and curry to partial but just doing the renames doesn't help. I get the error:
taxCurry = taxCurry(50000)
TypeError: <lambda>() takes exactly 3 arguments (1 given)
The following does work but do I really have to change it so much?
from functools import partial
taxcalc = lambda income,rate,deduct: (income-(deduct))*rate
taxCurry = partial(taxcalc)
taxCurry = partial(taxCurry, 50000)
taxCurry = partial(taxCurry, 0.30)
taxCurry = partial(taxCurry, 10000)
print "Curried taxes due =", taxCurry()
print "Curried expression taxes due =", \
taxcalc(50000, 0.30, 10000)
Is there a better way of preserving the mechanics of the original example? Lastly was the original example truly currying or just partial application? (as per http://www.uncarved.com/blog/not_currying.mrk)
Thanks for your time

I guess the reason why they changed it is because Python is dynamically typed. This means that it would be really hard to debug the original curry code if anything goes wrong - much harder than in a language like Haskell where you would directly get a nice type error. So I think it was a reasonable decision to replace it with the more explicit partial version (looks more pythonic to me).
Your example is also a bit strange, since you just reassigning the partially applied functions to the same name. Normally the partially applied function would be given to another function. At least that is the only reasonable use case in Python I can think of.

The implementation of curry in the toolz project should be a drop-in replacement.
$ pip install toolz
>>> from toolz import curry

I wrote an implementation of a curry decorator that works well:
def curry(func):
"""
Decorator to curry a function, typical usage:
>>> #curry
... def foo(a, b, c):
... return a + b + c
The function still work normally:
>>> foo(1, 2, 3)
6
And in various curried forms:
>>> foo(1)(2, 3)
6
>>> foo(1)(2)(3)
6
This also work with named arguments:
>>> foo(a=1)(b=2)(c=3)
6
>>> foo(b=1)(c=2)(a=3)
6
>>> foo(a=1, b=2)(c=3)
6
>>> foo(a=1)(b=2, c=3)
6
And you may also change your mind on named arguments,
But I don't know why you may want to do that:
>>> foo(a=1, b=0)(b=2, c=3)
6
Finally, if you give more parameters than expected, the exception
is the expected one, not some garbage produced by the currying
mechanism:
>>> foo(1, 2)(3, 4)
Traceback (most recent call last):
...
TypeError: foo() takes exactly 3 arguments (4 given)
"""
def curried(*args, **kwargs):
if len(args) + len(kwargs) >= func.__code__.co_argcount:
return func(*args, **kwargs)
return (lambda *args2, **kwargs2:
curried(*(args + args2), **dict(kwargs, **kwargs2)))
return curried
if __name__ == "__main__":
import doctest
doctest.testmod()

Related

Is it possible to pass the same optional arguments to multiple functions?

I want to ask if there is a way to prevent unnecessary duplicate of code when passing the same arguments into a function's optional arguments.
Hopefully the following example provides a good idea of what I am trying to do:
def f(arg1):
def g(optional_1=0, optional_2=0, optional_3=0):
return arg1+optional_1+optional_2+optional_3
return g
b, c = 2, 3
f1 = f(1)
f2 = f(2)
calc_f1 = f1(optional_2=b, optional_3=c)
calc_f2 = f2(optional_2=b, optional_3=c)
As you can see, f1 and f2 only differ in the arg1 passed into f and afterwards I call them with the same variables for the same optional arguments.
It is fine when the code is short, but when I have over 10 optional arguments, it becomes unnecessarily long and redundant.
Is it possible to do something like
optional_variable_pair = #some way to combine them
calc_f1 = f1(optional_variable_pair)
calc_f2 = f2(optional_variable_pair)
so I get a more succinct and easy to read code?
Any function with multiple optional arguments is a bit smelly because:
you get so many argument combinations that it requires a large amount of testing.
because of all the options the function has to have alot of conditionals and routes which increase its cyclomatic complexity.
You can apply a refactoring to extract the whole argument list into an Object and have the function work on that object. This works really well if you can find a unifying name that describes your argument list and fits whatever metaphor you are using around the function. You can even invert the call so that the function becomes a method of the Object, so you get some encapsulation.
To answer the question you asked, the answer is yes. You can do almost exactly what you want using keyword argument unpacking.
def f(arg1):
def g(optional_1=0, optional_2=0, optional_3=0):
return arg1+optional_1+optional_2+optional_3
return g
optional_variable_pair = {
'optional_2': 2,
'optional_3': 3
}
f1 = f(1)
f2 = f(2)
calc_f1 = f1(**optional_variable_pair)
calc_f2 = f2(**optional_variable_pair)
If I'm reading your intent correctly, though, the essence of your question is wanting to pass new first arguments with the same successive arguments to a function. Depending on your use case, the wrapper function g may be unnecessary.
def f(arg1, *, optional_1=0, optional_2=0, optional_3=0):
return optional_1 + optional_2+optional_3
optional_variable_pair = {
'optional_2': 2,
'optional_3': 3
}
calc_f1 = f(1, **optional_variable_pair)
calc_f2 = f(2, **optional_variable_pair)
Obviously, if the first argument continues incrementing by one, a for loop is in order. Obviously, if you are never using the optional_1 parameter, you do not need to include it. But, moreover, if you find yourself using numbered arguments, there is a good chance you really should be working with tuple unpacking instead of keyword unpacking:
def f(*args):
return sum(args)
optional_variable_pair = (2, 3)
for i in range(1, 3):
calc = f(i, *optional_variable_pair)
# ...do something with calc...
You may also be interested in researching functools.partial, as well, which can take the place of your wrapper function g, and allow this:
import functools
def f(*args):
return sum(args)
f1 = functools.partial(f, 1)
f2 = functools.partial(f, 2)
calc_f1 = f1(2, 3) # = 1 + 2 + 3 = 6
calc_f2 = f2(2, 3) # = 2 + 2 + 3 = 7
You use key-value pairs as function argsuments, for this purpose you can use *args and **kwargs:
optional_variable_pair = {
"optional_1": 1,
"optional_2": 2,
"optional_3": 3,
}
calc_f1 = f1(**optional_variable_pair)
calc_f2 = f2(**optional_variable_pair)

From Haskell to Python: how to do currying?

I recently started coding in Python and I was wondering if it's possible to return a function that specializes another function.
For example, in Haskell you can create a function that adds 5 to any given number like this:
sumFive = (+5)
Is it somehow possible in Python?
I think the other answers are misunderstanding the question. I believe the OP is asking about partial application of a function, in his example the function is (+).
If the goal isn't partial application, the solution is as simple as:
def sumFive(x): return x + 5
For partial application in Python, we can use this function: https://docs.python.org/2/library/functools.html#functools.partial
def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords):
newkeywords = keywords.copy()
newkeywords.update(fkeywords)
return func(*(args + fargs), **newkeywords)
newfunc.func = func
newfunc.args = args
newfunc.keywords = keywords
return newfunc
Then, we must turn the + operator into a function (I don't believe there's a lightweight syntax to do so like in Haskell):
def plus(x, y): return x + y
Finally:
sumFive = partial(plus, 5)
Not nearly as nice as in Haskell, but it works:
>>> sumFive(7)
12
Python's design does not naturally support the evaluation of a multi-variable function into a sequence of single-variable functions (currying). As other answers point out, the related (but distinct) concept of partial application is more straightforward to do using partial from the functools module.
However, the PyMonad library supplies you with the tools to make currying possible in Python, providing a "collection of classes for programming with functors, applicative functors and monads."
Use the curry decorator to decorate a function that accepts any number of arguments:
from pymonad import curry
#curry
def add(x, y):
return x + y
It is then very easy to curry add. The syntax is not too dissimilar to Haskell's:
>>> add5 = add(5)
>>> add5(12)
17
Note that here the add and add5 functions are instances of PyMonad's Reader monad class, not a normal Python function object:
>>> add
<pymonad.Reader.Reader at 0x7f7024ccf908>
This allows, for example, the possibility of using simpler syntax to compose functions (easy to do in Haskell, normally much less so in Python).
Finally, it's worth noting that the infix operator + is not a Python function: + calls into the left-hand operand's __add__ method, or the right-hand operand's __radd__ method and returns the result. You'll need to decorate these class methods for the objects you're working with if you want to curry using + (disclaimer: I've not tried to do this yet).
Yup. Python supports lambda expressions:
sumFive = lambda x: x + 5
for i in range(5):
print sumFive(i),
#OUTPUT 5,6,7,8,9
Python functions can return functions, allowing you to create higher-order functions. For example, here is a higher-order function which can specialize a function of two variables:
def specialize(f,a,i):
def g(x):
if i == 0:
return f(a,x)
else:
return f(x,a)
return g
Used like this:
>>> def subtract(x,y): return x - y
>>> f = specialize(subtract,5,0)
>>> g = specialize(subtract,5,1)
>>> f(7)
-2
>>> g(7)
2
But -- there is really no need to reinvent the wheel, the module functools has a number of useful higher-order functions that any Haskell programmer would find useful, including partial for partial function application, which is what you are asking about.
As it was pointed out, python does have lambda functions, so the following does solve the problem:
# Haskell: sumFive = (+5)
sumFive = lambda x : x + 5
I think this is more useful with the fact that python has first class functions (1,2)
def summation(n, term):
total, k = 0, 1
while k <= n:
total, k = total + term(k), k + 1
return total
def identity(x):
return x
def sum_naturals(n):
return summation(n, identity)
sum_naturals(10) # Returns 55
# Now for something a bit more complex
def pi_term(x):
return 8 / ((4*x-3) * (4*x-1))
def pi_sum(n):
return summation(n, pi_term)
pi_sum(1e6) # returns: 3.141592153589902
You can find more on functional programming and python here
For the most generic Haskell style currying, look at partial from the functools module.

Python, assign function to variable, change optional argument's value

Is it possible to assign a function to a variable with modified default arguments?
To make it more concrete, I'll give an example.
The following obviously doesn't work in the current form and is only meant to show what I need:
def power(a, pow=2):
ret = 1
for _ in range(pow):
ret *= a
return ret
cube = power(pow=3)
And the result of cube(5) should be 125.
functools.partial to the rescue:
Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords.
from functools import partial
cube = partial(power, pow=3)
Demo:
>>> from functools import partial
>>>
>>> def power(a, pow=2):
... ret = 1
... for _ in range(pow):
... ret *= a
... return ret
...
>>> cube = partial(power, pow=3)
>>>
>>> cube(5)
125
The answer using partial is good, using the standard library, but I think it's worth mentioning that the following approach is equivalent:
def cube(a):
return power(a, pow=3)
Even though this doesn't seem like assignment because there isn't a =, it is doing much the same thing (binding a name to a function object). I think this is often more legible.
In specific there's a special function for exponents:
>>> 2**3
8
But I also solved it with a lambda function, which is a nicer version of a function pointer.
# cube = power(pow=3) # original
cube = lambda x: power(x,3)

Partial function application with the original docstring in Python?

For partial function application, I know there are several ways to do that in Python. However, they seems not to preserve the original function's docstring.
Take functools.partial as example:
from functools import partial
def foo(a, b, c=1):
"""Return (a+b)*c."""
return (a+b)*c
bar10_p = partial(foo, b=10)
print bar10_p.__doc__
partial(func, *args, **keywords) - new function with partial application
of the given arguments and keywords.
Let's try fn.py:
from fn import F
def foo(a, b, c=1):
"""Return (a+b)*c."""
return (a+b)*c
bar10_F = F(foo, b=10)
print bar10_F.__doc__
Provide simple syntax for functions composition
(through << and >> operators) and partial function
application (through simple tuple syntax).
Usage example:
>>> func = F() << (_ + 10) << (_ + 5)
>>> print(func(10))
25
>>> func = F() >> (filter, _ < 6) >> sum
>>> print(func(range(10)))
15
Is there any Python package/module providing partial application with preserved docstring?
UPDATE
As #Kevin and #Martijn Pieters mentioned, the function signature has changed such that it is not suggested to stick to the original function's docstring. I realized that I'm looking for an updated docstring with something like foo() with a default b value of 10 (Thanks for Kevin's simple but direct example.).
__doc__ is writable, on partial objects as well as on functions; simply copy it over:
bar10_p = partial(foo, b=10)
bar10_p.__doc__ = func.__doc__
or use the functools.update_wrapper() function to do the copying for you; it'll copy a few other pieces of metadata for you too:
from functools import update_wrapper
bar10_p = partial(foo, b=10)
update_wrapper(bar10_p, foo)
Just write a new __doc__.
bar10_p = partial(foo, b=10)
bar10_p.__doc__ = """foo() with a default b value of 10.
See foo().
"""
Your function has a different interface from the original, so it should not copy the docstring exactly.
Partial has access to func method which is the original function. So through original function, you have access to original function docstring.
Try this:
from math import cos
from functools import partial
cos_partial = partial(cos, 0.5)
print(cos_partial.func.__doc__)
With makefun you can do it:
from makefun import partial
def foo(a, b, c=1):
"""Return (a+b)*c."""
return (a + b) * c
bar10_p = partial(foo, b=10)
assert bar10_p(0) == 10
assert bar10_p(0, c=2) == 20
help(bar10_p)
It yields:
Help on function foo in module makefun.tests.test_so:
foo(a, c=1)
<This function is equivalent to 'foo(a, c=1, b=10)', see original 'foo' doc below.>
Return (a+b)*c.
Note that if you have any comment on how the docstring should be updated, do not hesitate to propose an issue on the git repo !
(I'm the author by the way)

Hidden features of Python [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
What are the lesser-known but useful features of the Python programming language?
Try to limit answers to Python core.
One feature per answer.
Give an example and short description of the feature, not just a link to documentation.
Label the feature using a title as the first line.
Quick links to answers:
Argument Unpacking
Braces
Chaining Comparison Operators
Decorators
Default Argument Gotchas / Dangers of Mutable Default arguments
Descriptors
Dictionary default .get value
Docstring Tests
Ellipsis Slicing Syntax
Enumeration
For/else
Function as iter() argument
Generator expressions
import this
In Place Value Swapping
List stepping
__missing__ items
Multi-line Regex
Named string formatting
Nested list/generator comprehensions
New types at runtime
.pth files
ROT13 Encoding
Regex Debugging
Sending to Generators
Tab Completion in Interactive Interpreter
Ternary Expression
try/except/else
Unpacking+print() function
with statement
Chaining comparison operators:
>>> x = 5
>>> 1 < x < 10
True
>>> 10 < x < 20
False
>>> x < 10 < x*10 < 100
True
>>> 10 > x <= 9
True
>>> 5 == x > 4
True
In case you're thinking it's doing 1 < x, which comes out as True, and then comparing True < 10, which is also True, then no, that's really not what happens (see the last example.) It's really translating into 1 < x and x < 10, and x < 10 and 10 < x * 10 and x*10 < 100, but with less typing and each term is only evaluated once.
Get the python regex parse tree to debug your regex.
Regular expressions are a great feature of python, but debugging them can be a pain, and it's all too easy to get a regex wrong.
Fortunately, python can print the regex parse tree, by passing the undocumented, experimental, hidden flag re.DEBUG (actually, 128) to re.compile.
>>> re.compile("^\[font(?:=(?P<size>[-+][0-9]{1,2}))?\](.*?)[/font]",
re.DEBUG)
at at_beginning
literal 91
literal 102
literal 111
literal 110
literal 116
max_repeat 0 1
subpattern None
literal 61
subpattern 1
in
literal 45
literal 43
max_repeat 1 2
in
range (48, 57)
literal 93
subpattern 2
min_repeat 0 65535
any None
in
literal 47
literal 102
literal 111
literal 110
literal 116
Once you understand the syntax, you can spot your errors. There we can see that I forgot to escape the [] in [/font].
Of course you can combine it with whatever flags you want, like commented regexes:
>>> re.compile("""
^ # start of a line
\[font # the font tag
(?:=(?P<size> # optional [font=+size]
[-+][0-9]{1,2} # size specification
))?
\] # end of tag
(.*?) # text between the tags
\[/font\] # end of the tag
""", re.DEBUG|re.VERBOSE|re.DOTALL)
enumerate
Wrap an iterable with enumerate and it will yield the item along with its index.
For example:
>>> a = ['a', 'b', 'c', 'd', 'e']
>>> for index, item in enumerate(a): print index, item
...
0 a
1 b
2 c
3 d
4 e
>>>
References:
Python tutorial—looping techniques
Python docs—built-in functions—enumerate
PEP 279
Creating generators objects
If you write
x=(n for n in foo if bar(n))
you can get out the generator and assign it to x. Now it means you can do
for n in x:
The advantage of this is that you don't need intermediate storage, which you would need if you did
x = [n for n in foo if bar(n)]
In some cases this can lead to significant speed up.
You can append many if statements to the end of the generator, basically replicating nested for loops:
>>> n = ((a,b) for a in range(0,2) for b in range(4,6))
>>> for i in n:
... print i
(0, 4)
(0, 5)
(1, 4)
(1, 5)
iter() can take a callable argument
For instance:
def seek_next_line(f):
for c in iter(lambda: f.read(1),'\n'):
pass
The iter(callable, until_value) function repeatedly calls callable and yields its result until until_value is returned.
Be careful with mutable default arguments
>>> def foo(x=[]):
... x.append(1)
... print x
...
>>> foo()
[1]
>>> foo()
[1, 1]
>>> foo()
[1, 1, 1]
Instead, you should use a sentinel value denoting "not given" and replace with the mutable you'd like as default:
>>> def foo(x=None):
... if x is None:
... x = []
... x.append(1)
... print x
>>> foo()
[1]
>>> foo()
[1]
Sending values into generator functions. For example having this function:
def mygen():
"""Yield 5 until something else is passed back via send()"""
a = 5
while True:
f = (yield a) #yield a and possibly get f in return
if f is not None:
a = f #store the new value
You can:
>>> g = mygen()
>>> g.next()
5
>>> g.next()
5
>>> g.send(7) #we send this back to the generator
7
>>> g.next() #now it will yield 7 until we send something else
7
If you don't like using whitespace to denote scopes, you can use the C-style {} by issuing:
from __future__ import braces
The step argument in slice operators. For example:
a = [1,2,3,4,5]
>>> a[::2] # iterate over the whole list in 2-increments
[1,3,5]
The special case x[::-1] is a useful idiom for 'x reversed'.
>>> a[::-1]
[5,4,3,2,1]
Decorators
Decorators allow to wrap a function or method in another function that can add functionality, modify arguments or results, etc. You write decorators one line above the function definition, beginning with an "at" sign (#).
Example shows a print_args decorator that prints the decorated function's arguments before calling it:
>>> def print_args(function):
>>> def wrapper(*args, **kwargs):
>>> print 'Arguments:', args, kwargs
>>> return function(*args, **kwargs)
>>> return wrapper
>>> #print_args
>>> def write(text):
>>> print text
>>> write('foo')
Arguments: ('foo',) {}
foo
The for...else syntax (see http://docs.python.org/ref/for.html )
for i in foo:
if i == 0:
break
else:
print("i was never 0")
The "else" block will be normally executed at the end of the for loop, unless the break is called.
The above code could be emulated as follows:
found = False
for i in foo:
if i == 0:
found = True
break
if not found:
print("i was never 0")
From 2.5 onwards dicts have a special method __missing__ that is invoked for missing items:
>>> class MyDict(dict):
... def __missing__(self, key):
... self[key] = rv = []
... return rv
...
>>> m = MyDict()
>>> m["foo"].append(1)
>>> m["foo"].append(2)
>>> dict(m)
{'foo': [1, 2]}
There is also a dict subclass in collections called defaultdict that does pretty much the same but calls a function without arguments for not existing items:
>>> from collections import defaultdict
>>> m = defaultdict(list)
>>> m["foo"].append(1)
>>> m["foo"].append(2)
>>> dict(m)
{'foo': [1, 2]}
I recommend converting such dicts to regular dicts before passing them to functions that don't expect such subclasses. A lot of code uses d[a_key] and catches KeyErrors to check if an item exists which would add a new item to the dict.
In-place value swapping
>>> a = 10
>>> b = 5
>>> a, b
(10, 5)
>>> a, b = b, a
>>> a, b
(5, 10)
The right-hand side of the assignment is an expression that creates a new tuple. The left-hand side of the assignment immediately unpacks that (unreferenced) tuple to the names a and b.
After the assignment, the new tuple is unreferenced and marked for garbage collection, and the values bound to a and b have been swapped.
As noted in the Python tutorial section on data structures,
Note that multiple assignment is really just a combination of tuple packing and sequence unpacking.
Readable regular expressions
In Python you can split a regular expression over multiple lines, name your matches and insert comments.
Example verbose syntax (from Dive into Python):
>>> pattern = """
... ^ # beginning of string
... M{0,4} # thousands - 0 to 4 M's
... (CM|CD|D?C{0,3}) # hundreds - 900 (CM), 400 (CD), 0-300 (0 to 3 C's),
... # or 500-800 (D, followed by 0 to 3 C's)
... (XC|XL|L?X{0,3}) # tens - 90 (XC), 40 (XL), 0-30 (0 to 3 X's),
... # or 50-80 (L, followed by 0 to 3 X's)
... (IX|IV|V?I{0,3}) # ones - 9 (IX), 4 (IV), 0-3 (0 to 3 I's),
... # or 5-8 (V, followed by 0 to 3 I's)
... $ # end of string
... """
>>> re.search(pattern, 'M', re.VERBOSE)
Example naming matches (from Regular Expression HOWTO)
>>> p = re.compile(r'(?P<word>\b\w+\b)')
>>> m = p.search( '(((( Lots of punctuation )))' )
>>> m.group('word')
'Lots'
You can also verbosely write a regex without using re.VERBOSE thanks to string literal concatenation.
>>> pattern = (
... "^" # beginning of string
... "M{0,4}" # thousands - 0 to 4 M's
... "(CM|CD|D?C{0,3})" # hundreds - 900 (CM), 400 (CD), 0-300 (0 to 3 C's),
... # or 500-800 (D, followed by 0 to 3 C's)
... "(XC|XL|L?X{0,3})" # tens - 90 (XC), 40 (XL), 0-30 (0 to 3 X's),
... # or 50-80 (L, followed by 0 to 3 X's)
... "(IX|IV|V?I{0,3})" # ones - 9 (IX), 4 (IV), 0-3 (0 to 3 I's),
... # or 5-8 (V, followed by 0 to 3 I's)
... "$" # end of string
... )
>>> print pattern
"^M{0,4}(CM|CD|D?C{0,3})(XC|XL|L?X{0,3})(IX|IV|V?I{0,3})$"
Function argument unpacking
You can unpack a list or a dictionary as function arguments using * and **.
For example:
def draw_point(x, y):
# do some magic
point_foo = (3, 4)
point_bar = {'y': 3, 'x': 2}
draw_point(*point_foo)
draw_point(**point_bar)
Very useful shortcut since lists, tuples and dicts are widely used as containers.
ROT13 is a valid encoding for source code, when you use the right coding declaration at the top of the code file:
#!/usr/bin/env python
# -*- coding: rot13 -*-
cevag "Uryyb fgnpxbiresybj!".rapbqr("rot13")
Creating new types in a fully dynamic manner
>>> NewType = type("NewType", (object,), {"x": "hello"})
>>> n = NewType()
>>> n.x
"hello"
which is exactly the same as
>>> class NewType(object):
>>> x = "hello"
>>> n = NewType()
>>> n.x
"hello"
Probably not the most useful thing, but nice to know.
Edit: Fixed name of new type, should be NewType to be the exact same thing as with class statement.
Edit: Adjusted the title to more accurately describe the feature.
Context managers and the "with" Statement
Introduced in PEP 343, a context manager is an object that acts as a run-time context for a suite of statements.
Since the feature makes use of new keywords, it is introduced gradually: it is available in Python 2.5 via the __future__ directive. Python 2.6 and above (including Python 3) has it available by default.
I have used the "with" statement a lot because I think it's a very useful construct, here is a quick demo:
from __future__ import with_statement
with open('foo.txt', 'w') as f:
f.write('hello!')
What's happening here behind the scenes, is that the "with" statement calls the special __enter__ and __exit__ methods on the file object. Exception details are also passed to __exit__ if any exception was raised from the with statement body, allowing for exception handling to happen there.
What this does for you in this particular case is that it guarantees that the file is closed when execution falls out of scope of the with suite, regardless if that occurs normally or whether an exception was thrown. It is basically a way of abstracting away common exception-handling code.
Other common use cases for this include locking with threads and database transactions.
Dictionaries have a get() method
Dictionaries have a 'get()' method. If you do d['key'] and key isn't there, you get an exception. If you do d.get('key'), you get back None if 'key' isn't there. You can add a second argument to get that item back instead of None, eg: d.get('key', 0).
It's great for things like adding up numbers:
sum[value] = sum.get(value, 0) + 1
Descriptors
They're the magic behind a whole bunch of core Python features.
When you use dotted access to look up a member (eg, x.y), Python first looks for the member in the instance dictionary. If it's not found, it looks for it in the class dictionary. If it finds it in the class dictionary, and the object implements the descriptor protocol, instead of just returning it, Python executes it. A descriptor is any class that implements the __get__, __set__, or __delete__ methods.
Here's how you'd implement your own (read-only) version of property using descriptors:
class Property(object):
def __init__(self, fget):
self.fget = fget
def __get__(self, obj, type):
if obj is None:
return self
return self.fget(obj)
and you'd use it just like the built-in property():
class MyClass(object):
#Property
def foo(self):
return "Foo!"
Descriptors are used in Python to implement properties, bound methods, static methods, class methods and slots, amongst other things. Understanding them makes it easy to see why a lot of things that previously looked like Python 'quirks' are the way they are.
Raymond Hettinger has an excellent tutorial that does a much better job of describing them than I do.
Conditional Assignment
x = 3 if (y == 1) else 2
It does exactly what it sounds like: "assign 3 to x if y is 1, otherwise assign 2 to x". Note that the parens are not necessary, but I like them for readability. You can also chain it if you have something more complicated:
x = 3 if (y == 1) else 2 if (y == -1) else 1
Though at a certain point, it goes a little too far.
Note that you can use if ... else in any expression. For example:
(func1 if y == 1 else func2)(arg1, arg2)
Here func1 will be called if y is 1 and func2, otherwise. In both cases the corresponding function will be called with arguments arg1 and arg2.
Analogously, the following is also valid:
x = (class1 if y == 1 else class2)(arg1, arg2)
where class1 and class2 are two classes.
Doctest: documentation and unit-testing at the same time.
Example extracted from the Python documentation:
def factorial(n):
"""Return the factorial of n, an exact integer >= 0.
If the result is small enough to fit in an int, return an int.
Else return a long.
>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]
>>> factorial(-1)
Traceback (most recent call last):
...
ValueError: n must be >= 0
Factorials of floats are OK, but the float must be an exact integer:
"""
import math
if not n >= 0:
raise ValueError("n must be >= 0")
if math.floor(n) != n:
raise ValueError("n must be exact integer")
if n+1 == n: # catch a value like 1e300
raise OverflowError("n too large")
result = 1
factor = 2
while factor <= n:
result *= factor
factor += 1
return result
def _test():
import doctest
doctest.testmod()
if __name__ == "__main__":
_test()
Named formatting
% -formatting takes a dictionary (also applies %i/%s etc. validation).
>>> print "The %(foo)s is %(bar)i." % {'foo': 'answer', 'bar':42}
The answer is 42.
>>> foo, bar = 'question', 123
>>> print "The %(foo)s is %(bar)i." % locals()
The question is 123.
And since locals() is also a dictionary, you can simply pass that as a dict and have % -substitions from your local variables. I think this is frowned upon, but simplifies things..
New Style Formatting
>>> print("The {foo} is {bar}".format(foo='answer', bar=42))
To add more python modules (espcially 3rd party ones), most people seem to use PYTHONPATH environment variables or they add symlinks or directories in their site-packages directories. Another way, is to use *.pth files. Here's the official python doc's explanation:
"The most convenient way [to modify
python's search path] is to add a path
configuration file to a directory
that's already on Python's path,
usually to the .../site-packages/
directory. Path configuration files
have an extension of .pth, and each
line must contain a single path that
will be appended to sys.path. (Because
the new paths are appended to
sys.path, modules in the added
directories will not override standard
modules. This means you can't use this
mechanism for installing fixed
versions of standard modules.)"
Exception else clause:
try:
put_4000000000_volts_through_it(parrot)
except Voom:
print "'E's pining!"
else:
print "This parrot is no more!"
finally:
end_sketch()
The use of the else clause is better than adding additional code to the try clause because it avoids accidentally catching an exception that wasn’t raised by the code being protected by the try ... except statement.
See http://docs.python.org/tut/node10.html
Re-raising exceptions:
# Python 2 syntax
try:
some_operation()
except SomeError, e:
if is_fatal(e):
raise
handle_nonfatal(e)
# Python 3 syntax
try:
some_operation()
except SomeError as e:
if is_fatal(e):
raise
handle_nonfatal(e)
The 'raise' statement with no arguments inside an error handler tells Python to re-raise the exception with the original traceback intact, allowing you to say "oh, sorry, sorry, I didn't mean to catch that, sorry, sorry."
If you wish to print, store or fiddle with the original traceback, you can get it with sys.exc_info(), and printing it like Python would is done with the 'traceback' module.
Main messages :)
import this
# btw look at this module's source :)
De-cyphered:
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than right now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Interactive Interpreter Tab Completion
try:
import readline
except ImportError:
print "Unable to load readline module."
else:
import rlcompleter
readline.parse_and_bind("tab: complete")
>>> class myclass:
... def function(self):
... print "my function"
...
>>> class_instance = myclass()
>>> class_instance.<TAB>
class_instance.__class__ class_instance.__module__
class_instance.__doc__ class_instance.function
>>> class_instance.f<TAB>unction()
You will also have to set a PYTHONSTARTUP environment variable.
Nested list comprehensions and generator expressions:
[(i,j) for i in range(3) for j in range(i) ]
((i,j) for i in range(4) for j in range(i) )
These can replace huge chunks of nested-loop code.
Operator overloading for the set builtin:
>>> a = set([1,2,3,4])
>>> b = set([3,4,5,6])
>>> a | b # Union
{1, 2, 3, 4, 5, 6}
>>> a & b # Intersection
{3, 4}
>>> a < b # Subset
False
>>> a - b # Difference
{1, 2}
>>> a ^ b # Symmetric Difference
{1, 2, 5, 6}
More detail from the standard library reference: Set Types

Categories

Resources