Related
Consider the following python code:
class View:
pass
v = View()
print(list(v))
When run, this gives the following error:
Traceback (most recent call last):
File "testing.py", line 35, in <module>
print(list(v))
TypeError: 'View' object is not iterable
Makes sense. Let's add the __iter__ to View
class View:
def __iter__(self):
for i in range(5):
yield i
This works now, and correctly produces [0, 1, 2, 3, 4]
However, consider what happens if View has __len__ method defined:
class View:
def __len__(self):
print("len called")
return 2
def __iter__(self):
for i in range(5):
yield i
v = View()
print(list(v))
When run, this produces:
len called
len called
[0, 1, 2, 3, 4]
Not only __len__ is called twice, it seems it is also not "respected" (I am returning 2 as length, but my __iter__ produces 5 values, and the list correctly composes of 5 values)
What is going on? (Note that this is not an idle curiosity. In my application, I have an expensive __len__ method and calling it twice is slowing down my initialization time)
Per Why does list ask about __len__? the __len__ method is called to get an approximation of how much space needs to be allocated to the list. Note it's just an approximation, whether you're iterating using __iter__ or __getitem__ that length is not binding:
>>> class GetItemIter:
... def __len__(self):
... return 2
... def __getitem__(self, index):
... return [0, 1, 2][index]
...
>>> list(GetItemIter())
[0, 1, 2]
>>> class IterIter:
... def __len__(self):
... return 2
... def __iter__(self):
... return iter([0, 1, 2])
...
>>> list(IterIter())
[0, 1, 2]
From Python 3.8 the list constructor is checking the length twice. This is in the bug tracker, but may be considered an implementation detail:
The number of times we can __len__ on the constructor is an
implementation detail. The reason is called now twice is because there
is an extra check for the preallocation logic...
I am trying to add items into any iterable in a type-agnostic way, but not all python iterables seem to have a standard interface/method for doing so.
lists use append(), sets use add(), etc.
Basically, something like this -
def add(anylist, item):
# adds 'item' into 'anylist'
# possibly return 'newlist' if 'anylist' is immutable (such as tuple, str)
return newlist
x = add([1,2,3], 4) # x is now [1,2,3,4]
y = add('abcd', 'e') # y is now "abcde"
z = add((1,2,3), 4) # z is now (1,2,3,4)
Currently, I am making do by handling individual types on a case-by-case basis like so-
>>> def add(anylist, item):
... if isinstance(anylist, list):
... anylist.append(item)
... elif isinstance(anylist, set):
... anylist.add(item)
... elif isinstance(anylist, str):
... anylist += item
... elif isinstance(anylist, tuple):
... anylist += (item,)
... return anylist
...
>>> print add([1,2,3], 4)
[1, 2, 3, 4]
>>> print add((1,2,3), 4)
(1, 2, 3, 4)
>>> print add("123", '4')
1234
>>> print add({1,2,3}, 4)
set([1, 2, 3, 4])
This is obviously sub-optimal because it doesn't work with every iterable.
Am I missing something?
I tried to make use of python's concatenate(+) operator to achieve this, but that didn't help either.
>>> x = [1,2,3]
>>> x += [4]
>>> x
[1, 2, 3, 4]
>>> y = "123"
>>> y += "4"
>>> y
'1234'
as there's no easy way to convert 'item' to an iterable type either.
For eg, the following -
item = '4'
x = '123'
x += str(item) # x = '1234'
works, but lists do not behave the same way -
>>> item = 4
>>> x = [1,2,3]
>>> x += list(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
There's no generic way to take one sequence an automatically create another sequence of the same type with one extra item. That's because not all sequences allow arbitrary values. For instance, a Python 3 range object (or an xrange in Python 2) computes its values on the fly, so there's no way to add a new value of your choice.
However, if you're only going to iterate over your sequence after adding the value to the end, you may be able to get what you want using itertools.chain, which returns an iterator.
import itertools
def add(iterable, value):
return itertools.chain(iterable, [value])
You won't be able to pass the iterator directly to a print statement, but you can perhaps use some other kind of container, like a list to hold the values so you can print them.
print list(add((1,2,3), 4)) # prints [1, 2, 3, 4], doesn't try to produce a tuple
You can do this by overloading operators in iterable classes, but the effort probably outweighs the benefit.
In any case, if you want to use + as a way of extending the list, you can create your own container class inheriting from list, and overload the .__add__ method:
class my_list(list):
def __add__(self, o):
self.__getitem__(self).extend(o)
a = my_list()
a += [1] #a=[1]
a += [2] #a=[1,2]
Similarly, you extend all iterables.
Your proposed method of if-else is probably easier to maintain with language updates, so worth keeping this in mind if you decide to start extending the inbuilt classes.
I wrote a small Programm in python but it don't work like expected.
Here's the code:
puzzle = [8, 7, 5, 4, 1, 2, 3, 0, 6]
def count(p):
p[0] += 1
return p
def main(p):
print(p)
l = count(p)
print(l)
print(p)
b1 = main(puzzle)
I expect that print(p) will be different from print(l), but the result of both is the same, it's the result that print(l) should have. But p did change also, however I would need it to be unchanged… Is this a special python behavior? Is there something I missed?
I also tried to change the variable names in the functions, but that didn't help.
I restarted the Compiler, but that didn't help either.
Is there a solution to store a function output and than call the function again without let the function change the given parameters?
So that l will be the result after the calculation and p will stay the value before?
Kind Regards,
Joh.
You are passing a List parameter. Parameter passing is Call-by-Object. Since a List is a mutable object in this situation it is similar to pass by reference and changes to your List object will persist. If you were passing an immutable, such as an Integer or String, it would be akin to pass by copy/value, and changes would not persist. E.g.:
def s2asdf(s):
s = "asdf"
s = "hello world"
s2asdf(s)
print s
... results in:
$ python example.py
hello world
The reason for this is because Python passes function parameters by reference. When you call the count function it allows the function to modify the list inside the function and the changes will be applied to the original object.
If you want to have the function not modify the list but instead return a different list, you will have to make a copy of the list either by passing a copy to the function or make a copy inside the function itself. There are many ways to copy a list in Python, but I like to use the list() function to do it.
This should fix your problem:
puzzle = [8, 7, 5, 4, 1, 2, 3, 0, 6]
def count(p):
new_list = list(p) # copy values of p to new_list
new_list[0] += 1
return new_list
def main(p):
print(p)
l = count(p)
print(l) # l is the new_list returned from count
print(p) # p stays the original value
b1 = main(puzzle)
Let's say we have a function add as follows
def add(x, y):
return x + y
we want to apply map function for an array
map(add, [1, 2, 3], 2)
The semantics are I want to add 2 to every element of the array. But the map function requires a list in the third argument as well.
Note: I am putting the add example for simplicity. My original function is much more complicated. And of course option of setting the default value of y in add function is out of question as it will be changed for every call.
One option is a list comprehension:
[add(x, 2) for x in [1, 2, 3]]
More options:
a = [1, 2, 3]
import functools
map(functools.partial(add, y=2), a)
import itertools
map(add, a, itertools.repeat(2, len(a)))
The docs explicitly suggest this is the main use for itertools.repeat:
Make an iterator that returns object over and over again. Runs indefinitely unless the times argument is specified. Used as argument to map() for invariant parameters to the called function. Also used with zip() to create an invariant part of a tuple record.
And there's no reason for pass len([1,2,3]) as the times argument; map stops as soon as the first iterable is consumed, so an infinite iterable is perfectly fine:
>>> from operator import add
>>> from itertools import repeat
>>> list(map(add, [1,2,3], repeat(4)))
[5, 6, 7]
In fact, this is equivalent to the example for repeat in the docs:
>>> list(map(pow, range(10), repeat(2)))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
This makes for a nice lazy-functional-language-y solution that's also perfectly readable in Python-iterator terms.
Use a list comprehension.
[x + 2 for x in [1, 2, 3]]
If you really, really, really want to use map, give it an anonymous function as the first argument:
map(lambda x: x + 2, [1,2,3])
Map can contain multiple arguments, the standard way is
map(add, a, b)
In your question, it should be
map(add, a, [2]*len(a))
The correct answer is simpler than you think.
Simply do:
map(add, [(x, 2) for x in [1,2,3]])
And change the implementation of add to take a tuple i.e
def add(t):
x, y = t
return x+y
This can handle any complicated use case where both add parameters are dynamic.
Sometimes I resolved similar situations (such as using pandas.apply method) using closures
In order to use them, you define a function which dynamically defines and returns a wrapper for your function, effectively making one of the parameters a constant.
Something like this:
def add(x, y):
return x + y
def add_constant(y):
def f(x):
return add(x, y)
return f
Then, add_constant(y) returns a function which can be used to add y to any given value:
>>> add_constant(2)(3)
5
Which allows you to use it in any situation where parameters are given one at a time:
>>> map(add_constant(2), [1,2,3])
[3, 4, 5]
edit
If you do not want to have to write the closure function somewhere else, you always have the possibility to build it on the fly using a lambda function:
>>> map(lambda x: add(x, 2), [1, 2, 3])
[3, 4, 5]
If you have it available, I would consider using numpy. It's very fast for these types of operations:
>>> import numpy
>>> numpy.array([1,2,3]) + 2
array([3, 4, 5])
This is assuming your real application is doing mathematical operations (that can be vectorized).
If you really really need to use map function (like my class assignment here...), you could use a wrapper function with 1 argument, passing the rest to the original one in its body; i.e. :
extraArguments = value
def myFunc(arg):
# call the target function
return Func(arg, extraArguments)
map(myFunc, itterable)
Dirty & ugly, still does the trick
I believe starmap is what you need:
from itertools import starmap
def test(x, y, z):
return x + y + z
list(starmap(test, [(1, 2, 3), (4, 5, 6)]))
def func(a, b, c, d):
return a + b * c % d
map(lambda x: func(*x), [[1,2,3,4], [5,6,7,8]])
By wrapping the function call with a lambda and using the star unpack, you can do map with arbitrary number of arguments.
You can include lambda along with map:
list(map(lambda a: a+2, [1, 2, 3]))
To pass multiple arguments to a map function.
def q(x,y):
return x*y
print map (q,range(0,10),range(10,20))
Here q is function with multiple argument that map() calls.
Make sure, the length of both the ranges i.e.
len (range(a,a')) and len (range(b,b')) are equal.
In :nums = [1, 2, 3]
In :map(add, nums, [2]*len(nums))
Out:[3, 4, 5]
Another option is:
results = []
for x in [1,2,3]:
z = add(x,2)
...
results += [f(z,x,y)]
This format is very useful when calling multiple functions.
#multi argument
def joke(r):
if len(r)==2:
x, y = r
return x + y
elif len(r)==3:
x,y,z=r
return x+y+z
#using map
print(list(map(joke,[[2,3],[3,4,5]])))
output = [6,12]
if the case like above and just want use function
def add(x,y):
ar =[]
for xx in x:
ar.append(xx+y)
return ar
print(list(map(add,[[3,2,4]],[2]))[0])
output = [5,4,6]
Note: you can modified as you want.
The documentation for the built-in type set says:
class set([iterable])
Return a new set or frozenset object
whose elements are taken from
iterable. The elements of a set must
be hashable.
That is all right but why does this work:
>>> l = range(10)
>>> s = set(l)
>>> s
set([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
And this doesn't:
>>> s.add([10])
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
s.add([10])
TypeError: unhashable type: 'list'
Both are lists. Is some magic happening during the initialization?
When you initialize a set, you provide a list of values that must each be hashable.
s = set()
s.add([10])
is the same as
s = set([[10]])
which throws the same error that you're seeing right now.
In [13]: (2).__hash__
Out[13]: <method-wrapper '__hash__' of int object at 0x9f61d84>
In [14]: ([2]).__hash__ # nothing.
The thing is that set needs its items to be hashable, i.e. implement the __hash__ magic method (this is used for ordering in the tree as far as I know). list does not implement that magic method, hence it cannot be added in a set.
In this line:
s.add([10])
You are trying to add a list to the set, rather than the elements of the list. If you want ot add the elements of the list, use the update method.
Think of the constructor being something like:
class Set:
def __init__(self,l):
for elem in l:
self.add(elem)
Nothing too interesting to be concerned about why it takes lists but on the other hand add(element) does not.
It behaves according to the documentation: set.add() adds a single element (and since you give it a list, it complains it is unhashable - since lists are no good as hash keys). If you want to add a list of elements, use set.update(). Example:
>>> s = set([1,2,3])
>>> s.add(5)
>>> s
set([1, 2, 3, 5])
>>> s.update([8])
>>> s
set([8, 1, 2, 3, 5])
s.add([10]) works as documented. An exception is raised because [10] is not hashable.
There is no magic happening during initialisation.
set([0,1,2,3,4,5,6,7,8,9]) has the same effect as set(range(10)) and set(xrange(10)) and set(foo()) where
def foo():
for i in (9,8,7,6,5,4,3,2,1,0):
yield i
In other words, the arg to set is an iterable, and each of the values obtained from the iterable must be hashable.