I've just joined two arrays of unequal length together with the command:
allorders = map(None,todayorders, lastyearorders)
where "none" is given where today orders fails to have a value (as the todayorders array is not as long).
However, when I try to pass the allorders array into a matplotlib bar chart:
p10= plt.bar(ind, allorders[9], width, color='#0000DD', bottom=allorders[8])
..I get the following error:
TypeError: unsupported operand type(s) for +=: 'int' and 'NoneType'
So, is there a way for matplotlib to accept none datatypes? if not, how do I replace the 'Nones' with zeroes in my allorders array?
If you can, as I am a Python newbie (coming over from the R community), please provide detailed code from start to finish that I can use/test.
Use a list comprehension:
allorders = [i if i[0] is not None else (0, i[1]) for i in allorders]
With numpy:
import numpy as np
allorders = np.array(allorders)
This creates an arrray of objects due to the Nones. We can replace them with zeros:
allorders[allorders == None] = 0
Then convert the array to the proper type:
allorders.astype(int)
Since it sounds like you want this all to be in numpy, the direct answer to your question is really just an aside, and the right answer doesn't being until the "Of course…" paragraph.
If you think about it, you're using map with a None first parameter as a zip_longest, because Python doesn't have a zip_longest. But it does have one, in itertools—and it allows you to specify a custom fillvalue. So, you can do this all in one step with izip_longest:
>>> import itertools
>>> todayorders = [1, 2]
>>> lastyearorders = [1, 2, 3]
>>> allorders = itertools.izip_longest(todayorders, lastyearorders, fillvalue=0)
>>> list(allorders)
[(1, 1), (2, 2), (0, 3)]
This only fills in 0 for the Nones that show up as extra values for the shorter list; if you want to replace every None with a 0, you have to do it Martijn Pieters's way. But I think this is what you want.
Also, note that list(allorders) at the end: izip_longest, like most things in itertools, returns an iterator, not a list. Or, in terms you might be more familiar with, it returns a "lazy" sequence rather than a "strict" one. If you're just going to iterate over the result, that's actually better, but if you need to use it with some function that requires a list (like printing it out in human-readable form—or accessing allorders[9], as in your example), you need to explicitly convert it first.
If you actually want a numpy.array rather than a list, you can get there directly, without going through a list first. (If all you're ever going to do with it is matplotlib it, you probably do want an array.) The clearest way is to just use np.fromiter(allorders) instead of list(allorders). You might want to pass an explicit dtype=int (or whatever's appropriate). And, if you know the size (which you do—it's max(len(todayorders), len(lastyearorders))), in some cases it's faster or simpler to pass an explicit count as well.
Of course if any of the numpy stuff sounds appealing, you probably should stay within numpy in the first place, instead of using map or izip_longest:
>>> todayorders.resize(lastyearorders.shape)
>>> allorders = np.vstack(todayorders, lastyearorders).transpose()
Unfortunately, that mutates todayorders, and as far as I know, the equivalent immutable function numpy.resize doesn't give you any way to "zero-extend", but instead repeats the values. Hopefully I'm wrong and someone will suggest the easy way, but otherwise, you have to do it explicitly:
>>> extrazeros = np.zeros(len(lastyearorders) - len(todayorders), dtype=int)
>>> allorders = np.vstack(np.concatenate((todayorders, extrazeros)), lastyearorders)
>>> allorders = allorders.transpose()
array([[ 1, 1],
[ 2, 2],
[ 0, 3]])
Of course if you do a lot of that, I'd write a zeroextend function that takes a pair of arrays and extends one to match the other (or, if you're not just dealing with 1D, extends the shorter one on each axis to make the other).
At any rate, aside from being faster and using less temporary memory than using map, izip_longest, etc., this also means that you end up with a final array with the right dtype (int rather than object)—which means your result also uses less long-term memory, and everything you do from then on will also be faster and use less temporary memory.
For completeness: It is possible to have pyplot handle None values, but I don't think it's what you want. For example, you can pass it a Transform object whose transform method converts None to 0. But this will be effectively the same as Martijn Pieters's answer but much more verbose, and there's no advantage at all unless you need to plot tons of such arrays.
Related
Suppose I have a variable S with the string "1:3" (or for that matter, "1", or "1:" or ":3") and I want to use that as a slice specifier on list L. You cannot simply do L[S] since the required args for a slice are "int:int".
Now, I current have some ugly code that parses S into its two constituent ints and deals with all the edge cases (4 of them) to come up with the correct slice access but this is just plain ugly and unpythonic.
How do I elegantly take string S and use it as my slice specifier?
This can be done without much hacking by using a list comprehension. We split the string on :, passing the split items as arguments to the slice() builtin. This allows us to quite nicely produce the slice in one line, in a way which works in every case I can think of:
slice(*[int(i.strip()) if i else None for i in string_slice.split(":")])
By using the slice() builtin, we neatly avoid having to deal with too many edge cases ourselves.
Example usage:
>>> some_list = [1, 2, 3]
>>> string_slice = ":2"
>>> some_list[slice(*[int(i.strip()) if i else None for i in string_slice.split(":")])]
[1, 2]
>>> string_slice = "::-1"
>>> some_list[slice(*[int(i.strip()) if i else None for i in string_slice.split(":")])]
[3, 2, 1]
Here's another solution
eval("L[%s]" % S)
warning - It's not safe if S is coming from an external(unreliable) source.
I was hoping to use the colon operator with my deque but it didn't seem to work the same as a list.
I was trying something like:
myDeque = deque([0,1,2,3,4,5])
myDequeFunction(myDeque[3:])
This is the error I recieved:
"TypeError: sequence index must be integer, not 'slice'"
What is the best way to do array slicing with deques?
Iterating is probably faster than brute-force methods (note: unproven) due to the nature of a deque.
>>> myDeque = collections.deque([0,1,2,3,4,5])
>>> list(itertools.islice(myDeque, 3, sys.maxint))
[3, 4, 5]
deque objects don't support slicing themselves, but you can make a new deque:
sliced_deque = deque(list(old_deque)[3:])
collections.deque objects don't support slicing. It'd be more straightforward to make a new one.
n_deque = deque(list(d)[3:])
This question already has answers here:
Why do these list operations (methods: clear / extend / reverse / append / sort / remove) return None, rather than the resulting list?
(6 answers)
Closed 5 months ago.
I'm having an issue considering the built-in Python List-methods.
As I learned Python, I always thought Python mutators, as any value class mutators should do, returned the new variable it created.
Take this example:
a = range(5)
# will give [0, 1, 2, 3, 4]
b = a.remove(1)
# as I learned it, b should now be [0, 2, 3, 4]
# what actually happens:
# a = [0, 2, 3, 4]
# b = None
The main problem with this list mutator not returning a new list, is that you cannot to multiple mutations subsequently.
Say I want a list ranging from 0 to 5, without the 2 and the 3.
Mutators returning new variables should be able to do it like this:
a = range(5).remove(2).remove(3)
This sadly isn't possible, as range(5).remove(2) = None.
Now, is there a way to actually do multiple mutations on lists like I wanna do in my example? I think even PHP allows these types of subsequent mutations with Strings.
I also can't find a good reference on all the built-in Python functions. If anyone can find the actual definition (with return values) of all the list mutator methods, please let me know. All I can find is this page: http://docs.python.org/tutorial/datastructures.html
Rather than both mutating and returning objects, the Python library chooses to have just one way of using the result of a mutator. From import this:
There should be one-- and preferably only one --obvious way to do it.
Having said that, the more usual Python style for what you want to do is using list comprehensions or generator expressions:
[x for x in range(5) if x != 2 and x != 3]
You can also chain these together:
>>> [x for x in (x for x in range(5) if x != 2) if x != 3]
[0, 1, 4]
The above generator expression has the added advantage that it runs in O(n) time because Python only iterates over the range() once. For large generator expressions, and even for infinite generator expressions, this is advantageous.
Many methods of list and other mutable types intentionally return None so that there is no question in your mind as to whether you are creating a new object or mutating an existing object. The only thing that could be happening is mutation since, if a new object were created, it would have to be returned by the method, and it is not returned.
As you may have noticed, the methods of str that edit the string do return the new string, because strings are immutable and a new string is always returned.
There is of course nothing at all keeping you from writing a list subclass that has the desired behavior on .append() et al, although this seems like rather a heavy hammer to swing merely to allow you to chain method calls. Also, most Python programmers won't expect that behavior, making your code less clear.
In Python, essentially all methods that mutate the object return None.
That's so you don't accidentally think you've got a new object as a result.
You mention
I think even PHP allows these types of subsequent mutations with Strings.
While I don't remember about PHP, with string manipulations, you can chain method calls, because strings are immutable, so all string methods return a new string, since they don't (can't) change the one you call them on.
>>> "a{0}b".format(3).center(5).capitalize().join('ABC').swapcase()
'a A3B b A3B c'
Neither Python or php have built-in "mutators". You can simply write multiple lines, like
a = list(range(5))
a.remove(2)
a.remove(3)
If you want to generate a new list, copy the old one beforehand:
a = list(range(5))
b = a[:]
b.remove(2)
Note that in most cases, a singular call to remove indicates you should have used a set in the first place.
To remove multiple mutations on lists as you want to do on your example, you can do :
a = range(5)
a = [i for j, i in enumerate(a) if j not in [2, 3]]
a will be [0, 1, 4]
Is there a good, succinct/built-in way to see if all the values in an iterable are zeros? Right now I am using all() with a little list comprehension, but (to me) it seems like there should be a more expressive method. I'd view this as somewhat equivalent to a memcmp() in C.
values = (0, 0, 0, 0, 0)
# Test if all items in values tuple are zero
if all([ v == 0 for v in values ]) :
print 'indeed they are'
I would expect a built-in function that does something like:
def allcmp(iter, value) :
for item in iter :
if item != value :
return False
return True
Does that function exist in python and I'm just blind, or should I just stick with my original version?
Update
I'm not suggesting that allcmp() is the solution. It is an example of what I think might be more meaningful. This isn't the place where I would suggest new built-ins for Python.
In my opinion, all() isn't that meaningful. It doesn't express what "all" is checking for. You could assume that all() takes an iterable, but it doesn't express what the function is looking for (an iterable of bools that tests all of them for True). What I'm asking for is some function like my allcmp() that takes two parameters: an iterable and a comparison value. I'm asking if there is a built-in function that does something similar to my made up allcmp().
I called mine allcmp() because of my C background and memcmp(), the name of my made up function is irrelevant here.
Use generators rather than lists in cases like that:
all(v == 0 for v in values)
Edit:
all is standard Python built-in. If you want to be efficient Python programmer you should know probably more than half of them (http://docs.python.org/library/functions.html). Arguing that alltrue is better name than all is like arguing that C while should be call whiletrue. Is subjective, but i think that most of the people prefer shorter names for built-ins. This is because you should know what they do anyway, and you have to type them a lot.
Using generators is better than using numpy because generators have more elegant syntax. numpy may be faster, but you will benefit only in rare cases (generators like showed are fast, you will benefit only if this code is bottleneck in your program).
You probably can't expect nothing more descriptive from Python.
PS. Here is code if you do this in memcpm style (I like all version more, but maybe you will like this one):
list(l) == [0] * len(l)
If you know that the iterable will contain only integers then you can just do this:
if not any(values):
# etc...
If values is a numpy array you can write
import numpy as np
values = np.array((0, 0, 0, 0, 0))
all(values == 0)
The any() function may be the most simple and easy way to achieve just that. If the iterable is empty,e.g. all elements are zero, it will return False.
values = (0, 0, 0, 0, 0)
print (any(values)) # return False
The built-in set is given an iterable and returns a collection (set) of unique values.
So it can be used here as:
set(it) == {0}
assuming it is the iterable
{0} is a set containing only zero
More info on python set-types-set-frozenset here in docs.
I prefer using negation:
all(not v for v in values)
This question already has answers here:
How do I get the number of elements in a list (length of a list) in Python?
(11 answers)
Closed 5 years ago.
How can I count the number of elements in an array, because contrary to logic array.count(string) does not count all the elements in the array, it just searches for the number of occurrences of string.
The method len() returns the number of elements in the list.
Syntax:
len(myArray)
Eg:
myArray = [1, 2, 3]
len(myArray)
Output:
3
len is a built-in function that calls the given container object's __len__ member function to get the number of elements in the object.
Functions encased with double underscores are usually "special methods" implementing one of the standard interfaces in Python (container, number, etc). Special methods are used via syntactic sugar (object creation, container indexing and slicing, attribute access, built-in functions, etc.).
Using obj.__len__() wouldn't be the correct way of using the special method, but I don't see why the others were modded down so much.
If you have a multi-dimensional array, len() might not give you the value you are looking for. For instance:
import numpy as np
a = np.arange(10).reshape(2, 5)
print len(a) == 2
This code block will return true, telling you the size of the array is 2. However, there are in fact 10 elements in this 2D array. In the case of multi-dimensional arrays, len() gives you the length of the first dimension of the array i.e.
import numpy as np
len(a) == np.shape(a)[0]
To get the number of elements in a multi-dimensional array of arbitrary shape:
import numpy as np
size = 1
for dim in np.shape(a): size *= dim
Or,
myArray.__len__()
if you want to be oopy; "len(myArray)" is a lot easier to type! :)
Before I saw this, I thought to myself, "I need to make a way to do this!"
for tempVar in arrayName: tempVar+=1
And then I thought, "There must be a simpler way to do this." and I was right.
len(arrayName)