I was hoping to use the colon operator with my deque but it didn't seem to work the same as a list.
I was trying something like:
myDeque = deque([0,1,2,3,4,5])
myDequeFunction(myDeque[3:])
This is the error I recieved:
"TypeError: sequence index must be integer, not 'slice'"
What is the best way to do array slicing with deques?
Iterating is probably faster than brute-force methods (note: unproven) due to the nature of a deque.
>>> myDeque = collections.deque([0,1,2,3,4,5])
>>> list(itertools.islice(myDeque, 3, sys.maxint))
[3, 4, 5]
deque objects don't support slicing themselves, but you can make a new deque:
sliced_deque = deque(list(old_deque)[3:])
collections.deque objects don't support slicing. It'd be more straightforward to make a new one.
n_deque = deque(list(d)[3:])
Related
I've just joined two arrays of unequal length together with the command:
allorders = map(None,todayorders, lastyearorders)
where "none" is given where today orders fails to have a value (as the todayorders array is not as long).
However, when I try to pass the allorders array into a matplotlib bar chart:
p10= plt.bar(ind, allorders[9], width, color='#0000DD', bottom=allorders[8])
..I get the following error:
TypeError: unsupported operand type(s) for +=: 'int' and 'NoneType'
So, is there a way for matplotlib to accept none datatypes? if not, how do I replace the 'Nones' with zeroes in my allorders array?
If you can, as I am a Python newbie (coming over from the R community), please provide detailed code from start to finish that I can use/test.
Use a list comprehension:
allorders = [i if i[0] is not None else (0, i[1]) for i in allorders]
With numpy:
import numpy as np
allorders = np.array(allorders)
This creates an arrray of objects due to the Nones. We can replace them with zeros:
allorders[allorders == None] = 0
Then convert the array to the proper type:
allorders.astype(int)
Since it sounds like you want this all to be in numpy, the direct answer to your question is really just an aside, and the right answer doesn't being until the "Of course…" paragraph.
If you think about it, you're using map with a None first parameter as a zip_longest, because Python doesn't have a zip_longest. But it does have one, in itertools—and it allows you to specify a custom fillvalue. So, you can do this all in one step with izip_longest:
>>> import itertools
>>> todayorders = [1, 2]
>>> lastyearorders = [1, 2, 3]
>>> allorders = itertools.izip_longest(todayorders, lastyearorders, fillvalue=0)
>>> list(allorders)
[(1, 1), (2, 2), (0, 3)]
This only fills in 0 for the Nones that show up as extra values for the shorter list; if you want to replace every None with a 0, you have to do it Martijn Pieters's way. But I think this is what you want.
Also, note that list(allorders) at the end: izip_longest, like most things in itertools, returns an iterator, not a list. Or, in terms you might be more familiar with, it returns a "lazy" sequence rather than a "strict" one. If you're just going to iterate over the result, that's actually better, but if you need to use it with some function that requires a list (like printing it out in human-readable form—or accessing allorders[9], as in your example), you need to explicitly convert it first.
If you actually want a numpy.array rather than a list, you can get there directly, without going through a list first. (If all you're ever going to do with it is matplotlib it, you probably do want an array.) The clearest way is to just use np.fromiter(allorders) instead of list(allorders). You might want to pass an explicit dtype=int (or whatever's appropriate). And, if you know the size (which you do—it's max(len(todayorders), len(lastyearorders))), in some cases it's faster or simpler to pass an explicit count as well.
Of course if any of the numpy stuff sounds appealing, you probably should stay within numpy in the first place, instead of using map or izip_longest:
>>> todayorders.resize(lastyearorders.shape)
>>> allorders = np.vstack(todayorders, lastyearorders).transpose()
Unfortunately, that mutates todayorders, and as far as I know, the equivalent immutable function numpy.resize doesn't give you any way to "zero-extend", but instead repeats the values. Hopefully I'm wrong and someone will suggest the easy way, but otherwise, you have to do it explicitly:
>>> extrazeros = np.zeros(len(lastyearorders) - len(todayorders), dtype=int)
>>> allorders = np.vstack(np.concatenate((todayorders, extrazeros)), lastyearorders)
>>> allorders = allorders.transpose()
array([[ 1, 1],
[ 2, 2],
[ 0, 3]])
Of course if you do a lot of that, I'd write a zeroextend function that takes a pair of arrays and extends one to match the other (or, if you're not just dealing with 1D, extends the shorter one on each axis to make the other).
At any rate, aside from being faster and using less temporary memory than using map, izip_longest, etc., this also means that you end up with a final array with the right dtype (int rather than object)—which means your result also uses less long-term memory, and everything you do from then on will also be faster and use less temporary memory.
For completeness: It is possible to have pyplot handle None values, but I don't think it's what you want. For example, you can pass it a Transform object whose transform method converts None to 0. But this will be effectively the same as Martijn Pieters's answer but much more verbose, and there's no advantage at all unless you need to plot tons of such arrays.
Suppose I have a variable S with the string "1:3" (or for that matter, "1", or "1:" or ":3") and I want to use that as a slice specifier on list L. You cannot simply do L[S] since the required args for a slice are "int:int".
Now, I current have some ugly code that parses S into its two constituent ints and deals with all the edge cases (4 of them) to come up with the correct slice access but this is just plain ugly and unpythonic.
How do I elegantly take string S and use it as my slice specifier?
This can be done without much hacking by using a list comprehension. We split the string on :, passing the split items as arguments to the slice() builtin. This allows us to quite nicely produce the slice in one line, in a way which works in every case I can think of:
slice(*[int(i.strip()) if i else None for i in string_slice.split(":")])
By using the slice() builtin, we neatly avoid having to deal with too many edge cases ourselves.
Example usage:
>>> some_list = [1, 2, 3]
>>> string_slice = ":2"
>>> some_list[slice(*[int(i.strip()) if i else None for i in string_slice.split(":")])]
[1, 2]
>>> string_slice = "::-1"
>>> some_list[slice(*[int(i.strip()) if i else None for i in string_slice.split(":")])]
[3, 2, 1]
Here's another solution
eval("L[%s]" % S)
warning - It's not safe if S is coming from an external(unreliable) source.
.append
Function adds elements to the list.
How can I add elements to the list? In reverse? So that index zero is new value, and the old values move up in index?
What append does
[a,b,c,d,e]
what I would like.
[e,d,c,b,a]
Thank you very much.
Suppose you have a list a, a = [1, 2, 3]
Now suppose you wonder what kinds of things you can do to that list:
dir(a)
Hmmmm... wonder what this insert thingy does...
help(a.insert)
Insert object before index, you say? Why, that sounds a lot like what I want to do! If I want to insert something at the beginning of the list, that would be before index 0. What object do I want to insert? Let's try 7...
a.insert(0, 7)
print a
Well, look at that 7 right at the front of the list!
TL;DR: dir() will let you see what's available, help() will show you how it works, and then you can play around with it and see what it does, or Google up some documentation since you now know what the feature you want is called.
It would be more efficient to use a deque(double-ended queue) for this. Inserting at index 0 is extremely costly in lists since each element must be shifted over which requires O(N) running time, in a deque the same operation is O(1).
>>> from collections import deque
>>> x = deque()
>>> x.appendleft('a')
>>> x.appendleft('b')
>>> x
deque(['b', 'a'])
Use somelist.insert(0, item) to place item at the beginning of somelist, shifting all other elements down. Note that for large lists this is a very expensive operation. Consider using deque instead if you will be adding items to or removing items from both ends of the sequence.
Using Python's list insert command with 0 for the position value will insert the value at the head of the list, thus inserting in reverse order:
your_list.insert(0, new_item)
You can do
your_list=['New item!!']+your_list
But the insert method works as well.
lst=["a","b","c","d","e","f"]
lst_rev=[]
lst_rev.append(lst[::-1])
print(lst_rev)
Here's an example of how to add elements in a list in reverse order:
liste1 = [1,2,3,4,5]
liste2 = list()
for i in liste1:
liste2.insert(0,i)
Use the following (assuming x is what you want to prepend):
your_list = [x] + your_list
or:
your_list.insert(0, x)
This question already has answers here:
Why do these list operations (methods: clear / extend / reverse / append / sort / remove) return None, rather than the resulting list?
(6 answers)
Closed 5 months ago.
I'm having an issue considering the built-in Python List-methods.
As I learned Python, I always thought Python mutators, as any value class mutators should do, returned the new variable it created.
Take this example:
a = range(5)
# will give [0, 1, 2, 3, 4]
b = a.remove(1)
# as I learned it, b should now be [0, 2, 3, 4]
# what actually happens:
# a = [0, 2, 3, 4]
# b = None
The main problem with this list mutator not returning a new list, is that you cannot to multiple mutations subsequently.
Say I want a list ranging from 0 to 5, without the 2 and the 3.
Mutators returning new variables should be able to do it like this:
a = range(5).remove(2).remove(3)
This sadly isn't possible, as range(5).remove(2) = None.
Now, is there a way to actually do multiple mutations on lists like I wanna do in my example? I think even PHP allows these types of subsequent mutations with Strings.
I also can't find a good reference on all the built-in Python functions. If anyone can find the actual definition (with return values) of all the list mutator methods, please let me know. All I can find is this page: http://docs.python.org/tutorial/datastructures.html
Rather than both mutating and returning objects, the Python library chooses to have just one way of using the result of a mutator. From import this:
There should be one-- and preferably only one --obvious way to do it.
Having said that, the more usual Python style for what you want to do is using list comprehensions or generator expressions:
[x for x in range(5) if x != 2 and x != 3]
You can also chain these together:
>>> [x for x in (x for x in range(5) if x != 2) if x != 3]
[0, 1, 4]
The above generator expression has the added advantage that it runs in O(n) time because Python only iterates over the range() once. For large generator expressions, and even for infinite generator expressions, this is advantageous.
Many methods of list and other mutable types intentionally return None so that there is no question in your mind as to whether you are creating a new object or mutating an existing object. The only thing that could be happening is mutation since, if a new object were created, it would have to be returned by the method, and it is not returned.
As you may have noticed, the methods of str that edit the string do return the new string, because strings are immutable and a new string is always returned.
There is of course nothing at all keeping you from writing a list subclass that has the desired behavior on .append() et al, although this seems like rather a heavy hammer to swing merely to allow you to chain method calls. Also, most Python programmers won't expect that behavior, making your code less clear.
In Python, essentially all methods that mutate the object return None.
That's so you don't accidentally think you've got a new object as a result.
You mention
I think even PHP allows these types of subsequent mutations with Strings.
While I don't remember about PHP, with string manipulations, you can chain method calls, because strings are immutable, so all string methods return a new string, since they don't (can't) change the one you call them on.
>>> "a{0}b".format(3).center(5).capitalize().join('ABC').swapcase()
'a A3B b A3B c'
Neither Python or php have built-in "mutators". You can simply write multiple lines, like
a = list(range(5))
a.remove(2)
a.remove(3)
If you want to generate a new list, copy the old one beforehand:
a = list(range(5))
b = a[:]
b.remove(2)
Note that in most cases, a singular call to remove indicates you should have used a set in the first place.
To remove multiple mutations on lists as you want to do on your example, you can do :
a = range(5)
a = [i for j, i in enumerate(a) if j not in [2, 3]]
a will be [0, 1, 4]
I am a newbie to Python and trying out different ways to optimize and simplify my code.
I have a list of arrays(necessarily in this format) initially empty, which I need to update with arrays, making sure that duplicate entries are not added.
Right now I am doing it the following way, which is the only thing i tried out which works:
if len(where(((array(self.pop_next)-(self.pop[self.top_indv_indx[i]]))==0).sum(1)==len((self.pop[self.top_indv_indx[i]])))[0])<=0):
self.pop_next.append(self.pop[self.top_indv_indx[i]])
where self.pop_next is my list of arrays and self.pop[self.top_indv_indx[i]] is the array to be added.
I know this Unpythonic and guess that there are much better simple ways to do the same.
Please Help
Edit: I see from your comment that you're using numpy arrays. I've never used numpy so I have no idea how they work with sets.
One option would be to use a set. Sets are like lists but they are unordered and only allow each item to be added once:
>>> s = set()
>>> s.add(1)
>>> s.add(2)
>>> s.add(2)
>>> s.add(2)
>>> s
set([1, 2])
However, you'll run into problems if you try to add a list to a set:
>>> s.add(['my','list'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
An item must be hashable to add to set, and a list can't be hashable as it can't have an unchanging hash value since it can be modified at any time by adding or removing values.
If you don't need the lists you are checking to be mutable you can convert them to tuples which are fixed and so hashable and so set-friendly:
>>> mylist = ['my','list']
>>> s = set()
>>> s.add(tuple(mylist))
>>> s.add(tuple(mylist))
>>> s
set([('my', 'list')])
You may want to try with numpy.all(array1 == array2) as condition for an individual array comparison.
Extension in edit:
To loop over the list, you may use the following:
if all((numpy.all(array_to_add != a) for a in array_list)):
array_list.append(array_to_add)
This compares array_to_add to all elements of array_list by value. Note that all here is __builtin__.all, in contrast to numpy.all. If you did from numpy import * before, this will not work. Use import numpy instead and call functions by full name as in the example above.
If it is ok to compare by object (i.e. two arrays are only the same if the are the exact same object in memory), use the following simpler variant:
if array_to_add is not in array_list:
array_list.append(array_to_add)