I'm new to python and I am stuck. I've been playing with this a lot. I am trying to get my 3 lists to join and when I do python says that the new list only contains 1 item. How do I get them to merge completely?
here is the code I have now :
(where avg[] is some array containing lots of data)
q=avg[0:40]
p=avg[53:70]
u=avg[95:145]
pu=p+u
NF=[numpy.append(q,pu)]
>>>len(NF)
>>1
but the actual length of all the items is 107.
Please help
If avg is list, then q, p and u are slices of a list and therefore will be lists too. In that case, you could concatenate the lists using addition:
q+p+u
If you want a NumPy array, you could use np.concatenate:
In [48]: avg = np.arange(20)
In [49]: q = avg[0:4]
In [50]: p = avg[5:7]
In [51]: u = avg[9:14]
In [52]: np.concatenate([q,p,u])
Out[52]: array([ 0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13])
I made the arrays smaller so the result is easier to check.
Other alternatives include np.hstack and np.r_:
In [53]: np.hstack([q,p,u])
Out[53]: array([ 0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13])
In [54]: np.r_[q,p,u]
Out[54]: array([ 0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13])
In the above examples, q, p, and u may be NumPy arrays or Python lists. In each case a NumPy array is returned.
You put your array in another array. try this:
NF=numpy.append(q,pu)
q=avg[0:40]
p=avg[53:70]
u=avg[95:145]
pu=p+u
NF=[numpy.append(q,pu)] #problem right here just do NF = numpy.append(q,pu)
>>>len(NF)
>>1
Okay this is going to be horrifically slow, maybe numpy has its own way of doing it, but using python way and provided that you have one-dimensional arrays, try this:
from itertools import chain
items = chain.from_iterable([avg[0:40], avg[53:70], avg[95:145]])
The statements about will return a generator, that can be converted into a list that reports you its length.
item_list = [x for x in items]
len(item_list)
Related
I have a list of indices and a list of data. The list of indices says which elements should be removed from the list of data. I would like to use the list of indices efficiently, i.e. without loops. Is there a faster way to remove these elements?
Assuming you use numpy, np.delete does exactly what you want:
>>> a = np.array([1, 4, 9, 16, 25, 36])
>>> np.delete(a, [1, 2, 5])
array([ 1, 16, 25])
I have an numpy array a that I would like to replace some elements. I have the value of the new elements in a tuple/numpy array and the indexes of the elements of a that needs to be replaced in another tuple/numpy array. Below is an example of using python to do what I want.How do I do this efficiently in NumPy?
Example script:
a = np.arange(10)
print( f'a = {a}' )
newvalues = (10, 20, 35)
indexes = (2, 4, 6)
for n,i in enumerate( indexes ):
a[i]=newvalues[n]
print( f'a = {a}' )
Output:
a = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
a = array([ 0, 1, 10, 3, 20, 5, 35, 7, 8, 9])
I tried a[indexes]=newvalues but got IndexError: too many indices for array: array is 1-dimensional, but 3 were indexed
The list of indices indicating which elements you want to replace should be a Python list (or similar type), not a tuple. Different items in the selection tuple indicate that they should be selected from different axis dimensions.
Therefore, a[(2, 4, 6)] is the same as a[2, 4, 6], which is interpreted as the value at index 2 in the first dimension, index 4 in the second dimension, and index 6 in the third dimension.
The following code works correctly:
indexes = [2, 4, 6]
a[indexes] = newvalues
See also the page on Indexing from the numpy documentation, specifically the second 'Note' block in the introduction as well as the first 'Warning' under Advanced Indexing:
In Python, x[(exp1, exp2, ..., expN)] is equivalent to x[exp1, exp2, ..., expN]; the latter is just syntactic sugar for the former.
The definition of advanced indexing means that x[(1,2,3),] is fundamentally different than x[(1,2,3)]. The latter is equivalent to x[1,2,3] which will trigger basic selection while the former will trigger advanced indexing. Be sure to understand why this occurs.
If I have one numpy array that contains a set of indices, how would I apply those indices to a list of data, which in turn will extract the corresponding data values and append these into a new numpy array?
The code I tried to use was this:
second_index = np.zeros(len(first_index))
for i in range(len(first_index)):
second_index[i] = data[first_index[i]]
print(second_index)
Both first_index and second_index are 1D numpy arrays, and I would expect to see the same number of elements in both first_index and second_index. However, I receieve the following error message
TypeError: list indices must be integers or slices, not numpy.float64
Should I be using nested for-loops instead and perform the loop over two different ranges?
Thanks for any suggestions!
You can directly index using numpy arrays:
>>> import numpy as np
>>> data = np.array([2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36])
>>> first_index = np.array([7, 17])
>>> second_index = data[first_index]
>>> second_index
array([16, 36])
ex= [0.5,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]
newarr=[]
for i in range(len(ex)):
newarr.append(ex[i])
print(newarr)
[0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
Is this you looking for..?
Edited for the confusion in the problem, thanks for the answers!
My original problem was that I have a list [1,2,3,4,5,6,7,8], and I want to select every chunk of size x with gap of one. So if I want to select select every other chunk of size 2, the outcome would be [1,2,4,5,7,8]. A chunk size of three would give me [1,2,3,5,6,7].
I've searched a lot on slicing and I couldn't find a way to select chunks instead of element. Make multiple slice operations then join and sort seems a little too expensive. The input can either be a python list or numpy ndarray. Thanks in advance.
To me it seems, you want to skip one element between chunks until the end of the input list or array.
Here's one approach based on np.delete that deletes that single elements squeezed between chunks -
out = np.delete(A,np.arange(len(A)/(x+1))*(x+1)+x)
Here's another approach based on boolean-indexing -
L = len(A)
avoid_idx = np.arange(L/(x+1))*(x+1)+x
out = np.array(A)[~np.in1d(np.arange(L),avoid_idx)]
Sample run -
In [98]: A = [51,42,13,34,25,68,667,18,55,32] # Input list
In [99]: x = 2
# Thus, [51,42,13,34,25,68,667,18,55,32]
^ ^ ^ # Skip these
In [100]: np.delete(A,np.arange(len(A)/(x+1))*(x+1)+x)
Out[100]: array([ 51, 42, 34, 25, 667, 18, 32])
In [101]: L = len(A)
...: avoid_idx = np.arange(L/(x+1))*(x+1)+x
...: out = np.array(A)[~np.in1d(np.arange(L),avoid_idx)]
...:
In [102]: out
Out[102]: array([ 51, 42, 34, 25, 667, 18, 32])
First off, you can create an array of indices then use np.in1d() function in order to extract the indices that should be omit then with a simple not operator get the indices that must be preserve. And at last pick up them using a simple boolean indexing:
>>> a = np.array([1,2,3,4,5,6,7,8])
>>> range_arr = np.arange(a.size)
>>>
>>> a[~np.in1d(range_arr,range_arr[2::3])]
array([1, 2, 4, 6, 8])
General approach:
>>> range_arr = np.arange(np_array.size)
>>> np_array[~np.in1d(range_arr,range_arr[chunk::chunk+1])]
Using a pure python solution:
This assumes the desired items are: [yes, yes, no, yes, yes, no, ...]
Quicker to code, slower to run:
data = [1, 2, 3, 4, 5, 6, 7, 8]
filtered = [item for i, item in enumerate(data) if i % 3 != 2]
assert filtered == [1, 2, 4, 5, 7, 8]
Slightly slower to write, but faster to run:
from itertools import cycle, compress
data = [1, 2, 3, 4, 5, 6, 7, 8]
selection_criteria = [True, True, False]
filtered = list(compress(data, cycle(selection_criteria)))
assert filtered == [1, 2, 4, 5, 7, 8]
The second example runs in 66% of the time the first example does, and is also clearer and easier to change the selection criteria
A simple list solution
>> ll = [1,2,3,4,5,6,7,8]
>> list(itertools.chain(*zip(ll[::3],ll[1::3])))
[1, 2, 4, 5, 7, 8]
At least for this case of chunks of size 2, skipping one value between chunks. The number ll[] slicings determine the chunk size, and the slicing step determines the chunk spacing.
As I commented there is some ambiguity in the problem description, so I hesitate to generalize this solution more until that is cleared up.
It may be easier to generalize the numpy solutions, but they aren't necessarily faster. Conversion to arrays has a time overhead.
list(itertools.chain(*zip(*[ll[i::6] for i in range(3)])))
produces chunks of length 3, skipping 3 elements.
zip(*) is an idiomatic way of 'transposing' a list of lists
itertools.chain(*...) is an idiomatic way of a flattening a list of lists.
Another option is a list comprehension with a condition based on item count
[v for i,v in enumerate(ll) if i%3]
handily skips every 3rd item, same as your example. (0<(i%6)<4) keeps 3, skips 3.
This should do the trick:
step = 3
size = 2
chunks = len(input) // step
input = np.asarray(input)
result = input[:chunks*step].reshape(chunks, step)[:, :size]
A simple list comprehension can do the job:
[ L[i] for i in range(len(L)) if i%3 != 2 ]
For chunks of size n
[ L[i] for i in range(len(L)) if i%(n+1) != n ]
This supposedly simple task is driving me a bit mad.
Say you want to create an array by concatenating an integer to another array:
import numpy as np
a = 4
b = np.array([1, 10, 24, 12])
A = np.array(a, b)
gives me TypeError: data type not understood. Which I understand because I'm mixing an integer with a list. Now if I do A = np.array([a], b) I get the same result, if I get A = np.array([a] + b) I don't get the expected result.
Also, I tried A = np.array([a, *b]) with SyntaxError: can use starred expression only as assignment target.
How's the proper way to do this?
What's wrong with using np.append?:
In [20]:
a = 4
b = np.array([1, 10, 24, 12])
np.append(a,b)
Out[20]:
array([ 4, 1, 10, 24, 12])
You can use the concatenate function to do this
A = np.concatenate(([a], b))
For your case, I think using append is "better" since it is less error prone as it accepts scalars as well, (as my own mistake clearly shows!) and (arguably) slightly more readable.
You can also use hstack (http://docs.scipy.org/doc/numpy/reference/generated/numpy.hstack.html):
In [194]: a = 4
In [195]: b = np.array([1, 10, 24, 12])
In [196]: np.hstack((a,b))
Out[196]: array([ 4, 1, 10, 24, 12])
hstack has the advantage that it can concatenate as many arrays/lists as you want (e.g. np.hstack((a, b, a, b, [0, 2, 4, 6, 8])))