Is there a way to change all the values past the max value of a list to its own?
For example I have the given array arranged as followed:
values = [-10,-2,0,1,3,8,10,22,18,16,12,10]
where 22 is the max value of the list.
I have the following pseudocode:
max_value = max(values)
for i in range(len(values)):
if values[i]== max_value:
values[i+1] = max_value
values[i +2] == max_value
...etc.
break
therefore:
values = [-10,-2,0,1,3,8,10,22,22,22,22,22]
Try this:
import numpy as np
myvalues = np.array([-10, -2, 0, 1, 3, 8, 10, 22, 18, 16, 12, 10])
max_index = np.argmax(myvalues)
myvalues[max_index:] =myvalues[max_index]
print(myvalues)
The result is
[-10 -2 0 1 3 8 10 22 22 22 22 22]
In case the ascending-then-descending shape of your example array is not just a coincidence but that that's always the case for you, you could just accumulate by max:
values[:] = itertools.accumulate(values, max)
reference this post: Pythonic way to find maximum value and its index in a list?
import operator
values = [-10, -2, 0, 1, 3, 8, 10, 22, 18, 16, 12, 10]
# find the maximum value and its index
index, maximum_value = max(enumerate(values), key=operator.itemgetter(1))
# replace to maximum_value after the index
for i in range(index, len(values)):
values[i] = maximum_value
Related
So here is my problem: I have an array like this:
arr = array([0, 0, 1, 8, 10, 20, 26, 32, 37, 52, 0, 0, 46, 42, 30, 19, 8, 2, 0, 0, 0])
In this array I want to find n consecutive values, greater than zero with the biggest sum. In this example with n = 5 this would be array([20, 26, 32, 37, 52]) and the index would be 5.
What I tried is of course a loop:
n = 5
max_sum = 0
max_loc = 0
for i in range(arr.size - n):
if all(arr[i:i + n] > 0) and arr[i:i + n].sum() > max_sum:
max_sum = arr[i:i + n].sum()
max_loc = i
print(max_loc)
This is fine for not too many short arrays but of course I need to use this on many not so short arrays.
I was experimenting with numpy so I would only have to iterate non-zero value groups:
diffs = np.concatenate((np.array([False]), np.diff(arr > 0)))
groups = np.split(arr, np.where(diffs)[0])
for group in groups:
if group.sum() > 0 and group.size >= n:
...
but I believe this is nice but not the right direction. I am looking for a simpler and faster numpy / pandas solution that really uses the powers of these packages.
Using cross-correlation, numpy.correlate, is a possible, concise and fast solution:
n=5
arr[arr<0] = np.iinfo(arr.dtype).min # The greatest negative integer possible
#Thanks for the np.iinfo suggestion, #Corralien
idx = np.argmax(np.correlate(arr, np.ones(n), 'valid'))
idx, arr[idx:(idx+5)]
Another possible solution:
n, l = 5, arr.size
arr[arr<0] = np.iinfo(arr.dtype).min # The greatest negative integer possible
#Thanks for the np.iinfo suggestion, #Corralien
idx = np.argmax([np.sum(np.roll(arr,-x)[:n]) for x in range(l-n+1)])
idx, arr[idx:(idx+n)]
Output:
(5, array([20, 26, 32, 37, 52]))
You can use sliding_window_view:
from numpy.lib.stride_tricks import sliding_window_view
N = 5
win = sliding_window_view(arr, N)
idx = ((win.sum(axis=1)) * ((win>0).all(axis=1))).argmax()
print(idx, arr[idx:idx+N])
# Output
5 [20 26 32 37 52]
Answer greatly enhanced by chrslg to save memory and keep a win as a view.
Update
A nice bonus is this should work with Pandas Series just fine.
N = 5
idx = pd.Series(arr).where(lambda x: x > 0).rolling(N).sum().shift(-N+1).idxmax()
print(idx, arr[idx:idx+N])
# Output
5 [20 26 32 37 52]
Is there a simpler way, using e.g. numpy, to get samples for a given X and delta than the below code?
>>> X = [1, 4, 5, 6, 11, 13, 15, 20, 21, 22, 25, 30]
>>> delta = 5
>>> samples = [X[0]]
>>> for x in X:
... if x - samples[-1] >= delta:
... samples.append(x)
>>> samples
[1, 6, 11, 20, 25, 30]
If you are aiming to "vectorize" the process for performance reasons (e.g. using numpy), you could compute the number of elements that are less than each element plus the delta. This will give you indices for the items to select with the items that need to be skipped getting the same index as the preceding ones to be kept.
import numpy as np
X = np.array([1, 4, 5, 6, 11, 13, 15, 20, 21, 22, 25, 30])
delta = 5
i = np.sum(X<X[:,None]+delta,axis=1) # index of first to keep
i = np.insert(i[:-1],0,0) # always want the first, never the last
Y = X[np.unique(i)] # extract values as unique indexes
print(Y)
[ 1 6 11 20 25 30]
This assumes that the numbers are in ascending order
[EDIT]
As indicated in my comment, the above solution is flawed and will only work some of the time. Although vectorizing a python function does not fully leverage the parallelism (and is slower than the python loop), it is possible to implement the filter like this
X = np.array([1, 4, 5, 6, 10,11,12, 13, 15, 20, 21, 22, 25, 30])
delta = 5
fdelta = np.frompyfunc(lambda a,b:a if a+delta>b else b,2,1)
Y = X[X==fdelta.accumulate(X,dtype=np.object)]
print(Y)
[ 1 6 11 20 25 30]
I have list [1, 2, 5, 6, 9, 10, 14, 19], how can i get any number range index.
For example:
l = [1, 2, 5, 6, 9, 10, 14, 19]
value = 11
range_index = get_range_index(l)
range_index = (5, 6) # need like this
# give a value = 11, need to get value index like (5, 6), because 10 < value < 14.
# the list size may be very very long,can there have good method?
This i try to get left value and calculate index by returned left value.
It's not very good and not high performance.
def get_left_point(self, data, value):
if len(data) == 1:
return data[0]
mid_index, mid_value = len(data) // 2, data[len(data) // 2]
if value >= float(mid_value):
ret = self.get_left_point(data[mid_index:], value)
else:
ret = self.get_left_point(data[:mid_index], value)
return ret
my_range = (100, 101)
[x for x in l if (my_range[0] <= x) and (x <= my_range[1])]
I have two numpy arrays of integers A and B. The values in array A and B correspond to time-points at which events A and B occurred. I would like to transform A to contain the time since the most recent event b occurred.
I know I need to subtract each element of A by its nearest smaller the element of B but am unsure of how to do so. Any help would be greatly appreciated.
>>> import numpy as np
>>> A = np.array([11, 12, 13, 17, 20, 22, 33, 34])
>>> B = np.array([5, 10, 15, 20, 25, 30])
Desired Result:
cond_a = relative_timestamp(to_transform=A, reference=B)
cond_a
>>> array([1, 2, 3, 2, 0, 2, 3, 4])
You can use np.searchsorted to find the indices where the elements of A should be inserted in B to maintain order. In other words, you are finding the closest elemet in B for each element in A:
idx = np.searchsorted(B, A, side='right')
result = A-B[idx-1] # substract one for proper index
According to the docs searchsorted uses binary search, so it will scale fine for large inputs.
Here's an approach consisting on computing the pairwise differences. Note that it has a O(n**2) complexity so it might for larger arrays #brenlla's answer will perform much better.
The idea here is to use np.subtract.outer and then find the minimum difference along axis 1 over a masked array, where only values in B smaller than a are considered:
dif = np.abs(np.subtract.outer(A,B))
np.ma.array(dif, mask = A[:,None] < B).min(1).data
# array([1, 2, 3, 2, 0, 2, 3, 4])
As I am not sure, if it is really faster to calculate all pairwise differences, instead of a python loop over each array entry (worst case O(Len(A)+len(B)), the solution with a loop:
A = np.array([11, 12, 13, 17, 20, 22, 33, 34])
B = np.array([5, 10, 15, 20, 25, 30])
def calculate_next_distance(to_transform, reference):
max_reference = len(reference) - 1
current_reference = 0
transformed_values = np.zeros_like(to_transform)
for i, value in enumerate(to_transform):
while current_reference < max_reference and reference[current_reference+1] <= value:
current_reference += 1
transformed_values[i] = value - reference[current_reference]
return transformed_values
calculate_next_distance(A,B)
# array([1, 2, 3, 2, 0, 2, 3, 4])
I have list with values, and I'm looking for way to track if those values went below certain value, and then if they went back up above it (possibly number of times)
So let's say my lisy looks like:
list1 = [20, 18, 16, 15, 13, 12, 16, 17, 14, 11, 16]
And I need something that will tell me, that values went back up above 15 two or x number of times after it initially went below 15.
Anyone knows how to tackle this problem?
pos_count = 0
neg_count = 0
for x in range(len(list1)-1):
if list1[x] <= 15 and list1[x + 1] > 15:
neg_count += 1
elif list1[x] >= 15 and list1[x + 1] < 15:
pos_count += 1
print(pos_count)
print(neg_count)
You could simply check the negative/positive difference from 15.
Something like this:
l = [20, 18, 16, 15, 13, 12, 16, 17, 14, 11, 16] #Your list
import numpy as np
arr = np.asarray(l) #Convert to numpy-array
#Now apply the function to check the difference from 15
result = np.apply_along_axis(lambda x: x - 15, axis=0, arr=arr)
This results in:
array([ 5, 3, 1, 0, -2, -3, 1, 2, -1, -4, 1])
If you want to check all positive values, you can do it like this:
pos = result[result >= 0] #All positive values
neg = result[result < 0] #All negative values
If you want to count how often this happens, you can count the length:
len(pos) #Number of positives
len(neg) #Number of negatives