I want to build a matrix in NumPy in which the items add up to each other. So I have tried to build it with the following code:
StartpointRow = int(input("First number of row?:\n"))
EndpointRow = int(input("Last number of row?:\n"))
StepRow = int(input("Which steps should the row have?:\n"))
StartpointCol = int(input("First number of column?:\n"))
EndpointCol = int(input("Last number of column?:\n"))
StepCol = int(input("Which steps should the column have?:\n"))
x = np.array([[i+j for i in range(StartpointCol, EndpointCol , StepCol)]
for j in range(StartpointRow, EndpointRow , StepRow)])
print(x)
let's say that, for instance, I enter 1,4,1 and 1,4,1. I want the solution to be a matrix like this:
1 2 3 4
2 4 5 6
3 5 6 7
4 6 7 8
Not like that:
2 3 4
3 4 5
4 5 6
or If the user types in: 1,4,1 and 2,4,1.
0 1 2 3 4
2 3 4 5 6
3 4 5 6 7
4 5 6 7 8
not like that:
3 4
4 5
5 6
Could you help me out?
Use np.add.outer:
def build(r_start, r_stop, r_step, c_start, c_stop, c_step):
r = np.arange(r_start, r_stop + 1, r_step)
c = np.arange(c_start, c_stop + 1, c_step)
if r_start == c_start:
ret = np.empty((c.size, r.size), int)
ret[:, 0] = c
ret[0, :] = r
else:
ret = np.empty((c.size + 1, r.size + 1), int)
ret[0, 0] = 0
ret[1:, 0] = c
ret[0, 1:] = r
np.add.outer(ret[1:, 0], ret[0, 1:], out=ret[1:, 1:])
return ret
A little simplification:
def build(r_start, r_stop, r_step, c_start, c_stop, c_step):
r = np.arange(r_start, r_stop + 1, r_step)
c = np.arange(c_start, c_stop + 1, c_step)
ne = int(r_start != c_start)
ret = np.empty((c.size + ne, r.size + ne), int)
ret[0, 0] = 0
ret[ne:, 0] = c
ret[0, ne:] = r
np.add.outer(ret[1:, 0], ret[0, 1:], out=ret[1:, 1:])
return ret
Test:
>>> build(1, 4, 1, 1, 4, 1)
array([[1, 2, 3, 4],
[2, 4, 5, 6],
[3, 5, 6, 7],
[4, 6, 7, 8]])
>>> build(1, 4, 1, 2, 4, 1)
array([[0, 1, 2, 3, 4],
[2, 3, 4, 5, 6],
[3, 4, 5, 6, 7],
[4, 5, 6, 7, 8]])
I think your test cases are wrong.
What I understand you mean is that each row and each column have a starting number, what needs to be done is to add the two and generate the matrix according to step.
If the user types in: 1,4,1 and 1,4,1, what he can get is:
2 3 4 5
3 4 5 6
4 5 6 7
5 6 7 8
And if the user types in: 1,4,1 and 2,4,1, what he can get is:
3 4 5
4 5 6
5 6 7
6 7 8
And my code is:
import numpy as np
StartpointRow = int(input("First number of row?:\n"))
EndpointRow = int(input("Last number of row?:\n"))
StepRow = int(input("Which steps should the row have?:\n"))
StartpointCol = int(input("First number of column?:\n"))
EndpointCol = int(input("Last number of column?:\n"))
StepCol = int(input("Which steps should the column have?:\n"))
x = np.array([[i+j for i in range(StartpointCol, EndpointCol + 1 , StepCol)]
for j in range(StartpointRow, EndpointRow + 1, StepRow)])
print(x)
How to make a multiplication chart with nested lists and for ? I need to all numbers from first list multiply to from second list
chart = [
[],
[],
]
for i in range(1,len(chart)+1):
for j in range(i,i*len(chart)+1):
print(f'{i} * {j} = {i*j}')
In python positions of elements in a list start from 0.
The chart list contains 2 lists:
chart[0] = [1, 2, 3, 4, 5]
chart[1] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
What you want to do is access the elements of the first list and multiply by elements of the second list.
for i in range(len(chart[0])): # range(5) => 0, 1, 2, 3, 4
for j in range(len(chart[1])): # range(10) => 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
print(f'{chart[0][i]} * {chart[1][j]} = {chart[0][i] * chart[1][j]}
itertools.product will produce the desired output, creating 2-tuples consisting of one element from the first list and one element from the second list:
import itertools
chart = [
[1,2,3,4,5],
[1,2,3,4,5,6,7,8,9,10],
]
for i, j in itertools.product(*chart):
print(f'{i} * {j} = {i*j}')
Use a cartesian product:
chart = [[1,2,3,4,5],[1,2,3,4,5,6,7,8,9,10]]
>>> print('\n'.join([f'{i} * {j} = {i*j}' for i in chart[0] for j in chart[1]]))
1 * 1 = 1
1 * 2 = 2
1 * 3 = 3
1 * 4 = 4
...
4 * 10 = 40
5 * 1 = 5
5 * 2 = 10
5 * 3 = 15
5 * 4 = 20
5 * 5 = 25
5 * 6 = 30
5 * 7 = 35
5 * 8 = 40
5 * 9 = 45
5 * 10 = 50
I have large 1D NumPy array a of any comparable dtype, some of its elements may be repeated.
How do I find sorting indexes ix that will stable-sort (stability in a sense described here) a by frequencies of values in descending/ascending orders?
I want to find fastest and simplest way to do this. Maybe there is existing standard numpy function to do that.
There is another related question here but it was asking specifically to remove arrays duplicates, i.e. output only unique sorted values, I need all values of original array including duplicates.
I've coded my first trial to do the task, but it is not the fastest (uses Python's loop) and probably not shortest/simplest possible form. This python loop can be very expensive if repeating of equal elements is not high and array is huge. Also would be nice to have short function for doing this all if available in NumPy (e.g. imaginary np.argsort_by_freq()).
Try it online!
import numpy as np
np.random.seed(1)
hi, n, desc = 7, 24, True
a = np.random.choice(np.arange(hi), (n,), p = (
lambda p = np.random.random((hi,)): p / p.sum()
)())
us, cs = np.unique(a, return_counts = True)
af = np.zeros(n, dtype = np.int64)
for u, c in zip(us, cs):
af[a == u] = c
if desc:
ix = np.argsort(-af, kind = 'stable') # Descending sort
else:
ix = np.argsort(af, kind = 'stable') # Ascending sort
print('rows: i_col(0) / original_a(1) / freqs(2) / sorted_a(3)')
print(' / sorted_freqs(4) / sorting_ix(5)')
print(np.stack((
np.arange(n), a, af, a[ix], af[ix], ix,
), 0))
outputs:
rows: i_col(0) / original_a(1) / freqs(2) / sorted_a(3)
/ sorted_freqs(4) / sorting_ix(5)
[[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
[ 1 1 1 1 3 0 5 0 3 1 1 0 0 4 6 1 3 5 5 0 0 0 5 0]
[ 7 7 7 7 3 8 4 8 3 7 7 8 8 1 1 7 3 4 4 8 8 8 4 8]
[ 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 5 5 5 5 3 3 3 4 6]
[ 8 8 8 8 8 8 8 8 7 7 7 7 7 7 7 4 4 4 4 3 3 3 1 1]
[ 5 7 11 12 19 20 21 23 0 1 2 3 9 10 15 6 17 18 22 4 8 16 13 14]]
I might be missing something, but it seems that with a Counter you can then sort the indexes of each element according to the count of that element's value, using the element value and then the index to break ties. For example:
from collections import Counter
a = [ 1, 1, 1, 1, 3, 0, 5, 0, 3, 1, 1, 0, 0, 4, 6, 1, 3, 5, 5, 0, 0, 0, 5, 0]
counts = Counter(a)
t = [(counts[v], v, i) for i, v in enumerate(a)]
t.sort()
print([v[2] for v in t])
t.sort(reverse=True)
print([v[2] for v in t])
Output:
[13, 14, 4, 8, 16, 6, 17, 18, 22, 0, 1, 2, 3, 9, 10, 15, 5, 7, 11, 12, 19, 20, 21, 23]
[23, 21, 20, 19, 12, 11, 7, 5, 15, 10, 9, 3, 2, 1, 0, 22, 18, 17, 6, 16, 8, 4, 14, 13]
If you want to maintain ascending order of indexes with groups with equal counts, you can just use a lambda function for the descending sort:
t.sort(key = lambda x:(-x[0],-x[1],x[2]))
print([v[2] for v in t])
Output:
[5, 7, 11, 12, 19, 20, 21, 23, 0, 1, 2, 3, 9, 10, 15, 6, 17, 18, 22, 4, 8, 16, 14, 13]
If you want to maintain the ordering of elements in the order that they originally appeared in the array if their counts are the same, then rather than sort on the values, sort on the index of their first occurrence in the array:
a = [ 1, 1, 1, 1, 3, 0, 5, 0, 3, 1, 1, 0, 0, 4, 6, 1, 3, 5, 5, 0, 0, 0, 5, 0]
counts = Counter(a)
idxs = {}
t = []
for i, v in enumerate(a):
if not v in idxs:
idxs[v] = i
t.append((counts[v], idxs[v], i))
t.sort()
print([v[2] for v in t])
t.sort(key = lambda x:(-x[0],x[1],x[2]))
print([v[2] for v in t])
Output:
[13, 14, 4, 8, 16, 6, 17, 18, 22, 0, 1, 2, 3, 9, 10, 15, 5, 7, 11, 12, 19, 20, 21, 23]
[5, 7, 11, 12, 19, 20, 21, 23, 0, 1, 2, 3, 9, 10, 15, 6, 17, 18, 22, 4, 8, 16, 13, 14]
To sort according to count, and then position in the array, you don't need the value or the first index at all:
from collections import Counter
a = [ 1, 1, 1, 1, 3, 0, 5, 0, 3, 1, 1, 0, 0, 4, 6, 1, 3, 5, 5, 0, 0, 0, 5, 0]
counts = Counter(a)
t = [(counts[v], i) for i, v in enumerate(a)]
t.sort()
print([v[1] for v in t])
t.sort(key = lambda x:(-x[0],x[1]))
print([v[1] for v in t])
This produces the same output as the prior code for the sample data, for your string array:
a = ['g', 'g', 'c', 'f', 'd', 'd', 'g', 'a', 'a', 'a', 'f', 'f', 'f',
'g', 'f', 'c', 'f', 'a', 'e', 'b', 'g', 'd', 'c', 'b', 'f' ]
This produces the output:
[18, 19, 23, 2, 4, 5, 15, 21, 22, 7, 8, 9, 17, 0, 1, 6, 13, 20, 3, 10, 11, 12, 14, 16, 24]
[3, 10, 11, 12, 14, 16, 24, 0, 1, 6, 13, 20, 7, 8, 9, 17, 2, 4, 5, 15, 21, 22, 19, 23, 18]
I just figured myself probably very fast solution for any dtype using just numpy functions without python looping, it works in O(N log N) time. Used numpy functions: np.unique, np.argsort and array indexing.
Although wasn't asked in original question, I implemented extra flag equal_order_by_val if it is False then array elements with same frequencies are sorted as equal stable range, meaning that there could be c d d c d c output like in outputs dumps below, because this is the order as elements go in original array for equal frequency. When flag is True such elements are in addition sorted by value of original array, resulting in c c c d d d. In other words in case of False we sort stably just by key freq, and when it is True we sort by (freq, value) for ascending order and by (-freq, value) for descending order.
Try it online!
import string, math
import numpy as np
np.random.seed(0)
# Generating input data
hi, n, desc = 7, 25, True
letters = np.array(list(string.ascii_letters), dtype = np.object_)[:hi]
a = np.random.choice(letters, (n,), p = (
lambda p = np.random.random((letters.size,)): p / p.sum()
)())
for equal_order_by_val in [False, True]:
# Solving task
us, ui, cs = np.unique(a, return_inverse = True, return_counts = True)
af = cs[ui]
sort_key = -af if desc else af
if equal_order_by_val:
shift_bits = max(1, math.ceil(math.log(us.size) / math.log(2)))
sort_key = ((sort_key.astype(np.int64) << shift_bits) +
np.arange(us.size, dtype = np.int64)[ui])
ix = np.argsort(sort_key, kind = 'stable') # Do sorting itself
# Printing results
print('\nequal_order_by_val:', equal_order_by_val)
for name, val in [
('i_col', np.arange(n)), ('original_a', a),
('freqs', af), ('sorted_a', a[ix]),
('sorted_freqs', af[ix]), ('sorting_ix', ix),
]:
print(name.rjust(12), ' '.join([str(e).rjust(2) for e in val]))
outputs:
equal_order_by_val: False
i_col 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
original_a g g c f d d g a a a f f f g f c f a e b g d c b f
freqs 5 5 3 7 3 3 5 4 4 4 7 7 7 5 7 3 7 4 1 2 5 3 3 2 7
sorted_a f f f f f f f g g g g g a a a a c d d c d c b b e
sorted_freqs 7 7 7 7 7 7 7 5 5 5 5 5 4 4 4 4 3 3 3 3 3 3 2 2 1
sorting_ix 3 10 11 12 14 16 24 0 1 6 13 20 7 8 9 17 2 4 5 15 21 22 19 23 18
equal_order_by_val: True
i_col 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
original_a g g c f d d g a a a f f f g f c f a e b g d c b f
freqs 5 5 3 7 3 3 5 4 4 4 7 7 7 5 7 3 7 4 1 2 5 3 3 2 7
sorted_a f f f f f f f g g g g g a a a a c c c d d d b b e
sorted_freqs 7 7 7 7 7 7 7 5 5 5 5 5 4 4 4 4 3 3 3 3 3 3 2 2 1
sorting_ix 3 10 11 12 14 16 24 0 1 6 13 20 7 8 9 17 2 15 22 4 5 21 19 23 18
I'm trying to implement overlap-add in Tensorflow but I'm struggling to convert the numpy output_seq[start:end] += chunk to Tensorflow. Right now I'm output_seq = output_seq + tf.pad(chunk, [[start, length - end]]) but that's really slow on long sequences.
I also have a hunch there might be tricks you can do with gather/scatter, but I can't quite figure it out. Below is my brute force attempt:
import tensorflow as tf
input = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
def overlap_add(overlap):
with tf.Graph().as_default(), tf.Session() as sess:
x = tf.constant(input)
num_chunks = tf.shape(x)[0]
chunk_size = tf.shape(x)[1]
hop_length = chunk_size - overlap
out_len = chunk_size + hop_length * (num_chunks - 1)
y = tf.zeros((out_len,), dtype=tf.int32)
def body(i, y):
j = i * hop_length
padding = [[j, out_len - (j + chunk_size)]]
chunk = x[i]
y = y + tf.pad(chunk, padding)
return (i + 1, y)
i = tf.constant(0)
i, y = tf.while_loop(
cond=lambda i, _: tf.less(i, num_chunks),
body=body,
loop_vars=[i, y])
return sess.run(y)
for i in range(4):
print 'overlap_add(%d): %s' % (i, overlap_add(i))
# overlap_add(0): [ 1 2 3 4 5 6 7 8 9 10 11 12]
# overlap_add(1): [ 1 2 3 9 6 7 17 10 11 12]
# overlap_add(2): [ 1 2 8 10 16 18 11 12]
# overlap_add(3): [ 1 7 18 21 19 12]
UPDATE: There's now an overlap_and_add function in Tensorflow itself.
OLD ANSWER:
Trawled through the docs and found unsorted_segment_sum:
import tensorflow as tf
input = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
def tf_repeat(a, repeats):
return tf.reshape(tf.tile(tf.reshape(a, [-1, 1]),
[1, repeats]), [-1])
def overlap_add(overlap):
with tf.Graph().as_default(), tf.Session() as sess:
x = tf.constant(input)
x_flat = tf.reshape(x, [-1])
num_chunks = tf.shape(x)[0]
chunk_size = tf.shape(x)[1]
hop_len = chunk_size - overlap
flat_len = num_chunks * chunk_size
out_len = chunk_size + hop_len * (num_chunks - 1)
# e.g. [0,1,2,3, 2,3,4,5, 4,5,6,7] for overlap == 2
indexes = tf.range(flat_len) - tf_repeat(tf.range(num_chunks), chunk_size) * overlap
return sess.run(tf.unsorted_segment_sum(x_flat, indexes, out_len))
for i in range(4):
print 'overlap_add(%d): %s' % (i, overlap_add(i))
# overlap_add(0): [ 1 2 3 4 5 6 7 8 9 10 11 12]
# overlap_add(1): [ 1 2 3 9 6 7 17 10 11 12]
# overlap_add(2): [ 1 2 8 10 16 18 11 12]
# overlap_add(3): [ 1 7 18 21 19 12]
You can use slices in Tensorflow as well:
a[1:3].assign(a[1:3] + b[1:3]).eval()
For some reason assign_add is not implemented. It seems like a bug to me.
a[1:3].assign_add(b[1:3]).eval() # Doesn't work
I'm trying to create an algorithm to produce a decimal number by certain way.
a) I have an initial number say i = 2.
b) Then I have an incremental addition method, say f(n) { n * 2 }.
c) Then I have a slot length for digits say l = 2, that creates front zeros for small numbers and limits max length of the longer numbers. 2 becomes 02, 64 is 64, but 512 = (5)12 where 5 is moved backward on previous slot
d) Max slots is the fourth parameter, m = 10
e) Finally I want to compute value by summing up digit from slots and using it as a decimal part of the 0.
So with given example:
i=2
f(n)=n*2
l=2
m=10
outcome should be produced in this manner:
step 1)
02 04 08 16 32 64 128 256 512 1024
step 2)
02 04 08 16 32 64
1 28
2 56
5 12
10 24
->
slot: 1 2 3 4 5 6 7 8 9 10
computed: 02 04 08 16 32 65 30 61 22 24
step 3)
I have a number: 02040816326530612224 or 0.02040816326530612224 as stated on part e).
Note that if max slot is bigger in this example, then numbers on slots 9 and 10 will change. I also want to have part b) as a function, so I can change it to other like fib(nx) {n1+n2}.
I prefer Python as a computer language for algo, but anything that is easy to transform to Python is acceptable.
ADDED
This is a function I have managed to create so far:
# l = slot length, doesnt work with number > 2...
def comp(l = 2):
a = []
# how to pass a function, that produces this list?
b = [[0, 2], [0, 4], [0, 8], [1, 6], [3, 2], [6, 4], [1, 2, 8], [2, 5, 6], [5, 1, 2], [1, 0, 2, 4], [2, 0, 4, 8]]
r = 0
# main algo
for bb in b:
ll = len(bb)
for i in range(0, ll):
x = r + i - ll + l
# is there a better way to do following try except part?
try:
a[x] += bb[i]
except IndexError:
a.append(bb[i])
# moving bits backward, any better way to do this?
s = a[x] - 9
d = 0
while s > 0:
d += 1
a[x] -= 10
a[x-d] += 1
s = a[x-d] - 9
r += l
return '0.' + ''.join(map(str, a))
def doub(n):
return pow(2, n)
def fibo(n):
a, b = 0, 1
for i in range(n):
a, b = b, a + b
return a
def fixed_slot_numbers(f, l, m):
b = []
for n in range(1, m):
a = [int(c) for c in str(f(n))]
while len(a) < l:
a.insert(0, 0)
b.append(a)
return b
def algo(function, fixed_slot_length = 2, max_slots = 12):
a = []
slot_numbers = fixed_slot_numbers(function, fixed_slot_length, max_slots)
for r, b in enumerate(slot_numbers):
r *= fixed_slot_length
slot_length = len(b)
for bidx in range(0, slot_length):
aidx = r + bidx - slot_length + fixed_slot_length
try:
a[aidx] += b[bidx]
except IndexError:
a.append(b[bidx])
d = 0
while a[aidx-d] > 9:
a[aidx-d] -= 10
d += 1
a[aidx-d] += 1
return '0.%s' % ''.join(map(str, a))
algo(doub, 2, 28) -> 0.020408163265306122448979591836734693877551020405424128 = 1/49
algo(fibo, 1, 28) -> 0.112359550561797752808950848 = 10/89