So I'm trying to generate a list of possible adjacent movements within a 3d array (preferebly n-dimensional).
What I have works as it's supposed to, but I was wondering if there's a more numpythonic way to do so.
def adjacents(loc, bounds):
adj = []
bounds = np.array(bounds) - 1
if loc[0] > 0:
adj.append((-1, 0, 0))
if loc[1] > 0:
adj.append((0, -1, 0))
if loc[2] > 0:
adj.append((0, 0, -1))
if loc[0] < bounds[0]:
adj.append((1, 0, 0))
if loc[1] < bounds[1]:
adj.append((0, 1, 0))
if loc[2] < bounds[2]:
adj.append((0, 0, 1))
return np.array(adj)
Here are some example outputs:
adjacents((0, 0, 0), (10, 10, 10))
= [[1 0 0]
[0 1 0]
[0 0 1]]
adjacents((9, 9, 9), (10, 10, 10))
= [[-1 0 0]
[ 0 -1 0]
[ 0 0 -1]]
adjacents((5, 5, 5), (10, 10, 10))
= [[-1 0 0]
[ 0 -1 0]
[ 0 0 -1]
[ 1 0 0]
[ 0 1 0]
[ 0 0 1]]
Here's an alternative which is vectorized and uses a constant, prepopulated array:
# all possible moves
_moves = np.array([
[-1, 0, 0],
[ 0,-1, 0],
[ 0, 0,-1],
[ 1, 0, 0],
[ 0, 1, 0],
[ 0, 0, 1]])
def adjacents(loc, bounds):
loc = np.asarray(loc)
bounds = np.asarray(bounds)
mask = np.concatenate((loc > 0, loc < bounds - 1))
return _moves[mask]
This uses asarray() instead of array() because it avoids copying if the input happens to be an array already. Then mask is constructed as an array of six bools corresponding to the original six if conditions. Finally, the appropriate rows of the constant data _moves are returned.
But what about performance?
The vectorized approach above, while it will appeal to some, actually runs only half as fast as the original. If it's performance you're after, the best simple change you can make is to remove the line bounds = np.array(bounds) - 1 and subtract 1 inside each of the last three if conditions. That gives you a 2x speedup (because it avoids creating an unnecessary array).
Related
I have a Pandas Dataframe of 1's and 0's converted to an array :
[0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1]
I'm using the following function to amend the array:
def regressor(x, param):
new_array = x
for i in range(len(new_array)):
length = len(new_array)
current = new_array[(length-i)-1]
previous = new_array[(length-i)-2]
if current != 0:
if previous == 0:
new_array[(length-i)-2] = current*param
return new_array
However, my array is still unchanged. new_array[(length-i)-2] does not seem to actually amend that element of the array.
Could someone tell me what I'm missing?
Thanks
UPDATE: My problem was solved by converting the DataFrame to a pandas.Series and then converting this to a list within the function.
If I run your code I get this (for param=2):
x = [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]
def regressor(x, param):
new_array = x
for i in range(len(new_array)):
length = len(new_array)
current = new_array[(length-i)-1]
previous = new_array[(length-i)-2]
if current != 0:
if previous == 0:
new_array[(length-i)-2] = current * param
return new_array
new_array = regressor(x, 2)
print(new_array)
# Output:
# [8, 4, 2, 1, 2097152, 1048576, 524288, 262144, 131072, 65536, 32768, 16384, 8192, 4096, 2048, 1024, 512, 256, 128, 64, 32, 16, 8, 4, 2, 1]
Why do you not see changes in your new_array? Do you actually call the regressor function to update your array?
Also, copying x to new_array at the beginning of the function is redundant.
Just re-posting your function with some edits for better readability:
def regressor(arr, param):
for i in range(len(arr)):
length = len(arr)
current = arr[(length-i)-1]
previous = arr[(length-i)-2]
if (current != 0) and (previous == 0):
arr[(length-i)-2] = current * param
return arr
I am using music21 for handling MIDI and mXML files and converting them to a piano roll I am using in my project.
My piano roll is made up of sequence of 88-dimensional vectors where each element in a vector represents one pitch. One vector is one time step that can be 16th, 8th, 4th, and so on. Elements can obtain three values {0, 1, 2}. 0 means note is off. 1 means note is on. 2 means also that note is on but it always follows 1 - that is how I distinguish multiple key presses of same note. E.g., let time step be 8th and these two pitches be C and E:
[0 0 0 ... 1 0 0 0 1 ... 0]
[0 0 0 ... 1 0 0 0 1 ... 0]
[0 0 0 ... 2 0 0 0 2 ... 0]
[0 0 0 ... 2 0 0 0 2 ... 0]
[0 0 0 ... 1 0 0 0 0 ... 0]
[0 0 0 ... 1 0 0 0 0 ... 0]
We see that C and E are simultaneously played for quarter note, then again for quarter note, and we end with a C that lasts quarter note.
Right now, I am creating Stream() for every note and fill it as notes come. That gives me 88 streams and when I convert that to MIDI, and open that MIDI with MuseScore, that leaves me with a mess that is not readable.
My question is, is there some nicer way to transform this kind of piano roll to MIDI? Some algorithm, or idea which I could use would be appreciated.
In my opinion music21 is a very good library but too high-level for
this job. There is no such thing as streams, quarter notes or chords
in MIDI -- only messages. Try the
Mido library instead. Here
is sample code:
from mido import Message, MidiFile, MidiTrack
def stop_note(note, time):
return Message('note_off', note = note,
velocity = 0, time = time)
def start_note(note, time):
return Message('note_on', note = note,
velocity = 127, time = time)
def roll_to_track(roll):
delta = 0
# State of the notes in the roll.
notes = [False] * len(roll[0])
# MIDI note for first column.
midi_base = 60
for row in roll:
for i, col in enumerate(row):
note = midi_base + i
if col == 1:
if notes[i]:
# First stop the ringing note
yield stop_note(note, delta)
delta = 0
yield start_note(note, delta)
delta = 0
notes[i] = True
elif col == 0:
if notes[i]:
# Stop the ringing note
yield stop_note(note, delta)
delta = 0
notes[i] = False
# ms per row
delta += 500
roll = [[0, 0, 0, 1, 0, 0, 0, 1, 0],
[0, 0, 0, 1, 0, 0, 0, 1, 0],
[0, 0, 0, 2, 0, 0, 0, 2, 0],
[0, 1, 0, 2, 0, 0, 0, 2, 0],
[0, 0, 0, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0, 0]]
midi = MidiFile(type = 1)
midi.tracks.append(MidiTrack(roll_to_track(roll)))
midi.save('test.mid')
I have a feature matrix and a corresponding targets, which are ones or zeroes:
# raw observations
features = np.array([[1, 1, 0],
[1, 1, 0],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[0, 0, 1]])
targets = np.array([1, 0, 1, 1, 0, 0])
As you can see, each feature may correspond to both ones and zeros. I need to convert my raw observation matrix to probability matrix, where each feature will correspond to the probability of seeing one as a target:
[1 1 0] -> 0.5
[0 1 0] -> 0.67
[0 0 1] -> 0
I have constructed a quite straight-forward solution:
import numpy as np
# raw observations
features = np.array([[1, 1, 0],
[1, 1, 0],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[0, 0, 1]])
targets = np.array([1, 0, 1, 1, 0, 0])
from collections import Counter
def convert_obs_to_proba(features, targets):
features_ = []
targets_ = []
# compute unique rows (idx will point to some representative)
b = np.ascontiguousarray(features).view(np.dtype((np.void, features.dtype.itemsize * features.shape[1])))
_, idx = np.unique(b, return_index=True)
idx = idx[::-1]
zeros = Counter()
ones = Counter()
# collect row-wise number of one and zero targets
for i, row in enumerate(features[:]):
if targets[i] == 0:
zeros[tuple(row)] += 1
else:
ones[tuple(row)] += 1
# iterate over unique features and compute probabilities
for k in idx:
unique_row = features[k]
zero_count = zeros[tuple(unique_row)]
one_count = ones[tuple(unique_row)]
proba = float(one_count) / float(zero_count + one_count)
features_.append(unique_row)
targets_.append(proba)
return np.array(features_), np.array(targets_)
features_, targets_ = convert_obs_to_proba(features, targets)
print(features_)
print(targets_)
which:
extracts unique features;
counts number of zero and one observations targets for each unique feature;
computes probability and constructs the result.
Could it be solved in a prettier way using some advanced numpy magic?
Update. Previous code was pretty inefficient O(n^2). Converted it to more performance-friendly. Old code:
import numpy as np
# raw observations
features = np.array([[1, 1, 0],
[1, 1, 0],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[0, 0, 1]])
targets = np.array([1, 0, 1, 1, 0, 0])
def convert_obs_to_proba(features, targets):
features_ = []
targets_ = []
# compute unique rows (idx will point to some representative)
b = np.ascontiguousarray(features).view(np.dtype((np.void, features.dtype.itemsize * features.shape[1])))
_, idx = np.unique(b, return_index=True)
idx = idx[::-1]
# calculate ZERO class occurences and ONE class occurences
for k in idx:
unique_row = features[k]
zeros = 0
ones = 0
for i, row in enumerate(features[:]):
if np.array_equal(row, unique_row):
if targets[i] == 0:
zeros += 1
else:
ones += 1
proba = float(ones) / float(zeros + ones)
features_.append(unique_row)
targets_.append(proba)
return np.array(features_), np.array(targets_)
features_, targets_ = convert_obs_to_proba(features, targets)
print(features_)
print(targets_)
It's easy using Pandas:
df = pd.DataFrame(features)
df['targets'] = targets
Now you have:
0 1 2 targets
0 1 1 0 1
1 1 1 0 0
2 0 1 0 1
3 0 1 0 1
4 0 1 0 0
5 0 0 1 0
Now, the fancy part:
df.groupby([0,1,2]).targets.mean()
Gives you:
0 1 2
0 0 1 0.000000
1 0 0.666667
1 1 0 0.500000
Name: targets, dtype: float64
Pandas doesn't print the 0 at the leftmost part of the 0.666 row, but if you inspect the value there, it is indeed 0.
np.sum(np.reshape([targets[f] if tuple(features[f])==tuple(i) else 0 for i in np.vstack(set(map(tuple,features))) for f in range(features.shape[0])],features.shape[::-1]),axis=1)/np.sum(np.reshape([1 if tuple(features[f])==tuple(i) else 0 for i in np.vstack(set(map(tuple,features))) for f in range(features.shape[0])],features.shape[::-1]),axis=1)
Here you go, numpy magic! Although unnecceserily so, this could probably be cleaned up using some boring variables ;)
(And this is probably far from optimal)
I have this code:
gs = open("graph.txt", "r")
gp = gs.readline()
gp_splitIndex = gp.find(" ")
gp_nodeCount = int(gp[0:gp_splitIndex])
gp_edgeCount = int(gp[gp_splitIndex+1:-1])
matrix = [] # predecare the array
for i in range(0, gp_nodeCount):
matrix.append([])
for y in range(0, gp_nodeCount):
matrix[i].append(0)
for i in range(0, gp_edgeCount-1):
gp = gs.readline()
gp_splitIndex = gp.find(" ") # get the index of space, dividing the 2 numbers on a row
gp_from = int(gp[0:gp_splitIndex])
gp_to = int(gp[gp_splitIndex+1:-1])
matrix[gp_from][gp_to] = 1
print matrix
The file graph.txt contains this:
5 10
0 1
1 2
2 3
3 4
4 0
0 3
3 1
1 4
4 2
2 0
The first two number are telling me, that GRAPH has 5 nodes and 10 edges. The Following number pairs demonstrate the edges between nodes. For example "1 4" means an edge between node 1 and 4.
Problem is, the output should be this:
[[0, 1, 0, 1, 0], [0, 0, 1, 0, 1], [1, 0, 0, 1, 0], [0, 1, 0, 0, 1], [1, 0, 1, 0, 0]]
But instead of that, I get this:
[[0, 1, 0, 1, 0], [0, 0, 1, 0, 1], [0, 0, 0, 1, 0], [0, 1, 0, 0, 1], [1, 0, 1, 0, 0]]
Only one number is different and I can't understand why is this happening. The edge "3 1" is not present. Can someone explain, where is the problem?
Change for i in range(0, gp_edgeCount-1): to
for i in range(0, gp_edgeCount):
The range() function already does the "-1" operation. range(0,3) "==" [0,1,2]
And it is not the "3 1" edge that is missing, it is the "2 0" edge that is missing, and that is the last edge. The matrices start counting at 0.
Matthias has it; you don't need edgeCount - 1 since the range function doesn't include the end value in the iteration.
There are several other things you can do to clean up your code:
The with operator is preferred for opening files, since it closes them automatically for you
You don't need to call find and manually slice, split already does what you want.
You can convert and assign directly to a pair of numbers using a generator expression and iterable unpacking
You can call range with just an end value, the 0 start is implicit.
The multiplication operator is handy for initializing lists
With all of those changes:
with open('graph.txt', 'r') as graph:
node_count, edge_count = (int(n) for n in graph.readline().split())
matrix = [[0]*node_count for _ in range(node_count)]
for i in range(edge_count):
src, dst = (int(n) for n in graph.readline().split())
matrix[src][dst] = 1
print matrix
# [[0, 1, 0, 1, 0], [0, 0, 1, 0, 1], [1, 0, 0, 1, 0], [0, 1, 0, 0, 1], [1, 0, 1, 0, 0]]
Just to keep your code and style, of course it could be much more readable:
gs = open("graph.txt", "r")
gp = gs.readline()
gp_splitIndex = gp.split(" ")
gp_nodeCount = int(gp_splitIndex[0])
gp_edgeCount = int(gp_splitIndex[1])
matrix = [] # predecare the array
for i in range(0, gp_nodeCount):
matrix.append([])
for y in range(0, gp_nodeCount):
matrix[i].append(0)
for i in range(0, gp_edgeCount):
gp = gs.readline()
gp_Index = gp.split(" ") # get the index of space, dividing the 2 numbers on a row
gp_from = int(gp_Index[0])
gp_to = int(gp_Index[1])
matrix[gp_from][gp_to] = 1
print matrix
Exactly is the last instance not used..the 2 0 from your file. Thus the missed 1. Have a nice day!
The other answers are correct, another version similar to the one of tzaman:
with open('graph.txt', mode='r') as txt_file:
lines = [l.strip() for l in txt_file.readlines()]
number_pairs = [[int(n) for n in line.split(' ')] for line in lines]
header = number_pairs[0]
edge_pairs = number_pairs[1:]
num_nodes, num_edges = header
edges = [[0] * num_nodes for _ in xrange(num_nodes)]
for edge_start, edge_end in edge_pairs:
edges[edge_start][edge_end] = 1
print edges
I'm trying to get pixel values within contours. I've followed along with answers to similar questions but my results are off.
This block of code finds contours for an image and then iterates through them to find the contour containing the largest area. I added the ending if statement that tries to get the RGB value of the code if it is during daylight hours. The original image (video frame) is passed to a function I wrote (grab_rgb), along with the contour.
thresh = cv2.dilate(thresh, None, iterations=2)
(_, cnts, _) = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# look for motion
motion_found = False
biggest_area = 0
# examine the contours, looking for the largest one
for c in cnts:
(x, y, w, h) = cv2.boundingRect(c)
# get an approximate area of the contour
found_area = w * h
# find the largest bounding rectangle
if (found_area > MIN_AREA) and (found_area > biggest_area):
biggest_area = found_area
motion_found = True
if not is_nighttime():
rgb = grab_rgb(image, c)
else:
rgb = 'nighttime'
This is the function I wrote:
def grab_rgb(image, c):
pixels = []
# TODO: Convert to real code
# Detect pixel values (RGB)
mask = np.zeros_like(image)
cv2.drawContours(mask, c, -1, color=255, thickness=-1)
points = np.where(mask == 255)
for point in points:
pixel = (image[point[1], point[0]])
pixel = pixel.tolist()
pixels.append(pixel)
pixels = [tuple(l) for l in pixels]
car_color = (pixels[1])
r = car_color[0]
g = car_color[1]
b = car_color[2]
pixel_string = '{0},{1},{2}'.format(r, g, b)
return pixel_string
The code runs, but returns only three RGB values, with only the second value containing anything meaningful (values 0 and 2 are [0,0,0],[0,0,0]. There should definitely be more than three pixels within the contours, so I'm not sure where I went wrong.
EDIT: I realized it might be helpful to include what is actually being stored in the variables.
mask:
[[[ 0 0 0]
[ 0 0 0]
[ 0 0 0]
...,
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]]
[[ 0 0 0]
[255 0 0]
[ 0 0 0]
...,
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]]
[[ 0 0 0]
[ 0 0 0]
[ 0 0 0]
...,
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]]
...,
[[ 0 0 0]
[ 0 0 0]
[ 0 0 0]
...,
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]]
[[ 0 0 0]
[ 0 0 0]
[ 0 0 0]
...,
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]]
[[ 0 0 0]
[ 0 0 0]
[ 0 0 0]
...,
[ 0 0 0]
[ 0 0 0]
[ 0 0 0]]]
points:
(array([ 1, 1, 3, 5, 10, 11, 11, 12, 12, 13, 13, 14, 14], dtype=int32), array([ 1, 22, 22, 24, 24, 21, 23, 16, 20, 9, 15, 1, 8], dtype=int32), array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32))
pixel:
[0, 0, 0] [136, 89, 96] [0, 0, 0]
pixels:
[(0, 0, 0), (136, 89, 96), (0, 0, 0)]
car_color:
(136, 89, 96)
It seems like what you've asked the code to return is the RGB value of just the second point in the pixel values list (called 'pixels' here ) of the points in every contour passed to grab_rgb, with
car_color = (pixels[1])
r = car_color[0]
g = car_color[1]
b = car_color[2]
So the output should mean that your image has atleast three detected contours satisfying your area constraints, and that the RGB values of the second point in the contours' point lists are what you mentioned([0,0,0],[x,y,z] and [0,0,0]).