I'm trying to implement the Warshall algorithm in python 3 to create a matrix with the shortest distance between each point.
This is supposed to be a simple implementation, I make a matrix and fill it with the distance between each point.
However, I'm getting the wrong result, and I dont know what is the problem with my implementation.
#number of vertex (N), number of connections(M)
N, M = 4,4;
#my matrix [A,B,C] where A and B indicates a connection
#from A to B with a distance C
A = [[0,1,2],[0,2,4],[1,3,1],[2,3,5]];
#matrix alocation
inf = float("inf");
dist = [[inf for x in range(N)] for y in range(M)];
#set distances from/to the same vertex as 0
for vertex in range(N):
dist[vertex][vertex] = 0;
#set the distances from each vertex to the other
#they are bidirectional.
for vertex in A:
dist[vertex[0]][vertex[1]] = vertex[2];
dist[vertex[1]][vertex[0]] = vertex[2];
#floyd warshall algorithm
for k in range(N):
for i in range(N):
for j in range(N):
if dist[i][j] > dist[i][k] + dist[k][j]:
dist[1][j] = dist[i][k] + dist[k][j];
print(dist);
Expected Matrix on the first index (dist[0]):
[0, 2, 4, 3]
Actual result:
[0, 2, 4, inf]
for some reason I keep getting inf instead of 3 on dist[0][3].
What am I missing?
It's a little tricky to spot, but a simple change-by-change trace of your program spots the problem:
if dist[i][j] > dist[i][k] + dist[k][j]:
dist[1][j] = dist[i][k] + dist[k][j];
^ This should be i, not 1
You're changing the distance from node 1 to the target node; rather than from the source node. Your resulting distance matrix is
[0, 2, 4, 3]
[2, 0, 6, 1]
[4, 6, 0, 5]
[3, 1, 5, 0]
See this lovely debug blog for help.
Related
I am looking for a way to speed up the specific operation on tensors in PyTorch. Since it is a general operation on matrices, I am open to answers in NumPy as well.
Let's say I have a tensor with values from 0 to N-1 (N=4) where each value repeats the same number of times (R=2).
import torch
x = torch.Tensor([0, 0, 1, 1, 2, 2, 3, 3])
In this case, it is sorted, but any permutation of x is also in the set of considered tensors X.
I am getting an input tensor with values from 0 to N-1 but without any constraints on the repetition.
z = torch.tensor([3, 2, 3, 0, 2, 3, 1, 2])
And I would like to find an efficient implementation of foo such that y = foo(z). y should be some permutation of x (from the set X) that tries to do as few changes in z as possible (in terms of Hamming distance), for example
y = torch.tensor([3, 2, 3, 0, 2, 0, 1, 1])
The trivial solution is to keep counting the number elements with the same value, but it is extremely inefficient to process elements one-by-one for larger tensors:
def foo(z):
R = 2
N = 4
counters = [0] * N
# first, we replace extra elements with -1
y = []
for elem in z:
if counters[elem] < R:
counters[elem] += 1
y.append(elem)
else:
y.append(-1)
y = torch.tensor(y)
assert torch.equal(y, torch.tensor([3, 2, 3, 0, 2, -1, 1, -1]))
# second, we replace -1 by "unfilled" counters
for i in range(len(y)):
if y[i] == -1:
first_unfilled = [n for n in range(N) if counters[n] < R][0]
counters[first_unfilled] += 1
y[i] = first_unfilled
return y
assert torch.equal(y, foo(z))
I want to convert [0, 0, 1, 0, 1, 0, 1, 0] to [2, 4, 6] using ortools.
Where "2", "4", "6" in the second list are the index of "1" in the first list.
Using the below code I could get a list [0, 0, 2, 0, 4, 0, 6, 0]. How can I get [2, 4, 6]?
from ortools.sat.python import cp_model
model = cp_model.CpModel()
solver = cp_model.CpSolver()
work = {}
days = 8
horizon = 7
for i in range(days):
work[i] = model.NewBoolVar("work(%i)" % (i))
model.Add(work[0] == 0)
model.Add(work[1] == 0)
model.Add(work[2] == 1)
model.Add(work[3] == 0)
model.Add(work[4] == 1)
model.Add(work[5] == 0)
model.Add(work[6] == 1)
model.Add(work[7] == 0)
v1 = [model.NewIntVar(0, horizon, "") for _ in range(days)]
for d in range(days):
model.Add(v1[d] == d * work[d])
status = solver.Solve(model)
print("status:", status)
vec = []
for i in range(days):
vec.append(solver.Value(work[i]))
print("work",vec)
vec = []
for v in v1:
vec.append(solver.Value(v))
print("vec1",vec)
You should see this output on the console,
status: 4
work [0, 0, 1, 0, 1, 0, 1, 0]
vec1 [0, 0, 2, 0, 4, 0, 6, 0]
Thank you.
Edit:
I also wish to get a result as [4, 6, 2].
For just 3 variables, this is easy. In pseudo code:
The max index is max(work[i] * i)
The min index is min(horizon - (horizon - i) * work[i])
The medium is sum(i * work[i]) - max_index - min_index
But that is cheating.
If you want more that 3 variable, you will need parallel arrays of Boolean variables that indicate the rank of each variable.
Let me sketch the full solution.
You need to build a graph. The X axis are the variables. The why axis are the ranks. You have horizontal arcs going right, and diagonal arcs going right and up. If the variable is selected, you need to use a diagonal arc, otherwise an horizontal arc.
If using a diagonal arc, you will assign the current variable to the rank of the tail of the arc.
Then you need to add constraints to make it a contiguous path:
mass conservation at each node
variable is selected -> one of the diagonal arc must be selected
variable is not selected -> one of the horizontal arc must be selected
bottom left node has one outgoing arc
top right node has one incoming arc
For example this array of GPS Coordinates:
GPSS = [{"Lat":40.641099,"Lon": -73.917094},{"Lat":40.60442,"Lon": -74.054873},{"Lat":40.779582,"Lon": -73.920213},{"Lat":40.651616,"Lon": -73.89097},{"Lat":40.755183,"Lon": -73.846248}]
I have already calculated the Distances below for each possible combination:
Distances = [{'GPSS': [0, 1], 'Distance': 12.34895151892164}, {'GPSS': [0, 2], 'Distance': 15.380561959360797}, {'GPSS': [0, 3], 'Distance': 2.499303143635897}, {'GPSS': [0, 4], 'Distance': 14.012560598709298}, {'GPSS': [1, 2], 'Distance': 22.53687775052488}, {'GPSS': [1, 3], 'Distance': 14.824576927209662}, {'GPSS': [1, 4], 'Distance': 24.318038568441654}, {'GPSS': [2, 3], 'Distance': 14.423642658224264}, {'GPSS': [2, 4], 'Distance': 6.807346029310139}, {'GPSS': [3, 4], 'Distance': 12.106031672624894}]
0,1 = referring to 40.641099,-73.917094 and 40.60442,-74.054873
1,4 = 40.641099,-73.917094 and 40.755183,-73.846248
I would now like to find out the shortest Distance (route) to visit each set of coordinates, so it's most likely not going to be point 0 to 1 to 2 to 3 to 4.
But something like 1 to 3 to 4 to 2 to 0.
How would I accomplish something like this?
This is as far as I got:
for index, d in enumerate(Distances):
print(d['GPSS'])
Total = d['Distance']
Start = d['GPSS'][1] #[0]
CheckPoints = []
CheckPoints.append(d['GPSS'][0])
CheckPoints.append(d['GPSS'][1])
for index2, d2 in enumerate(Distances):
if index != index2:
if Start == d2['GPSS'][0]: #0-1, 1-2, 2-3
Total += d2['Distance']
Start += 1
if d2['GPSS'][0] not in CheckPoints:
CheckPoints.append(d2['GPSS'][0])
if d2['GPSS'][1] not in CheckPoints:
CheckPoints.append(d2['GPSS'][1])
#print(CheckPoints)
print("+"+str(d2['Distance'])+" = "+str(Total)+" | "+str(Start)+" - "+str(d2['GPSS']))
if len(CheckPoints) <= len(GPSS)-1: #GPPS - is from above
for x in range(len(GPSS)-1):
if x not in CheckPoints:
for d3 in Distances:
if d3['GPSS'][0] == x and d3['GPSS'][1] == CheckPoints[-1]:
print("HERE")
print(d3)
Total += d3['Distance']
break
print(Total)
Any help would be much appreciated.
Thanks
The best way to do what you are looking for is to create a Graph. If you do not know what that is, you should look it up as it's a very important data structure. You will probably also need to know what it is to fully understand the following code. Python does not have a built in graph so you need to create your own.
The type of graph you are going to need is a un-directed weighted graph with all of the nodes, or in your case GPS coordinates, connected to each other. Then you can sort the graph by using a form of "Dijkstra's Algorithm" to find the shortest path to all of the points.
Below is an implementation of what you are looking for. However I coded this to work with a list containing lists of paired coordinates. It also includes a driver, driver(), you can call to test it out.
I wrote this up quick and didn't code it as a class, but in the real world you most definitely should.
As a note, when you run the driver function it will execute the code and print out all of the possible paths and their weights for the provided coordinate list. "Weight" in your case refers to the distance between the points. The list printed shows the path it took with "1" referring to the pair of points at index "0" of the coordinate list. The next number in the list is the pair of points it went to next.
If you have any further questions feel free to ask
from collections import defaultdict
from math import sqrt
# Shortest path to all coordinates from any node
# Coordinates must be provided as a list containing lists of
# x/y pairs. ie [[23.2321, 58.3123], [x.xxx, y.yyy]]
def distance_between_coords(x1, y1, x2, y2):
distance = sqrt(((x2 - x1) ** 2) + ((y2 - y1) ** 2))
return distance
# Adds "names" to coordinates to use as keys for edge detection
def name_coords(coords):
coord_count = 0
for coord in coords:
coord_count += 1
coord.append(coord_count)
return coords
# Creates a weighted and undirected graph
# Returns named coordinates and their connected edges as a dictonary
def graph(coords):
coords = name_coords(coords)
graph = defaultdict(list)
edges = {}
for current in coords:
for comparer in coords:
if comparer == current:
continue
else:
weight = distance_between_coords(current[0], current[1],
comparer[0], comparer[1])
graph[current[2]].append(comparer[2])
edges[current[2], comparer[2]] = weight
return coords, edges
# Returns a path to all nodes with least weight as a list of names
# from a specific node
def shortest_path(node_list, edges, start):
neighbor = 0
unvisited = []
visited = []
total_weight = 0
current_node = start
for node in node_list:
if node[2] == start:
visited.append(start)
else:
unvisited.append(node[2])
while unvisited:
for index, neighbor in enumerate(unvisited):
if index == 0:
current_weight = edges[start, neighbor]
current_node = neighbor
elif edges[start, neighbor] < current_weight:
current_weight = edges[start, neighbor]
current_node = neighbor
total_weight += current_weight
unvisited.remove(current_node)
visited.append(current_node)
return visited, total_weight
def driver():
coords = [[1.7592675, 92.4836507], [17.549836, 32.457398],
[23.465896, 45], [25.195462, 37.462742],
[42.925274, 63.234028], [2.484631, 5.364871],
[50.748376, 36.194797]]
coords, edges = graph(coords)
shortest_path(coords, edges, 3)
shortest_path_taken = []
shortest_path_weight = 0
for index, node in enumerate(coords):
path, weight = shortest_path(coords, edges, index + 1)
print('--------------------------------------')
print("Path", index + 1, "=", path)
print("Weight =", weight)
if index == 0:
shortest_path_weight = weight
shortest_path_taken = path
elif weight < shortest_path_weight:
shortest_path_weight = weight
shortest_path_taken = path
print('--------------------------------------')
print("The shortest path to all nodes is:", shortest_path_taken)
print("The weight of the path is:", shortest_path_weight)
Edit:
Here is what the output will look like when you call the driver function:
--------------------------------------
Path 1 = [1, 5, 3, 4, 2, 7, 6]
Weight = 386.3252849770695
--------------------------------------
Path 2 = [2, 4, 3, 6, 7, 5, 1]
Weight = 189.3710721663407
--------------------------------------
Path 3 = [3, 4, 2, 5, 7, 6, 1]
Weight = 173.99235180101968
--------------------------------------
Path 4 = [4, 3, 2, 7, 5, 6, 1]
Weight = 172.86112533927678
--------------------------------------
Path 5 = [5, 3, 7, 4, 2, 1, 6]
Weight = 247.08415835699554
--------------------------------------
Path 6 = [6, 2, 4, 3, 7, 5, 1]
Weight = 330.1567215845902
--------------------------------------
Path 7 = [7, 4, 5, 3, 2, 6, 1]
Weight = 247.70066871941674
--------------------------------------
The shortest path to all nodes is: [4, 3, 2, 7, 5, 6, 1]
The weight of the path is: 172.86112533927678
[Finished in 0.1s]*
I'm attempting to convert a double summation formula into code, but can't figure out the correct matrix/vector representation of it.
The first summation is i to n, and the second is over j > i to n.
I'm guessing there is a much more efficient & pythonic way of writing this?
I resorted to nested for loops to just get it working but, as expected, it runs very slowly with a large dataset:
def wapc_denom(weights, vols):
x = []
y = []
for i, wi in enumerate(weights):
for j, wj in enumerate(weights):
if j > i:
x.append(wi * wj * vols[i] * vols[j])
y.append(np.sum(x))
return np.sum(y)
Edit:
Using guidance from smci's answer I think I have a potential solution:
def wapc_denom2(weights, vols):
return np.sum(np.tril(np.outer(weights, vols.T)**2, k=-1))
Assuming you want to count every term only once (for that you have to move the x = [] into the outer loop) one cheap way of computing the sum would be
Create mock data
weights = np.random.random(10)
vols = np.random.random(10)
Do the calculation
wv = weights * vols
result = (wv.sum()**2 - wv#wv) / 2
Check that it's the same
def wapc_denom(weights, vols):
y = []
for i, wi in enumerate(weights):
x = []
for j, wj in enumerate(weights):
if j > i:
x.append(wi * wj * vols[i] * vols[j])
y.append(np.sum(x))
return np.sum(y)
assert np.allclose(result, wapc_denom(weights, vols))
Why does it work?
What we are doing is compute the sum of the full matrix, subtract the diagonal and divide by two. This is cheap because it is easy to verify that the sum of an outer product is just the product of the summed factors.
wi * wj * vols[i] * vols[j] is a telltale. vols is another vector, so first you want to compute the vector wv = w * vols
then (wj * vols[j]) * (wi * vols[i]) = wv^T * wv is your (matrix outer product) expression; that's a column vector * a row vector. But actually you only want the sum. So I don't see a need to construct a vector y.append(np.sum(x)), you're only going to sum it anyway np.sum(y)
also the if j > i part means you only want the sum of the Lower Triangular part, and exclude the diagonal.
EDIT: the result is fully determined just from wv, I didn't think we needed the matrix to get the sum, and we didn't need the diagonal; #PaulPanzer found the most compact expression.
You can use triangulations in numpy, check np.triu and np.meshgrid. Do:
np.product(np.triu(np.meshgrid(weights,weights), 1) * np.triu(np.meshgrid(vols,vols), 1),0).sum(1).cumsum().sum()
Example:
w = np.arange(4) +1
v = np.array([1,3,2,2])
print(np.triu(np.meshgrid(w,w), k=1))
>>array([[[0, 2, 3, 4],
[0, 0, 3, 4],
[0, 0, 0, 4],
[0, 0, 0, 0]],
[[0, 1, 1, 1],
[0, 0, 2, 2],
[0, 0, 0, 3],
[0, 0, 0, 0]]])
# example of product + triu + meshgrid (your x values):
print(np.product(np.triu(np.meshgrid(w,w), 1) * np.triu(np.meshgrid(v,v), 1),0))
>>array([[ 0, 6, 6, 8],
[ 0, 0, 36, 48],
[ 0, 0, 0, 48],
[ 0, 0, 0, 0]])
print(np.product(np.triu(np.meshgrid(w,w), 1) * np.triu(np.meshgrid(v,v), 1),0).sum(1).cumsum().sum())
>> 428
print(wapc_denom(w, v))
>> 428
I'm struggling to work out how to effectively implement this, even though I know what I'm doing wrong. I'm trying to get my code to read an adjacency list for example an undirected, weighted graph:
[(1,5)], [(0,5), (2,7)], [(1,7)]
And then convert that to an adjacency matrix, which would return:
[0, 5, inf], [5, 0, 7], [inf, 7, 0]
The code below however returns [0, 5, inf], [5, inf, 0, inf, 7], [inf, 7, 0], and I know why this is. However, I only want to append 'inf' to the adjacency matrix in cases like [0, 5, inf] because 0 is not adjacent to 2 and thus its weight is 'inf'. What's the best solution?
def adjacency_matrix(graph_string):
adj_list = adjacency_list(graph_string)
n = len(adj_list)
adj_mat = [[] for _ in range(n)]
for i in range(n):
for j in range(n):
if j == i:
adj_mat[i].append(0)
else:
for neighbour, weight in adj_list[i]:
if j == neighbour:
adj_mat[i].append(weight)
break
elif j != neighbour:
adj_mat[i].append(float('inf'))
return adj_mat
The problem seems to be in the elif part
elif j != neighbour:
adj_mat[i].append(float('inf'))
Because you only want to fill the inf for the missing edges. Using condition elif j < neighbour would be correct if you have your adj_list sorted.
However, a better solution would be initializing the adjacency matrix with zero diagonal and inf values elsewhere. And only filling the weights from adjacency list. This way you avoid thinking about non-edges.
Here is a short example how that could be implemented using numpy.
import numpy as np
def adj_list_to_matrix(adj_list):
n = len(adj_list)
adj_matrix = np.nan * np.ones((n,n))
np.fill_diagonal(adj_matrix,0)
for i in range(n):
for j, w in adj_list[i]:
adj_matrix[i,j] = w
return adj_matrix
Usage:
adj_list = [(1,5)], [(0,5), (2,7)], [(1,7)]
adj_list_to_matrix(adj_list)