Related
I have following script:
import pandas as pd
from igraph import *
df_p_c = pd.read_csv('data/edges.csv')
...
edges = list_edges
vertices = list(dict_case_to_number.keys())
g = Graph(edges=edges, directed=True)
plot(g, bbox=(6000, 6000))
I have 2300 edges with rare connection. This is my plot of it:
And here are zooms of a few parts of it:
This plot is not readable because the distance between edges is too small. How can I have a bigger distance between edges? Only edges from the same 'family' have small distance.
Is there any other way to improve plots with a lot of edges?
I'm looking for any way to visualize parent-child correlation, it could be another python packet.
You seem to have a lot of small, disconnected components. If you want an informative graph, I think you should sort and group the connected components by size. Furthermore, the underlying assumption of many network layout algorithms is that there is a single giant component. Hence if you want sensible coordinates, you will often need to compute the layout for each component separately and then arrange the components with respect to each other. I would re-plot your graph in this way:
I have written the code for this graph using networkx as that is my module of choice. However, it would be very easy to substitute the networkx functions with igraph functions. The two functions that you need to replace are networkx.connected_component_subgraphs and whatever you want to use for the component_layout_func.
#!/usr/bin/env python
import numpy as np
import matplotlib.pyplot as plt
import networkx
def layout_many_components(graph,
component_layout_func=networkx.layout.spring_layout,
pad_x=1., pad_y=1.):
"""
Arguments:
----------
graph: networkx.Graph object
The graph to plot.
component_layout_func: function (default networkx.layout.spring_layout)
Function used to layout individual components.
You can parameterize the layout function by partially evaluating the
function first. For example:
from functools import partial
my_layout_func = partial(networkx.layout.spring_layout, k=10.)
pos = layout_many_components(graph, my_layout_func)
pad_x, pad_y: float
Padding between subgraphs in the x and y dimension.
Returns:
--------
pos : dict node : (float x, float y)
The layout of the graph.
"""
components = _get_components_sorted_by_size(graph)
component_sizes = [len(component) for component in components]
bboxes = _get_component_bboxes(component_sizes, pad_x, pad_y)
pos = dict()
for component, bbox in zip(components, bboxes):
component_pos = _layout_component(component, bbox, component_layout_func)
pos.update(component_pos)
return pos
def _get_components_sorted_by_size(g):
subgraphs = list(networkx.connected_component_subgraphs(g))
return sorted(subgraphs, key=len)
def _get_component_bboxes(component_sizes, pad_x=1., pad_y=1.):
bboxes = []
x, y = (0, 0)
current_n = 1
for n in component_sizes:
width, height = _get_bbox_dimensions(n, power=0.8)
if not n == current_n: # create a "new line"
x = 0 # reset x
y += height + pad_y # shift y up
current_n = n
bbox = x, y, width, height
bboxes.append(bbox)
x += width + pad_x # shift x down the line
return bboxes
def _get_bbox_dimensions(n, power=0.5):
# return (np.sqrt(n), np.sqrt(n))
return (n**power, n**power)
def _layout_component(component, bbox, component_layout_func):
pos = component_layout_func(component)
rescaled_pos = _rescale_layout(pos, bbox)
return rescaled_pos
def _rescale_layout(pos, bbox):
min_x, min_y = np.min([v for v in pos.values()], axis=0)
max_x, max_y = np.max([v for v in pos.values()], axis=0)
if not min_x == max_x:
delta_x = max_x - min_x
else: # graph probably only has a single node
delta_x = 1.
if not min_y == max_y:
delta_y = max_y - min_y
else: # graph probably only has a single node
delta_y = 1.
new_min_x, new_min_y, new_delta_x, new_delta_y = bbox
new_pos = dict()
for node, (x, y) in pos.items():
new_x = (x - min_x) / delta_x * new_delta_x + new_min_x
new_y = (y - min_y) / delta_y * new_delta_y + new_min_y
new_pos[node] = (new_x, new_y)
return new_pos
def test():
from itertools import combinations
g = networkx.Graph()
# add 100 unconnected nodes
g.add_nodes_from(range(100))
# add 50 2-node components
g.add_edges_from([(ii, ii+1) for ii in range(100, 200, 2)])
# add 33 3-node components
for ii in range(200, 300, 3):
g.add_edges_from([(ii, ii+1), (ii, ii+2), (ii+1, ii+2)])
# add a couple of larger components
n = 300
for ii in np.random.randint(4, 30, size=10):
g.add_edges_from(combinations(range(n, n+ii), 2))
n += ii
pos = layout_many_components(g, component_layout_func=networkx.layout.circular_layout)
networkx.draw(g, pos, node_size=100)
plt.show()
if __name__ == '__main__':
test()
EDIT
If you want the subgraphs tightly arranged, you need to install rectangle-packer (pip install rectangle-packer), and substitute _get_component_bboxes with this version:
import rpack
def _get_component_bboxes(component_sizes, pad_x=1., pad_y=1.):
dimensions = [_get_bbox_dimensions(n, power=0.8) for n in component_sizes]
# rpack only works on integers; sizes should be in descending order
dimensions = [(int(width + pad_x), int(height + pad_y)) for (width, height) in dimensions[::-1]]
origins = rpack.pack(dimensions)
bboxes = [(x, y, width-pad_x, height-pad_y) for (x,y), (width, height) in zip(origins, dimensions)]
return bboxes[::-1]
Edit #2
I have written a library for visualising networks, which is called netgraph. It automatically handles networks with multiple components in the way outlined above. It is fully compatible with networkx and igraph Graph objects, so it should be easy and fast to make great looking graphs of graphs (at least that is the idea).
import itertools
import matplotlib.pyplot as plt
import networkx as nx
# installation easiest via pip:
# pip install netgraph
from netgraph import Graph
# construct the graph as before:
g = nx.Graph()
# add 30 unconnected nodes
g.add_nodes_from(range(30))
# add 15 2-node components
g.add_edges_from([(ii, ii+1) for ii in range(30, 60, 2)])
# add 10 3-node components
for ii in range(60, 90, 3):
g.add_edges_from([(ii, ii+1), (ii, ii+2), (ii+1, ii+2)])
# add a couple of larger components
n = 90
for ii in [10, 20, 40]:
g.add_edges_from(itertools.combinations(range(n, n+ii), 2))
n += ii
# if there are any disconnected components, netgraph automatically handles them separately
Graph(g, node_layout='circular', node_size=1, node_edge_width=0.1, edge_width=0.1, edge_color='black', edge_alpha=1.)
plt.show()
You could checkout networkx, which is a pretty nice graph library. Networkx has direct plotting support for matplotlib.
It supports various layout types, for example spring layout, random layout, and a few more
You should especially look at spring layout, which has a few interesting parameters for your use-case:
k (float (default=None)) – Optimal distance between nodes. If None the
distance is set to 1/sqrt(n) where n is the number of nodes. Increase
this value to move nodes farther apart.
Or both of these in combination with a custom layout:
pos (dict or None optional (default=None)) – Initial positions for
nodes as a dictionary with node as keys and values as a coordinate
list or tuple. If None, then use random initial positions.
fixed (list or None optional (default=None)) – Nodes to keep fixed at initial
position.
The edge weight might also be something you can tune in order to get results you like:
weight (string or None optional (default=’weight’)) – The edge
attribute that holds the numerical value used for the edge weight. If
None, then all edge weights are 1.
I would recommend combining networkx with bokeh, which is a new plotting library that creates web-based html/js plots. It has direct support for networkx, and has some nice features like easy integration of node hover tools. If your graph isn't too big, the performance is pretty good. (I've plotted graphs with about 20000 nodes and a few thousand edges).
With both libraries combined, all you need is the following bit of code for a simple example (from the documentation) that tries to build an optimized layout:
import networkx as nx
from bokeh.io import show, output_file
from bokeh.plotting import figure
from bokeh.models.graphs import from_networkx
G=nx.karate_club_graph() # Replace with your own graph
plot = figure(title="Networkx Integration Demonstration", x_range=(-1.1,1.1), y_range=(-1.1,1.1),
tools="", toolbar_location=None)
graph = from_networkx(G, nx.spring_layout, scale=2, center=(0,0))
plot.renderers.append(graph)
output_file("networkx_graph.html")
show(plot)
Do you know what meaning you are looking for? Or are you exploring? Or is this a specific question about zooming issues?
So far, you have done a good job of seeing the overall structure. Some ideas you might consider making new vocabulary with a few routines to support it. For example, if you make a small cluster be the set of points and edges that are together, then you can plot histograms, visualizations of clusters overlayed on each other, compare clusters with and without long nodes, and so one.
Developing the A* algorithm for path planning, I am trying to get a list called edges in which all connections from one pixel to its neighbour pixels that occur in a non-occupied space (where the pixel value is 1).
The image from which I compute this connections is a 351x335 pixels image.
Pixels P2,4,6,8 are at a distance=1 from the center, while pixels P1,3,5,7 are at a approximate distance=1.4 from the center (Pythagoras theorem); see image:
The code written for the edges loop is never ending. Is this taking too much computational time due to the loops? The vertices loop ends in a second or so.
Note: I am initializing the lists as two very big arrays and cutting them at the end to not use dynamic allocation.
EDIT: The image (imOut) is the following one:
Link to image used as map
EDIT: The full code is the following:
'''
IMPORTS
'''
import cv2 as cv # Import OpenCV
import numpy as np # Import Numpy
from skimage.color import rgb2gray
import math
from datetime import datetime
import matplotlib.pyplot as plt
from scipy import arange
'''
CODE SETTINGS
'''
# Allowing to print full array without truncation
np.set_printoptions(threshold=np.nan)
'''
MAIN PROGRAM
'''
im = rgb2gray(cv.imread('map1.png'))
imOut = im # Making a copy of the image to output
# plt.imshow(imOut)
# plt.show()
vertices = np.zeros((imOut.shape[0]*imOut.shape[1], 3)) # 1st col to x, 2nd col to y, 3rd col to heuristic (euclidean distance to QGoal)
edges = np.zeros((100*imOut.shape[0]*imOut.shape[1], 3)) # 1st col 1st vertex, 2nd col 2nd vertex, 3rd col edge length
# Initialization of vertices with start pos.
# CREATE VERTICES AND EDGES LISTS FROM THE MAP PROVIDED
'''
Vertices List -> Add all obstacle-free configurations to the vertices list.
Edges List -> Go pixel by pixel in the map and, if they are obstacle-free configurations,
add, out of the 8 neighbouring pixels, the ones that are obstacle-free as feasible edges.
'''
# Vertices list creation
indexVertices = 0
for i in range(0, imOut.shape[0]):
for j in range(0, imOut.shape[1]):
if imOut[i,j] == 1: # If it's in free space
# Compute heuristic to goal node (euclidean distance).
heuristic = math.sqrt(pow(i-QGoal[0],2)
+ pow(j-QGoal[1],2))
vertices[indexVertices,:] = [i, j, heuristic]
indexVertices = indexVertices + 1
vertices = vertices[0:indexVertices,:]
# Edges list creation
# I loop over the same vertices array, as it only contains the free pixels.
indexEdges = 0
for i in range(0, vertices.shape[0]):
for k in range(0, vertices.shape[0]):
# If it is not the same pixel that we are checking
if i != k:
# Check if it is a neighbouring pixel and, if so,
#add it to the list of edges with its distance(path cost).
pathCost = (math.sqrt(pow(vertices[i,0]
- vertices[k,0], 2)
+ pow(vertices[i,1]
- vertices[k,1], 2))
if pathCost == 1 or round(pathCost,1) == 1.4:
edges[indexEdges,:] = [i, k, pathCost]
indexEdges = indexEdges + 1
edges = edges[0:indexEdges,:]
I'm using NetworkX in python. Given any undirected and unweighted graph, I want to loop through all the nodes. With each node, I want to add a random edge and/or delete an existing random edge for that node with probability p. Is there a simple way to do this? Thanks a lot!
Create a new random edge in networkx
Let's set up a test graph:
import networkx as nx
import random
import matplotlib.pyplot as plt
graph = nx.Graph()
graph.add_edges_from([(1,3), (3,5), (2,4)])
nx.draw(graph, with_labels=True)
plt.show()
Now we can pick a random edge from a list of non-edge from the graph. It is not totally clear yet what is the probability you mentioned. Since you add a comment stating that you want to use random.choice I'll stick to that.
def random_edge(graph, del_orig=True):
'''
Create a new random edge and delete one of its current edge if del_orig is True.
:param graph: networkx graph
:param del_orig: bool
:return: networkx graph
'''
edges = list(graph.edges)
nonedges = list(nx.non_edges(graph))
# random edge choice
chosen_edge = random.choice(edges)
chosen_nonedge = random.choice([x for x in nonedges if chosen_edge[0] == x[0]])
if del_orig:
# delete chosen edge
graph.remove_edge(chosen_edge[0], chosen_edge[1])
# add new edge
graph.add_edge(chosen_nonedge[0], chosen_nonedge[1])
return graph
Usage exemple:
new_graph = random_edge(graph, del_orig=True)
nx.draw(new_graph, with_labels=True)
plt.show()
We can still add a probability distribution over the edges in random.choiceif you need to (using numpy.random.choice() for instance).
Given a node i, To add edges without duplication you need to know (1) what edges from i already exist and then compute (2) the set of candidate edges that don't exist from i. For removals, you already defined a method in the comment - which is based simply on (1).
Here is a function that will provide one round of randomised addition and removal, based on list comprehensions
def add_and_remove_edges(G, p_new_connection, p_remove_connection):
'''
for each node,
add a new connection to random other node, with prob p_new_connection,
remove a connection, with prob p_remove_connection
operates on G in-place
'''
new_edges = []
rem_edges = []
for node in G.nodes():
# find the other nodes this one is connected to
connected = [to for (fr, to) in G.edges(node)]
# and find the remainder of nodes, which are candidates for new edges
unconnected = [n for n in G.nodes() if not n in connected]
# probabilistically add a random edge
if len(unconnected): # only try if new edge is possible
if random.random() < p_new_connection:
new = random.choice(unconnected)
G.add_edge(node, new)
print "\tnew edge:\t {} -- {}".format(node, new)
new_edges.append( (node, new) )
# book-keeping, in case both add and remove done in same cycle
unconnected.remove(new)
connected.append(new)
# probabilistically remove a random edge
if len(connected): # only try if an edge exists to remove
if random.random() < p_remove_connection:
remove = random.choice(connected)
G.remove_edge(node, remove)
print "\tedge removed:\t {} -- {}".format(node, remove)
rem_edges.append( (node, remove) )
# book-keeping, in case lists are important later?
connected.remove(remove)
unconnected.append(remove)
return rem_edges, new_edges
To see this function in action:
import networkx as nx
import random
import matplotlib.pyplot as plt
p_new_connection = 0.1
p_remove_connection = 0.1
G = nx.karate_club_graph() # sample graph (undirected, unweighted)
# show original
plt.figure(1); plt.clf()
fig, ax = plt.subplots(2,1, num=1, sharex=True, sharey=True)
pos = nx.spring_layout(G)
nx.draw_networkx(G, pos=pos, ax=ax[0])
# now apply one round of changes
rem_edges, new_edges = add_and_remove_edges(G, p_new_connection, p_remove_connection)
# and draw new version and highlight changes
nx.draw_networkx(G, pos=pos, ax=ax[1])
nx.draw_networkx_edges(G, pos=pos, ax=ax[1], edgelist=new_edges,
edge_color='b', width=4)
# note: to highlight edges that were removed, add them back in;
# This is obviously just for display!
G.add_edges_from(rem_edges)
nx.draw_networkx_edges(G, pos=pos, ax=ax[1], edgelist=rem_edges,
edge_color='r', style='dashed', width=4)
G.remove_edges_from(rem_edges)
plt.show()
And you should see something like this.
Note that you could also do something similar with the adjacency matrix,
A = nx.adjacency_matrix(G).todense() (it's a numpy matrix so operations like A[i,:].nonzero() would be relevant). This might be more efficient if you have extremely large networks.
I've been studying the other networkx plotting posts, but I've been having a hard time adapting them to my problem.
1) How do I create subplots with network graphs, without a PREDEFINED number of objects to plot? The function grabs this dynamically.
2) Is there an easy way to filter the network graph by limiting only those edges with over a weight of 2, for instance? or do I have to edit the network object itself to do so?
UPDATE #2: I figured out a way to filter by degree (see below). I'm wondering more generally if there are better ways to make my network data more understandable?
nol comes in the format [ [Year, networkobject], [Year, networkobject]]
def standardgraph_multiyear(nol, minimumdegree):
"""
Plots multiple graphs based on year
nol = takes in a LIST of [year, network object]
minimum = takes in a digit to filter nodes by degree
"""
#Each iteration prints a new subplot
numrows = len(nol)
fig = plt.figure(figsize=(10,60))
for i, val in enumerate(nol):
gloc = numrows,1,i+1
plt.subplot(numrows, 1, i+1)
if minimumdegree > 0:
outdeg = val[1].degree()
to_keep = [n for n in outdeg if outdeg[n] > minimumdegree]
mingraph = val[1].subgraph(to_keep)
pos = nx.spring_layout(mingraph, iterations=200)
nx.draw(mingraph, pos, font_size=8, with_labels=True)
nx.draw_networkx_edges(mingraph, pos, alpha=.2)
nx.draw_networkx_nodes(mingraph, pos, node_size=60, font_size =8, labels=True)
nx.draw_networkx_labels(mingraph, pos, font_color='k', font_size=8)
plt.title("Year {0}".format(val[0]), loc = 'center', fontsize=20)
if minimumdegree == 0:
outdeg = val[1].degree()
to_keep = [n for n in outdeg if outdeg[n] > minimumdegree]
mingraph = val[1].subgraph(to_keep)
pos = nx.spring_layout(mingraph, iterations=200)
nx.draw(mingraph, pos, font_size=8, with_labels=True)
nx.draw_networkx_edges(mingraph, pos, alpha=.2)
nx.draw_networkx_nodes(mingraph, pos, node_size=60, font_size =8, labels=True)
nx.draw_networkx_labels(mingraph, pos, font_color='k', font_size=8)
plt.title("Year {0}".format(val[0]), loc = 'center', fontsize=20)
return
fig.savefig('out.png', dpi=100)
Your out of range error likely comes from the call to plt.subplot(221+i), since you don't seem to limit i to be <4; thus matplotlib will not know what subplot you intend to refer to?
(You also seem to have some conflicting code assembling the plots: a call to plt.subplots(1,1) and a later which requests a 2x2 grid).
In a different question I used the more basic plt.subplot(xxx) syntax to generate multiple subplots (following the four grids example from networkx). But you can also do it as shown below, setting the ax= keyword argument to an already existing set of axes. Note the call to sca() before rendering to each axis, which I needed to get this to work.
I've also shown one way to filter the edges that are shown below, and it does not require modifying the underlying graph: instead, you construct the edge lineweights you want based on the data from your graph, and use that as argument to draw_networkx_edges.
Edit (re updated question): the example code now includes a more explicit illustration of how to handle an unknown number of networks.
import matplotlib.pyplot as plt
import networkx as nx
import numpy as np
n = 15; m = 40 # graph size
L = np.random.choice(xrange(n), 2*m) # select some edge destinations
weights = 0.5 + 5 * np.random.rand(m) # give each edge a weight
G = nx.Graph() # create a graph object
G.add_nodes_from(xrange(n)) # add n nodes to it
for i, (fr, to) in enumerate(zip(L[1::2], L[::2])):
G.add_edge(fr, to, weight=weights[i]) # add each edge
# use one of the edge properties to control line thickness
edgewidth = [ d['weight'] for (u,v,d) in G.edges(data=True)]
# and create a filtered version (still need an entry for each edge)
w_threshold = 2
edgewidth_filtered = [ d['weight'] if d['weight'] > w_threshold else 0
for (u,v,d) in G.edges(data=True)]
# alt. filtering - all edges that meet some criterion are displayed uniformly
binary_filtered_edges = [ 1 if d['weight'] > w_threshold else 0
for (u,v,d) in G.edges(data=True)]
titles = [ 'edges drawn with lineweight=1', 'edge width from edge weight',
'edge width from edge weight, only strong edges',
'strong edges shown as lineweight=1', ]
edge_display_params = [ np.ones(len(edgewidth),), edgewidth,
edgewidth_filtered, binary_filtered_edges,]
# to illustrate drawing an unknown number of graphs, add some repeats repeats
n_extra = np.random.randint(0, 5)
indices = range(len(edge_display_params)) * 3
indices = indices[len(edge_display_params) + n_extra:]
# layout
pos = nx.spring_layout(G, iterations=50)
pos = nx.circular_layout(G)
#pos = nx.random_layout(G)
# rendering
fig = plt.figure(1); plt.clf()
# compute a grid size that will fit all graphs on it (couple blanks likely)
nr = int(np.ceil(np.sqrt(len(indices))))
fig, ax = plt.subplots(nr, nr, num=1)
for i, j in enumerate(indices):
# dereference index into valid data (needed here since some repeated rather
# than creating more, to illustrate handling unknown amount of data)
k = indices[j]
widths = edge_display_params[k]
# compute index for the subplot, and set this subplot as current
ix = np.unravel_index(i, ax.shape)
plt.sca(ax[ix])
# draw all nodes homogeneously, and edge weights as filtered
nx.draw_networkx_nodes(G, pos, ax=ax[ix])
nx.draw_networkx_edges(G, pos, width=widths, ax=ax[ix],)
ax[ix].set_title(titles[k], fontsize=10)
ax[ix].set_axis_off()
plt.show()
This example uses the same input graph four times over but obviously you could apply a single filter to different graphs (by filtering within the plotting loop) instead of applying different filters.
Below shows one run that created an extra 4 graphs, and so we have one unused pane:
Is there an easy way to get the (x,y) values of a contour line that was plotted like this:
import matplotlib.pyplot as plt
x = [1,2,3,4]
y = [1,2,3,4]
m = [[15,14,13,12],[14,12,10,8],[13,10,7,4],[12,8,4,0]]
cs = plt.contour(x,y,m, [9.5])
plt.show()
Look at the collections property of the returned ContourSet. In particular the get_paths() method of the first collection returns paired points making up each line segment.
cs.collections[0].get_paths()
To get a NumPy array of the coordinates, use the Path.vertices attribute.
p1 = cs.collections[0].get_paths()[0] # grab the 1st path
coor_p1 = p1.vertices
Going through the collections and extracting the paths and vertices is not the most straight forward or fastest thing to do. The returned Contour object actually has attributes for the segments via cs.allsegs, which returns a nested list of shape [level][element][vertex_coord]:
num_levels = len(cs.allsegs)
num_element = len(cs.allsegs[0]) # in level 0
num_vertices = len(cs.allsegs[0][0]) # of element 0, in level 0
num_coord = len(cs.allsegs[0][0][0]) # of vertex 0, in element 0, in level 0
Hence the vertices of an all paths can be extracted as:
cs.allsegs[i][j] # for element j, in level i
See reference:
https://matplotlib.org/3.1.1/api/contour_api.html