I'm having trouble figuring out how to add attributes to nodes in my network from columns in my dataframe.
I have provided an example of my dataframe below, there are around 10 columns in total, but I only use the 5 columns shown below when creating my network.
Unfortunately at the moment I can only get edge attributes working with my network, I am doing this as shown below:
g = nx.from_pandas_dataframe(df, 'node_from', 'node_to', edge_attr=['attribute1','attribute2','attribute3'])
The network will be a directed network. The attributes shown in the below dataframe are the attributes for the 'node_from' nodes. The 'node_to' nodes sometimes appear as 'node_from' nodes. All the nodes that can possibly be shown in the network and their respective attributes are shown in the df_attributes_only table.
df_relationship:
node_from: node_to: ........ attribute1: attribute2: attribute3:
jim john ........ tall red fat
...
All of the columns have words as their values, not digits.
I also have another dataframe which has each possible node and their attributes:
df_attributes_only:
id: attribute1: attribute2: attribute3:
jim tall red fat
john small blue fat
...
I essentially need to assign the above three attributes to their respective id, so every node has their 3 attributes attached.
Any help on how I could get node attributes working with my network is greatly appreciated.
As of Networkx 2.0, you can input a dictionary of dictionaries into nx.set_node_attributes to set attributes for multiple nodes. This is a much more streamlined approach compared to iterating over each node manually. The outer dictionary keys represent each node, and the inner dictionaries keys correspond to the attributes you want to set for each node. Something like this:
attrs = {
node0: {attr0: val00, attr1: val01},
node1: {attr0: val10, attr1: val11},
node2: {attr0: val20, attr1: val21},
}
nx.set_node_attributes(G, attrs)
You can find more detail in the documentation.
Using your example, assuming your index is id, you can convert your dataframe df_attributes_only of node attributes to this format and add to your graph:
df_attributes_only = pd.DataFrame(
[['jim', 'tall', 'red', 'fat'], ['john', 'small', 'blue', 'fat']],
columns=['id', 'attribute1', 'attribute2', 'attribute3']
)
node_attr = df_attributes_only.set_index('id').to_dict('index')
nx.set_node_attributes(g, node_attr)
g.nodes['jim']
>>> {'attribute1': 'tall', 'attribute2': 'red', 'attribute3': 'fat'}
nx.from_pandas_dataframe (and from_pandas_edgelist in latest stable version 2.2), conceptually converts an edgelist to a graph. I.e., each row in the dataframe represents an edge, which is a pair of 2 different nodes.
Using this API it is not possible to read nodes' attributes. It makes sense, because each row has two different nodes and keeping specific columns for the different nodes would be cumbersome and can cause discrepancies. For example, consider the following dataframe:
node_from node_to src_attr_1 tgt_attr_1
a b 0 3
a c 2 4
What should be the 'src_attr_1' value for node a? Is it 0 or 2? Moreover, we need to keep two columns for each attribute (since it's a node attribute both of the nodes in each edge should have it). In my opinion it would be bad design to support it, and I guess that's why NetworkX API doesn't.
You can still read nodes' attributes, after converting the df to a graph, as follows:
import networkx as nx
import pandas as pd
# Build a sample dataframe (with 2 edges: 0 -> 1, 0 -> 2, node 0 has attr_1 value of 'a', node 1 has 'b', node 2 has 'c')
d = {'node_from': [0, 0], 'node_to': [1, 2], 'src_attr_1': ['a','a'], 'tgt_attr_1': ['b', 'c']}
df = pd.DataFrame(data=d)
G = nx.from_pandas_edgelist(df, 'node_from', 'node_to')
# Iterate over df rows and set the source and target nodes' attributes for each row:
for index, row in df.iterrows():
G.nodes[row['node_from']]['attr_1'] = row['src_attr_1']
G.nodes[row['node_to']]['attr_1'] = row['tgt_attr_1']
print(G.edges())
print(G.nodes(data=True))
Edit:
In case you want to have a large list of attributes for the source node, you can extract the dictionary of this columns automatically as follows:
#List of desired source attributes:
src_attributes = ['src_attr_1', 'src_attr_2', 'src_attr_3']
# Iterate over df rows and set source node attributes:
for index, row in df.iterrows():
src_attr_dict = {k: row.to_dict()[k] for k in src_attributes}
G.nodes[row['node_from']].update(src_attr_dict)
This is building off of #zohar.kom's answer. There is a way to solve this problem without iteration. That answer can be optimized. I'm assuming that the attributes describe the node_from.
Start with a graph from an edgelist (like in #zohar.kom's anser):
G = nx.from_pandas_edgelist(df, 'node_from', 'node_to')
You can add the nodes and attributes first.
# Create a mask with only the first records
mask = ~df['node_from'].duplicated()
# Get a list of nodes with attributes
nodes = df[mask][['node_from','attribute1','attribute2','attribute3']]
This method for adding nodes from a dataframe comes from this answer.
# Add the attributes one at a time.
attr_dict = nodes.set_index('node_from')['attribute1'].to_dict()
nx.set_node_attributes(G,attr_dict,'attr1')
attr_dict = nodes.set_index('node_from')['attribute2'].to_dict()
nx.set_node_attributes(G,attr_dict,'attr2')
attr_dict = nodes.set_index('node_from')['attribute3'].to_dict()
nx.set_node_attributes(G,attr_dict,'attr3')
Similar result to #zohar.kom, but with less iterating.
Answer:
Objective: From dataframe object, generate network with nodes, edges, and node-attributes.
Lets consider, we want to generate a network with nodes and node-attributes. Each node has 3 attributes .e., attr1, attr2, and attr3.
Given a dataframe df with 1st and 2nd column as from_node and to_node respectively; and has attribute columns namely attr1, attr2, and attr3.
Below code will add required edge, node, and node-attributes from dataframe.
#%%time
g = nx.Graph()
# Add edges
g = nx.from_pandas_edgelist(df_5, 'from_node','to_node')
# Iterate over df rows and set the target nodes' and node-attributes for each row:
for index, row in df.iterrows():
g.nodes[row[0]]['attr_dict'] = row.iloc[2:].to_dict()
list(g.edges())[0:5]
list(g.nodes(data=True))[0:5]
Related
I am trying to learn networkx and have run into an issue.
I have a dataset that has individuals and multiple columns of dummy variables.
I am trying to have each individual as a node and each dummy variable as a node with edges that connect individuals to each dummy variable they have a 1 for.
Code I have:
G = nx.Graph()
G = nx.from_pandas_edgelist(df, "NAME", dummy_variables)
nx.draw_shell(G, with_labels=True)
where dummy_variables is a list containing the dummy variables from the dataframe I am interested in
Here is a sample of the dataframe I am working with:
dummy variable dataframe
Where HSSAFETYX and other variables are dummy variables. Name is the individual.
What I want to see is each row represented by a node with edges connecting them to the dummy variables they have a 1 for.
I am trying to create a table displaying the attributes of nodes in a NetworkX (using Python) graph. The functions used to get these attributes generate a list of the nodes with their assigned attribute and are generated like in the following:
import networkx as nx
# degrees of nodes
pprint(g.degree())
# clustering coefficient
pprint(nx.clustering(g))
I should like to be able to compile these into a table with intuitive overview in the following format:
node degree clustering coefficient
----------------------------------------
a 2 3
b 1 1
...
Any tips on how to do so? Perhaps using a dict and tabulate but I am not quite sure... Would be grateful for any tips!
Edit
I found a way to do some simple printing, but it isn't formatted very nicely and doesn't make for nice syntax:
for n in g.nodes():
print(n, g.degree(n), nx.clustering(g, n))
I would use a Pandas dataframe (pd.DataFrame) to store this data, which I would construct in a list comprehension of dictionaries (each dictionary corresponding to a row in the final data frame). Here's how I would do that with two attributes, in_degree and out_degree for each node:
import pandas as pd
import networkx as nx
g = nx.DiGraph()
# ... populate the Digraph ...
def compute_indegree(node, digraph):
return digraph.in_degree(node)
def compute_outdegree(node, digraph):
return digraph.out_degree(node)
attr_dicts = [
{ 'node': node,
'in_degree': compute_indegree(node, g), \
'out_degree': compute_outdegree(node, g)} \
for node in g.nodes
]
dataframe = pd.DataFrame(attr_dicts)
dataframe.set_index('node', inplace=True)
print(dataframe)
The final print line neatly formats the resulting DataFrame:
If you modify or add the above functions compute_indegree and compute_outdegree to return other stats like your clustering coefficient, the table will still populate as above.
I have a table that has two columns, 'parent' and 'child'. This is a download from SAP (ERP) for SETNODE table. Need to create a dataframe in python that has each level as it's own column in respect to it's parent and all levels before.
In python 3+.
There are an unknown (or always changing) number of levels for the full relationship so that max level can't always be defined. I would like to create a full dataframe table that shows ALL parent/child relationships for all levels. Right now it's about 15 levels but it can probably go up to 20 or more with other data I work with.
For example (example_df) of the two columns:
example_df = pd.DataFrame({'parent:['a','a','b','c','c','f'],'child':['b','c','d','f','g','h']})
To give output dataframe (solution_example):
solution_example = pd.DataFrame({'child':['h','f','d'],'parent_1':['a','a','a'],'parent_2':['c','c','b'],'parent_3':['f', 'none', 'none']})
This can be solved using the networkx library. First, build a directed graph from the DataFrame, and then find all ancestors of the leaf nodes.
import networkx as nx
leaves = set(df.child).difference(df.parent)
g = nx.from_pandas_edgelist(df, 'parent', 'child', create_using=nx.DiGraph())
ancestors = {
n: nx.algorithms.dag.ancestors(g, n) for n in leaves
}
(pd.DataFrame.from_dict(ancestors, orient='index')
.rename(lambda x: 'parent_{}'.format(x+1), axis=1)
.rename_axis('child')
.fillna(''))
parent_1 parent_2 parent_3
child
h a c f
g a c
d a b
I am using networkx to build an email network structure from a txt file where each row represents an "edge." I first loaded the txt file (3 columns: {'#Sender', 'Recipient', 'time'}) into Python and then converted to an networkx object using the following code:
import networkx as nx
import pandas as pd
email_df = pd.read_csv('email_network.txt', delimiter = '->')
email = nx.from_pandas_dataframe(email_df, '#Sender', 'Recipient', edge_attr = 'time')
The email.txt data can be accessed here.
However, email_df (a pandas DataFrame object) has a length of 82927, while email (a Networkx object) has a length of 3251.
In [1]: len(email_df)
In [2]: 82927
In [3]: len(email.edges())
In [4]: 3251
I got really confused because even if for rows containing the same two nodes in the first two columns of email_df with the same sequence of direction (say, '1' to '2'), the third column ('time', meaning timestamped) should distinguish them from each other, hence, no replicated edges would appear. Then why does the number of edges dramatically decreased from 82927 to 3251 after I used nx.from_pandas_dataframe to read from `email_df'?
Would anyone help explain this to me?
Thank you.
Your line here is saying to take the Sender column as the source node, the Recipient column as the Target and add the time as edge attributes. So you are only creating a single (directed) edge between Sender and Recipient, and only the time of the last row will be added as an attribute of the edge.
email = nx.from_pandas_dataframe(email_df, '#Sender', 'Recipient', edge_attr = 'time')
You can only have one edge defined for a pair of nodes - you could group the dataframe before constructing your network and use the count as the weights for the edges,
edge_groups = email_df.groupby(["#Sender", "Recipient"], as_index=False).count().rename(columns={"time":"weight"})
email = nx.from_pandas_dataframe(edge_groups, '#Sender', 'Recipient', edge_attr = 'weight')
I am building an itinerary of vegetation types for a given location. Data is passed to me as a CSV and I want a way in which I can automatically re-classify items in one column, into broader classes that I provide. I can already read the data in with pandas, do a bit of housekeeping and then write out the data frame to a new file.
However, given I am provided with a column such that:
species = ['maple', 'oak', holly, 'sawgrass', 'cat tails'...... 'birch']
I would like to be able to automatically, reclassify these into broad categories using another list like:
VegClass = ['Tree', 'Bush', 'Grass']
The only way I know to do this would be to iterate through the species list, in a manner similar to:
out = []
for i in species:
if species[i]=='Oak':
out.append('Tree')
but this would require that I write a lot of code if the species list becomes very large and I don't imagine it would be very efficient with large datasets.
Is there a more direct way of doing this? I understand that I would need to list all the species manually (in separate classes) e.g.:
TreeSpecies = ['oak'....'birch']
GrassSpecies = ['Sawgrass....']
but I would only have to do this once to build a dictionary of species names. Im expecting more data so may have to add an additional species name or two in future, but this would not be considered too time intensive if I could process a lot of the data quickly.
You need to create a dict of classifier mappings for your different items, for instance,
classifier = {'oak': 'Tree',
'maple': 'Tree',
'holly': 'Tree',
'sawgrass': 'Grass',
'cat tails': 'Bush',
'birch': 'Tree'}
Then getting a column of groups is as simple as calling map on your column.
>>> df.species.map(classifier)
0 Tree
1 Tree
2 Tree
3 Grass
4 Bush
5 Tree
Name: species, dtype: object
so you can set a new column with
df['classification'] = df.species.map(classifier)
You need a dictionary like
VegClass = {'oak': 'Tree', 'seagrass': 'Grass'}
df['class'] = df['species'].map(VegClass)
I don't know if I follow you, but since you will have to create some sort of associative list, in the form
plant | type
oak | tree
sawgrass | grass
kkk | bush
...
Just create a hash table and get the type from the hash table.
You may read the table from an external file so it is not hardcoded in your program.