I have a graph having
957267 Number of nodes ;
1860761 Number of edges and
Average degree is 3.8877
G=nx.from_edgelist(edge)
print(nx.info(G))
nx.draw_spring(G)
plt.draw()
plt.show()
till print function, every thing work, but after that when I draw graph, I got the error
All the values in edge list are integers.
I am getting memory error. Though I have 8 GB ram, and SSD hard disk Is there a way to deal with it or alternative of NetworkX
Traceback (most recent call last):
File "", line 1, in
nx.draw_spring(G)
File "C:\ProgramData\Anaconda3\lib\site-packages\networkx\drawing\nx_pylab.py", line 965, in draw_spring
draw(G, spring_layout(G), **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\networkx\drawing\layout.py", line 333, in fruchterman_reingold_layout
pos = _fruchterman_reingold(A, k, pos_arr, fixed, iterations, dim)
File "C:\ProgramData\Anaconda3\lib\site-packages\networkx\drawing\layout.py", line 388, in _fruchterman_reingold
distance = np.linalg.norm(delta, axis=-1)
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\linalg\linalg.py", line 2198, in norm
return sqrt(add.reduce(s, axis=axis, keepdims=keepdims))
MemoryError
Related
I am building a timeseries PyBats model using a Poisson distribution to signify the distribution of observations. My model instantiation looks like this
model = define_dglm(
Y=data.actual.values,
X=None,
family="poisson",
k=1,
prior_length=8,
dates=data["month"],
ntrend=2,
seasPeriods=[],
seasHarmComponents=[],
nsamps=10000,
)
Where data.actual.values is a numpy array of integers. After instantiating the model, in order to forecast into the future with pybats I run
forecast_samples = model.forecast_path(k=steps_to_forecast, X=X_future, nsamps=10000)
and get the following error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/conda/lib/python3.8/site-packages/pybats/dglm.py", line 289, in forecast_path
return forecast_path_copula(self, k, X, nsamps, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/pybats/forecast.py", line 211, in forecast_path_copula
return forecast_path_copula_sim(mod, k, lambda_mu, lambda_cov, nsamps, t_dist, nu)
File "/opt/conda/lib/python3.8/site-packages/pybats/forecast.py", line 326, in forecast_path_copula_sim
return np.array(list(map(lambda prior: mod.simulate_from_sampling_model(prior, nsamps),
File "/opt/conda/lib/python3.8/site-packages/pybats/forecast.py", line 326, in <lambda>
return np.array(list(map(lambda prior: mod.simulate_from_sampling_model(prior, nsamps),
File "/opt/conda/lib/python3.8/site-packages/pybats/dglm.py", line 477, in simulate_from_sampling_model
return np.random.poisson(rate, [nsamps])
File "mtrand.pyx", line 3573, in numpy.random.mtrand.RandomState.poisson
File "_common.pyx", line 824, in numpy.random._common.disc
File "_common.pyx", line 621, in numpy.random._common.discrete_broadcast_d
File "_common.pyx", line 355, in numpy.random._common.check_array_constraint
ValueError: lam value too large
I have tried converting my Y array to floats, and have tried replacing all 0 values with 1 and get the same error. What is causing this error?
The issue is in exceeding the maximum value allowed in numpy.random.poisson. It looks like any value larger than np.random.poisson(1E19) will cause this error.
A couple things you can try:
Use a longer prior length than 8 when defining the model. This will help produce more stable estimates of the coefficients. After defining your model, check what the coefficient mean vector (model.a) and covariance matrix (model.R) are, to make sure they're reasonable. If they're not, you can change them manually.
If some of your 'Y' values are truly that large, a Poisson model is probably not appropriate. I would suggest modeling log(Y) using the normal dlm model in Pybats.
I hope that this help!
Thanks,
Isaac
Get following error when try I to use io.imsave("image.jpg",array)
Traceback (most recent call last):
File "Fourer.py", line 37, in <module>
io.imsave( "test.jpg", fImage2)
File "C:\ProgramData\Miniconda3\lib\site-packages\skimage\io\_io.py", line 131, in imsave
if is_low_contrast(arr):
File "C:\ProgramData\Miniconda3\lib\site-packages\skimage\exposure\exposure.py", line 503, in is_low_contrast
dlimits = dtype_limits(image, clip_negative=False)
File "C:\ProgramData\Miniconda3\lib\site-packages\skimage\util\dtype.py", line 49, in dtype_limits
imin, imax = dtype_range[image.dtype.type]
KeyError: <class 'numpy.complex128'>
it's a 2n complex array I use
array = [[ 3.25000000e+02+0.00000000e+00j -1.25000000e+01+1.72047740e+01j
-1.25000000e+01+4.06149620e+00j -1.25000000e+01-4.06149620e+00j
-1.25000000e+01-1.72047740e+01j]
[-6.25000000e+01+8.60238700e+01j -8.88178420e-16+8.88178420e-16j
0.00000000e+00+1.29059879e-15j 0.00000000e+00+1.29059879e-15j
-8.88178420e-16-8.88178420e-16j]
[-6.25000000e+01+2.03074810e+01j -8.88178420e-16+4.44089210e-16j
-3.55271368e-15+5.46706420e-15j -3.55271368e-15+5.46706420e-15j
-8.88178420e-16-4.44089210e-16j]
[-6.25000000e+01-2.03074810e+01j -8.88178420e-16+4.44089210e-16j
-3.55271368e-15-5.46706420e-15j -3.55271368e-15-5.46706420e-15j
-8.88178420e-16-4.44089210e-16j]
[-6.25000000e+01-8.60238700e+01j -8.88178420e-16+8.88178420e-16j
0.00000000e+00-1.29059879e-15j 0.00000000e+00-1.29059879e-15j
-8.88178420e-16-8.88178420e-16j]]
How can i save i complex array as image?
If that matrix was obtained from the FFT of an image, then you first need to do Inverse FFT. Only then, you can save it using io.imsave.
If that is the case, take a look at skimage's:
----> Inverse Fourier Transform
I'm trying to visualize graph in NetworkX. I need to colorize the graph like this: center node needs to be colored dark. Then, all nodes that are further away will need to be colored lighter, but when i run the code i get this error :
error: Cannot convert argument type < class 'numpy.ndarray' > to rgba array on the line :
nx.draw_networkx_nodes(G,pos,nodelist=p.keys(),node_size=90,
node_color=p.values(),cmap=plt.cm.Reds_r)
I think the problem is in:
node_color=p.values()
The code is:
import numpy
import pandas
import networkx as nx
import unicodecsv as csv
import community
import matplotlib.pyplot as plt
# Generate the Graph
G=nx.davis_southern_women_graph()
# Create a Spring Layout
pos=nx.spring_layout(G)
# Find the center Node
dmin=1
ncenter=0
for n in pos:
x,y=pos[n]
d=(x-0.5)**2+(y-0.5)**2
if d<dmin:
ncenter=n
dmin=d
""" returns a dictionary of nodes and their distance to the node
supplied as an argument. We will then use these distances
to determine colors"""
p=nx.single_source_shortest_path_length(G,ncenter)
plt.figure(figsize=(8,8))
nx.draw_networkx_edges(G,pos,nodelist=[ncenter],alpha=0.4)
nx.draw_networkx_nodes(G,pos,nodelist=p.keys(),node_size=90,
node_color=p.values(),cmap=plt.cm.Reds_r)
plt.show()
Full Traceback
Traceback (most recent call last):
File "<ipython-input-4-da1414ba5e14>", line 1, in <module>
runfile('C:/Users/Desktop/Marvel/finding_key_players.py', wdir='C:/Users/Desktop/Marvel')
File "C:\Users\Anaconda33\lib\site- packages\spyderlib\widgets\externalshell\sitecustomize.py", line 685, in runfile
execfile(filename, namespace)
File "C:\Users\Anaconda33\lib\site packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)
File "C:/Users/Desktop/Marvel/finding_key_players.py", line 70, in
<module>
cmap=plt.cm.Reds_r)
File "C:\Users\Anaconda33\lib\site-packages\networkx\drawing\nx_pylab.py", line 399, in draw_networkx_nodes
label=label)
File "C:\Users\Anaconda33\lib\site-packages\matplotlib\axes\_axes.py", line 3606, in scatter
colors = mcolors.colorConverter.to_rgba_array(c, alpha)
File "C:\Users\Anaconda33\lib\site-packages\matplotlib\colors.py", line 391, in to_rgba_array
if alpha > 1 or alpha < 0:
ValueError: Cannot convert argument type <class 'numpy.ndarray'> to rgba array
The error is in the function for drawing nodes.
p.keys()values must be put in a list for nodelist, and node_color, otherwise it's not working.
So the correct line is:
nx.draw_networkx_nodes(G,pos,nodelist=list(p.keys()),node_size=80,node_color=list(p.values()), cmap=plt.cm.Reds_r)
plt.axis('off')
plt.show()
I'm trying to write a short script that takes a .csv file with some distance data, and outputs the psd file for it. the code is here:
import math
import matplotlib.pyplot as plt
name = raw_input('File:')
data = open(name + '.csv', 'r')
distances = []
for row in data:
distances.append(row.replace("\n",""))
for i in range(len(distances)):
distances[i] = float(distances[i])
Pxx, freqs = plt.psd(distances, NFFT=16,Fs=2,detrend='detrend_mean',window='window_none',noverlap=128,sides='onesided',scale_by_freq=True)
plot(Pxx,freqs)
plt.savefig(name + 'psd.png', bbox_inches = 'tight')
As you can see, it's pretty simple. the csv file just features one column of numbers, so distances is a vector.
The error I'm getting is as follows:
Traceback (most recent call last):
File "C:psdplot.py", line 15, in <module>
Pxx, freqs = plt.psd(distances, NFFT=16,Fs=2,detrend='detrend_mean',window='window_none',noverlap=128,sides='onesided',scale_by_freq=True)
File "C:\Python27\lib\site-packages\matplotlib\pyplot.py", line 3029, in psd
sides=sides, scale_by_freq=scale_by_freq, **kwargs)
File "C:\Python27\lib\site-packages\matplotlib\axes.py", line 8696, in psd
sides, scale_by_freq)
File "C:\Python27\lib\site-packages\matplotlib\mlab.py", line 389, in psd
scale_by_freq)
File "C:\Python27\lib\site-packages\matplotlib\mlab.py", line 423, in csd
noverlap, pad_to, sides, scale_by_freq)
File "C:\Python27\lib\site-packages\matplotlib\mlab.py", line 251, in _spectral_helper
assert(len(window) == NFFT)
AssertionError
Could someone direct me on how to fix this? I'm sure it's rather obvious, but I haven't been able to find anything on fixing it in this particular context.
Thanks in advance!
I am interpolating data from satellite imagery. My initial data is not on a regular grid. The data e.g. ch1 refers to positions given by longitude and latitude. When I try nearest neighbour interpolation the result I get is good.
from scipy import interpolate
lats = np.arange(latitude.max(), latitude.min(),-.1)
lons = np.arange(longitude.min(),longitude.max(),.1)
all_lon,all_lat = np.meshgrid(lons,lats)
ch1_all = interpolate.griddata((longitude.reshape(-1),latitude.reshape(-1)),ch1.reshape(-1),(all_lon,all_lat),'nearest')
However when I request bilinear interpolation I raise a Qhull error.
ch1_all = interpolate.griddata((longitude.reshape(-1),latitude.reshape(-1)),ch1.reshape(-1),(all_lon,all_lat),'linear')
The error I get is:
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "C:\Python27\lib\site-packages\scipy\interpolate\ndgriddata.py", line 206, in griddata
rescale=rescale)
File "interpnd.pyx", line 239, in scipy.interpolate.interpnd.LinearNDInterpolator.__init__ (scipy\interpolate\interpnd.c:4549)
File "qhull.pyx", line 1736, in scipy.spatial.qhull.Delaunay.__init__ (scipy\spatial\qhull.c:13719)
File "qhull.pyx", line 328, in scipy.spatial.qhull._Qhull.__init__ (scipy\spatial\qhull.c:3602)
QhullError: Qhull error
I have read the post at griddata runtime error -- Python / SciPy (Interpolation) but in my case, it interpolates data with one method but not with the other.
What am I doing wrong?
In the image below I have plotted the final point positions (red) and overlaid the initial positions (blue)