Hi experienced python community.
I often collect magnetic data as part of my job but I have to wait until I am back from the field to process the data to check it for quality. This data comes in this format:-
Time,F,Ef,FP,Easting,Northing,Height
21:51:02,53169.31,-14.3,-17.79,386330.362,7371876.155,540.939
This can be outputted in different formats such as txt, xls or in this case a csv. My aim is to be able to plot it quickly on a laptop and check that there is no contamination to the data.
Using Google lead me to stackoverflow and looking through various posts I have come up with the script which is below. Thank you to all for these posts.
My problems is that I can read the csv file but I can't understand how to then get that data into the plotting section and removing the unwanted numbers from line 27 onwards. I am sure you will find it rather simple but I have been going around in circles for the last 2 weeks due to my lack of experience. Thank you to all who reply.
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.mlab as ml
f = open("filename.csv")
data = np.genfromtxt('filename.csv', dtype=[('Time',float),('F',float),('Ef',float),
('FP',float),('E',float),('N',float),('H',float)],
comments='"', delimiter=',')
#only here so that I can see the file is being
#read so will probably remove it later
for line in f:
print line
#below is copied from elsewhere in stackoverflow and trying to adapt
#to my needs so at the moment I get this a Duplicate Point Warning.
#So I need to call the above into what is below.
ndata = 100
ny, nx = 100, 200
xmin, xmax = 1, 50
ymin, ymax = 1, 50
x = np.random.randint(xmin, xmax, ndata)
y = np.random.randint(ymin, ymax, ndata)
z = np.random.random(ndata)
xi = np.linspace(xmin, xmax, nx) #
yi = np.linspace(ymin, ymax, ny) #
zi = ml.griddata(x, y, z, xi, yi) #
plt.contour(xi, yi, zi, 15, linewidths = 0.5, colors = 'k')
plt.pcolormesh(xi, yi, zi, cmap = plt.get_cmap('rainbow'))
plt.colorbar()
plt.scatter(x, y, marker = 'o', c = 'b', s = 5, zorder = 10)
plt.xlim(xmin, xmax)
plt.ylim(ymin, ymax)
plt.show()
Two things. First, you're doing a good deal more work than necessary when reading in the data. As long as you always have exactly one header line, you should just do something simple like
data = np.genfromtxt('filename.csv', skip_header=1, delimiter=',')
Here, the skip_header=1 just says to skip the first line. Note that your data will have nan in the first column. That's fine; it just says that numpy doesn't recognize your time string. But I assume you don't need that for plotting. Note that you don't need to do f = open("filename.csv") at all, and if you ever do want to, be sure to use f.close() once you're done with it.
Second, to plot, you need to reshape your data. The plt.contour function takes three main arguments. The first and second specify your x and y coordinates, while the third specifies the z values. If there are N_x and N_y coordinate values, then z has to have N_x * N_y values.
I'll have to assume that your CSV file is in some particular order. Here, I'll assume that it first goes through values of Easting, then repeats the values of Easting for different values of Northing. Then your data will be something like
x = data[:N_x,4]
y = data[::N_x,5]
z = data[:,6].reshape(N_y,N_x)
Here, data[:N_x,4] takes the first N_x values from the 5th column (which is number 4 when you start with 0), which should give you all the different x values. Then, data[::N_x,5] takes all the numbers from the 6th column, but skips N_x at a time, so that it gets the different y values. Finally, the reshape command takes your Height data, and makes it into a rectangular array for plotting. If you want something other than Height, use a value other than 6.
Then, you simply plot the data with something like
plt.contour(x,y,z)
plt.show()
Everything else in that lower section in your code is either to construct some random sample data, or to add other bells and whistles to the plot. It's probably best to play around with those only after you get something basic working.
Related
I am trying to plot (magnetic) field lines. I came to know stream function is the tool used to plot field lines. However, an error comes up telling ValueError: 'x' values must be equally spaced.
Please note that the code uses Fourier basis along x so it is equally spaced along x and uses Chebyshev basis along y and hence it is non-uniform grid along y. I do not understand why the code says "x must be uniformly spaced" when it's uniform along x.
The other thing I want to know is, how to plot field lines for a non-uniform grid?
Also is there any other method to plot field lines apart from streamplot?
Code that I am running is attached below for reference:
import numpy as np
import h5py
import matplotlib.pyplot as plt
f = h5py.File('./B_mm/Re_10k/Bx_v24/Bx_v24_s1.h5', 'r')
y = f['/scales/y/1.0'][:]
x = f['/scales/x/1.0'][:]
Bxtotal = f['/tasks/Bxtotal'][:]
Bytotal = f['/tasks/Bytotal'][:]
t = f['scales']['sim_time'][:]
print(np.shape(y))
print('y = ', y)
print(np.shape(x))
print('x= ', x)
X, Y = np.meshgrid(y,x)
print(np.shape(X))
print(np.shape(Y))
Bx = Bxtotal[len(Bxtotal)-1, :, :]
By = Bytotal[len(Bytotal)-1, :, :]
print(np.shape(Bx))
print(np.shape(By))
plt.figure(figsize=(4,4),facecolor="w")
plt.streamplot(X, Y, Bx, By, color='k', density=1.3, minlength=0.9, arrowstyle='-')
plt.show()
And this is my error message;
I would like to plot in 3D with Pandas / MatplotLib (Wireframe or other, I do not care) but in a specific way..
I'm using RFID sensors and I'm trying to record the signal I receive at different distance + different angles. And I want to see the correlation between the rising of the distance and the angle.
So that's why I want to plot in 3D :
X Axis -> the Distance, Y Axis -> the Angle, Z Axis -> the signal received which means a float
My CSV file from where I generate my DataFrame is organized like this a double entry table :
Distance;0;23;45;90;120;180
0;-53.145;-53.08;-53.1;-53.035;-53.035;-53.035
5;-53.145;-53.145;-53.05;-53.145;-53.145;-53.145
15;-53.145;-53.145;-53.145;-53.145;-53.145;-53.145
25;-53.145;-52.145;-53.145;-53.002;-53.145;-53.145
40;-53.145;-53.002;-51.145;-53.145;-54.255;-53.145
60;-53.145;-53.145;-53.145;-53.145;-53.145;-53.145
80;-53.145;-53.145;-53.145;-53.145;-60;-53.145
100;-53.145;-52;-53.145;-54;-53.145;-53.145
120;-53.145;-53.145;-53.145;-53.145;-53.002;-53.145
140;-51.754;-53.145;-51.845;-53.145;-53.145;-53.145
160;-53.145;-53.145;-49;-53.145;-53.145;-53.145
180;-53.145;-53.145;-53.145;-53.145;-53.145;-53.002
200;-53.145;-53.145;-53.145;-53.145;-53.145;-53.145
On the first label row we've different angles : 0°, 23°, 45°, ...
And the index of the DataFrame is the distance : 0 cm, 15 cm...
And the matrix inside represents the signal, so, values of Z Axis...
But I do not know how to generate a 3D Scatter, WireFrame... because in every tutorial I see people that use specific columns as axis.
Indeed, in my CSV file on the first row I've the label of all columns
Distance;0 ;23 ;45 ;90 ;120;180
And I do not know how to generate a 3D plot with a double entry table.
Do you know how to do it ? Or, to generate my CSV file in a better way to see the same result at the end !
I would be grateful if you would help me about this !
Thank you !
maybe contour is enough
b = np.array([0,5,15,25,40,60,80,100,120,140,160,180,200])
a = np.array([0,23,45,90,120,180])
x, y = np.meshgrid(a, b)
z = np.random.randint(-50,-40, (x.shape))
scm = plt.contourf(x, y, z, cmap='inferno')
plt.colorbar(scm)
plt.xticks(a)
plt.yticks(b)
plt.xlabel('Distance')
plt.ylabel('Angle')
plt.show()
displays
You can get a contour plot with something like this (but for the data shown it is not very interesting since all the values are constant at -45):
df = pd.read_csv(sep=';')
df = df.set_index('Distance')
x = df.index
y = df.columns.astype(int)
z = df.values
X,Y = np.meshgrid(x,y)
Z = z.T
plt.contourf(X,Y,Z,cmap='jet')
plt.colorbar()
plt.show()
Welcome to stackoverflow, your question can be split into several steps:
Step 1 - read the data
I have stored your data in a file called data.txt.
I don't know Pandas very well but this can also be handled with the nice simple function of Numpy called loadtxt. Your data is a bit problematic because of the text 'Distance' value in the first column and first row. But don't panic we load the file as a matrix of strings:
raw_data = np.loadtxt('data.txt', delimiter=';', dtype=np.string_)
Step 2 - transform the raw data
To extract the wanted data from the raw data we can do the following:
angle = raw_data[0 , 1:].astype(float)
distance = raw_data[1:, 0 ].astype(float)
data = raw_data[1:, 1:].astype(float)
With indexing the raw data we select the data that we want and with astype we change the string values to numbers.
Intermediate step - making the data a bit fancier
Your data was a bit boring, only the value -45, i took the liberty to make it a bit fancier:
data = (50 + angle[np.newaxis,:]) / (10 + np.sqrt(distance[:,np.newaxis]))
Step 4 - make a wireframe plot
The example at matplotlib.org looks easy enough:
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(X, Y, Z)
plt.show()
But the trick is to get the X, Y, Z parameters right...
Step 3 - make the X and Y data
The Z data is simply our data values:
Z = data
The X and Y should also be 2D array's such that plot_wireframe can find the x and y for each value of Z in the 2D arrays X an Y at the same array locations. There is a Numpy function to create these 2D array's:
X, Y = np.meshgrid(angle, distance)
Step 5 - fancing it up a bit
ax.set_xticks(angle)
ax.set_yticks(distance[::2])
ax.set_xlabel('angle')
ax.set_ylabel('distance')
Putting it together
All steps together in the right order:
# necessary includes...
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
import numpy as np
raw_data = np.loadtxt('data.txt', delimiter=';', dtype=np.string_)
angle = raw_data[0 , 1:].astype(float)
distance = raw_data[1:, 0 ].astype(float)
data = raw_data[1:, 1:].astype(float)
# make the example data a bit more interesting...
data = (50 + angle[np.newaxis,:]) / (10 + np.sqrt(distance[:,np.newaxis]))
# setting up the plot
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
# the trickey part creating the data that plot_wireframe wants
Z = data
X, Y = np.meshgrid(angle, distance)
ax.plot_wireframe(X, Y, Z)
# fancing it up a bit
ax.set_xticks(angle)
ax.set_yticks(distance[::2])
ax.set_xlabel('angle')
ax.set_ylabel('distance')
# and showing the plot ...
plt.show()
Is there a way to extract the data from an array, which corresponds to a line of a contourplot in python? I.e. I have the following code:
n = 100
x, y = np.mgrid[0:1:n*1j, 0:1:n*1j]
plt.contour(x,y,values)
where values is a 2d array with data (I stored the data in a file but it seems not to be possible to upload it here). The picture below shows the corresponding contourplot. My question is, if it is possible to get exactly the data from values, which corresponds e.g. to the left contourline in the plot?
Worth noting here, since this post was the top hit when I had the same question, that this can be done with scikit-image much more simply than with matplotlib. I'd encourage you to check out skimage.measure.find_contours. A snippet of their example:
from skimage import measure
x, y = np.ogrid[-np.pi:np.pi:100j, -np.pi:np.pi:100j]
r = np.sin(np.exp((np.sin(x)**3 + np.cos(y)**2)))
contours = measure.find_contours(r, 0.8)
which can then be plotted/manipulated as you need. I like this more because you don't have to get into the deep weeds of matplotlib.
plt.contour returns a QuadContourSet. From that, we can access the individual lines using:
cs.collections[0].get_paths()
This returns all the individual paths. To access the actual x, y locations, we need to look at the vertices attribute of each path. The first contour drawn should be accessible using:
X, Y = cs.collections[0].get_paths()[0].vertices.T
See the example below to see how to access any of the given lines. In the example I only access the first one:
import matplotlib.pyplot as plt
import numpy as np
n = 100
x, y = np.mgrid[0:1:n*1j, 0:1:n*1j]
values = x**0.5 * y**0.5
fig1, ax1 = plt.subplots(1)
cs = plt.contour(x, y, values)
lines = []
for line in cs.collections[0].get_paths():
lines.append(line.vertices)
fig1.savefig('contours1.png')
fig2, ax2 = plt.subplots(1)
ax2.plot(lines[0][:, 0], lines[0][:, 1])
fig2.savefig('contours2.png')
contours1.png:
contours2.png:
plt.contour returns a QuadContourSet which holds the data you're after.
See Get coordinates from the contour in matplotlib? (which this question is probably a duplicate of...)
I have got this code to generate a surface plot. But it gives a zero division error. I am not able to figure out what is wrong. Thank you.
import pylab, csv
import numpy
from mayavi.mlab import *
def getData(fileName):
try:
data = csv.reader(open(fileName,'rb'))
except:
print 'File not found'
else:
data = [[float(row[0]), float(row[1]),float(row[2])] for row in data]
x = [row[0] for row in data]
y = [row[1] for row in data]
z = [row[2] for row in data]
return (x, y, z)
def plotData(fileName):
xVals, yVals, zVals = getData(fileName)
xVals = pylab.array(xVals)
yVals = pylab.array(yVals)
zVals = (pylab.array(zVals)*10**3)
x, y = numpy.mgrid[-0.5:0.5:0.001, -0.5:0.5:0.001]
s = surf(x, y, zVals)
return s
plotData('data')
If I have understood the code correctly, there is a problem with zVals in mayavi.mlab.surf.
According to the documentation of the function, s is the elevation matrix, a 2D array, where indices along the first array axis represent x locations, and indices along the second array axis represent y locations. Your file reader seems to return a 1D vector instead of an array.
However, this may not be the most difficult problem. Your file seems to contain triplets of x, y, and z coordinates. You can use mayavi.mlab.surf only if your x and y coordinates in the file form a regular square grid. If this is the case, then you just have to recover that grid and form nice 2D arrays of all three parts. If the points are in the file in a known order, it is easy, otherwise it is rather tricky.
Maybe you would want to start with mayavi.mlab.points3d(xVals, yVals, zVals). That will give you an overall impression of your data. (Or if already know more about your data, you might give us a hint by editing your question and adding more information!)
Just to give you an idea of probably slightly pythonic style of writing this, your code is rewritten (and surf replaced) in the following:
import mayavi.mlab as ml
import numpy
def plot_data(filename):
data = numpy.loadtxt(filename)
xvals = data[:,0]
yvals = data[:,1]
zvals = data[:,2] * 1000.
return ml.points3d(x, y, z)
plot_data('data')
(Essential changes: the use of numpy.loadtxt, get rid of pylab namespace here, no import *, no CamelCase variable or function names. For more information, see PEP 8.)
If you only need to see the shape of the surface, and the data in the file is ordered row-by-row and with the same number of data points in each row (i.e. fixed number of columns), then you may use:
import mayavi.mlab as ml
import numpy
importt matplotlib.pyplot as plt
# whatever you have as the number of points per row
columns = 13
data = numpy.loadtxt(filename)
# draw the data points into a XY plane to check that they really for a rectangular grid:
plt.plot(data[:,0], data[:,1])
# draw the surface
zvals = data[:,2].reshape(-1,columns)
ml.surf(zvals, warp_scale='auto')
As you can see, this code allows you to check that your values really are in the right kind of grid. It does not check that they are in the correct order, but at least you can see they form a nice grid. Also, you have to input the number of columns manually. The keyword warp_scale takes care of the surface scaling so that it should look reasonable.
I have the following script which reads the ascii file of two columns and generates 1D plot. The graph has several peaks. What I want is to give all the peak a number like first peak 1, second peak 2 and so on. The peaks appear in an equidistant position in X axis. Can someone tell me how to do that in python. The code-
from pylab import*
# Read the file.
f2 = open('d012_SAXS-recomb.txt', 'r')
# read the whole file into a single variable, which is a list of every row of the file.
lines = f2.readlines()[2:-100]
f2.close()
# initialize some variable to be lists:
x1 = []
y1 = []
# scan the rows of the file stored in lines, and put the values into some variables:
for line in lines:
p = line.split()
x1.append(float(p[0]))
y1.append(float(p[1]))
x = np.array(x1)
y = np.array(y1)
xlim(0.0,4.0)
# now, plot the data:
#subplot(211)
plt.plot(x, y, color='orange',linewidth=2.0, linestyle='-', label='Arabic - LPP''\nRoman - SPP''\nAsterisk - CHOL')
legend(loc='upper right')
xlabel('q')
ylabel('Intensity')
plt.show()
Here's some example code that finds the first (highest) peak. (BTW, I'm using pylab here, so the plot and numpy modules are already imported).
x = linspace(0,10,501)
y = exp(-0.2*x)*sin(x)
k = y.argmax()
plot(x,y)
text(x[k],y[k],'Peak1')
Try that to get started.