Develop Reference

Fitting a line through 3D x,y,z scatter plot data

I have a handful of data points that cluster along a line in 3d space. I have the x,y,z data in a csv file that I want to import. I would like to find an equation that represents that line, or the plane perpendicular to that line, or whatever is mathematically correct. These data are independent of each other. Maybe there are better ways to do this than what I tried to do but...
I attempted to replicate an old post here that seemed to be doing exactly what I'm trying to do
Fitting a line in 3D
but it seems that maybe updates over the past decade have left the second part of the code not working? Or maybe I'm just doing something wrong. I've included the entire thing that I frankensteined together from this at the bottom. There are two lines that seem to be giving me a problem.
I've snippeted them out here...
import numpy as np
pts = np.add.accumulate(np.random.random((10,3)))
x,y,z = pts.T
# this will find the slope and x-intercept of a plane
# parallel to the y-axis that best fits the data
A_xz = np.vstack((x, np.ones(len(x)))).T
m_xz, c_xz = np.linalg.lstsq(A_xz, z)[0]
# again for a plane parallel to the x-axis
A_yz = np.vstack((y, np.ones(len(y)))).T
m_yz, c_yz = np.linalg.lstsq(A_yz, z)[0]
# the intersection of those two planes and
# the function for the line would be:
# z = m_yz * y + c_yz
# z = m_xz * x + c_xz
# or:
def lin(z):
x = (z - c_xz)/m_xz
y = (z - c_yz)/m_yz
return x,y
#verifying:
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
fig = plt.figure()
ax = Axes3D(fig)
zz = np.linspace(0,5)
xx,yy = lin(zz)
ax.scatter(x, y, z)
ax.plot(xx,yy,zz)
plt.savefig('test.png')
plt.show()
They return this, but no values...
FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1.
m_xz, c_xz = np.linalg.lstsq(A_xz, z)[0]
FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1.
m_yz, c_yz = np.linalg.lstsq(A_yz, z)[0]
I don't know where to go from here. I don't even actually need the plot, I just needed an equation and am ill-equipped to move forward. If anyone knows an easier way to do this, or can point me in the right direction, I'm willing to learn, but I'm very, very lost. Thank you in advance!!
Here is my entire frankensteined code in case that is what is causing the issue.
import pandas as pd
import numpy as np
mydataset = pd.read_csv('line1.csv')
x = mydataset.iloc[:,0]
y = mydataset.iloc[:,1]
z = mydataset.iloc[:,2]
data = np.concatenate((x[:, np.newaxis],
y[:, np.newaxis],
z[:, np.newaxis]),
axis=1)
# Calculate the mean of the points, i.e. the 'center' of the cloud
datamean = data.mean(axis=0)
# Do an SVD on the mean-centered data.
uu, dd, vv = np.linalg.svd(data - datamean)
# Now vv[0] contains the first principal component, i.e. the direction
# vector of the 'best fit' line in the least squares sense.
# Now generate some points along this best fit line, for plotting.
# we want it to have mean 0 (like the points we did
# the svd on). Also, it's a straight line, so we only need 2 points.
linepts = vv[0] * np.mgrid[-100:100:2j][:, np.newaxis]
# shift by the mean to get the line in the right place
linepts += datamean
# Verify that everything looks right.
import matplotlib.pyplot as plt
import mpl_toolkits.mplot3d as m3d
ax = m3d.Axes3D(plt.figure())
ax.scatter3D(*data.T)
ax.plot3D(*linepts.T)
plt.show()
# this will find the slope and x-intercept of a plane
# parallel to the y-axis that best fits the data
A_xz = np.vstack((x, np.ones(len(x)))).T
m_xz, c_xz = np.linalg.lstsq(A_xz, z)[0]
# again for a plane parallel to the x-axis
A_yz = np.vstack((y, np.ones(len(y)))).T
m_yz, c_yz = np.linalg.lstsq(A_yz, z)[0]
# the intersection of those two planes and
# the function for the line would be:
# z = m_yz * y + c_yz
# z = m_xz * x + c_xz
# or:
def lin(z):
x = (z - c_xz)/m_xz
y = (z - c_yz)/m_yz
return x,y
print(x,y)
#verifying:
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
fig = plt.figure()
ax = Axes3D(fig)
zz = np.linspace(0,5)
xx,yy = lin(zz)
ax.scatter(x, y, z)
ax.plot(xx,yy,zz)
plt.savefig('test.png')
plt.show()

As was proposed in the old post you refer to, you could also make use of principal component analysis instead of a least squares approach. For that I suggest sklearn.decomposition.PCA from the sklearn package.
An example can be found below using the csv-file you provided.
import pandas as pd
import numpy as np
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
mydataset = pd.read_csv('line1.csv')
x = mydataset.iloc[:,0]
y = mydataset.iloc[:,1]
z = mydataset.iloc[:,2]
coords = np.array((x, y, z)).T
pca = PCA(n_components=1)
pca.fit(coords)
direction_vector = pca.components_
print(direction_vector)
# Create plot
origin = np.mean(coords, axis=0)
euclidian_distance = np.linalg.norm(coords - origin, axis=1)
extent = np.max(euclidian_distance)
line = np.vstack((origin - direction_vector * extent,
origin + direction_vector * extent))
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(coords[:, 0], coords[:, 1], coords[:,2])
ax.plot(line[:, 0], line[:, 1], line[:, 2], 'r')

You can get rid of the complaint from leastsquares by adding rcond=None like this:
m_xz, c_xz = np.linalg.lstsq(A_xz, z, rcond=None)[0]
Is this the right decision for your situation? I have no idea. But there's more about it in the docs.
When I run your code with your inputs it seems to run just fine and I get values assigned to m_xz, c_xz, etc. If you don't call them explicitly with print('m_xz') (or whatever) then you won't see them.
m_xz
Out[42]: 5.186132604596112
c_xz
Out[43]: 62.5764694106141
Also, you reference your data in kind of two different ways. You get x, y, and z from your csv, but also put it into a numpy array. You can get rid of the duplication and pandas by just using numpy:
data = np.genfromtxt('line1.csv', delimiter=',', skip_header=1)
x = data[:,0]
y = data[:,1]
z = data[:,2]

Python script for plotting the evolution of charts such as Paraview does

I want to code a python script that generates a plot like the one shown to the right in the next screenshot from Paraview:
I have a series of files generated with the command foamToVTK:
Is there in VTK any function similar to PlotOverLine method of Paraview?

The ParaView PlotOverLine is in vtk as vtkProbeFilter.
Minimal working example:
import vtk
# Original data
source = vtk.vtkRTAnalyticSource()
# the line to plot over
line = vtk.vtkLineSource()
# filter
probeFilter = vtk.vtkProbeFilter()
probeFilter.SetInputConnection(line.GetOutputPort())
probeFilter.SetSourceConnection(source.GetOutputPort())
# rendering
plot = vtk.vtkXYPlotActor()
plot.AddDataSetInputConnection(probeFilter.GetOutputPort())
plot.SetTitle("My plot")
window = vtk.vtkRenderWindow()
interactor = vtk.vtkRenderWindowInteractor()
interactor.SetRenderWindow(window)
renderer = vtk.vtkRenderer()
renderer.AddActor2D(plot)
window.AddRenderer(renderer)
window.Render()
interactor.Start()
You can find a more complex (multi plot, colors ...) c++ example here: https://lorensen.github.io/VTKExamples/site/Cxx/Annotation/XYPlot/
The python API is the same.

Solution with vtkplotter:
from vtkplotter import *
img = load(datadir+'vase.vti').imagedata()
# write a test file, return a vtkMultiBlockData
mblock = write([img], "multiblock.vtm")
# load back from file into a list of meshes/volumes
mobjs = load("multiblock.vtm")
p1, p2 = (10,10,10), (80,80,80)
pl = probeLine(mobjs[0], p1, p2).lw(3)
vals = pl.getPointArray()
xvals = pl.points()[:,0]
plt = plotxy([xvals, vals],
lc="r", # line color
marker="*", # marker style
mc="dr", # marker color
ms=0.6, # marker color
)
show([(img, pl), plt], N=2, sharecam=False, axes=1, bg='w')

I figured out a solution to this problem. It is not likely the optimal one, though.
For this solution, I firstly converted the mesh to a vtkUnstructuredGrid (in this case using a resolution of 256 points.)
Following I include the code I used to carry this out:
import matplotlib.pyplot as plt
from scipy.interpolate import griddata
import numpy as np
import vtk
from vtk.util.numpy_support import vtk_to_numpy
from os import walk, path, system
import pandas as pd
### Create the VTK files
system("foamToVTK")
### Initialization of variables
cnt=1
fig= plt.figure()
npts = 256 #dimensions of the grid
### Get the file names of each step of the simulation
(dirpath, dirnames, filenames) = next(walk('VTK'))
ids=[]
for dir in dirnames:
ids.append(int(dir.split("_")[1]))
ids = sorted(ids)
basename = dirnames[0].split("_")[0]
### Iteration of time steps
for id in ids[1:]:
### Read values from the file of this time step
filename = "%s/%s_%d/internal.vtu" % (dirpath, basename, id)
reader = vtk.vtkXMLUnstructuredGridReader()
reader.SetFileName(filename)
reader.Update()
### Get the coordinates of nodes in the mesh using VTK methods
nodes_vtk_array= reader.GetOutput().GetPoints().GetData()
vtk_array = reader.GetOutput().GetPointData().GetArray('U') #Velocity (3 dimensions)
numpy_array = vtk_to_numpy(vtk_array)
nodes_nummpy_array = vtk_to_numpy(nodes_vtk_array)
x,y,z= nodes_nummpy_array[:,0] , nodes_nummpy_array[:,1] , nodes_nummpy_array[:,2]
xmin, xmax = min(x), max(x)
ymin, ymax = min(y), max(y)
### Define grid
xi = np.linspace(xmin, xmax, npts)
yi = np.linspace(ymin, ymax, npts)
### Grid the data
interpolated = griddata((x, y), numpy_array, (xi[None,:], yi[:,None]), method='cubic')
### Create the list of points
plotOverLine=[]
for point in range(len(interpolated[0])):
plotOverLine.append(interpolated[127][point])
### Update and plot the chart for this time step
df = pd.DataFrame(plotOverLine, columns=['X', 'Y', 'Z'])
plt.clf()
plt.title('Frame %d' % cnt)
plt.plot(df)
plt.legend(df.columns)
axes = plt.gca()
axes.set_ylim([-15,10])
plt.draw()
plt.pause(.05)
For each time step, it updates and shows a plot like this:

Inspired by #Nico Vuaille and https://stackoverflow.com/a/21990296/4286662 answers, I came across another different solution based on probe filters. This solution remains faithfully to the Paraview results. This is the working code:
from vtk.util import numpy_support as VN
from matplotlib import pyplot as plt
import vtk
import numpy as np
from os import walk, path, system
import pandas as pd
def getFilenames():
### Get the file names of each step of the simulation
(dirpath, dirnames, filenames) = next(walk('VTK'))
ids=[]
for dir in dirnames:
ids.append(int(dir.split("_")[1]))
ids = sorted(ids)
basename = dirnames[0].split("_")[0]
return ids, basename, dirpath
def createLine(p1, p2):
# Create the line along which you want to sample
line = vtk.vtkLineSource()
line.SetResolution(numPoints)
line.SetPoint1(p1)
line.SetPoint2(p2)
line.Update()
return line
def probeOverLine(line,reader):
### Interpolate the data from the VTK-file on the created line.
probe = vtk.vtkProbeFilter()
probe.SetInputConnection(line.GetOutputPort())
probe.SetSourceConnection(reader.GetOutputPort())
probe.Update()
### Get the velocity from the probe
return VN.vtk_to_numpy(probe.GetOutput().GetPointData().GetArray('U'))
### Initialization of variables
cnt=1
fig= plt.figure()
numPoints=100
ids, basename, dirpath = getFilenames()
### Iteration of time steps
for id in ids[1:]:
### Read values from the file of this time step
filename = "%s/%s_%d/internal.vtu" % (dirpath, basename, id)
reader = vtk.vtkXMLUnstructuredGridReader()
reader.SetFileName(filename)
reader.Update()
if cnt==1:
### Get points for a line in the center of the object
bounds = reader.GetOutput().GetBounds()
p1 = [(bounds[1] - bounds[0]) / 2.0 + bounds[0], bounds[2], 0]
p2 = [(bounds[3] - bounds[2]) / 2.0 + bounds[2], bounds[3], 0]
line = createLine(p1, p2)
plotOverLine = probeOverLine(line, reader)
df = pd.DataFrame(plotOverLine, columns=['X', 'Y', 'Z'])
plt.clf()
plt.title('Frame %d' % cnt)
plt.plot(df)
plt.legend(df.columns)
axes = plt.gca()
plt.draw()
plt.pause(.05)
cnt+=1
plt.show()
With the following result:

Basemap subplots with pandas groupby data issue

Problem: struggling to get subplots with Basemap and pandas group-by function. I've worked around several SO similar questions, the closest being: Trouble with basemap subplots and How to make grouper and axis the same length?. Nothing wrong when I plot individually, or manually set the axis (i.e ax = ax[0], ax [1] etc). I could set axis on a counter, but surely within the loop i, ax = ax[i] should work. Greatly appreciate any help or suggestions. Just can't seem to see it...
Desired outcome: Use the pandas group-by function to plot one basemap for each day, for the selected df column. In this sample, plot 4 basemaps (4 days) for T1.
Here is the code so far (apologies, tried to cut excess off but leave enough to follow + txt):
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.mlab import griddata
import pandas as pd
from matplotlib.path import Path
from matplotlib.patches import PathPatch
"""grab data"""
colnames = ['Date','Day','Lat','Lon','Location','T1','T2']
data = pd.read_table('sampledata.txt',sep = '\,',header=0,names=colnames)
"""set up plot"""
ncol = 2
nrow = 2
fig, ax = plt.subplots(nrows=nrow, ncols=ncol)
"""define map extent"""
lonmin=31.034
latmin=-29.87226
lonmax=31.06757
latmax=-29.76993
"""Contour plot area"""
lon_l = 31.034458
lon_r = 31.063841
lat_top = -29.772105
lat_bot = -29.866729
"""transform lon / lat coordinates to map projection"""
data['projected_lon'], data['projected_lat'] = map(*(data.Lon.values, data.Lat.values))
"""grid data"""
numcols, numrows = 100, 100
xi = np.linspace(data['projected_lon'].min(), data['projected_lon'].max(), numcols)
yi = np.linspace(data['projected_lat'].min(), data['projected_lat'].max(), numrows)
xi, yi = np.meshgrid(xi, yi)
"""Set Boundary for clip"""
verts = [
(3413.68264053, 12562.754945193),(2529.90577973, 10518.39901756),( 1836.04976419, 8720.5262622),
( 1511.02714665, 7828.73600079),(1164.54391838, 5708.6428411),(1262.72899238, 3495.89719006),
(1504.13306445, 2376.68833083),(2119.70788849, 1417.91132045),(2828.7976018 , 1093.77254193),
(2687.91369608, 812.19595702), (1553.61478351, 738.21055305),(870.98945027, 2237.5816522 ),
(515.94421668, 3930.99046256),(580.43724377, 6163.91186676),(682.62533323, 6802.055938 ),
(771.02525829, 7354.17422723),(743.67131922, 7644.57787453), (107.19185881, 7843.3455092 ),
(142.88541346, 8263.93842556),(1078.47908569, 8510.110874 ), (1503.57709008, 9693.10793354),
(2152.06559691, 11221.41133494),(2926.64909117, 12784.761329246),(3413.68264053, 12562.754945193),
]
codes = [Path.MOVETO, Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO, Path.LINETO,Path.LINETO,Path.LINETO, Path.LINETO, Path.LINETO,
Path.LINETO, Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO, Path.LINETO,Path.LINETO,Path.LINETO,Path.LINETO, Path.CLOSEPOLY,
]
path = Path(verts, codes)
clip = Path(verts, codes)
clip = PathPatch(clip, transform=ax.transData)
""" Setup Plots """
for i, group in data.groupby(['Day']):
plt.figure()
group[i].plot(title=str(i),ax=ax[i])
map = Basemap(projection='merc', lat_0=-29, lon_0=31,
resolution = 'l', area_thresh = 0.1,
llcrnrlon = lonmin, llcrnrlat = latmin ,
urcrnrlon = lonmax, urcrnrlat =latmax ,
ax=ax[i])
map.drawcoastlines()
map.fillcontinents(color='burlywood')
"""interpolate on defined grid"""
x, y, z = data['projected_lon'].values, data['projected_lat'].values, data['T1'].values
zi = griddata(x, y, z, xi, yi,interp='linear')
"""contour plot + clip"""
con = map.contourf(xi, yi, zi)
clip = PathPatch(clip, transform=ax[i].transData)
for contour in con.collections:
contour.set_clip_path(clip)
"""Map Extras"""
map.drawmapboundary(fill_color = 'white')
map.scatter(
data['projected_lon'],
data['projected_lat'],
color='#545454',
edgecolor='#ffffff',
vmin=zi.min(), vmax=zi.max(), zorder=4)
"""Finally Plot all figures generated"""
plt.show()
Sample text file:
Date,Day,Lat,Lon,Location,T1,T2
27-Feb-15,1,-29.86198,31.04568,A1,7,4
27-Feb-15,1,-29.86151,31.0472,A2,4,22
27-Feb-15,1,-29.86098,31.04907,A3,9,24
27-Feb-15,1,-29.85482,31.04243,A4,2,17
27-Feb-15,1,-29.8547,31.04394,A5,2,29
27-Feb-15,1,-29.8544,31.04598,A6,10,27
27-Feb-15,1,-29.8416,31.03864,A7,8,11
27-Feb-15,1,-29.84162,31.04041,A8,1,13
27-Feb-15,1,-29.84161,31.04219,A9,1,14
27-Feb-15,1,-29.83229,31.03868,A10,14,53
27-Feb-15,1,-29.83246,31.04055,A11,8,10
27-Feb-15,1,-29.83256,31.04251,A12,1,13
27-Feb-15,1,-29.82418,31.03922,A13,46,71
27-Feb-15,1,-29.82448,31.04116,A14,4,53
27-Feb-15,1,-29.82461,31.04331,A15,7,5
27-Feb-15,1,-29.81028,31.04301,A16,39,29
27-Feb-15,1,-29.81083,31.0449,A17,12,15
27-Feb-15,1,-29.81114,31.0467,A18,2,9
27-Feb-15,1,-29.79983,31.04648,A19,30,15
27-Feb-15,1,-29.80046,31.04805,A20,37,38
27-Feb-15,1,-29.80078,31.04983,A21,4,23
27-Feb-15,1,-29.78945,31.05083,A22,11,63
27-Feb-15,1,-29.79008,31.05287,A23,5,47
27-Feb-15,1,-29.79047,31.05408,A24,4,15
27-Feb-15,1,-29.78094,31.05577,A25,6,37
27-Feb-15,1,-29.78131,31.05677,A26,13,41
27-Feb-15,1,-29.78178,31.05827,A27,12,46
27-Feb-15,1,-29.77251,31.06032,A28,14,64
27-Feb-15,1,-29.77316,31.0618,A29,5,36
27-Feb-15,1,-29.77348,31.06291,A30,12,45
27-Feb-15,1,-29.86373,31.05944,A31,0,0
27-Feb-15,1,-29.865902,31.058159,A32,0,0
27-Feb-15,1,-29.810559,31.035082,A33,0,0
27-Feb-15,1,-29.866335,31.049232,A34,7,4
28-Feb-15,2,-29.86198,31.04568,A1,3,2
28-Feb-15,2,-29.86151,31.0472,A2,1,2
28-Feb-15,2,-29.86098,31.04907,A3,2,2
28-Feb-15,2,-29.85482,31.04243,A4,62,490
28-Feb-15,2,-29.8547,31.04394,A5,1,1
28-Feb-15,2,-29.8544,31.04598,A6,5,28
28-Feb-15,2,-29.8416,31.03864,A7,1,23
28-Feb-15,2,-29.84162,31.04041,A8,1,46
28-Feb-15,2,-29.84161,31.04219,A9,1,6
28-Feb-15,2,-29.83229,31.03868,A10,12,44
28-Feb-15,2,-29.83246,31.04055,A11,1,18
28-Feb-15,2,-29.83256,31.04251,A12,1,18
28-Feb-15,2,-29.82418,31.03922,A13,6,25
28-Feb-15,2,-29.82448,31.04116,A14,1,19
28-Feb-15,2,-29.82461,31.04331,A15,4,34
28-Feb-15,2,-29.81028,31.04301,A16,6,55
28-Feb-15,2,-29.81083,31.0449,A17,9,52
28-Feb-15,2,-29.81114,31.0467,A18,9,28
28-Feb-15,2,-29.79983,31.04648,A19,33,20
28-Feb-15,2,-29.80046,31.04805,A20,29,20
28-Feb-15,2,-29.80078,31.04983,A21,21,20
28-Feb-15,2,-29.78945,31.05083,A22,9,20
28-Feb-15,2,-29.79008,31.05287,A23,1,11
28-Feb-15,2,-29.79047,31.05408,A24,1,14
28-Feb-15,2,-29.78094,31.05577,A25,4,6
28-Feb-15,2,-29.78131,31.05677,A26,1,42
28-Feb-15,2,-29.78178,31.05827,A27,1,10
28-Feb-15,2,-29.77251,31.06032,A28,22,82
28-Feb-15,2,-29.77316,31.0618,A29,65,30
28-Feb-15,2,-29.77348,31.06291,A30,1,8
28-Feb-15,2,-29.86373,31.05944,A31,24,39
28-Feb-15,2,-29.865902,31.058159,A32,24,39
28-Feb-15,2,-29.810559,31.035082,A33,70,17000
28-Feb-15,2,-29.866335,31.049232,A34,3,2
01-Mar-15,3,-29.86198,31.04568,A1,1,1
01-Mar-15,3,-29.86151,31.0472,A2,1,1
01-Mar-15,3,-29.86098,31.04907,A3,1,1
01-Mar-15,3,-29.85482,31.04243,A4,1,2
01-Mar-15,3,-29.8547,31.04394,A5,1,5
01-Mar-15,3,-29.8544,31.04598,A6,1,1
01-Mar-15,3,-29.8416,31.03864,A7,1,1
01-Mar-15,3,-29.84162,31.04041,A8,1,1
01-Mar-15,3,-29.84161,31.04219,A9,1,1
01-Mar-15,3,-29.83229,31.03868,A10,4,10
01-Mar-15,3,-29.83246,31.04055,A11,1,5
01-Mar-15,3,-29.83256,31.04251,A12,1,2
01-Mar-15,3,-29.82418,31.03922,A13,6,26
01-Mar-15,3,-29.82448,31.04116,A14,3,7
01-Mar-15,3,-29.82461,31.04331,A15,1,1
01-Mar-15,3,-29.81028,31.04301,A16,70,70
01-Mar-15,3,-29.81083,31.0449,A17,70,70
01-Mar-15,3,-29.81114,31.0467,A18,90,70
01-Mar-15,3,-29.79983,31.04648,A19,27,210
01-Mar-15,3,-29.80046,31.04805,A20,54,600
01-Mar-15,3,-29.80078,31.04983,A21,90,70
01-Mar-15,3,-29.78945,31.05083,A22,27,160
01-Mar-15,3,-29.79008,31.05287,A23,45,250
01-Mar-15,3,-29.79047,31.05408,A24,53,580
01-Mar-15,3,-29.78094,31.05577,A25,19,70
01-Mar-15,3,-29.78131,31.05677,A26,37,180
01-Mar-15,3,-29.78178,31.05827,A27,60,400
01-Mar-15,3,-29.77251,31.06032,A28,6,28
01-Mar-15,3,-29.77316,31.0618,A29,1,32
01-Mar-15,3,-29.77348,31.06291,A30,6,38
01-Mar-15,3,-29.86373,31.05944,A31,3,6
01-Mar-15,3,-29.865902,31.058159,A32,3,6
01-Mar-15,3,-29.810559,31.035082,A33,120,30
01-Mar-15,3,-29.866335,31.049232,A34,1,1
02-Mar-15,4,-29.86198,31.04568,A1,12,11
02-Mar-15,4,-29.86151,31.0472,A2,8,5
02-Mar-15,4,-29.86098,31.04907,A3,6,3
02-Mar-15,4,-29.85482,31.04243,A4,14,14
02-Mar-15,4,-29.8547,31.04394,A5,16,13
02-Mar-15,4,-29.8544,31.04598,A6,3,4
02-Mar-15,4,-29.8416,31.03864,A7,37,27
02-Mar-15,4,-29.84162,31.04041,A8,10,7
02-Mar-15,4,-29.84161,31.04219,A9,9,7
02-Mar-15,4,-29.83229,31.03868,A10,200,30
02-Mar-15,4,-29.83246,31.04055,A11,25,11
02-Mar-15,4,-29.83256,31.04251,A12,52,23
02-Mar-15,4,-29.82418,31.03922,A13,400,43
02-Mar-15,4,-29.82448,31.04116,A14,360,70
02-Mar-15,4,-29.82461,31.04331,A15,420,62
02-Mar-15,4,-29.81028,31.04301,A16,1000,110
02-Mar-15,4,-29.81083,31.0449,A17,1100,100
02-Mar-15,4,-29.81114,31.0467,A18,900,120
02-Mar-15,4,-29.79983,31.04648,A19,6300,170
02-Mar-15,4,-29.80046,31.04805,A20,4800,190
02-Mar-15,4,-29.80078,31.04983,A21,4700,110
02-Mar-15,4,-29.78945,31.05083,A22,1000,68
02-Mar-15,4,-29.79008,31.05287,A23,18,7
02-Mar-15,4,-29.79047,31.05408,A24,21,6
02-Mar-15,4,-29.78094,31.05577,A25,,
02-Mar-15,4,-29.78131,31.05677,A26,20,2
02-Mar-15,4,-29.78178,31.05827,A27,15,10
02-Mar-15,4,-29.77251,31.06032,A28,27,14
02-Mar-15,4,-29.77316,31.0618,A29,34,18
02-Mar-15,4,-29.77348,31.06291,A30,22,6
02-Mar-15,4,-29.86373,31.05944,A31,800,200
02-Mar-15,4,-29.865902,31.058159,A32,800,200
02-Mar-15,4,-29.810559,31.035082,A33,2400,600
02-Mar-15,4,-29.866335,31.049232,A34,12,11

Lines not plotting on graph using Python/Basemap

Sorry if this question is simple I'm a newb to using Python and Basemap. Anyway I'm trying to plot the path of 20 hurricanes on a map (graph). The map itself and the legend show up perfectly but the paths of the hurricanes do not. Also I'm not getting any traceback messages but I think I have an idea of where my problem may be. Could someone please tell me where I went wrong.
Here's a sample of the csv file:
Year, Name, Type, Latitude, Longitude
1957,AUDREY,HU, 21.6, 93.3
1957,AUDREY,HU,22.0, 93.4
1957,AUDREY,HU,22.6, 93.5
1969,AUDREY,HU,28.2,99.6
1957,AUDREY,HU,26.5,93.8
1957,AUDREY,HU,27.9,93.8
1957,AUDREY,HU,29.3,95
1957,AUDREY,HU,27.9,93.8
1957,AUDREY,HU,29.3,93.8
1957,AUDREY,HU,30.7,93.5
1969,CAMILLE,HU, 21.6,99.3
1969,CAMILLE,HU,22.0,98.4
1969,CAMILLE,HU,22.6,90.5
1969,CAMILLE,HU,23.2,93.6
Here's the code I have so far:
import numpy as np
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import csv, os, scipy
import pandas
from PIL import *
data = np.loadtxt('louisianastormb.csv',dtype=np.str,delimiter=',',skiprows=1)
'''print data'''
fig = plt.figure(figsize=(12,12))
ax = fig.add_axes([0.1,0.1,0.8,0.8])
m = Basemap(llcrnrlon=-100.,llcrnrlat=0.,urcrnrlon=-20.,urcrnrlat=57.,
projection='lcc',lat_1=20.,lat_2=40.,lon_0=-60.,
resolution ='l',area_thresh=1000.)
m.bluemarble()
m.drawcoastlines(linewidth=0.5)
m.drawcountries(linewidth=0.5)
m.drawstates(linewidth=0.5)
# Creates parallels and meridians
m.drawparallels(np.arange(10.,35.,5.),labels=[1,0,0,1])
m.drawmeridians(np.arange(-120.,-80.,5.),labels=[1,0,0,1])
m.drawmapboundary(fill_color='aqua')
color_dict = {'AUDREY': 'red', 'ETHEL': 'white', 'BETSY': 'yellow','CAMILLE': 'blue', 'CARMEN': 'green','BABE': 'purple', }
colnames = ['Year','Name','Type','Latitude','Longitude']
data = pandas.read_csv('louisianastormb.csv', names=colnames)
names = list(data.Name)
lat = list(data.Latitude)
long = list(data.Longitude)
colorName = list(data.Name)
#print lat
#print long
lat.pop(0)
long.pop(0)
colorName.pop(0)
latitude= map(float, lat)
longitude = map(float, long)
x, y = m(latitude,longitude)
#Plots points on map
for colorName in color_dict.keys():
plt.plot(x,y,linestyle ='-',label=colorName,color=color_dict[colorName], linewidth=5 )
lg = plt.legend()
lg.get_frame().set_facecolor('grey')
plt.show()

two (okay I lied, should be there) problems with your code
i, your input longitude should be negative to be within the boundary you defined for your basemap, so add this after before converting to x and y
longitude = [-i for i in longitude]
ii, your coordinate conversion line is wrong, you should swap lon and lat in the argument list
x, y = m(longitude, latitude)
instead of
x, y = m(latitude,longitude)
EDIT:
okay, the second question that OP posted in the comments, please check the complete code below and please pay attention to the changes I've made compared to yours
# Last-modified: 21 Oct 2013 05:35:16 PM
import numpy as np
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import csv, os, scipy
import pandas
from PIL import *
data = np.loadtxt('louisianastormb.csv',dtype=np.str,delimiter=',',skiprows=1)
'''print data'''
fig = plt.figure(figsize=(12,12))
ax = fig.add_axes([0.1,0.1,0.8,0.8])
m = Basemap(llcrnrlon=-100.,llcrnrlat=0.,urcrnrlon=-20.,urcrnrlat=57.,
projection='lcc',lat_1=20.,lat_2=40.,lon_0=-60.,
resolution ='l',area_thresh=1000.)
m.drawcoastlines(linewidth=0.5)
m.drawcountries(linewidth=0.5)
m.drawstates(linewidth=0.5)
# m.bluemarble(ax=ax)
# Creates parallels and meridians
m.drawparallels(np.arange(10.,35.,5.),labels=[1,0,0,1])
m.drawmeridians(np.arange(-120.,-80.,5.),labels=[1,0,0,1])
m.drawmapboundary(fill_color='aqua')
color_dict = {'AUDREY': 'red', 'ETHEL': 'white', 'BETSY': 'yellow','CAMILLE': 'blue', 'CARMEN': 'green','BABE': 'purple', }
colnames = ['Year','Name','Type','Latitude','Longitude']
data = pandas.read_csv('louisianastormb.csv', names=colnames)
names = list(data.Name)
lat = list(data.Latitude)
long = list(data.Longitude)
colorNames = list(data.Name)
#print lat
#print long
lat.pop(0)
long.pop(0)
colorNames.pop(0)
latitude= map(float, lat)
longitude = map(float, long)
# added by nye17
longitude = [-i for i in longitude]
# x, y = m(latitude,longitude)
x, y = m(longitude,latitude)
# convert to numpy arrays
x = np.atleast_1d(x)
y = np.atleast_1d(y)
colorNames = np.atleast_1d(colorNames)
#Plots points on map
for colorName in color_dict.keys():
plt.plot(x[colorName == colorNames],y[colorName == colorNames],linestyle ='-',label=colorName,color=color_dict[colorName], linewidth=5 )
lg = plt.legend()
lg.get_frame().set_facecolor('grey')
plt.show()

I think your difficulty is not so much in Basemap as in the plotting. Instead of plotting the entire x/y data set you need to find the x/y points corresponding on hurricane Z. Then plot only those points in a certain color c. Then find the points corresponding to the next hurricane etc...
The below, while not using the Basemap data structure should provide a starting point for plotting subsets of points based on some selector vector.
#given a list of x,y coordinates with a label we'll plot each line individually
#first construct some points to plot
x1 = [1,1.1,1.2,1.3, 2.0,2.2,2.3, 4,3.9,3.8,3.7]
y1 = [5,5.1,5.2,5.3, 6.0,6.2,6.3, 2,2.1,2.2,2.3]
pointNames = []
#generate some labels
pointNames.extend(['a']*4)
pointNames.extend(['b']*3)
pointNames.extend(['c']*4)
#make things easy by casting to numpy arrays to allow for easier indexing
x1 = numpy.array(x1)
y1 = numpy.array(y1)
pointNames = numpy.array(pointNames)
for elem in ['a','b','c']:
selector = pointNames==elem
subsetX = x1[selector]
subsetY = y1[selector]
#now plot subsetX vs subsetY in color Z
plot(subsetX,subsetY,'*-')
show()

Matplotlib contour plot with intersecting contour lines

I am trying to make a contour plot of the following data using matplotlib in python. The data is of this form -
# x y height
77.23 22.34 56
77.53 22.87 63
77.37 22.54 72
77.29 22.44 88
The data actually consists of nearly 10,000 points, which I am reading from an input file. However the set of distinct possible values of z is small (within 50-90, integers), and I wish to have a contour lines for every such distinct z.
Here is my code -
import matplotlib
import numpy as np
import matplotlib.cm as cm
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import csv
import sys
# read data from file
data = csv.reader(open(sys.argv[1], 'rb'), delimiter='|', quotechar='"')
x = []
y = []
z = []
for row in data:
try:
x.append(float(row[0]))
y.append(float(row[1]))
z.append(float(row[2]))
except Exception as e:
pass
#print e
X, Y = np.meshgrid(x, y) # (I don't understand why is this required)
# creating a 2D array of z whose leading diagonal elements
# are the z values from the data set and the off-diagonal
# elements are 0, as I don't care about them.
z_2d = []
default = 0
for i, no in enumerate(z):
z_temp = []
for j in xrange(i): z_temp.append(default)
z_temp.append(no)
for j in xrange(i+1, len(x)): z_temp.append(default)
z_2d.append(z_temp)
Z = z_2d
CS = plt.contour(X, Y, Z, list(set(z)))
plt.figure()
CB = plt.colorbar(CS, shrink=0.8, extend='both')
plt.show()
Here is the plot of a small sample of data -
Here is a close look to one of the regions of the above plot (note the overlapping/intersecting lines) -
I don't understand why it doesn't look like a contour plot. The lines are intersecting, which shouldn't happen. What can be possibly wrong? Please help.

Try to use the following code. This might help you -- it's the same thing which was in the Cookbook:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.mlab import griddata
# with this way you can load your csv-file really easy -- maybe you should change
# the last 'dtype' to 'int', because you said you have int for the last column
data = np.genfromtxt('output.csv', dtype=[('x',float),('y',float),('z',float)],
comments='"', delimiter='|')
# just an assigning for better look in the plot routines
x = data['x']
y = data['y']
z = data['z']
# just an arbitrary number for grid point
ngrid = 500
# create an array with same difference between the entries
# you could use x.min()/x.max() for creating xi and y.min()/y.max() for yi
xi = np.linspace(-1,1,ngrid)
yi = np.linspace(-1,1,ngrid)
# create the grid data for the contour plot
zi = griddata(x,y,z,xi,yi)
# plot the contour and a scatter plot for checking if everything went right
plt.contour(xi,yi,zi,20,linewidths=1)
plt.scatter(x,y,c=z,s=20)
plt.xlim(-1,1)
plt.ylim(-1,1)
plt.show()
I created a sample output file with an Gaussian distribution in 2D. My result with using the code from above:
NOTE:
Maybe you noticed that the edges are kind of cropped. This is due to the fact that the griddata-function create masked arrays. I mean the border of the plot is created by the outer points. Everything outside the border is not there. If your points would be on a line then you will not have any contour for plotting. This is kind of logical. I mention it, cause of your four posted data points. It seems likely that you have this case. Maybe you don't have it =)
UPDATE
I edited the code a bit. Your problem was probably that you didn't resolve the dependencies of your input-file correctly. With the following code the plot should work correctly.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.mlab import griddata
import csv
data = np.genfromtxt('example.csv', dtype=[('x',float),('y',float),('z',float)],
comments='"', delimiter=',')
sample_pts = 500
con_levels = 20
x = data['x']
xmin = x.min()
xmax = x.max()
y = data['y']
ymin = y.min()
ymax = y.max()
z = data['z']
xi = np.linspace(xmin,xmax,sample_pts)
yi = np.linspace(ymin,ymax,sample_pts)
zi = griddata(x,y,z,xi,yi)
plt.contour(xi,yi,zi,con_levels,linewidths=1)
plt.scatter(x,y,c=z,s=20)
plt.xlim(xmin,xmax)
plt.ylim(ymin,ymax)
plt.show()
With this code and your small sample I get the following plot:
Try to use my snippet and just change it a bit. For example, I had to change for the given sample csv-file the delimitter from | to ,. The code I wrote for you is not really nice, but it's written straight foreword.
Sorry for the late response.