Develop Reference

Python: Creating a Grid of X,Y coordinates and corresponding calculated Z values to result in a 3D array of XYZ

I have a function that calculates a z value from a given x and y coordinate. I then want to combine these values together to get a 3D array of x,y,z. I'm attempting to do this with the code below:
#import packages
import pandas as pd
import math
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.tri as tri
import matplotlib.pyplot as plt
from matplotlib import rcParams
%matplotlib inline
import matplotlib as mpl
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1.axes_divider import make_axes_locatable
from mpl_toolkits.mplot3d import Axes3D
#Define function to calculate z over a grid
def func(X, Y, x, y, Q):
return (Q / (2 * np.pi)) * np.arctan((y-Y)/(x-X))
#For initial testing just defining the IW explicitly, last step will be to read the input file and pull this data
X1=2417743.658
Y1=806346.704
Q1=5
X2=2417690.718
Y2=806343.693
Q2=5
X3=2417715.221
Y3=806309.685
Q3=5
#initiate the XY grid
xi = np.linspace(2417675,2417800,625)
yi = np.linspace(806300,806375,375)
#mesh the grid in to x,y space
x,y = np.meshgrid(xi,yi)
#calculate the values over the grid at every x,y using the defined function above
zi = (func(X1,Y1,x,y,Q1)+func(X2,Y2,x,y,Q2)+func(X3,Y3,x,y,Q3))
#reshape the xy space into 3d space - when i plot this grid it looks correct
xy = np.array([[(x, y) for x in xi] for y in yi])
#reshape z into 3d space - this appears to be where the issue begins
z = np.array(zi).reshape(xy.shape[0],xy.shape[1], -1)
#combined xyz into a single grid
xyz = np.concatenate((xy, z), axis = -1)
# Create figure and add axis
fig = plt.figure(figsize=(4,4))
ax = fig.add_subplot(111)
img = ax.imshow((xyz*255).astype(np.uint8))
output:
I do get an XYZ array and when i print it the values appear to be mapping correctly, however when I plot the data, it shows the y values "upside down" essentially. This is what the output should look like but "flipped" over the x over axis. Additionally the axes show node numbers and not the X,Y values. I want the 0,0 point to be the lower left hand corner like cartesian coordinates, and each x,y have a corresponding z which is calculated from that given x,y. I know there must be an easier way to go about this. Does anyone know a better way? or maybe what i'm doing wrong here?
Thanks

There is an option for ax.imshow() that allows to specify the origin point.
https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.imshow.html
origin{'upper', 'lower'}, default: rcParams["image.origin"] (default:
'upper') Place the [0, 0] index of the array in the upper left or
lower left corner of the Axes. The convention (the default) 'upper' is
typically used for matrices and images.
Note that the vertical axis points upward for 'lower' but downward for
'upper'.
See the origin and extent in imshow tutorial for examples and a more
detailed description.
Try to modify to this:
img = ax.imshow((xyz*255).astype(np.uint8), origin='lower')
For the axis labels they can be changed with the following commands
ax.set_xticks(LIST_OF_INDICIES)
ax.set_xticklabels(LIST_OF_VALUES)

Basemap plotting a contour figure over coastlines

I am trying to superimpose a contour plot onto a basemap plot of coastlines. Right now it either plots both separately or just the basemap.
Xa = np.linspace(-93.6683,-93.2683,25)
Ya = np.linspace(29.005,29.405,25)
plt.figure()
m = Basemap(width=1200000,height=900000,projection='lcc',resolution='f',lat_1=29.205,lat_2=29.5,lat_0=29.205,lon_0=-93,4683)
m.drawcoastlines()
plt.contourf(Ya,Xa,Result.reshape(len(Xa),len(Ya)))
plt.colorbar()
plt.show()
The Result in the code are the concentrations that are plotted as a contour. I would add them, but there are 625 values for concentration from running my code.
Im wondering how I can write the plotting part of my code to be able to superimpose the two graphs. Thanks!

You would want to use basemap's contour function here. This has however some particularities.
It accepts only 2D arrays as input, meaning, you need to create a meshgrid of coordinates first.
and the input must be the mapped coordinates, x,y = m(X,Y)
Also pay attention to the dimensions. The first dimension of a numpy array is the y axis, the second dimension is the x axis.
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
Xa = np.linspace(-93.6683,-93.2683,25)
Ya = np.linspace(29.005,29.405,25)
X,Y = np.meshgrid(Xa,Ya)
Result = np.random.rand(len(Ya)*len(Xa))
m = Basemap(width=1200000,height=900000,projection='lcc',resolution='c',
lat_1=29.205,lat_2=29.5,lat_0=29.205,lon_0=-93.4683)
m.drawcoastlines()
mx,my = m(X,Y)
m.contourf(mx,my,Result.reshape(len(Ya),len(Xa)))
plt.colorbar()
plt.show()

Plotting dataset using griddata without cancelling outliers

I have an x, y, z dataset which contains a rather large number of points.
x and y are the positions while z is the actual observable at those coordinates.
Most coordinates have a zero value for z, while only a few of them define lines (with smoothly changing z) in the 2D map.
If I plot it with
scatter(x,y,c=z))
I get only very faint lines as the scatterpoints with color defined by z=0 are overlapping with the nonzero values of z. If I decrease the size of the points to reduce overlap, I can't see them anymore.
Here an example of the best I could get using scatter (blue is zero z, other colors are non-zero z).
So, I thought of instead using
data = np.genfromtxt('data')
x=data[:,0]
y=data[:,1]*3.0
z=data[:,2]
grid_x, grid_y = np.mgrid[min(x):max(x):100, min(y):max(y):1000]
from scipy.interpolate import griddata
grid_z0 = griddata((x, y),z, (grid_x, grid_y), method='cubic')
im = imshow(grid_z0,origin="lower",extent=[0,0.175,-0.15,0.15]) # zoom in on specific part of data
to get a denser grid of points and possibly get wider lines due to the cubic interpolation of points around them.
However, then it seems like griddata is removing the non-zero z, considering them as outliners, thus hiding any possible features and the whole grid plots a zero z.
Is there any python/matplotlib/... feature or trick I am missing to plot this in a nice way?
I am trying to make plots that would look something like the ones you can see in Fig. 2 (2) of [https://journals.aps.org/prb/abstract/10.1103/PhysRevB.93.0854092 (you can see the figure without downloading the paper) with possibly some kind of glow around the lines.
The data I used is in this dropbox link.

Of course you may change the scatter, e.g. to set the size of the points without energy to 0.
import matplotlib.pyplot as plt
import numpy as np
data = np.genfromtxt('data/some_solidstate_physics_data.txt')
x=data[:,0]; y=data[:,1]*3.0; z=data[:,2]
plt.scatter(x,y,c=z, s=np.log10(z+1), cmap="PuRd", vmin=-500)
plt.show()
Since the data is already gridded, there is for sure no need to use griddata, this will only smooth out the data. Instead just reshaping the data into a grid is enough.
import matplotlib.pyplot as plt
import numpy as np
data = np.genfromtxt('data/some_solidstate_physics_data.txt')
x=data[:,0]; y=data[:,1]*3.0; z=data[:,2]
ux = np.unique(x); uy = np.unique(y)
Z = z.reshape(len(ux),len(uy)).T
dx = np.diff(ux[:2])[0]; dy = np.diff(uy[:2])[0]
ext = [ux.min()-dx/2.,ux.max()+dx/2.,uy.min()-dy/2., uy.max()+dy/2.]
plt.imshow(Z, extent=ext, aspect="auto", cmap="magma")
plt.show()
Since the grid is very dense, it looks somehow pixelated.
You may of course also bin your data into larger chunks. For example joining the data of 3x3 pixels into one and taking the maximum value, using scipy.stats.binned_statistic_2d
import matplotlib.pyplot as plt
import numpy as np
from scipy.stats import binned_statistic_2d
data = np.genfromtxt('data/some_solidstate_physics_data.txt')
x=data[:,0]; y=data[:,1]*3.0; z=data[:,2]
ux = np.unique(x); uy = np.unique(y)
h, ex, ey,_ = binned_statistic_2d(x, y, z, bins=[ux[::3],uy[::3]], statistic='max')
dx = np.diff(ex[:2])[0]; dy = np.diff(ey[:2])[0]
ext = [ux.min()-dx/2.,ux.max()+dx/2.,uy.min()-dy/2., uy.max()+dy/2.]
plt.imshow(h.T, extent=ext, aspect="auto", cmap="magma")
plt.show()
Having those techniques at your disposal you may then decide to beautify your result at the expense of quantitative accuracy.
E.g. applying a gaussian filter, scipy.ndimage.filters.gaussian_filter as well as interpolation="gaussian" in the plotting.
import matplotlib.pyplot as plt
import numpy as np
import scipy.ndimage.filters
data = np.genfromtxt('data/some_solidstate_physics_data.txt')
x=data[:,0]; y=data[:,1]*3.0; z=data[:,2]
ux = np.unique(x); uy = np.unique(y)
Z = z.reshape(len(ux),len(uy)).T
Z = scipy.ndimage.filters.gaussian_filter(Z, 3)
dx = np.diff(ux[:2])[0]; dy = np.diff(uy[:2])[0]
ext = [ux.min()-dx/2.,ux.max()+dx/2.,uy.min()-dy/2., uy.max()+dy/2.]
plt.imshow(Z, extent=ext, aspect="auto", cmap="magma", interpolation="gaussian")
plt.show()

Making a contour plot with solutions from systems of differential equations with pylab

So, I'm solving a system of differential equations numerically i have x,y,z each a solution. Each array is one dimensional and and for example x[0],y[0],z[0] goes with a point in space. i want to graph these in a contour like the usual x y z coordinates, it says i need z to be a 2d array, i know how to make a mesh from x and y, but how do i do this to z?
I have made a mesh out of the x,y, but for z i don't know what to do.
if someone could give me insight it would be much appreciated.

It is not enough to just mesh in x and y, you need to grid your data on a regular grid to be able to do a contour plot. To do this you should look into matplotlib.mlab.griddata (http://matplotlib.org/examples/pylab_examples/griddata_demo.html).
I'll paste the example code from the link below with some extra comments:
from numpy.random import uniform, seed
from matplotlib.mlab import griddata
import matplotlib.pyplot as plt
import numpy as np
# Here the code generates some x and y coordinates and some corresponding z values.
seed(0)
npts = 200
x = uniform(-2,2,npts)
y = uniform(-2,2,npts)
z = x*np.exp(-x**2-y**2)
# Here you define a grid (of arbitrary dimensions, but equal spacing) onto which your data will be mapped
xi = np.linspace(-2.1,2.1,100)
yi = np.linspace(-2.1,2.1,200)
# Map the data to the grid to get a 2D array of remapped z values
zi = griddata(x,y,z,xi,yi,interp='linear')
# contour the gridded data, plotting dots at the nonuniform data points.
CS = plt.contour(xi,yi,zi,15,linewidths=0.5,colors='k')
CS = plt.contourf(xi,yi,zi,15,cmap=plt.cm.rainbow,
vmax=abs(zi).max(), vmin=-abs(zi).max())
plt.colorbar() # draw colorbar
# Plot the original sampling
plt.scatter(x,y,marker='o',c='b',s=5,zorder=10)
plt.xlim(-2,2)
plt.ylim(-2,2)
plt.title('griddata test (%d points)' % npts)
plt.show()

It looks like you are looking for line or scatter plots instead of contour.

How can I rotate vectors onto a Basemap map projection?

I'm making wind vector barb plots using the matplotlib barb function and basemap in python.
I have a list of vectors (wind observations) at arbitrary latitudes and longitudes, i.e. not on a regular grid.
I need to rotate the vectors onto the map projection before plotting or the barbs point in the wrong direction. What is the best way to do this?
e.g.
import numpy
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
# Define locations of my vectors
lat = numpy.array([50.1,46.2,51.6,52.2,54.4])
lon = numpy.array([-3.3,-1.0,-5.2,-1.2,0.2])
# Define some east-west vectors to illustrate the problem
u = numpy.array([5,5,5,5,5])
v = numpy.array([0,0,0,0,0])
# Set up map projection
m = Basemap(llcrnrlon=-15.,llcrnrlat=46.,urcrnrlon=15.,urcrnrlat=59.,
projection='lcc',lat_1=40.,lat_2=50.,lon_0=-50.,
resolution ='l')
# Calculate positions of vectors on map projection
x,y = m(lon,lat)
# Draw barbs
m.barbs(x,y,u,v, length=7, color='red')
# Draw some grid lines for reference
parallels = numpy.arange(-80.,90,20.)
meridians = numpy.arange(0.,360.,20.)
m.drawparallels(parallels)
m.drawmeridians(meridians)
m.drawcoastlines(linewidth=0.5)
plt.show()
Note that in the plot, the vectors do not point east-west.
I have tried using the rotate_vector and transform_vector routines, but these only work for gridded vector data.
Is there a routine to rotate the vectors onto the map projection for an arbitrary list of lat,lon u,v pairs?
Any help would be much appreciated!

For people with gridded data who stumpled upon this question
Rather use the built-in function rotate_vector, you can find it here:
http://matplotlib.org/basemap/api/basemap_api.html

Your problem is that you're specifying your u and v in lat, long. At the same time, you're specifying your x and y in map coordinates. barbs seems to expect both of them in map coordinates, rather than a mix.
The simplest way is to just calculate the endpoints to get the components. (My description makes no sense, so here's what I had in mind:)
x, y = m(lon, lat)
x1, y1 = m(lon+u, lat+v)
u_map, v_map = x1-x, y1-y
You'll then need to rescale the magnitudes, as well. As a full example:
import numpy
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
# Define locations of my vectors
lat = numpy.array([50.1,46.2,51.6,52.2,54.4])
lon = numpy.array([-3.3,-1.0,-5.2,-1.2,0.2])
# Define some east-west vectors to illustrate the problem
u = numpy.array([5,5,5,5,5])
v = numpy.array([0,0,0,0,0])
# Set up map projection
m = Basemap(llcrnrlon=-15.,llcrnrlat=46.,urcrnrlon=15.,urcrnrlat=59.,
projection='lcc',lat_1=40.,lat_2=50.,lon_0=-50.,
resolution ='l')
# Calculate positions of vectors on map projection
x,y = m(lon,lat)
# Calculate the orientation of the vectors
x1, y1 = m(lon+u, lat+v)
u_map, v_map = x1-x, y1-y
# Rescale the magnitudes of the vectors...
mag_scale = np.hypot(u_map, v_map) / np.hypot(u, v)
u_map /= mag_scale
v_map /= mag_scale
# Draw barbs
m.barbs(x,y,u_map,v_map, length=7, color='red')
# Draw some grid lines for reference
parallels = numpy.arange(-80.,90,20.)
meridians = numpy.arange(0.,360.,20.)
m.drawparallels(parallels)
m.drawmeridians(meridians)
m.drawcoastlines(linewidth=0.5)
plt.show()