OMI (Ozone Monitoring Instrument) measures the key air quality components such as nitrogen dioxide(NO2), ozone(O3). The daily columnO3 file which I downloaded here represented the global distribution of ozone column concentration of troposphere.
The file's size is about 90Mb. Anyone interested can download any of them.
The data was uploaded here in the shape of (15, 720, 1440)
15 is the Number of candidate scenes
1440 is the X-dimension, longitudes [-180:180] from left to right
720 is the Y-dimension, latitudes [-90:90] from bottom to top
Wiht h5py and matplotlib.basemap, here is my attempt:
import h5py
import numpy as np
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import sys
file = h5py.File("OMI-Aura_L2-OMNO2_2016m0529t1759-o63150_v003-2016m0531t023832.he5", 'r')
dataFields=file['HDFEOS']['GRIDS']['ColumnAmountNO2']['Data Fields']
SDS_NAME='ColumnAmountNO2'
data=dataFields[SDS_NAME]
map_label=data.attrs['Units'].decode()
fv=data.attrs['_FillValue']
mv=data.attrs['MissingValue']
offset=data.attrs['Offset']
scale=data.attrs['ScaleFactor']
lat=dataFields['Latitude'][:][0]
min_lat=np.min(lat)
max_lat=np.max(lat)
lon=dataFields['Longitude'][:][0]
min_lon=np.min(lon)
max_lon=np.max(lon)
dataArray=data[:][1]
dataArray[dataArray==fv]=np.nan
dataArray[dataArray==mv]=np.nan
dataArray = scale * (dataArray - offset)
fig = plt.figure()
data_mask = np.ma.masked_array(data[0], np.isnan(data[0]))
m = Basemap(projection='cyl', resolution='l',llcrnrlat=-90, urcrnrlat = 90,llcrnrlon=-180, urcrnrlon = 180)
m.drawcoastlines(linewidth=0.5)
m.drawparallels(np.arange(-90., 120., 30.), labels=[1, 0, 0, 0])
m.drawmeridians(np.arange(-180, 180., 45.), labels=[0, 0, 0, 1])
my_cmap = plt.cm.get_cmap('gist_stern_r')
my_cmap.set_under('w')
m.pcolormesh(lon, lat, data_mask,latlon=True, cmap=my_cmap)
cb = m.colorbar()
cb.set_label(map_label)
plt.autoscale()
plt.show()
Figure shows like this:
Using Panoply, with candidate 0, figure shows like this:
My question
How to set the candidate scenes to represent the global distribution daily(What is the meaning of candidate scenes? Does it correspond to orbit tracks?)
What's wrong with my code which not showing the correct figure
My target
The figure below was clipped from Internet. That's my target style!
Any advice or tutorial guide would be appreciate!
The Latitude and Longitude variables have missing values too, which are -1.2676506e+30 and thus cause the large xrange and yrange in your plots. Also, note that the _FillValue and MissingValue attributes are lists, so your replacement with NaNs went wrong.
import h5py
import numpy as np
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import sys
def array_with_nans(h5var):
""" Extracts the array and replaces fillvalues and missing values with Nans
"""
array = h5var[:] # not very efficient
# _FillValue and MissingValue attributes are lists
for value in h5var.attrs['MissingValue']:
array[array==value]=np.nan
for value in h5var.attrs['_FillValue']:
array[array==value]=np.nan
return array
#file = h5py.File("OMI-Aura_L2-OMNO2_2016m0529t1759-o63150_v003-2016m0531t023832.he5", 'r')
file = h5py.File("OMI-Aura_L2G-OMNO2G_2004m1001_v003-2012m0714t175148.he5", 'r')
dataFields=file['HDFEOS']['GRIDS']['ColumnAmountNO2']['Data Fields']
SDS_NAME='ColumnAmountNO2'
data=dataFields[SDS_NAME]
map_label=data.attrs['Units'].decode()
offset=data.attrs['Offset'][0]
print("offset: {}".format(offset))
scale=data.attrs['ScaleFactor'][0]
print("scale: {}".format(scale))
candidate = 0
dataArray=array_with_nans(data)[candidate]
dataArray = scale * (dataArray - offset)
lat = array_with_nans(dataFields['Latitude'])[candidate]
lon = array_with_nans(dataFields['Longitude'])[candidate]
fig = plt.figure()
data_mask = np.ma.masked_array(dataArray, np.isnan(dataArray))
m = Basemap(projection='cyl', resolution='l',llcrnrlat=-90, urcrnrlat = 90,llcrnrlon=-180, urcrnrlon = 180)
m.drawcoastlines(linewidth=0.5)
m.drawparallels(np.arange(-90., 120., 30.), labels=[1, 0, 0, 0])
m.drawmeridians(np.arange(-180, 180., 45.), labels=[0, 0, 0, 1])
my_cmap = plt.cm.get_cmap('gist_stern_r')
my_cmap.set_under('w')
m.pcolormesh(lon, lat, data_mask,latlon=True, cmap=my_cmap)
cb = m.colorbar()
cb.set_label(map_label)
plt.autoscale()
plt.show()
It's better to ask only one question per post and you are unlikely to get an answer on what the candidate scene means. You may find an answer to this in the product documentation
Related
I currently have 2 "data maps" for different time periods (1901-2021, 1979-2021). I would like to subtract the gridded points in the maps to show a difference between where areas have had more precipitation. I am unsure what I can look up to research this. Please advise.
Thank you
import xarray as xr
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
import cartopy.crs as ccrs
from cartopy import feature as cf
from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER
import netCDF4
import scipy
from netCDF4 import Dataset
import netCDF4
in_file = r'/Users/shawnpreston/Desktop/my_project_DO_NOT_ERASE/precip/cru_ts4.06.1901.2021.pre.dat.nc'
Data=xr.open_dataset(in_file)
precip_data=Data['pre']
NHNA=precip_data.sel(lon = np.arange(-128.715, -64.557, 0.5), lat = np.arange(24, 50, 0.5), method='nearest')
#taking the years from Dec 1978 - December 2021
study_date = NHNA[935:1452]
#Dec 1949-Dec 1979
study_date_1950_1980 = NHNA[587:948]
#Dec 1989- Dec 2020
study_date_1990_2020 = NHNA[1067:1440]
seas_nhna = study_date.groupby('time.season').mean('time')
#1950-1980
season_nhna = study_date_1950_1980.groupby('time.season').mean('time')
#1990-2020
seasons_nhna = study_date_1990_2020.groupby('time.season').mean('time')
#Taking all the years to show difference
Seas_NHNA = NHNA.groupby('time.season').mean('time')
#plotting 1901-2021 precip
Seas_NHNA.plot.contourf(x = 'lon', y = 'lat', col = 'season', col_wrap = 2, cmap = 'Blues', vmin = 0, vmax = 200)
#plotting 1979-2021 precip
seas_nhna.plot.contourf(x = 'lon', y = 'lat', col = 'season', col_wrap = 2, cmap = 'Blues', vmin = 0, vmax = 200)
I have tried setting the plots to a variable and subtracting them.
You might want to rename your variables, as they are a bit confusing.
According to the Xarray documentation, you should be able to subtract one data set from the other.
If you want to compare the 1950-1980 data to the mean:
season_all = study_date.groupby('time.season').mean('time')
season_5080 = study_date_1950_1980.groupby('time.season').mean('time')
differences = season_5080 - season_all
differences.plot.contourf(...)
You can look at the following example where they do something similar: Xarray monthly-means
To reply to your comment, Xarray has a Facetgrid object used when plotting, so that might be the source of your error message.
I re edited the question in order to make it more clear with what I need help. So I have the following code so far
#import the libraries
import netCDF4 as nc
from mpl_toolkits.basemap import Basemap
import numpy as np
import matplotlib.pyplot as plt
#check what variables are included
# in our case : dict_keys(['time', 'time_bnds', 'lat', 'lat_bnds', 'lon', 'lon_bnds', 'height', 'tas'])
print(data.variables.keys())
#extract the data
lats = data.variables['lat'][:]
lons = data.variables['lon'][:]
time = data.variables['time'][:]
te = data.variables['tas'][:]-273.5
#extract the coordinates we want to plot (Italy)by inputting lower #left and top right coordinates
italy = Basemap(projection='merc',
#ITALY
llcrnrlon= 6.63,
llcrnrlat= 35.29,
urcrnrlon= 18.78,
urcrnrlat= 47.09,
resolution= 'f')
#create the plot
lon,lat =np.meshgrid(lons,lats)
x,y=italy(lon,lat)
c_scheme = italy.pcolor(x,y,np.squeeze(te[0,:,:]), cmap ='jet')
italy.drawcoastlines()
italy.drawstates()
italy.drawcounties()
cbar = italy.colorbar(c_scheme, location='right',pad='10%')
plt.title('KIOST-ESM')
plt.show()
which creates the following heat map
Now what I want is to make the plot identify(colour) smaller differences between the temperatures. However I do not know exactly how to find the min and max value of my area that I am investigating (this time it is Italy). My variable te that has the temperatures is of the type : numpy.ma.core.MaskedArray with shape (9125, 160, 320) [time,lon,lat]
Setting vmin and vmax values should solve this. please try
min_height = int(np.min(np.squeeze(te[0,:,:])))
max_height = int(np.max(np.squeeze(te[0,:,:])))
c_scheme = italy.pcolor(x,y,np.squeeze(te[0,:,:]), cmap ='jet', vmin=min_height, vmax=max_height)
In the case that the te array holds is bigger than the mesh in x and y, I would try doing the following:
min_height = int(np.min(np.squeeze(te[0,x,y])))
max_height = int(np.max(np.squeeze(te[0,x,y])))
I have to confess I still have problems understanding the proper setup and relation of the plots and the parts of it with matplotlib, is still confusing how fig with plt with ax relates each other so I just has gone trial and error, docs are sometimes more confusing to me. :-(
I am plotting weather values, from a json and got points. that I can plot with the following code like the image below
fig=plt.figure(figsize=(10,8))
ax=fig.add_subplot(1,1,1,projection=mapcrs)
ax.set_extent([-93,-86,13,19],datacrs)
ax.add_feature(cfeature.COASTLINE)
ax.add_feature(cfeature.BORDERS, linestyle=':')
ax.scatter(lon,lat,c=dat,transform=datacrs)
and I am able to plot the map
Then I generate interpolation using metpy with this code
gridx, gridy, gridz = interpolate_to_grid(lon, lat, dat, interp_type='rbf', hres=.1, rbf_func='linear', rbf_smooth=0)
fig=plt.figure(figsize=(15,15))
ax=fig.add_subplot(1,1,1,projection=mapcrs)
#ax = fig.add_axes([0, 0, 1, 1], projection=mapcrs)
#ax.set_extent([-93,-86,13,19])
#ax.add_feature(cfeature.COASTLINE)
#ax.add_feature(cfeature.BORDERS, linestyle=':')
ax.contourf(gridx,gridy,gridz,levels=np.arange(10,60,2),cmap='viridis')
plt.plot(lon,lat,'k.',color='white')
I got the interpolation of points as desired but cannot show the features, how is the way to do it? If I uncomment the ax.extent all I see is an empty white figure. If I uncomment the ax.features the interpolation show as the below image but not the map.
thanks for any help and guidance.
You are missing the transform keyword argument in the contourf function in order to give the coordinate system of the interpolated data. Here is a minimal working example with random data, with the obtained output below:
import numpy as np
from cartopy import crs, feature
from matplotlib import pyplot as plt
from scipy.interpolate import griddata
# figure
fig = plt.figure(figsize=(5, 5))
# coordinate systems
crs_map = crs.Mercator()
crs_data = crs.PlateCarree()
# random data
np.random.seed(42) # for repro.
n = 100
lon = -89 + 2 * np.random.randn(n)
lat = 16 + 2 * np.random.randn(n)
dat = np.random.rand(n)
# interpolated data
ilon = np.linspace(-93, -86, 200)
ilat = np.linspace(13, 19, 200)
ilon, ilat = np.meshgrid(ilon, ilat)
idat = griddata((lon, lat), dat, (ilon, ilat), method="linear")
# show up
ax = fig.add_subplot(1, 1, 1, projection=crs_map)
ax.set_extent([-93, -86, 13, 19], crs_data)
ax.add_feature(feature.COASTLINE)
ax.add_feature(feature.BORDERS, ls=":", lw=0.5)
ax.scatter(lon, lat, c=dat, transform=crs_data) # this is invisible with contour
ax.plot(lon, lat, "k.", transform=crs_data) # in order to see the points
ax.contourf(ilon, ilat, idat, levels=np.linspace(0, 1, 10), transform=crs_data)
When plotting with Basemap's readshapefile, if the defined map is centered anywhere else than the longitudinal center of the shapefile, only a portion of it it's plotted. Here's an example using Natural Earth's coastlines:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
shpf = './NaturalEarth/ne_50m_land/ne_50m_land'
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=100)
m = Basemap(
ax = ax,
projection = 'cyl',
llcrnrlon = 0, llcrnrlat = -90,
urcrnrlon = 360, urcrnrlat = 90
)
m.readshapefile(shpf,'ne_50m_land')
m.drawmeridians(np.arange(0,360,45),labels=[True,False,False,True])
Which produces:
Is there a workaround for this with Basemap or Python? I know some people re-center the shapefile in QGIS or similar, but it seems unpractical to do so every time you create a new map, and my QGIS skills are extremely basic.
One way to do it would be to tell readshapefile not to plot the coastlines directly and then to manipulate the line segments before plotting them yourself. Here an example based on your use case:
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
shpf = 'shapefiles/ne_50m_land'
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=100)
m = Basemap(
ax = ax,
projection = 'cyl',
llcrnrlon = 0, llcrnrlat = -90,
urcrnrlon = 360, urcrnrlat = 90
)
m.readshapefile(shpf,'ne_50m_land', drawbounds = False)
boundary = 0.0
for info, shape in zip(m.ne_50m_land_info, m.ne_50m_land):
lons, lats = map(np.array, zip(*shape))
sep = (lons <= boundary).astype(int)
roots = np.where(sep[:-1]+sep[1:] == 1)[0]+1
lower = np.concatenate([[0],roots]).astype(int)
upper = np.concatenate([roots,[len(lons)]]).astype(int)
for low, high in zip(lower,upper):
lo_patch = lons[low:high]
la_patch = lats[low:high]
lo_patch[lo_patch<0] += 360
x,y = m(lo_patch,la_patch)
ax.plot(x,y,'k',lw=0.5)
m.drawmeridians(np.arange(0,360,45),labels=[True,False,False,True])
plt.show()
In the example above, I iterate through the line segments of the shape file the way it is explained in the Basemap documentation. First I thought it would be enough to just add 360 to each point with a longitude smaller 0, but then you would get horizontal lines whenever a coast line crosses the 0 degree line. So, instead, one has to cut the lines into smaller segments whenever such a crossing appears. This is quite easily accomplished with numpy. I then use the plot command to draw the coast lines. If you want to do something more complex have a look at the Basemap documentation.
The final result looks like this:
Hope this helps.
I have a set of satellite data file here, I created a grid for lat & lon and a 2D array for Ozone values.
I know that in order to plot the contourf of the data in a map I need the projection coordinates, but I can't get find a way around it as my grid is not square (144x24). I am covering the geographical area (0 to 360; -30 to 30) and I require square pixels.
The data is quite long to post it but this is my code so far,
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from mpl_toolkits.basemap import Basemap, cm
%matplotlib inline
path = '/home/rafaella/month_files_CSV/O3_COLUMNS_MATCHED_fv0005_200306.csv'
df = pd.read_csv(path, skiprows=1)
df = pd.read_csv(path, delim_whitespace=True)
lat = np.array(df['AVG_LAT'])
lon = np.array(df['AVG_LON'])
toc = np.array(df['TROP_COL'])
#new grid for lon[0,360] lat[-30,30]
lomin = 0
lomax = 360
lamin = -30
lamax = 30
stp = 2.5
loc_lon = np.zeros(int((lomax-lomin)/stp))
loc_lat = np.zeros(int((lamax-lamin)/stp))
for i in range(0,len(loc_lon)):
loc_lon[i] = i*stp +lomin
for j in range(0,len(loc_lat)):
loc_lat[j] = j*stp +lamin
mtoc_local = np.zeros((len(loc_lon),len(loc_lat)))
sdtoc_local = np.zeros((len(loc_lon),len(loc_lat)))
mtoc_local[:,:] = np.nan
sdtoc_local[:,:] = np.nan
for i in range (0, len(loc_lon)):
for j in range (0,len(loc_lat)):
ix = np.where((lon>=loc_lon[i])& (lat>=loc_lat[j]) & (lon<loc_lon[i]+stp) & (lat<loc_lat[j]+stp))[0]
mtoc_local[i,j]=np.nanmean(toc[ix])
sdtoc_local[i,j]=np.nanstd(toc[ix])
fig = plt.figure(figsize=(20, 5))
map = Basemap(llcrnrlon=0,llcrnrlat=-30, urcrnrlon=360.,urcrnrlat=30.,\
rsphere=(6378137.00,6356752.3142),\
resolution='l',projection='merc',\
lat_0=0,lon_0=-30.,lat_ts=30.)
map.drawcoastlines()
# draw parallels
map.drawparallels(np.arange(-30,30,10),labels=[1,1,0,1])
# draw meridians
map.drawmeridians(np.arange(-180,180,20),labels=[1,1,0,1])
map = plt.contourf(loc_lon, loc_lat , mtoc_local.T, vmin=210, vmax=350, cmap='RdPu')
plt.colorbar(orientation='horizontal', ticks=[200, 220, 240, 260, 280, 300, 320, 340] )
plt.title('Tropical TOC monthly mean 06,2009')
plt.show()
It plots very well the map OR the data but not both. here an image of both separately
map
real data
I am very new to python, I started a month ago, so it is still not familiar to me all the functions and libraries.
Your code has two problems. First you have to apply the projection on your coordinates which is done using x,y = map(lon, lat). However, this will raise an error in your case since the dimensions of loc_lon and loc_lat are different. Instead of passing x and y vectors to the contourf function you can pass arrays with the same shape as z (mtoc_local.T). You can use np.meshgrid to create those. Long story short, replace the line with the contourf command with the following three lines
X, Y = np.meshgrid(loc_lon, loc_lat)
x,y = map(X,Y)
map = plt.contourf(x, y , mtoc_local.T, vmin=210, vmax=350, cmap='RdPu')
and the result looks like this