Develop Reference

TypeError: '.dt' accessor only available for DataArray with datetime64 timedelta64 dtype or for arrays containing cftime datetime objects

Having searched through similar questions for a possible solution on Stackoverflow, non seems to address this particular challenge. After running the code, I was getting the TypeError: **'.dt' accessor only available for DataArray with datetime64 timedelta64 dtype or for arrays containing cftime datetime objects.. I have tried tweaking the code differently to no avail, any help at this point will be greatly appreciated, please. Thanks
import os
from netCDF4 import Dataset
import xarray as xr
import numpy as np
import pandas as pd
from datetime import datetime
#import ffmpeg
from IPython.display import HTML
from matplotlib import pyplot as plt
from matplotlib import animation
import ipynb
import cartopy.crs as ccrs
from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER
%run ../functions.ipynb
# Load Metop-A GOME-2 Level 3 AAI data
file = 'ESACCI-AEROSOL-L3-AAI-GOME2A-1D-20210205-fv1.8.nc'
aai_gome2a = xr.open_dataset(file)
# Load Metop Metop-A, -B & -C netcdf data
ds_a = xr.open_mfdataset('ESACCI-AEROSOL-L3-AAI-GOME2A-1D-2021020*.nc',
concat_dim='time',
combine='nested')
ds_b = xr.open_mfdataset('ESACCI-AEROSOL-L3-AAI-GOME2B-1D-2021020*.nc',
concat_dim='time',
combine='nested')
ds_c = xr.open_mfdataset('ESACCI-AEROSOL-L3-AAI-GOME2C-1D-2021020*.nc',
concat_dim='time',
combine='nested')
# display the variable of interest, i.e absorbing_aerosol_index
aai_a=ds_a['absorbing_aerosol_index']
aai_b=ds_b['absorbing_aerosol_index']
aai_c=ds_c['absorbing_aerosol_index']
# Concatenate the data from the three satellites Metop-A, -B and -C
aai_concat = xr.concat([aai_a,aai_b,aai_c], dim='satellite')
# Retrieve time coordinate information and assign time coordinates for the time dimension
start_day = aai_gome2a.description.split()[4]
time_coords = pd.date_range(datetime.strptime(start_day,'%d-%m-%Y'), periods=len(aai_concat.time), freq='d').strftime("%Y-%m-%d").astype('datetime64[ns]')
# Combine AAI data from the three satellites Metop-A, -B and -C onto one single grid
aai_combined = aai_concat.mean(dim='satellite')
# Visualize AAI data with data from the three satellites Metop-A, -B and C combined on one single grid
visualize_pcolormesh(data_array=aai_combined[1,:,:],
longitude=aai_combined.longitude,
latitude=aai_combined.latitude,
projection=ccrs.PlateCarree(),
color_scale='afmhot_r',
unit=' ',
long_name=aai_a.long_name + ' ' + str(aai_combined.time[0].dt.strftime('%Y-%m-%d').data),
vmin=0,
vmax=5,
lonmin=-50,
lonmax=36,
latmin=0,
latmax=70.,
set_global=False)
See expected output here: https://www.canva.com/design/DAE-vuWH6Ak/view

Python: how can I read data from files and assign it to an array? error":could not broadcast input array from shape"

I have the following MATLAB code that works fine using text data files, now I am trying to rewrite it using Python but running into errors. I have results that I am trying to apply some calculations on (perform data analysis). My results are in the format of binary files and I have a specific package I am using to help me import the data. For example, here ne is a 1024x256 array with 159 number of files printed per each iteration. So, in MATLAB I can simply do the following:
% Load data:
frame = 6; % how many number of output files
ne_bg = load([DirPath '/ne_unpert.txt']);
ne_p = load([DirPath '/ne_' num2str(frame) '.txt']);
% perform calculations on data:
ne = ne_bg + ne_p;
dn_over_n = ne_p ./ ne;
Since MATLAB deals easily with multi-dimensional arrays and matrices, I am struggling to interpret that to python.
My Python code:
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import matplotlib.gridspec as gridspec
import matplotlib.colors as colors
import matplotlib.patches as patches
import scipy.optimize as opt
from scipy.special import erf, comb, gamma, gammainc
import scipy.constants as const
from scipy.integrate import odeint
import sys
from glob import glob
from mpl_toolkits.axes_grid1 import make_axes_locatable
import Package as pg
# Initialize sizes
ne = np.zeros((1024,256))
ne_p = np.zeros((1024,256))
# Data
data = pg.GData('ne_p.bp')
dg = pg.GInterpModal(data, 2, 'ms')
#dg.interpolate(overwrite=True)
ne_p = data.getValues()
data = pg.GData('ne0.gkyl')
dg = pg.GInterpModal(data, 2, 'ms')
#dg.interpolate(overwrite=True)
ne_bg = data.getValues()
for i in range(1,159): # would like to look at files start from 1 to 159 not 0
data = pg.GData('ne{:d}.gkyl'.format(i))
dg = pg.GInterpModal(data, 2, 'ms')
ne[i,:] = data.getValues() # ERROR HERE
dn_over_n = ne_p/ne # get
....
Error message:
ValueError Traceback (most recent call last)
<ipython-input-35-d6134fb807e8> in <module>
48 dg = pg.GInterpModal(data, 2, 'ms')
49 #dg.interpolate(overwrite=True)
---> 50 ne[i,:] = data.getValues()
ValueError: could not broadcast input array from shape (1024,256,1) into shape (256)
Can someone show me how to fix this and explain what it means?

How do I make a for loop to each individual array?

I have a function that I created and I want the function to be applied to these different values using a for loop or something.
How do I create a for loop that takes each value but stores them in different arrays?
I have this so far:
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
import matplotlib.patches as patches
import xarray as xr
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import netCDF4 as s
import numpy.ma as ma
fwf_tot = fwf_ice + ds.runoff_tundra*ds.LSMGr #data input i am using
# function i want to apply to the data
def ob_annual(ob_monthly, id_number):
ann_sum = ob_monthly.where(ds.ocean_basins == id_number).resample(TIME='1AS').sum().sum(dim=('X','Y'))
return ann_sum
This is where my problem is to create the for loop to save for these different values. I think this for loop is just saving the function applied to the last value (87) and not the others. How might I fix this? I expected there to be an output of 7 arrays with each a size of 59.
obs = np.array([26,28,29,30,76,84,87])
total_obs = []
for i in obs:
total_obs = ob_annual(fwf_tot_grnl, i)
print(total_obs.shape)
(59)

You replace your list total_obs at each iteration. You must append each value into it:
for i in obs:
total_obs.append(ob_annual(fwf_tot_grnl, i))
or use a comprehension list
total_obs = [ob_annual(fwf_tot_grnl, i) for i in obs]

How to plot efficient frontier with pypfopt plotting

Hey guysm someone could help me?
I'm new here and in python codying to, so i'm beginner level.
I'm was trying plot my efficient frontier using pypfopt lib and i'm got some troubles with the parameters in Plotting.plot_efficient_frontier command that is used to plot the graph.
I added the picture of my code in the post
#importando as libs
import pandas as pd
import numpy as np
import pandas_datareader.data as web
import datetime as dt
import matplotlib as plt
from pypfopt.expected_returns import mean_historical_return
from pypfopt.risk_models import CovarianceShrinkage
#definindo o período de análise
start = dt.datetime(2019,1,1)
end = dt.datetime(2019,12,31)
#asset tickers
tickers = ["ITUB4.SA","B3SA3.SA","PETR4.SA","ABEV3.SA","ELET3.SA"]
#getting the data
carteira = web.DataReader(tickers,"yahoo",start,end)
#craindo outro dataframe só com os proços de feachamento ajustados
prices = carteira['Adj Close']
#expected returns
#covariance matrix
e_r = mean_historical_return(prices)
cov_matrix = CovarianceShrinkage(prices).ledoit_wolf()
from pypfopt.efficient_frontier import EfficientFrontier
#border problem solution
#selectiong portfolio with better sharpe ratio
ef = EfficientFrontier(e_r, cov_matrix)
weights = ef.max_sharpe()
from pypfopt import Plotting
Plotting.plot_efficient_frontier(ef ,points=100 , show_assets = True)
[n ][1]
[1]: https://i.stack.imgur.com/NaeJI.png

try it in lowercase() like:
from pypfopt import plotting

Importing images for manifold Isomap

There are 192 x 144 pixel images. They should be imported to a Python list so that the items in the list are NDArray instances. New dataframe should be created from the list and that dataframe should be given to Isomap. iso.fit(df) fails with the errors
array = array.astype(np.float64)
ValueError: setting an array element with a sequence.
I have spent more than one day trying to figure out how the NDArrays should be processed and the dataframe loaded with them. No luck. Any help would be appreciated.
import pandas as pd
from scipy import misc
import glob
from sklearn import manifold
samples = []
for filename in glob.glob('Datasets/ALOI/32/*.png'):
img = misc.imread(filename, mode='I')
samples.append(img)
df = pd.DataFrame.from_records(samples, coerce_float=True)
iso = manifold.Isomap(n_neighbors=6, n_components=3)
iso.fit(df)

If those are gray scale images from the ALOI, you probably want to treat each pixel's brightness as a feature. Therefore, you should flatten the img array with img.reshape(-1). The revised code follows:
import pandas as pd
from scipy import misc
import glob
from sklearn import manifold
samples = []
for filename in glob.glob('Datasets/ALOI/32/*.png'):
img = misc.imread(filename, mode='I')
# the following line changed
samples.append(img.reshape(-1))
df = pd.DataFrame.from_records(samples, coerce_float=True)
iso = manifold.Isomap(n_neighbors=6, n_components=3)
iso.fit(df)