I'm trying to plot some meteorological data in NetCDF format accessed via the Unidata siphon package.
I've imported what the MetPy docs suggest are the relevant libraries
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import matplotlib.pyplot as plt
from netCDF4 import num2date
import numpy as np
import xarray as xr
from siphon.catalog import TDSCatalog
from datetime import datetime
import metpy.calc as mpcalc
from metpy.units import units
and I've constructed a query for data as per the Siphon docs
best_gfs = TDSCatalog('http://thredds.ucar.edu/thredds/catalog/grib/NCEP/GFS/Global_0p25deg/catalog.xml?dataset=grib/NCEP/GFS/Global_0p25deg/Best')
best_ds = best_gfs.datasets[0]
ncss = best_ds.subset()
query = ncss.query()
query.lonlat_box(north=55, south=20, east=-60, west=-90).time(datetime.utcnow())
query.accept('netcdf4')
query.variables('Vertical_velocity_pressure_isobaric','Relative_humidity_isobaric','Temperature_isobaric','u-component_of_wind_isobaric','v-component_of_wind_isobaric','Geopotential_height_isobaric')
data = ncss.get_data(query)
Unfortunately, when I attempt to parse the dataset using the code from the Metpy docs
data = data.metpy.parse_cf()
I get an error: "AttributeError: NetCDF: Attribute not found"
When attempting to fix this problem, I came across another SO post that seems to have the same issue, but the solution suggested there -- to update my metpy to the latest version, -- did not work for me. I updated metpy using Conda but got the same problem as before I updated. Any other ideas on how to get this resolved?
Right now the following code in Siphon
data = ncss.get_data(query)
will return a Dataset object from netcdf4-python. You need one extra step to hand this to xarray, which will make MetPy's parse_cf available:
from xarray.backends import NetCDF4DataStore
ds = xr.open_dataset(NetCDF4DataStore(data))
data = ds.metpy.parse_cf()
Related
i am following this tutorial
M/EEG analysis with MNE Python
and i have a little errors in this fragment : 01:35:57 Working with BIDS data
i followed all the steps before and also implemented code :
import matplotlib.pyplot as plt
import pathlib
import matplotlib
import mne
import mne_bids
matplotlib.use('Qt5Agg')
directory ='C:/Users/User/mne_data/MNE-sample-data/MEG\sample/sample_audvis_raw.fif'
raw =mne.io.read_raw(directory)
#raw.plot()
#plt.show()
events =mne.find_events(raw)
#print(events)
event_id ={
"Auditory/Left":1,
"Auditory/Right":2,
"Visual/Left":3,
"Visual/Right":4,
"Smiley":5,
"Button":32
}
raw.info['line_freq']=60
raw.load_data()
out_path =pathlib.Path("out_data/sample_bids")
bids_path =mne_bids.BIDSPath(subject='01',session='01',task='audiovisual',run='01',root=out_path)
mne_bids.write_raw_bids(raw,bids_path=bids_path,events_data=events,event_id=event_id,overwrite=True)
but when i am running this code, i am getting issue :
ValueError: The data is already loaded from disk and may be altered. See warning for "allow_preload".
i can't understand reason of this error, how can i fix it?please helo me
I want to implement the butterworthfilter with python in jupyter Notebook. Python is new to me and i dont know why i get a error. I search here but i didnt find a solution.
The data are from a CSV-File, it calls Samples.csv
The Data in Samples.csv are like
998,4778415
1009,209592
1006,619094
1001,785406
993,9426543
990,1408991
992,736118
995,8127334
1002,381664
1006,094429
1000,634799
999,3287747
1002,318812
999,3287747
1004,427698
1008,516733
1007,964781
1002,680906
1000,14449
994,257009
The column calls Euclidian Norm. The range of the data are from 0 to 1679.286158 and theyre are 1838 rows.
I wrote this code, it was from a tutorial.
from scipy.signal import filtfilt
from scipy import stats
import csv
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import scipy
def plot():
data=pd.read_csv('Samples.csv',sep=";")
sensor_data=data[['Euclidian Norm']]
sensor_data=np.array(sensor_data)
time=np.linspace(0,1679.286158,1838)
plt.plot(time, sensor_data)
plot.show()
plot()
I get the error TypeError: unhashable type: 'numpy.ndarray'and the line of plt.plot(time, sensor_data) it marks yellow.
I dont know what is wrong, because i dont see a type failure in the code, does anyone know what could be wrong in the code?
The problem is that you are using , as the decimal separator in your CSV file but you haven't told Pandas that you are doing that.
Try replacing the line
data=pd.read_csv('Samples.csv',sep=";")
with
data=pd.read_csv('Samples.csv',sep=";", decimal=",")
Really difficult to find anyone using markdown in a python IDE (I am using pycharm), with both R and python chunks.
Here is my code so far; I am just trying to set up my markdown to use both R and python code; it seems like my python chunk doesn't work; any idea why? Thanks!
R environment
library(readODS) # excel data
library(glmmTMB) # mixed models
library(car) # ANOVA on mixed models
library(DHARMa) # goodness of fit of the model
library(emmeans) # post hoc
library(ggplot2) # plots
library(reticulate) # link between R and python
use_python('C:/Users/saaa/anaconda3/envs/Python_projects/python.exe')
Python environment
import pandas as pd
import os
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
here the exemplary code:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import pickle5 as pickle
#Read
output = pd.read_pickle("Energy_data.pkl")
plt.figure()
#print(output)
output.plot()
I am using Python 3.7 and that is probably the reason for the error message, because this .pkl files were created in Python 3.8 . If my colleague runs it (he created the .pkl-Files), it'll work.
I tried to use this solution (maybe I did not do it correctly) shown here, but it did not work anyway. Can someone show me how to import the pkl files using the example above in Python 3.7?
Thank you very much in advance!
I am working with the classic titanic dataset and trying to apply NNs. My data comes already split into train and dev sets. However, I want to merge the datasets together for many things (for example, my own splitting, etc..)
Is there a way I can merge both datasets?
I have looked around and only found information about how to split a dataset, but I was unable to find how to merge them back together.
Any help?
A MWE is provided below!
from __future__ import absolute_import,division,print_function,unicode_literals
import numpy as np
import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
from IPython.display import clear_output
from six.moves import urllib
import tensorflow.compat.v2.feature_column as fc
import tensorflow as tf
import seaborn as sns
# URL address of data
TRAIN_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/train.csv"
TEST_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/eval.csv"
# Downloading data
train_file_path = tf.keras.utils.get_file("train.csv", TRAIN_DATA_URL)
test_file_path = tf.keras.utils.get_file("eval.csv", TEST_DATA_URL)
# Reading data
data_train = pd.read_csv(train_file_path)
data_test = pd.read_csv(test_file_path)
MY_DATA= MERGE HERE????? # merge(data_train,data_test)??
I assume data_train and data_test have the same number of columns and the column names are the same. Then just do
merged_df= pd.concat([data_train, data_test], axis=0)