I'm trying to use pyProj to convert UTM coordinates from WGS84 UTM 30N (epsg 36230) to ED50 (epsg 23030).
My WGS 85 coordinate is 456859.248 E, 6212329.347 N. When I convert using this online convertor [https://tool-online.com/en/coordinate-converter.php] I should get this: 456961.342 E, 6212551.085 N. Instead I get this: 456959.3405 E, 6212550.44 N, which as you can see is around 2 metres from where it should be.
In my code, I import a simple CSV with two columns called 'Easting' and 'Northing'. The CSV has one row which is the Easting and Northing values I posted above.
CODE BELOW
import pandas as pd
from pyproj import Transformer
class Transform:
def __init__(self, in_epsg, out_epsg, input_file):
self.in_epsg = in_epsg
self.out_epsg = out_epsg
self.transformer = Transformer.from_crs(f"epsg:{self.in_epsg}", f"epsg:{self.out_epsg}")
self.df = pd.read_csv(input_file)
self.x_list, self.y_list = [], []
def iterate_rows(self, column1, column2):
for index, row in self.df.iterrows():
x = (row[column1])
y = (row[column2])
x_new, y_new = self.transformer.transform(x, y)
self.x_list.append(x_new)
self.y_list.append(y_new)
self.df[f'Easting_converted'], self.df[f'Northing_converted'] = self.x_list, self.y_list
self.df.to_csv('converted.csv')
test = Transform(32630, 23030, 'small.csv')
test.iterate_rows('Easting', 'Northing')
I am trying to curve fit a sinusoidal shaped data set, but I a getting an error saying 'Only size-1 arrays can be converted to Python scalars'. How do I correctly pass my two series for X and Y values to fit the curve?
def objective(x, a, b, c, d):
return a * math.sin(b - x) + c * x**2 + d
# choose the input and output variables
x = moon_data["Full Moon"].values.squeeze()
y = moon_data["Full Moon Price"].values.squeeze()
plt.scatter(x,y) # This works!
# curve fit
popt, _ = curve_fit(objective, x, y) # This is the line causing the error
Moon Data is a Dataframe that I turned into a Series using .squeeze(). The original data looks like this (first 3 rows):
Full Moon
Full Moon Price
1488
2020-05-07
10001.0
1489
2020-06-05
9617.17
1490
2020-07-05
9083.8
Only Size 1 Arrays Error is a TypeError that gets triggered when you enter an array as a parameter in a function or method which accepts a single scalar value. So, the problem in your function is here: math.sin(b - x)
x can only be a single scalar value but what you are assigning is moon_data["Full Moon"] as x which is an array. Change it to np.sin(b*x) it will work but I guess you want something else so you will have to change the function accordingly.
You can't put datetime object inside sin(). Math.sin() takes a single number, not array. So, use np.sin() instead. Convert the moon_data["Full Moon"] column to datetime then toordinal so that you can put it into sin.
import pandas as pd
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
def objective(x, a, b, c,d):
return a * np.sin(b*x) + c * x**2+d
# choose the input and output variables
x = moon_data["Full Moon"]
y = moon_data["Full Moon Price"]
X=pd.to_datetime(x).apply(lambda x:x.toordinal())
plt.scatter(x,y) # This works!
# curve fit
popt, _ = curve_fit(objective, X, y)
I am using some code from a workshop to extract data from netCDF files by the coordinates closest to my specified coordinates. When using just one set of coordinates I am able to extract the values I need without trouble as below:
import numpy as np
import netCDF4
from math import pi
from numpy import cos, sin
def tunnel_fast(latvar,lonvar,lat0,lon0):
'''
Find closest point in a set of (lat,lon) points to specified point
latvar - 2D latitude variable from an open netCDF dataset
lonvar - 2D longitude variable from an open netCDF dataset
lat0,lon0 - query point
Returns iy,ix such that the square of the tunnel distance
between (latval[it,ix],lonval[iy,ix]) and (lat0,lon0)
is minimum.
'''
rad_factor = pi/180.0 # for trignometry, need angles in radians
# Read latitude and longitude from file into numpy arrays
latvals = latvar[:] * rad_factor
lonvals = lonvar[:] * rad_factor
ny,nx = latvals.shape
lat0_rad = lat0 * rad_factor
lon0_rad = lon0 * rad_factor
# Compute numpy arrays for all values, no loops
clat,clon = cos(latvals),cos(lonvals)
slat,slon = sin(latvals),sin(lonvals)
delX = cos(lat0_rad)*cos(lon0_rad) - clat*clon
delY = cos(lat0_rad)*sin(lon0_rad) - clat*slon
delZ = sin(lat0_rad) - slat;
dist_sq = delX**2 + delY**2 + delZ**2
minindex_1d = dist_sq.argmin() # 1D index of minimum element
iy_min,ix_min = np.unravel_index(minindex_1d, latvals.shape)
return iy_min,ix_min
ncfile = netCDF4.Dataset('E:\wind_level2_1.nc', 'r')
latvar = ncfile.variables['latitude']
lonvar = ncfile.variables['longitude']
#_________GG turbine_________GAD10 Latitude 51.735516, GAD10 Longitude 1.942656
iy,ix = tunnel_fast(latvar, lonvar, 51.735516, 1.942656)
print('Closest lat lon:', latvar[iy,ix], lonvar[iy,ix])
refLAT=latvar[iy,ix]
refLON = lonvar[iy,ix]
#try to find the data for this location
SARwind = ncfile.variables['sar_wind'][:,:]
ModelWind = ncfile.variables['model_speed'][:,:]
print 'iy,ix' #appears to be the index of the value of Lat,lon
print SARwind[iy,ix]
ncfile.close()
Now I am trying to loop through a text file containing coordinates coord_list to extract sets of coordinates, find the data then move to the next set of coordinates in the list. This code works on it's own as below:
import csv
from decimal import Decimal
with open('Turbine_locs_no_header.csv','rb') as f:
reader = csv.reader(f)
#coord_list = list(reader)
coord_list = [reader]
end_row = len(coord_list)
lon_ind=1
lat_ind=2
for row in range(0, end_row-1):#end_row - 1 due to the 0 index
turbine_lat = coord_list[row][lat_ind]
turbine_lon = coord_list[row][lon_ind]
turbine_lat = [Decimal(turbine_lat)]
print 'lat',turbine_lat, 'lon',turbine_lon, row
However, I want to pass coordinates from the text file to this part of the original code iy,ix = tunnel_fast(latvar, lonvar, 51.94341, 1.922094888), replacing the numbers with variables iy, ix = tunnel_fast(latvar, lonvar, turbine_lat, turbine_lon). I try to combine the two codes by creating a function get_coordinates, I get the following errors
File "C:/Users/mm/test_nc_bycoords_GG_turbines_AGW.py", line 65, in <module>
get_coordinates(coord_list, latvar, lonvar)
File "C:/Users/mm/test_nc_bycoords_GG_turbines_AGW.py", line 51, in get_coordinates
iy, ix = tunnel_fast(latvar, lonvar, turbine_lat, turbine_lon)
File "C:/Users/mm/test_nc_bycoords_GG_turbines_AGW.py", line 27, in tunnel_fast
lat0_rad = lat0 * rad_factor
TypeError: can't multiply sequence by non-int of type 'float'
I thought this is because the turbine_lat and turbine_lon are list items so cannot be used, but this doesn't seem to be connected to the errors. I know this code needs more work anyway, but if anyone could help me spot where I am going wrong that would be very helpful. My attempt to combine the two codes is below.
import numpy as np
import netCDF4
from math import pi
from numpy import cos, sin
import csv
# edited from https://github.com/Unidata/unidata-python-workshop/blob/a56daa50d7b343c7debe93968683613642d6b9f7/notebooks/netcdf-by-coordinates.ipynb
def tunnel_fast(latvar,lonvar,lat0,lon0):
'''
Find closest point in a set of (lat,lon) points to specified point
latvar - 2D latitude variable from an open netCDF dataset
lonvar - 2D longitude variable from an open netCDF dataset
lat0,lon0 - query point
Returns iy,ix such that the square of the tunnel distance
between (latval[it,ix],lonval[iy,ix]) and (lat0,lon0)
is minimum.
'''
rad_factor = pi/180.0 # for trignometry, need angles in radians
# Read latitude and longitude from file into numpy arrays
latvals = latvar[:] * rad_factor
lonvals = lonvar[:] * rad_factor
ny,nx = latvals.shape
lat0_rad = lat0 * rad_factor
lon0_rad = lon0 * rad_factor
# Compute numpy arrays for all values, no loops
clat,clon = cos(latvals),cos(lonvals)
slat,slon = sin(latvals),sin(lonvals)
delX = cos(lat0_rad)*cos(lon0_rad) - clat*clon
delY = cos(lat0_rad)*sin(lon0_rad) - clat*slon
delZ = sin(lat0_rad) - slat;
dist_sq = delX**2 + delY**2 + delZ**2
minindex_1d = dist_sq.argmin() # 1D index of minimum element
iy_min,ix_min = np.unravel_index(minindex_1d, latvals.shape)
return iy_min,ix_min
#________________my edits___________________________________________________
def get_coordinates(coord_list, latvar, lonvar):
"this takes coordinates from a .csv and assigns them to variables"
end_row = len(coord_list)
lon_ind=1
lat_ind=2
for row in range(0, end_row-1):#end_row - 1 due to the 0 index
turbine_lat = coord_list[row][lat_ind]
turbine_lon = coord_list[row][lon_ind]
iy, ix = tunnel_fast(latvar, lonvar, turbine_lat, turbine_lon)
print('Closest lat lon:', latvar[iy, ix], lonvar[iy, ix])
#________________________________________________________________________________________________________________________
ncfile = netCDF4.Dataset('NOGAPS_wind_level2_1.nc', 'r')
latvar = ncfile.variables['latitude']
lonvar = ncfile.variables['longitude']
#____added in to pass to get coordinates function
with open('Turbine_locs_no_header.csv','rb') as f:
reader = csv.reader(f)
coord_list = list(reader)
#_________take latitude from coordinateas function
get_coordinates(coord_list, latvar, lonvar)
#iy,ix = tunnel_fast(latvar, lonvar, turbine_lat, turbine_lon)#get these from the 'assign_coordinates_fromlist.py
#print('Closest lat lon:', latvar[iy,ix], lonvar[iy,ix])
SARwind = ncfile.variables['sar_wind'][:,:]
ModelWind = ncfile.variables['model_speed'][:,:]
print 'iy,ix' #appears to be the index of the value of Lat,lon
print SARwind[iy,ix]
ncfile.close()
When I try to convert
You can unpack an argument list using *args (see the docs). In your case you could do tunnel_fast(latvar, lonvar, *coord_list[row]). You need to make sure that the order of arguments in coord_list[row] is correct and if coord_list[row] contains more than the two values then you need to slice it appropriately.
Thanks to help from a_guest
It was a simple problem of lat0 and lon0 being passed as
<type 'str'> to tunnel_fast when it requires <type 'float'>. This appears to come from loading the coord_list as a list.
with open('Turbine_locs_no_header.csv','rb') as f:
reader = csv.reader(f)
coord_list = list(reader)
The workaround I used was to convert lat0 and lon0 to floats at the beginning of tunnel_fast
lat0 = float(lat0)
lon0 = float(lon0)
I am sure there is a more elegant way to do this, but it works.
I am trying to smooth/filter data from USB to serial. The data contains force value from two servos. It's not showing exact X and Y direction now. So I need a use full example code to smooth/filter my data from serial port
My function which gives the data from serial is given below
def get_force(self):
torque1 = self.servo1.get_servo_torque()
torque2 = self.servo2.get_servo_torque()
angle1, angle2 = self.get_jointpos()
angle1 = math.radians(angle1)
angle2 = math.radians(angle2)
A=-(L1*math.cos(angle1)+L2*math.cos(angle1+angle2))
B=L1*math.sin(angle1)+L2*math.sin(angle1+angle2)
C=L2*math.sin(angle1+angle2)
D=L2*math.cos(angle1+angle2)
self.a = np.array([[A, B], [C, D]])
ainv = inv(self.a)
#print np.allclose(np.dot(self.a, ainv), np.eye(2))
#print np.allclose(np.dot(ainv, self.a), np.eye(2))
#F=[F1,F2]
c = ainv
b = [torque1, torque2]
F=np.dot(c, b)
#print F[0]
self.G=F[0]
self.H=F[1]
return(self.G,self.H)
I want to filter the G and H data and plot in matplotlib animation .....Expecting sample code and useful link.
I am trying to convert the following code from Trefethen's Spectral Methods in MATLAB to Python.
% p6.m - variable coefficient wave equation
% Grid, variable coefficient, and initial data:
N = 128; h = 2*pi/N; x = h*(1:N); t = 0; dt = h/4;
c = .2 + sin(x-1).^2;
v = exp(-100*(x-1).^2); vold = exp(-100*(x-.2*dt-1).^2);
% Time-stepping by leap frog formula:
tmax = 8; tplot = .15; clf, drawnow, set(gcf,'renderer','zbuffer')
plotgap = round(tplot/dt); dt = tplot/plotgap;
nplots = round(tmax/tplot);
data = [v; zeros(nplots,N)]; tdata = t;
for i = 1:nplots
for n = 1:plotgap
t = t+dt;
v_hat = fft(v);
w_hat = 1i*[0:N/2-1 0 -N/2+1:-1] .* v_hat;
w = real(ifft(w_hat));
vnew = vold - 2*dt*c.*w; vold = v; v = vnew;
end
data(i+1,:) = v; tdata = [tdata; t];
end
waterfall(x,tdata,data), view(10,70), colormap(1e-6*[1 1 1]);
axis([0 2*pi 0 tmax 0 5]), ylabel t, zlabel u, grid off
For the most part it is going smoothly except for this line of code
data = [v; zeros(nplots,N)]
After reading how to convert between Numpy and Matlab here Link I tried to convert it by doing the following
data = np.array(v,zeros(nplots,N))
but I get this error
data = np.array(v,zeros(nplots,N));
TypeError: data type not understood
Which I assume is because a numpy array has this structure
numpy.array(object,dtype=none)
I would appreciate any help with converting that line. Thank you in advance!
data = [v; zeros(nplots,N)] this is concatenating two matrices and stacken them up, note the ; in numpy you can use numpy.concatenate((v, zeros((nplots,N))), axis = 0) where axis is by which axis you want to concatenate by ...
data = np.array(v,zeros(nplots,N));
TypeError: data type not understood
basically when you call np.array the fist argument must be iterable object, list, tuple and on the second argument must be the type ie 'int', 'float32', 'float32' and so on ... but you set the type to zeros(nplots,N) numpy is complaining that it isn't a type ...
numpy.zeros is the same the first argument must be a tuple and the second the type, sorry about that I didn't properly include ()
it should be data = numpy.concatenate((v, numpy.zeros((nplots,N))), axis = 0) assuming that you want to use double type which is the standard type.