Develop Reference

Get address using Latitude and Longitude from two columns in DataFrame?

I have a dataframe with the longitude column and latitude column. When I try to get the address using geolocator.reverse() I get the error ValueError: Must be a coordinate pair or Point
I can't for the life of me insert the lat and long into the reverse function without getting that error. I tried creating a tuple using list(zip(zips['Store_latitude'], zips['Store_longitude'])) but I get the same error.
Code:
import pandas as pd
from geopy.geocoders import Nominatim
from decimal import Decimal
from geopy.point import Point
zips = pd.read_excel("zips.xlsx")
geolocator = Nominatim(user_agent="geoapiExercises")
zips['Store_latitude']= zips['Store_latitude'].astype(str)
zips['Store_longitude'] = zips['Store_longitude'].astype(str)
zips['Location'] = list(zip(zips['Store_latitude'], zips['Store_longitude']))
zips['Address'] = geolocator.reverse(zips['Location'])
What my DataFrame looks like
Store_latitude
Store_longitude
34.2262225
-118.4508349
34.017667
-118.149135

I think you might try with a tuple or a geopy.point.Point before going to a list to see whether the package works all right.
I tested just now as follows (Python 3.9.13, command line style)
import geopy
p = geopy.point.Point(51.4,3.45)
gl = geopy.geocoders.Nominatim(user_agent="my_test") # Without the user_agent it raises a ConfigurationError.
gl.reverse(p)
output:
Location(Vlissingen, Zeeland, Nederland, (51.49433865, 3.415005767601362, 0.0))
This is as expected.
Maybe you should cast your dataframe['Store_latitude'] and dataframe['Store_longitude'] before/after you convert to list? They are not strings?
More information on your dataframe and content would be required to further assist, I think.
Good luck!
EDIT: added information after OP's comments below.
When you read your excel file as zips = pd.read("yourexcel.xlsx") you will get a pandas dataframe.
The content of the dataframe is two columns (which will be of type Series) and each element will be a numpy.float64 (if your excel has real values as input and not strings!). You can check this using the type() command:
>>> type(zips)
<class 'pandas.core.frame.DataFrame'>
>>> type(zips['Lat'])
<class 'pandas.core.series.Series'>
>>> type(zips['Lat'][0])
<class 'numpy.float64'>
What you then do is convert these floats (=decimal numbers) to a string (=text) by performing zips[...] = zips[...].astype(str). There is no reason to do that, because your geolocator requires numbers, not text.
As shown in the comment by #Derek, you need to iterate over each row and while doing so, you can put the resulting Locations you receive from the geolocator in a new column.
So in the next block, I first create a new (empty) list. Then i iterate over couples of lat,lon by combining your zips['Lat'] and zips['lon'] using the zip command (so the naming of zips is a bit unlucky if you don't know the zip command; it thus may be confusing you). But don't worry, what it does is just combining the entries of each row in the varables lat and lon. Within the for-each loop, I append the result of the geolocator lookup. Note that the argument of the reverse command is a tuple (lat,lon), so the complete syntax is reverse( (lat,lon) ). Instead of (lat,lon), you could also have created a Point as in my original example. But that is not necessary imo. (note: for brevity I just write 'Lat' and 'Lon' instead of your Store...).
Finally, assign the result list as a new column in your zip pandas dataframe.
import geopy as gp
# instiate a geolocator
gl = gp.geocoders.Nominatim(user_agent="my_test")
locations = [] # Create empty list
# For loop over each couple of lat, lon
for lat,lon in zip(zips['Lat'], zips['Lon']):
locations.append(gl.reverse((lat,lon))
# Add extra column to your pandas table (address will be the column name)
zips = zips.assign(address=locations)
One thing you still may want, is just have the text string instead of the complete geopy.Location() string in your table.
To get that you write the for loop with this small modification ([0] as the first element of the Location object). Note that this won't work if the result of the lookup of a given row is empty (None). Then the [0] will raise an error.
# For loop over each couple of lat, lon
for lat,lon in zip(zips['Lat'], zips['Lon']:
locations.append(gl.reverse((lat,lon)[0])
I hope this gets you going!

Spatially joining a series of netCDF data to a polygon object in python

I have a set of Polygon instances (Loaded via geopandas), which comprise of historical severe-weather based watches and warnings, and netCDF data at 15-minute intervals containing parameters such as tornado / hail forecast parameters and updraft helicity (These data are point data). Each polygon has a set start time and an end time. What I'm trying to do is for each of the 15-minute interval within the polygon's time range (start time -> end time), I want to spatially join in from the netCDF files, the highest value of each of these forecast parameter within the polygon.
I already have the code that pulls the range of time steps (time_list) required to analyze from the netCDF files for each polygon (TOR):
# Pull the warning issue time, locate the closest index in our subhour files
issue_time = datetime.strptime(str(TOR.ISSUED), "%Y%m%d%H%M")
expire_time = datetime.strptime(str(TOR.EXPIRED), "%Y%m%d%H%M")
closest_first = min(time_list, key=lambda d: abs(d - issue_time))
closest_last = min(time_list, key=lambda d: abs(d - expire_time))
# Pull the timesteps in between the issue time and the expire time
before = time_list.index(closest_first) if closest_first < issue_time else time_list.index(closest_first)-1
after = time_list.index(closest_last) if closest_last > issue_time else time_list.index(closest_last)+1
The TOR object is generated by slicing the geoDataFrame of the full polygon data. Right now I am using itertuples on the below slice to get the TOR object:
TORNADOES_IN_WINDOW = df_slice_TO_WRN[(df_slice_TO_WRN.ISSUED > int(first.strftime("%Y%m%d%H%M")))
& (df_slice_TO_WRN.ISSUED < int(last.strftime("%Y%m%d%H%M")))]
Inside the loop, I then iterate over the list of netCDF files in the range of timesteps found (before -> after), and load these into geopandas as well such that I can perform a spatial join, this is done with this block of code:
xr_i = xr.open_dataset(subhr_files[i])
ds_i = xr_i.to_dataframe()
ds_i = ds_i.reset_index()
geom = [Point(x,y) for x, y in zip(ds_i['XLONG'], ds_i['XLAT'])]
gdf = gpd.GeoDataFrame(ds_i, geometry=geom)
The set of problems I'm encountering right now comprise of the following. When trying to run itertuples on the geoDataFrame object (TORNADOES_IN_WINDOW), each iterable returns as a Pandas object, not a GeoPandas object, and therefore I cannot run sjoin to join the attributes. Also, this is not a very efficient methodology, and I'm wondering if perhaps there is a better way to do it.

Create list of dictionary items from lists

I am working on a project that involves going through two columns of latitude and longitude values. If the lat/long in one pair of columns are blank, then I need to figure out which pair of lat/long values in another two columns are (geographically) closest to those in the destination. The dataframe looks like this:
origin_lat | origin_lon | destination_lat | destination_lon
----------------------------------------------------------------
20.291326 -155.838488 25.145242 -98.491404
25.611236 -80.551706 25.646763 -81.466360
26.897654 -75.867564 nan nan
I am trying to build two dictionaries, one with the origin lat and long, and the other with the destination lat and long, in this format:
tmplist = [{'origin_lat': 39.7612992, 'origin_lon': -86.1519681},
{'origin_lat': 39.762241, 'origin_lon': -86.158436 },
{'origin_lat': 39.7622292, 'origin_lon': -86.1578917}]
What I want to do is for every row where the destination lat/lon are blank, compare the origin lat/lon in the same row to a dictionary of all the non-nan destination lat/lon values, then print the geographically closest lat/lon from the dictionary of destination lat/lon to the row in place of the nan values. I've been playing around with creating lists of dictionary objects but can't seem to build a dictionary in the correct format. Any help would be appreciated!

If df is your pandas.DataFrame, you can generate the requested dictionaries by iterating through the rows of df:
origin_dicts = [{'origin_lat': row['origin_lat'], 'origin_long': row['origin_lon']} for _, row in df.iterrows()]
and analogously for destination_dicts.
Remark: if the only reason for creating the dictionaries is the calculation of values replacing the nan-entries, it might be easier to do this directly on the data frame, e.g.
df['destination_lon'] = df.apply(find_closest_lon, axis=1)
df['destination_lat'] = df.apply(find_closest_lat, axis=1)
where find_closest_lon, find_closes_lat are functions receiving a data frame row as an argument and having access to the values of the origin-columns of the data frame.

The format that you want is the built-in 'records' format:
df[['origin_lat','origin_lon']].to_dict(orient = 'records')
produces
[{'origin_lat': 20.291326, 'origin_lon': -155.83848799999998},
{'origin_lat': 25.611235999999998, 'origin_lon': -80.55170600000001},
{'origin_lat': 26.897654, 'origin_lon': -75.867564}]
and of course you can equally have
df[['destination_lat','destination_lon']].to_dict(orient = 'records')
But I agree with #ctenar that you do not need to generate dictionaries for your ultimate task, Pandas provide enough functionality for that

Changing North/South latitude values in Python

In Python I'm dealing with a couple of large csv's containing geographical data in different kinds of formats for latitude and longitude. I settled on converting them to decimal degrees. My issue is that some files are already formatted this way, but with a direction (N,S,E,W) attached at the end of each individual coordinate. Also, the south and west coordinates are not yet negative, and they should be when in decimal degrees.
I was initially using regex to filter these directions out, but can't figure out a way to attach a negative to South and West coordinates before dropping them. I am using pandas to read the csv in.
Example coordinates:
Latitude, Longitude
30.112342N, 10.678982W
20.443459S, 30.678997E
import *pandas* as pd
df = pd.read_csv("mydataset.csv")
if df['Latitude'].str.endswith('S'):
df.Latitude = -float(df['Latitude'].str.strip('S'))
else:
df.Latitude = float(df['Latitude'].str.strip('N'))
Depending on how I tweak it, I get different errors, the most common being:
Attribute error: 'Latitude' object has no attribute 'strip'.
I've tried changing the dtype to string, among other methods, with no luck. I can filter out the directions with regular expressions, but can't discern what the direction was to change to negative if necessary. Any help is appreciated.

Look into .apply(). The value df['Latitude'] is the full column, so you can't work on all of them at once for this sort of operation.
Instead, do something like this:
def fix_latitude(x):
"""Work on individual latitude value X."""
if x.endswith('S'):
x = -float(x.strip('S'))
else:
x = x.strip('N')
return x
df['fixedLatitude'] = df.Latitude.apply(fix_latitude)

How do I save centroids in a shapefile, together with a geometry?

I calculate a centroid for some GeoDataFrame
M["centroid"] = M.centroid
now, I'd like to save that column in a shp file, and do
M[["centroid","geometry"]].to_file("mfile.shp")
But the driver complains I cannot save a Point together with my geometry. I guess that's ok, but I'm wondering what's the best way to store controid information alongside other geo info using geopandas

Because GeoPandas will not allow you to save two geometry columns (when create shapefiles), I would recommend saving xy coordinates into columns for each centroid. From that you can always easily get shapely.Point.
for index, row in M.iterrows():
centroid_coords = row.geometry.centroid.coords.xy
M.loc[index, 'cen_x'] = centroid_coords[0][0]
M.loc[index, 'cen_y'] = centroid_coords[1][0]
M[["cen_x", "cen_y", "geometry"]].to_file("mfile.shp")
EDIT based on comment below (thanks, didn't realise it):
temp = M.centroid
M['cen_x'] = temp.x
M['cen_y'] = temp.y
M[["cen_x", "cen_y", "geometry"]].to_file("mfile.shp")