I have a list of position coordinates given in UK grid reference format (X,Y) and I would like to convert them into latitude and longitude.
I am using OSGridConverter (python library) but is not converting it correctly. For example, one input for one location is X = 517393.6563 and Y = 194035.5469.
from OSGridConverter import grid2latlong
l=grid2latlong('TG 517393.6563 194035.5469')
The above, gives me an error: OSGridConverter error: Invalid grid reference
Therefore, which is wrong, I try:
>>>l=grid2latlong('TG 517393 194035')
>>>(l.latitude,l.longitude)
(52.71367793063314, 1.7297510074170983)
Which ends with a location out in UK, which is not correct. Most probably is due data formats, but I am not sure about how to solve it.
You should probably use something like pyproj:
import pyproj
crs_british = pyproj.Proj(init='EPSG:27700')
crs_wgs84 = pyproj.Proj(init='EPSG:4326')
long, lat = pyproj.transform(crs_british, crs_wgs84, 517393.6563, 194035.5469)
print(lat, long)
In your case this will give: 51.63289090467179, -0.3052119183057834
Firstly, a definition:
OS Grid Grid references (OSGGRs) refer to grid references prepended with the two letter grid identifiers, as explained rather well here.
As you found OSGGRs arn't floating point numbers. See the link above.
Moving on, the numbers for OSGGRs have their origin at the south westerly point of the grid - which is very different from the origin of the OSGB BNG projection (coords are easting and northings). See above link for great explanation.
You seem to be taking the BGGRs numeric portion as an Easting/Northing, which is incorrect.
So, armed with this knowledge, we can use OSGridConverter to convert from OSGGRs to eastings and northings using a conversion to lat long as an intermediary. But note the authors comments on errors at the bottom of the PyPI package page
import OSGridConverter
cvt_wgs84 = OSGridConverter.grid2latlong('SN5117013189')
print(cvt_wgs84.latitude, cvt_wgs84.longitude)
51.79749210128498 -4.160451113839529
cvt_EN = OSGridConverter.latlong2grid(cvt_wgs84.latitude, cvt_wgs84.longitude)
print(cvt_EN.E,cvt_EN.N)
251116 213191
# Back to OSGGR, however, note the error of tens of meters.
str(cvt_EN)
'SN 51116 13191'
I did a quick 'by-eye' check by-map on the agreement between the OSGGR coords and the lat/long, and the error is in meters. May be acceptable to some.
Related
I am using this Mapillary endpoint: https://tiles.mapillary.com/maps/vtp/mly1_public/2/{zoom_level}/{x}/{y}?access_token={} and getting such responses back (see photo). Also, here is the Mapillary documentation.
It is not quite clear to me what the nested coordinate lists in the response represent. By the looks of it, I initially thought it may have to do with pixel coordinates. But judging by the context (the API documentation) and the endpoint I am using, I would say that is not the case. Also, I am not sure if the json response you see in the picture is valid geojson. Some online formatters did not accept it as valid.
I would like to find the bounding box of the "sequence". For context, that would be the minimal-area rectangle defined by two lat, lon positions that fully encompasses the geometry of the so-called "sequence"; and a "sequence" is basically a series of photos taken during a vehicle/on-foot trip, together with the metadata associated with the photos (metadata is available using another endpoint, but that is just for context).
My question is: is it possbile to turn the coordinates you see in the pictures into (lat,lon)? Having those, it would be easy for me to find the bounding box of the sequence. And if so, how? Also, please notice that some of the nested lists are of type LineString while others are MultiLineString (which I read about the difference here: help.arcgis.com, hope this helps)
Minimal reproducible code snippet:
import json
import requests
import mercantile
import mapbox_vector_tile as mvt
ACCESS_TOKEN = 'XXX' # can be provided from here: https://www.mapillary.com/dashboard/developers
z6_tiles = list(mercantile.tiles( #us_west_coast_bbox
west=-125.066423,
south=42.042594,
east=-119.837770,
north=49.148042,
zooms=6
))
# pprint(z6_tiles)
vector_tiles_url = 'https://tiles.mapillary.com/maps/vtp/mly1_public/2/{}/{}/{}?access_token={}'
for tile in z6_tiles:
res = requests.get(vector_tiles_url.format(tile.z,tile.x,tile.y,ACCESS_TOKEN))
res_json = mvt.decode(res.content)
with open('idea.json','w+') as f:
json.dump(res_json, f, indent=4)
I think this get_normalized_coordinates is the solution I was looking for. Please take this with a grain of salt, as I did not fully test it yet. Will try to and then I will update my answer. Also, please be cautious, because for tiles closer to either the South or the North Pole, the Z14_TILE_DMD_WIDTH constant will not be the one you see, but something more like: 0.0018958715374282065.
Z14_TILE_DMD_WIDTH = 0.02197265625
Z14_TILE_DMD_HEIGHT = 0.018241950298914844
def get_normalized_coordinates(bbox: mercantile.LngLatBbox,
target_lat: int,
target_lon: int,
extent: int=4096): # 4096 is Mapillary's default
"""
Returns lon,lat tuple representing real position on world map of a map feature.
"""
min_lon, min_lat, _, _ = bbox
return min_lon + target_lon / extent * Z14_TILE_DMD_WIDTH,
min_lat + target_lat / extent * Z14_TILE_DMD_HEIGHT
And if you are wondering how I came with the constants that you see, I simply iterated over the list of tiles that I am interested in and checked to make sure they all have the same width/height size (this might have not been the case, keeping in mind what I mentioned above about tiles closer to one of the poles - I think this is called "distortion", not sure). Also, for context: these tiles I iterated over are within this bbox: (-125.024414, 31.128199, -108.896484, 49.152970) (min_lon, min_lat, max_lon, max_lat; US west coast) which I believe is also why all the tiles have the same width/height sizes.
set_test = set()
for tile in relevant_tiles_set:
curr_bbox = mercantile.bounds(list_relevant_tiles_set[i])
dm_width_diff: float = curr_bbox.east - curr_bbox.west
dm_height_diff: float = curr_bbox.north - curr_bbox.south
set_test.add((dm_width_diff, dm_height_diff))
set_test
output:
{(0.02197265625, 0.018241950298914844}
UPDATE: forgot to mention that you actually do not need to compute those WIDTH, HEIGHT constants. You just replace those with (max_lon - min_lon) and (max_lat - min_lat) respectively. What I did with those constants was something for testing purposes only
I am attempting to calculate the distance between two Positions using SRID(32148)
Here are my two points
Point-1:
Lat:46.489767 Long:-119.043221
Point-2:
Lat:47.610902 Long:-122.336422
This website states that the distance in miles b/w these two points is 173.388 however from the code below the result I am getting is 0.002161632093865483
This is the code that I am using
employeeloc = modelEmployee.objects.filter(user__first_name="adam")[0].location
employerloc = modelEmployer.objects.filter(user__first_name="adam")[0].location
meters = employerloc.distance(employeeloc)
#Caluclate in miles
dobj = Distance(m=meters)
mi = dobj.mi
This is a little more detail with debugging results attached
Any suggestions on why my result is so different ?
Update:
I tried transforming the position using the following code using SRID 4326. However the results are still incorrect
You appear to have used the lon / lat coordinates as SRID(32148) ones; you need to transform them.
This incorrect query gives your result 3.47m, because the coordinates don't match the SRID:
select
st_distance(
st_setsrid(st_point(-122.336422,47.610902),32148),
st_setsrid(st_point(-119.043221,46.489767),32148))
-- 3.47880964046985
This query gives you the 173.71 mi result you expect:
select
st_distance(
st_transform(st_setsrid(st_point(-122.336422,47.610902),4326),32148),
st_transform(st_setsrid(st_point(-119.043221,46.489767),4326),32148))
--279558.106935732m (=173.71mi)
And that is similar to the result of this query:
select
st_distance(
st_setsrid(st_point(-122.336422,47.610902),4326)::geography,
st_setsrid(st_point(-119.043221,46.489767),4326)::geography)
--279522.55326056 m (= 173.69 mi)
I have a polygon shapefile of the U.S. made up of individual states as their attribute values. In addition, I have arrays storing latitude and longitude values of point events that I am also interested in. Essentially, I would like to 'spatial join' the points and polygons (or perform a check to see which polygon [i.e., state] each point is in), then sum the number of points in each state to find out which state has the most number of 'events'.
I believe the pseudocode would be something like:
Read in US.shp
Read in lat/lon points of events
Loop through each state in the shapefile and find number of points in each state
print 'Here is a list of the number of points in each state: '
Any libraries or syntax would be greatly appreciated.
Based on what I can tell, the OGR library is what I need, but I am having trouble with the syntax:
dsPolygons = ogr.Open('US.shp')
polygonsLayer = dsPolygons.GetLayer()
#Iterating all the polygons
polygonFeature = polygonsLayer.GetNextFeature()
k=0
while polygonFeature:
k = k + 1
print "processing " + polygonFeature.GetField("STATE") + "-" + str(k) + " of " + str(polygonsLayer.GetFeatureCount())
geometry = polygonFeature.GetGeometryRef()
#Read in some points?
geomcol = ogr.Geometry(ogr.wkbGeometryCollection)
point = ogr.Geometry(ogr.wkbPoint)
point.AddPoint(-122.33,47.09)
point.AddPoint(-110.11,33.33)
#geomcol.AddGeometry(point)
print point.ExportToWkt()
print point
numCounts=0.0
while pointFeature:
if pointFeature.GetGeometryRef().Within(geometry):
numCounts = numCounts + 1
pointFeature = pointsLayer.GetNextFeature()
polygonFeature = polygonsLayer.GetNextFeature()
#Loop through to see how many events in each state
I like the question. I doubt I can give you the best answer, and definitely can't help with OGR, but FWIW I'll tell you what I'm doing right now.
I use GeoPandas, a geospatial extension of pandas. I recommend it — it's high-level and does a lot, giving you everything in Shapely and fiona for free. It is in active development by twitter/#kajord and others.
Here's a version of my working code. It assumes you have everything in shapefiles, but it's easy to generate a geopandas.GeoDataFrame from a list.
import geopandas as gpd
# Read the data.
polygons = gpd.GeoDataFrame.from_file('polygons.shp')
points = gpd.GeoDataFrame.from_file('points.shp')
# Make a copy because I'm going to drop points as I
# assign them to polys, to speed up subsequent search.
pts = points.copy()
# We're going to keep a list of how many points we find.
pts_in_polys = []
# Loop over polygons with index i.
for i, poly in polygons.iterrows():
# Keep a list of points in this poly
pts_in_this_poly = []
# Now loop over all points with index j.
for j, pt in pts.iterrows():
if poly.geometry.contains(pt.geometry):
# Then it's a hit! Add it to the list,
# and drop it so we have less hunting.
pts_in_this_poly.append(pt.geometry)
pts = pts.drop([j])
# We could do all sorts, like grab a property of the
# points, but let's just append the number of them.
pts_in_polys.append(len(pts_in_this_poly))
# Add the number of points for each poly to the dataframe.
polygons['number of points'] = gpd.GeoSeries(pts_in_polys)
The developer tells me that spatial joins are 'new in the dev version', so if you feel like poking around in there, I'd love to hear how that goes! The main problem with my code is that it's slow.
import geopandas as gpd
# Read the data.
polygons = gpd.GeoDataFrame.from_file('polygons.shp')
points = gpd.GeoDataFrame.from_file('points.shp')
# Spatial Joins
pointsInPolygon = gpd.sjoin(points, polygons, how="inner", op='intersects')
# Add a field with 1 as a constant value
pointsInPolygon['const']=1
# Group according to the column by which you want to aggregate data
pointsInPolygon.groupby(['statename']).sum()
**The column ['const'] will give you the count number of points in your multipolygons.**
#If you want to see others columns as well, just type something like this :
pointsInPolygon = pointsInPolygon.groupby('statename').agg({'columnA':'first', 'columnB':'first', 'const':'sum'}).reset_index()
[1]: https://geopandas.org/docs/user_guide/mergingdata.html#spatial-joins
[2]: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html
I'm looking to determine the alt/az of (un-famous) stars at given RA/Dec at specific times from Mauna Kea. I'm trying to compute these parameters using pyephem, but the resulting alt/az don't agree with other sources. Here's the calculation for HAT-P-32 from Keck:
import ephem
telescope = ephem.Observer()
telescope.lat = '19.8210'
telescope.long = '-155.4683'
telescope.elevation = 4154
telescope.date = '2013/1/18 10:04:14'
star = ephem.FixedBody()
star._ra = ephem.degrees('02:04:10.278')
star._dec = ephem.degrees('+46:41:16.21')
star.compute(telescope)
print star.alt, star.az
which returns -28:43:54.0 73:22:55.3, though according to Stellarium, the proper alt/az should be: 62:26:03 349:15:13. What am I doing wrong?
EDIT: Corrected latitude and longitude, which were formerly reversed.
First, you've got long and latitude backwards; second, you need to provide the strings in hexadecimal form; and third, you need to provide the RA as hours, not degrees:
import ephem
telescope = ephem.Observer()
# Reversed longitude and latitude for Mauna Kea
telescope.lat = '19:49:28' # from Wikipedia
telescope.long = '-155:28:24'
telescope.elevation = 4154.
telescope.date = '2013/1/18 00:04:14'
star = ephem.FixedBody()
star._ra = ephem.hours('02:04:10.278') # in hours for RA
star._dec = ephem.degrees('+46:41:16.21')
star.compute(telescope)
This way, you get:
>>> print star.alt, star.az
29:11:57.2 46:43:19.6
PyEphem always uses UTC for time, so that programs operate the same and give the same output wherever they are run. You simply need to convert the date you are using to UTC, instead of using your local time zone, and the results agree fairly closely with Stellarium; use:
telescope.date = '2013/1/18 05:04:14'
The result is this alt/az:
62:27:19.0 349:26:19.4
To know where the small remaining difference comes from, I would have to look into how the two programs handle each step of their computation; but does this get you close enough?
I've tried the following,
input: lat/lon data
then I'll calculate a box around it by, let's say 50 m, so +/- 50 m on easting/northing value.
Now I reconvert it to lat/lon and with a script:
http://robotics.ai.uiuc.edu/~hyoon24/LatLongUTMconversion.py I get a result that just can't be, lon before is around 7, afterwards around 2.
zone, easting, northing = LLtoUTM(23, location.get_lat(), location.get_lon())
topUTM = northing + error
bottomUTM = northing - error
leftUTM = easting - error
rightUTM = easting + error
left, top = UTMtoLL(23, leftUTM, topUTM, zone)
Is the error in my code, or might be the script flawed?
So I've tried to use pyproj, just lat/lon to utm to lat/lon to see what happens
>>> p = pyproj.Proj(proj='utm', zone=32, ellps='WGS84')
>>> p
<pyproj.Proj object at 0x7ff9b8487dd0>
>>> x,y = p(47.9941214, 7.8509671)
>>> print x,y
5159550.36822 1114087.43925
>>> print p(x,y,inverse=True)
(47.971558538495991, 7.8546573140162605)
And here it's not as extremely far off as with the script from above, but it still seems strongly enough incorrect as to not be able to use it. How come? What can I do to get more exact results?
EDIT:
I ran test() and it all tests passed.
in epsg file there is no such thing. The closest I've found was this:
<32632> +proj=utm +zone=32 +ellps=WGS84 +datum=WGS84 +units=m +no_defs <>
no tmerc. Also What would I need to pass the towgs84 as parameters? The ones above?
I've created a small UTM conversion library for Python last week and uploaded it to the Python Package Index: http://pypi.python.org/pypi/utm
I have compared it to using pyproj and it is faster and more accurate. Given your sample data, this is the result:
>>> import utm
>>> u = utm.from_latlon(47.9941214, 7.8509671)
>>> print u
(414278, 5316285, 32, 'T')
>>> print utm.to_latlon(*u)
(47.994157948891505, 7.850963967574302)
UPDATE: Richards answer below describes the real solution for this issue.
The error is in your code.
First off, the PyProj issue listed in one of the other answers is real. You should check your epsg file and make sure it includes the line
<2392> +proj=tmerc +lat_0=0 +lon_0=24 +k=1.000000 +x_0=2500000 +y_0=0 +ellps=intl +towgs84=-90.7,-106.1,-119.2,4.09,0.218,-1.05,1.37 +units=m +no_defs no_defs <>
Note the towgs84 parameter.
Your problem with PyProj stems from mis-using the projection command.
If we take 47.9941214N, 7.8509671E and convert to UTM we get Zone 32, 414278 Easting, 5316286 Northing.
You perform the following PyProj operations:
p = pyproj.Proj(proj='utm', zone=32, ellps='WGS84')
>>> x,y = p(47.9941214, 7.8509671)
>>> print x,y
5159550.36822 1114087.43925
>>> print p(x,y,inverse=True)
(47.971558538495991, 7.8546573140162605)
But, if we consult the PyProj documentation, we see the following:
Calling a Proj class instance with the arguments lon, lat will convert
lon/lat (in degrees) to x/y native map projection coordinates (in
meters).
Let's try running the OP's PyProj operations again, but switch the order of the lon/lat arguments:
p = pyproj.Proj(proj='utm', zone=32, ellps='WGS84')
>>> x,y = p(7.8509671, 47.9941214)
>>> print x,y
414278.16731 5316285.59492
>>> print p(x,y,inverse=True)
(7.850967099999812, 47.994121399999784)
The operation inverts itself (pretty much) perfectly!
To answer the first part of your question, if you look in http://robotics.ai.uiuc.edu/~hyoon24/LatLongUTMconversion.py at the definition of UTMtoLL, you find the following:
UTMtoLL(ReferenceEllipsoid, northing, easting, zone)
Yet you use UTMtoLL(23, leftUTM, topUTM, zone) where leftUTM is an Easting and topUTM is a Northing.
Therefore, in the case of both your first script and PyProj, you've used the wrong order of arguments.
It's a good reminder to always double- (or triple-) check your work before suggesting that someone else's is wrong. That said, Python's documentation is not the greatest and PyProj's documentation in this instance is cryptic at best. A nice web-based explanation of this command and accompanied by examples of its usage would have probably prevented angst on your part.
I have no problem with pyproj, try the following code
from pyproj import Proj
Lat = 52.063098675
Lon = -114.132980348 #Calgary
ZoneNo = "11" #Manually input, or calcuated from Lat Lon
myProj = Proj("+proj=utm +zone="+ZoneNo+",\
+north +ellps=WGS84 +datum=WGS84 +units=m +no_defs") #north for north hemisphere
UTMx, UTMy = myProj(Lon, Lat)
########################################
#UTM ==> Lat Lon:
ZoneNo = "11" #Manually input or from other sources
myProj = Proj("+proj=utm +zone="+\
ZoneNo+", +north +ellps=WGS84 +datum=WGS84 +units=m +no_defs")
Lon2, Lat2 = myProj(UTMx, UTMy,inverse=True)
print Lat2
print Lon2
Your issue with pyProj sounds just like the one described here:
http://code.google.com/p/pyproj/issues/detail?id=3
which is resolved:
solved! in epsg file there must be
<2392> +proj=tmerc +lat_0=0 +lon_0=24 +k=1.000000 +x_0=2500000 +y_0=0 +ellps=intl
+towgs84=-90.7,-106.1,-119.2,4.09,0.218,-1.05,1.37 +units=m +no_defs no_defs <>
note the towgs84 parameter!
Check that thread out if you want to continue to use pyproj.
Also, does the test() function of the module work? Have you tried any of the scripts that come with it in the test directory?