This should be very simple:
I have an Excel table with pollution detection sites, and their corresponding Latitudes and Longitudes. I want to create a point shapefile for this table.
I have written a script to create the file, but cannot figure out how to create the points in it:
#Creates Observation Site Shapefile
import arcpy
import fileinput
import string
import os
from arcpy import env
env.workspace = "F:\GEOG 487B\Project"
env.overwriteOutput = True
outpath = env.workspace
newfc = "sites.shp"
infile = "site_loc.xls"
arcpy.CreateFeatureclass_management(outpath, newfc, "Point")
The shapefile is created, but the attribute table is empty. How can I get ArcMap to recognize my Latutude and Longitude columns?
The table is very simple:
Column A = Site ID#, B = Latitude, C = Longitude
Thanks!!
I know there is a library which can read/write shape files. I've used it to read in National Weather Service Shapefiles in the past.
http://code.google.com/p/pyshp/
From the examples on that webpage, writing points into a shape file looks fairly easy.
My answer is probably late, but you can use Make XY Event Layer and then save the output layer to a shapefile or feature class with e.g. Copy Features.
You can also do this without Python by right-clicking your xls sheet in ArcCatalog > Create feature class from XY table.
Related
My goal is to get a so-called "choropleth map" (I guess) of the zip code areas in Germany. I have found the python package "folium" but it seems like it takes a .json file as input:
https://github.com/python-visualization/folium
On OpenStreetMap I only see shp.zip and .osm.pbf files. Inside the shp.zip archive I find all sorts of file endings which I have never heard of but no .json file. How do I use the data from OpenStreetMap to feed folium? Am I running into the wrong direction?
If you want to create a choropleth map you must follow these steps:
First you need a file containing info about the regions of that country. A sample .json file has been supplied with this answer, however, there are actually many file formats commonly used for maps. In your case, you need to convert your OSM shape file (.shp) into a more modern file type like .geojson. Thankfully we have ogr2ogr to do this last part:
ogr2ogr -f GeoJSON -t_srs EPSG:4326 -simplify 1000 [name].geojson [name].shp
Advice: You can also extract the administrative borders from these web sites:
* [OSM Boundaries Map 4.2][2]
* [Mapzen][3]
* [Geofabrik][4]
Download data based on it (a .csv file, for example). Obviously, the file must have a column with the ZIP Codes of that country.
Once you get these files the rest is straightforward, Follium will create the choropleth map automatically.
I wrote a simple example of this about the unemployment rate in the US:
Code:
import folium
import pandas as pd
osm = folium.Map([43, -100], zoom_start=4)
osm.choropleth(
geo_str = open('US_states.json').read(),
data = pd.read_csv("US_unemployment.csv"),
columns = ['State', 'Unemployment'],
key_on = 'feature.id',
fill_color = 'YlGn',
)
Output:
I haven't done this myself but there are various solutions for converting OSM files (.osm or .pbf) to (geo)json. For example osmtogeojson. More tools can be found at the GeoJSON page in the OSM wiki.
I went to https://overpass-turbo.eu/ (which retrieves data from openstreetmap via a specific Query Language QL) and hit run on the following code:
[timeout:900];
area[name="Deutschland"][admin_level=2][boundary=administrative]->.myarea;
rel(area.myarea)["boundary"="postal_code"];
out geom;
You can "export to geojson" but in my case that didn't work because it's too much data which cannot be processed inside the browser. But exporting the "raw data" works. So I did that and then I used "osmtogeojson" to get the right format. After that I was able to feed my openstreetmap data to folium as described in the tutorial of folium.
This answer was posted as an edit to the question Choropleth map with OpenStreetMap data by the OP user3182532 under CC BY-SA 3.0.
I have csv data containing latitude and longitude. I wish to create a 'heatmap' whereby some placemarks are certain colours based on values corresponding to each lat+long point in the csv.
import simplekml
import csv
kml = simplekml.Kml()
kml.document.name = "Test"
with open('final.csv', 'rb') as f:
reader = csv.reader(f)
first_row = reader.next() # removes csv header string
long = col[1]
lat = col[2]
for col in reader:
pnt = kml.newpoint()
pnt.name = 'test-name'
pnt.coords = [(long, lat, 10000)]
pnt.description = col[0] # timestamp data
pnt.style.labelstyle.color = 'ff0000ff'
kml.save("test.kml")
This script creates a kml file that on inspection with google earth presents the data points but I want some kind of graphical input too.
It doesn't seem like simplekml package supports such things.. Any advice on what python package is best to get a 'heatmap' or something along those lines? Or even I can add kml elements directly in the script but the documentation it seems is rather limited for the solutions I require.
Thanks
Sounds like you're looking to create a "thematic map", not a "heatmap".
Looks like you're using pnt.style.labelstyle.color = ... to add colors. LabelStyle refers to the text labels associated with icons, not the icons themselves. What you probably want is to refer to differently colored icons based on your attribute values. You should be able to do that with: pnt.style.iconstyle.icon.href = .... Either specify a colored icon image, or use a white image and apply a color with style.iconstyle.color = ....
Best practice for KML would be to set up several shared styles, one for each icon color, and then apply the relevant style to each point with pnt.style = ....
Implementation details are in the simplekml documentation.
I have a stack of CT-scan images. After processing (one image from those stack) CT-scan image using Matlab, I saved XY coordinates for each different boundary region in different Excel sheets as follows:
I = imread('myCTscan.jpeg');
BW = im2bw(I);
[coords, labeledImg] = bwboundaries(BW, 4, 'holes');
sheet = 1;
for n=1:length(coords);
xlswrite('fig.xlsx',coords{n,1},sheet,'A1');
sheet = sheet+1;
end
The next step is then to import this set of coordinates and plot it into Abaqus CAE Sketch for finite element analysis.
I figure out that my workflow is something like this:
Import Excel workbook
For each sheet in workbook:
2.1. For each row: read both column to get xy coordinates (each row has two column, x and y coordinate)
2.2. Put each xy coordinates inside a list
2.3. From list, sketch using spline method
Repeat step 2 for other sheets within the workbook
I searched for a while and found something like this:
from abaqus import *
lines= open('fig.xlsx', 'r').readlines()
pointList= []
for line in lines:
pointList.append(eval('(%s)' %line.strip()))
s1= mdb.models['Model-1'].ConstrainedSketch(name='mySketch', sheetSize=500.0)
s1.Spline(points= pointList)
But this only read XY coordinates from only one sheet and I'm stuck at step 3 above. Thus my problem is that how to read these coordinates in different sheets using Abaqus/Python (Abaqus 6.14, Python 2.7) script?
I'm new to Python programming, I can read and understand the syntax but can't write very well (I'm still struggling on how to import Python module in Abaqus). Manually type each coordinates (like in Abaqus' modelAExample.py tutorial) is practically impossible since each of my CT-scan image can have 100++ of boundary regions and 10k++ points.
I'm using:
Windows 7 x64
Abaqus 6.14 (with built in Python 2.7)
Excel 2013
Matlab 2016a with Image Processing Toolbox
You are attempting to read excel files as comma separated files. CSV files by definition can not have more than one tab. Your read command is interpreting the file as a csv and not allowing you to iterate over the tabs in your file (though it begs the question how your file is opening properly in the first place as you are saving an xlsx and reading a csv).
There are numerous python libraries that will parse and process XLS/XLSX files.
Take a look at pyxl and use it to read your file in.
You would likely use something like
from openpyxl import Workbook
(some commands to open the workbook)
listofnames=wb.sheetnames
for k in listofnames:
ws=wb.worksheets(k)
and then input your remaining commands.
I'm trying to do some work on a complex Excel Workbook which has a large number of variables which have been created and used using the Name Box feature. See picture attached for example/detail.
I'd like to store or change DeathRate or maybe read all the Name Boxes and create a dictionary between names and locations of the cell from outside Excel.
I'm using the win32com library in Python but I guess I could switch to another Excel reader as long as it copes with XLSX files.
Has someone come across this before?
Found the solution, see code below:
import os
from win32com.client import Dispatch #win32com is based around cells beginning at one.
app_xl = Dispatch("Excel.Application")
WORKING_DIR = os.getcwd()
excelPath = WORKING_DIR + "\SampleModel.xls"
wb = app_xl.Workbooks.Open(excelPath)
# Get Named Boxes
name_box_list = [x for x in app_xl.ActiveWorkbook.Names]
name_box_map = {x.Name:x.Value for x in name_box_list}
print name_box_list
print name_box_map
# Change Named Boxes
name_box_list[0].Name = u'NewName'
name_box_list[0].Value = u'=model!$B$5'
name_box_map = {x.Name:x.Value for x in name_box_list}
I'll try to give a brief background here. I recently received a large amount of data that was all digitized from paper maps. Each map was saved as an individual file that contains a number of records (polygons mostly). My goal is to merge all of these files into one shapefile or geodatabase, which is an easy enough task. However, other than spatial information, the records in the file do not have any distinguishing information so I would like to add a field and populate it with the original file name to track its provenance. For example, in the file "505_dmg.shp" I would like each record to have a "505_dmg" id in a column in the attribute table labeled "map_name". I am trying to automate this using Python and feel like I am very close. Here is the code I'm using:
# Import system module
import arcpy
from arcpy import env
from arcpy.sa import *
# Set overwrite on/off
arcpy.env.overwriteOutput = "TRUE"
# Define workspace
mywspace = "K:/Research/DATA/ADS_data/Historic/R2_ADS_Historical_Maps/Digitized Data/Arapahoe/test"
print mywspace
# Set the workspace for the ListFeatureClass function
arcpy.env.workspace = mywspace
try:
for shp in arcpy.ListFeatureClasses("","POLYGON",""):
print shp
map_name = shp[0:-4]
print map_name
arcpy.AddField_management(shp, "map_name", "TEXT","","","20")
arcpy.CalculateField_management(shp, "map_name","map_name", "PYTHON")
except:
print "Fubar, It's not working"
print arcpy.GetMessages()
else:
print "You're a genius Aaron"
The output I receive from running this script:
>>>
K:/Research/DATA/ADS_data/Historic/R2_ADS_Historical_Maps/Digitized Data/Arapahoe/test
505_dmg.shp
505_dmg
506_dmg.shp
506_dmg
You're a genius Aaron
Appears successful, right? Well, it has been...almost: a field was added and populated for both files, and it is perfect for 505_dmg.shp file. Problem is, 506_dmg.shp has also been labeled "505_dmg" in the "map_name" column. Though the loop appears to be working partially, the map_name variable does not seem to be updating. Any thoughts or suggestions much appreciated.
Thanks,
Aaron
I received a solution from the ESRI discussion board:
https://geonet.esri.com/thread/114520
Basically, a small edit in the Calculate field function did the trick. Here is the new code that worked:
arcpy.CalculateField_management(shp, "map_name","\"" + map_name + "\"", "PYTHON")