I have a fairly simple python loop that calls a few functions, and writes the output to a file. To do this is creates a folder, and saves the file in this folder.
When I run the program the first time with a unique file name, it runs fine. However, if I try to run it again, it will not work and I do not understand why. I am quite certain that it is not a problem of overwriting the file, as I delete the folder before re-running, and this is the only place that the file is stored. Is there a concept that I am mis-understanding?
The problematic file is 'buff1.shp'. I am using Python 2.5 to run some analysis in ArcGIS
Thanks for any advice (including suggestions about how to improve my coding style). One other note is that my loops currently only use one value as I am testing this at the moment.
# Import system modules
import sys, string, os, arcgisscripting, shutil
# Create the Geoprocessor object
gp = arcgisscripting.create()
# Load required toolboxes...
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Spatial Statistics Tools.tbx")
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Analysis Tools.tbx")
# specify workspace
gp.Workspace = "C:/LEED/Cities_20_Oct/services"
path = "C:\\LEED\\Cities_20_Oct\\services\\"
results = 'results\\'
os.mkdir( path + results )
newpath = path + results
# Loop through each file (0 -> 20)
for j in range(0,1):
in_file = "ser" + str(j) + ".shp"
in_file_2 = "ser" + str(j) + "_c.shp"
print "Analyzing " + str(in_file) + " and " + str(in_file_2)
#Loop through a range of buffers - in this case, 1,2
for i in range(1,2):
print "Buffering....."
# Local variables...
center_services = in_file_2
buffer_shp = newpath + "buff" + str(i) + ".shp"
points = in_file_2
buffered_analysis_count_shp = newpath + "buffered_analysis_count.shp"
count_txt = newpath + "count.txt"
# Buffer size
b_size = 1000 + 1000 * i
b_size_input = str(b_size) + ' METERS'
print "Buffer:" + b_size_input + "\n"
# Process: Buffer...
gp.Buffer_analysis(center_services, buffer_shp, b_size_input, "FULL", "ROUND", "ALL", "")
print "over"
(To clarify this question I edited a few parts that did not make sense without the rest of the code. The error still remains in the program.)
Error message:
ExecuteError: ERROR 000210: Cannot create output C:\LEED\Cities_20_Oct\services\results\buff1.shp Failed to execute (Buffer).
I can't see how the file name in the error message blahblah\buff1.shp can arise from your code.
for i in range(0,1):
buffer_shp = newpath + "buff" + str(i) + ".shp"
gp.Buffer_analysis(center_services, buffer_shp, etc etc)
should produce blahblah\buff0.shp not blahblah\buff1.shp... I strongly suggest that the code you display should be the code that you actually ran. Throw in a print statement just before the gp.Buffer_analysis() call to show the value of i and repr(buffer_shp). Show all print results.
Also the comment #Loop through a range of buffers (1 ->100) indicates you want to start at 1, not 0. It helps (you) greatly if the comments match the code.
Don't repeat yourself; instead of
os.mkdir( path + results )
newpath = path + results
do this:
newpath = path + results # using os.path.join() is even better
os.mkdir(newpath)
You might like to get into the habit of constructing all paths using os.path.join().
You need to take the call to os.mkdir() outside the loops i.e. do it once per run of the script, not once each time round the inner loop.
The results of these statements are not used:
buffered_analysis_count_shp = newpath + "buffered_analysis_count.shp"
count_txt = newpath + "count.txt"
Update
Googling with the first few words in your error message (always a good idea!) brings up this: troubleshooting geoprocessing errors which provides the following information:
geoprocessing errors that occur when
reading or writing ArcSDE/DBMS data
receive a generic 'catch-all' error
message, such as error 00210 when
writing output
This goes on to suggest some ways of determining what your exact problem is. If that doesn't help you, you might like to try asking in the relevant ESRI forum or on GIS StackExchange.
I see this is a 3 year old posting, but for others will add:
As I generate python script to work with Arc, I always include right after my import:
arcpy.env.overwriteOutput=True # This allows the script to overwrite files.
Also you mentioned you delete your "folder"?. That would be part of your directory, and I do not see where you are creating a directory in the script. You would want to clear the folder, not delete it (maybe you meant you delete the file though).
JJH
I'd be tempted to look again at
path = "C:\LEED\Cities_20_Oct\services\"
Surely you want double front slashes, not double back slashes?
Related
Problem
Referring to the code below, whenever I run this piece of code, it will find the charts all the sheets and exporting as an image. However, when it comes to chartObject.Chart.Export("chart" + str(i) + ".png") , it always exports to document folder in window.
Question
Is there a way to declare a path for it to be exported? (P.s) I have tried reading the documentation but there is no information regarding to my question. Thanks.
Code
1 thing to note: this piece of code was answered in 1 of the stackoverflow question way back in 2018. Hence, I do not own the credit for it. But I am working on something similar. Thanks.
from win32com.client import Dispatch
app = Dispatch("Excel.Application")
workbook_file_name = 'Programmes.xlsx'
workbook = app.Workbooks.Open(Filename=workbook_file_name)
# WARNING: The following line will cause the script to discard any unsaved changes in your workbook
app.DisplayAlerts = False
i = 1
for sheet in workbook.Worksheets:
for chartObject in sheet.ChartObjects():
# print(sheet.Name + ':' + chartObject.Name)
chartObject.Chart.Export("chart" + str(i) + ".png")
i += 1
workbook.Close(SaveChanges=False, Filename=workbook_file_name)
Solution
Hey guys, sorry for me posting my answer. But after some trial and error, I have figure out the solution
chartObject.Chart.Export("chart" + str(i) + ".png")
Simply add the path along with the file name. Example:
chartObject.Chart.Export(os.getcwd() + "/chart" + str(i) + ".png")
For those who are not sure what is os.getcwd() it basically means the current directory where the file of your code is.
Note: Never use os.path.join() cause you will get an error hence is safe to use as stated above.
I am trying to concatenate model output files, the model run was broken up in 5 and each output corresponds to one of those partial run, due to the way the software outputs to file it start relabelling from 0 on each of the file outputs. I wrote some code to:
1) concatenate all the output files together
2) edit the merged file to re-label all timesteps, starting at 0 and increasing by an increment at each one.
The aim is that I can load this single file into my visualization software in one chunk, rather than open 5 different windows.
So far my code throws a memory error due to the large files I am dealing with.
I have a few ideas of how I could try and get rid of it but I'm not sure what will work or/and might slow things down to a crawl.
Code so far:
import os
import time
start_time = time.time()
#create new txt file in smae folder as python script
open("domain.txt","w").close()
"""create concatenated document of all tecplot output files"""
#look into file number 1
for folder in range(1,6,1):
folder = str(folder)
for name in os.listdir(folder):
if "domain" in name:
with open(folder+'/'+name) as file_content_list:
start = ""
for line in file_content_list:
start = start + line# + '\n'
with open('domain.txt','a') as f:
f.write(start)
# print start
#identify file with "domain" in name
#extract contents
#append to the end of the new document with "domain" in folder level above
#once completed, add 1 to the file number previously searched and do again
#keep going until no more files with a higher number exist
""" replace the old timesteps with new timesteps """
#open folder named domain.txt
#Look for lines:
##ZONE T="0.000000000000e+00s", N=87715, E=173528, F=FEPOINT, ET=QUADRILATERAL
##STRANDID=1, SOLUTIONTIME=0.000000000000e+00
# if they are found edits them, otherwise copy the line without alteration
with open("domain.txt", "r") as combined_output:
start = ""
start_timestep = 0
time_increment = 3.154e10
for line in combined_output:
if "ZONE" in line:
start = start + 'ZONE T="' + str(start_timestep) + 's", N=87715, E=173528, F=FEPOINT, ET=QUADRILATERAL' + '\n'
elif "STRANDID" in line:
start = start + 'STRANDID=1, SOLUTIONTIME=' + str(start_timestep) + '\n'
start_timestep = start_timestep + time_increment
else:
start = start + line
with open('domain_final.txt','w') as f:
f.write(start)
end_time = time.time()
print 'runtime : ', end_time-start_time
os.remove("domain.txt")
So far, I get the memory error at the concatenation stage.
To improve I could:
1) Try and do the corrections on the go as I read each file, but since it's already failing to go through an entire one I don't think that would make much of a difference other than computing time
2) Load all the file as into an array and make a function of the checks and run that function on the array:
Something like:
def do_correction(line):
if "ZONE" in line:
return 'ZONE T="' + str(start_timestep) + 's", N=87715, E=173528, F=FEPOINT, ET=QUADRILATERAL' + '\n'
elif "STRANDID" in line:
return 'STRANDID=1, SOLUTIONTIME=' + str(start_timestep) + '\n'
else:
return line
3) keep it as is and ask Python to indicate when it is about to run out of memory and write to the file at that stage. Anyone knows if that is possible ?
Thank you for your help
It is not necessary to read the entire contents of each file into memory before writing to the output file. Large files will just consume, possibly all, available memory.
Simply read and write one line at a time. Also open the output file once only... and choose a name that will not be picked up and treated as an input file itself, otherwise you run the risk of concatenating the output file onto itself (not yet a problem, but could be if you also process files from the current directory) - if loading it doesn't already consume all memory.
import os.path
with open('output.txt', 'w') as outfile:
for folder in range(1,6,1):
for name in os.listdir(folder):
if "domain" in name:
with open(os.path.join(str(folder), name)) as file_content_list:
for line in file_content_list:
# perform corrections/modifications to line here
outfile.write(line)
Now you can process the data in a line oriented manner - just modify it before writing to the output file.
I'm trying to create a program that duplicates itself to another location and creates a batch file on the desktop. I can make it duplicate itself and I can create the batch file but I need some help with the paths.
I can find the path that my program is currently in. Both the direct path and the path to the directory. My problem lies in the fact that I want to place the file in (let's just say for simplicity) 'C:\Users\Me\Documents'. How would I edit the path? I want to be able to place this on a generic windows computer so I can't hard code the path in because each user will be different. This goes the same for placing the batch file and setting it for the right directory to run the python script in documents.
I have tried both
import os
print os.path.dirname(os.path.abspath(__file__))
and
import os
print os.path.abspath(__file__)
but am clueless as to how to edit the path. When I try googling for it and searching this wonderful site, all I get is stuff about configuring the Python path on windows and other stuff that I can't quite understand at my current level of Python.
Now I turn to you, can you help? Any input would be appreciated, if you could explain how it worked that would be even better!
<>
Due to some questions about my code (and a specific one to post it) Here it is
from sys import argv # Imports module
import os
script, create = argv # Gets script name and desired amount of copies
data = open(script) # Creates a variable to store the script
indata = copy.read() # Creates the data to be copied from the script
batData = """
echo off
%s
""" % # This is not finished, creating that batch file
createT = int(create) + 1
for i in range(1, createT): # Runs a set amount of times
copyName = "%s.py" % str(i) # Creates the name for the file
copy = open(copyName, 'w+') # Opens/creates the file for editing
copy.write(indata) # Writies the indata to the file opened
copy.close # Closes that file
batName = "%s.bat" % str(i)
bat = open(batName, 'w+')
It is not finished but hopefully you get the gist. The argv at the beginning is so I can change the amount of copies made, that will be deleted later as I evolve the code but for now I like it there.
I have currently tried the following to find the path:
import os
print os.path.abspath(__file__)
print os.path.dirname(os.path.abspath(__file__))
print os.path.dirname(__file__)
test = os.path.dirname(__file__)
a, b, c, d, e, f, g, h, i = test.split("\\")
print c
What I want to happen (or think I want to happen) is for the path to be found, then split into pieces (either each directory or break off everything after the username). Then I want to append the document folder tag to the end. For the batch instead of the document tag it will be for the desktop.
among a few others that people have posted. I hope this helps!
Your code snippet returns a string. Just take that string and edit to make the path you want. I'm on a mac so I can't test with an actual windows directory but I'll try to make it look Windows-ish. For instance, lets say this code:
directory_path = os.path.dirname(os.path.abspath(__file__))
print(directory_path)
gives you:
C:\Users\username\AppData
You can use the split function to break the path into pieces (docs).
stuff = path_string.split('\')
print(stuff)
Code output:
['C:', 'Users', 'username', 'AppData']
You can use the pieces create the path you want and then use it to write the file. So, if you want the username folder just loop until you find it. Some example code is below (just an example to get you started - read up on Python if you need help understanding the code).
username = ""
for i in range(0, len(stuff)):
if stuff[i] == "Users":
username = stuff[i + 1]
Not sure if that answers your question but hope it helps.
Am I correct that you are trying to figure out how to make a file path to some location on the user's directory without knowing who the user is going to be that is executing the program?
You may be able to take advantage of environment variables for this. For instance, I can get a file path to the C://Users/username/ directory of whoever is executing the code with:
my_path = os.path.join("C:\\", "Users", os.getenv("USERNAME"))
Where os.getenv("USERNAME") returns the value of the USERNAME environment variable (should be the name of the user that is currently logged on).
You can list all of the available environment variables and their values in Python with:
for key, val in os.environ.items():
print("{} \t {}\n".format(key, val)) # or however you want to prettify it
You may get lucky and find an environment variable that gives you most of the path to begin with.
In general, os.path.join(), os.path.relpath(), and os.path.abspath() combined with some environment variables might be able to help you. Check out the os documentation and os.path documentation.
Many times when modifying a path, I am looking to add/remove folders. Here is my simple method for adding a path, e.g. if I want to move the path of an object into a folder added_path='/train/.
Since my paths are usually uniform, I check the last split characters in the first file location. Usually, my experience is that windows have \\ at the end while Mac and Linux have `/', which makes this work across operating systems. (note: if the paths are not uniform, you obviously place the if-else in the for-loop.)
if '\\' in data[0].img_file:
split_char = '\\'
else:
split_char = '/'
for img in data:
img_path = img.img_file.split(split_char)
label_path = img.label_file.split(split_char)
img.img_file = '/'.join(img_path[:-1]) + added_path + img_path[-1]
img.label_file = '/'.join(label_path[:-1]) + added_path + label_path[-1]
So, this for loop uses all the folders up until the file name, where we insert the extra folder, and then, at last, add the file name.
Example input path: 'data/combined/18.png'
Example output path: 'data/combined/train/18.png'
The script receives two variables from a previous web page. From those variables, the code determines which images are desired. It sends those images to a temp folder, zips up that folder and places it in an output folder for pickup. That's where things go south. I'm trying to allow the webpage to provide a button for the user to click on and download the zip file. Because the zip file's name needs to change based on the variables the script receives, I cannot just make a generic link to the zip file.
import arcpy, sys, shutil, os
path = "C:/output/exportedData/raw/"
pathZip = "C:/output/exportedData/zip/"
#First arg is the mxd base filename which is the same as the geodatabase name
geodatabaseName = "C:/output/" + sys.argv[1] + ".gdb"
#this is where the images are determined and sent to a folder
zipFileName = sys.argv[1]
zipFile = shutil.make_archive(path + zipFileName,"zip")
movedZip = os.rename(zipFile, pathZip + zipFileName + ".zip")
shutil.rmtree(path + zipFileName)
print """<h3>Download zip file</h3>""".format(movedZip)
And the last line indicates where the problem comes in. Firebug indicates the link made is
Download zip file
The string substitution isn't working in this case and I'm at a loss as to why. Thank you, in advance for any assistance you can provide.
os.rename() doesn't return anything, which means that movedZip becomes None.
Here's what you probably want to do instead:
movedZip = pathZip + zipFileName + ".zip"
os.rename(zipFile, movedZip)
The os.rename method does not return any value. You could see the official document here. It rename the file or directory src to dst. Some exceptions might be thrown. But does not return anything.
I'm currently playing a trading card game called Hearthstone which is made by blizzard. The game is pretty good, but lacks basic features that any game that calls itself "competitive" should have, like stat tracking and replay.
So as I said in the title, I'm trying to create a (very crude and poorly done) script that let's me record every match I play. Due to my lack of programming skills, 80% of the script is just a bunch of code that I borrowed from all sorts of places and adapted to make it do what I wanted.
The idea is to make it work like this:
I take a picture of every turn I play. It might become annoying, but I do not dare to think about implementing OCR as to make the script take a picture at the start of every turn by itself. Would be awesome but I just can't do it...
The game sends every picture to the desktop (no need to code that).
At the end of the game I run the script
2.1 Every match is going to have a numbered folder so the script creates that. The folders are going to be called "Match1", "Match2", etc. You can see how poorly written that is because I made it on my own :P
import sys
import os
import shutil
def checkFolder():
os.path.join('D:\Hearthstone\Replays\Match1')
matchNumber=1
while os.path.exists("D:\\Hearthstone\\Replays\\Match"+ str(matchNumber)) is True:
matchNumber=matchNumber + 1
else:
os.makedirs("D:\Hearthstone\Replays\Match"+str(matchNumber))
2.2 Script sends the photos from Desktop to the recently created folder. The Problem is that I do not know how to make the script change the destination folder to the newest folder created. I did not write this part of the code, i merely adapted it. Source: http://tinyurl.com/srcbh
folder = os.path.join('C:\\Users\\Felipe\\', 'Desktop') # Folder in which the images are in.
destination = os.path.join('D:\\Hearthstone\\Replays\\', 'match9999') #**Destination needs to be the newest folder and I dont know how to implement that...
extmove = 'png' # The extension you wish to organize.
num = 0 # Variable simply to use after to count images.
for filename in os.listdir(folder): #Run through folder.
extension = filename.split(".")[-1] # This strips the extensions ready to check and places into the extension
if extension == extmove: # If statement. If the extension of the file matches the one set previously then..
shutil.move(folder + "\\" + filename, destination) # Move the file from the folder to the destination folder. Also previously set.
num = num + 1
print(num)
print (filename, extension)
And that's it! I need help with step 2.2. I'd certainly appreciate the help!
Now, the reason I made such a big post is because I wanted to expose my idea and hopefully inspire someone to take on a similar project seriously. Hearthstone has thousands of players that could benefit from it, not to mention that this seems to be a fairly easy task to someone with more experience.
Ok, I finally got it to work!
import sys
import os
import shutil
def sendPhotos():
matchNumber=1
photos_dest = "D:\\Hearthstone\\Replays\\Match"
while os.path.exists(photos_dest+ str(matchNumber)): #creates the name of the folder "Match1", "Match2", etc.
matchNumber=matchNumber + 1
else:
photos_destination = photos_dest+str(matchNumber)
os.makedirs(photos_destination)
for files in os.listdir('C:\\Users\\Felipe\\Desktop'):#only png files are moved
if files.endswith(".png"):
shutil.move(files, photos_destination)
sendPhotos()
Thank you to those who gave me some answers! I really appreciated it!
Well, first off, the fact that you identified a problem and put together a solution shows that you certainly don't lack programming skills. Give yourself some credit. You just need more practice. :)
This should be better, I haven't run it, so there might be some errors :P
def checkFunction(base_dir='D:\\Hearthstone\\Replays\\'): #set this as a parameter with a default
match_number = 1
if os.path.exists(base_dir): #if the base directory doesn't exist you'll have a bad time
while os.path.exists(os.path.join(base_dir, 'Match{0}'.format(match_number)))
match_number += 1
new_dir = os.path.join(base_dir, 'Match{0}'.format(match_number))
os.makedirs(new_dir)
return new_dir
For the function checkFolder, I suggest having it return the new directory name (as above). You'll also need to indent all the lines below it so python knows those lines are part of that function (this might just be a formatting issue on SO though).
Then, once the checkFolder function is working properly, all you have the change in 2.2 is:
destination = checkFolder()
This sees which matches are in the folder and takes the lowest number for the folder.
folder = os.path.join('C:\\Users\\Felipe\\', 'Desktop') # Folder in which the images are in.
recorded_matches_location = 'D:\\Hearthstone\\Replays\\'
match_number = 1
match_name = 'match1'
while match_name in os.listdir(recorded_matches_location):
match_number = 1 + match_number
match_name = 'match' + str(match_number) # corrected it! there must be a string and not a variable
destination = os.path.join(recorded_matches_location, match_name) #**Destination needs to be the newest folder and I dont know how to implement that...
extmove = 'png' # The extension you wish to organize.
num = 0 # Variable simply to use after to count images.
for filename in os.listdir(folder): #Run through folder.
extension = filename.split(".")[-1] # This strips the extensions ready to check and places into the extension
if extension == extmove: # If statement. If the extension of the file matches the one set previously then..
shutil.move(folder + "\\" + filename, destination) # Move the file from the folder to the destination folder. Also previously set.
num = num + 1
print(num)
print (filename, extension)