I have a script that is supposed to read a directory from its second argument and do various processes on it, and record the information to a second file, compressed.xml. Here's the code that causes errors:
targetpath = os.getcwd()
if len(sys.argv) >= 2:
targetpath = sys.argv[1]
with open("compressed.xml", "w") as f:
base = ntpath.basename(targetpath)
f.write("<" + base + ">\n")
f.write(getXML(targetpath, 1))
f.write("</" + base + ">\n")
When I run the script on the command line with a directory path as an argument, it works perfectly fine. But if I try to drag a folder onto the script from within File Explorer, it causes a "permission denied" error on compressed.xml. I can't really get more information about the traceback than that, because I don't really know how to get more info about it than using this simple catch;
except Exception as e:
raw_input(e)
Without it, the shell closes instantly upon causing an error, and I'm not able to see the traceback.
Related
I'm working on a task where I'm supposed to synchronize 2 files without using libraries that implement folder synchronization and this is what I've been able to do so far. This works as intended for all .txt files and folders at the moment and even for Excel and Word files as long as they are already in the folder before running the code.
While the code is running, a text file can be created and the script handles it without problems, but whenever a Word or Excel type file is created in the src folder, I get this error:
Traceback (most recent call last):
File "c:\Users\zaire\Desktop\hello.py", line 55, in <module>
shutil.copytree(source_folder, destination_folder, dirs_exist_ok=True)
File "C:\Users\zaire\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 560, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\zaire\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 514, in _copytree
raise Error(errors)
shutil.Error: [('D:\\src\\Nový List Microsoft Excelu.xlsx', 'D:\\dst\\Nový List Microsoft Excelu.xlsx', "[Errno 13] Permission denied: 'D:\\\\src\\\\Nový List Microsoft Excelu.xlsx'")]
Here is the code:
import shutil
import os
source_folder = "D:\src\\"
destination_folder = "D:\dst\\"
log_file = "D:\logfile.txt"
starting_source_folder = os.listdir(source_folder)
while True:
# check if a file or folder has been created by checking if starting_source_folder differs from current source_folder
if starting_source_folder != os.listdir(source_folder):
# check if a file has been removed from the source_folder by comparing it to starting_source_folder
for i in starting_source_folder:
if i not in os.listdir(source_folder):
starting_source_folder.remove(i)
print(i , 'was removed by user')
# check if a file has been created in the source_folder by comparing it to starting_source_folder
for i in os.listdir(source_folder):
if i not in starting_source_folder:
print('created', i)
starting_source_folder.append(i)
else: pass
# if there is no new created/removed file or folder, proceed with copying to dst
# print what files are gonna be copied
for file_name in os.listdir(source_folder):
if file_name not in os.listdir(destination_folder):
print('copied', file_name)
shutil.copytree(source_folder, destination_folder, dirs_exist_ok=True)
# remove files in destination_folder that are not in source_folder
for file_name2 in os.listdir(destination_folder):
if file_name2 not in os.listdir(source_folder):
if os.path.isfile(destination_folder + file_name2):
os.remove(destination_folder + file_name2)
print('removedd', file_name2)
else:
shutil.rmtree(destination_folder + file_name2)
print('removed folder', file_name2)
I've tried googling around for a solution for this permission error, but the answer to it seems to be to avoid using the shutil.copy function on directories, which I'm not doing in my code as I'm using shutil.copytree to copy the whole directory. When I run the code again, the Excel/Word file that had been created before and had caused the error gets instantly copied into the dst folder and the code keeps running until I try to create another Word/Excel type file (it could be more types but these are the types I've tested so far).
I've managed to get it to work a couple of times where I created 1-2 excel files when the code was running and it didn't break it, but the moment I created another one the code broke again. I'm not sure why it worked at times and now it doesn't as I haven't made changes to the code that should cause this behavior to change. I'm also having trouble otherwise googling this permission error because the problem occurs when the code is running, but if the while loop is gone and the code runs just once it works fine even for Excel and Word type files (it copies them from src to dst folder with no error), but I need the code to be periodically synchronizing the files. I'm aware that the code could be done way better but I am just a beginner.
Issue: Unable to save file in directory (/root/Notion/Image) when using Cron schedule
This is what my code is trying to do:
Check email
Download image attachment
Store in a directory - root/Notion/Image
Retrieve file path
The script is working when I run it manually in Google Cloud terminal. The problem is when I try to schedule it on Cron, it's unable to access the folder to save the file locally.
This is the error when the script failed and require permission:
Traceback (most recent call last):
File "Notion/test.py", line 121, in <module>
path = get_attachments(email.message_from_bytes(msg[0][1]))
File "Notion/test.py", line 47, in get_attachments
with open(filePath, 'wb') as f:
PermissionError: [Errno 13] Permission denied: '/root/Notion/Image/3.jpeg'
This is the code to retrieve attachment from email
def get_attachments(msg):
for part in msg.walk():
if part.get_content_maintype()=='multipart':
continue
if part.get('Content-Disposition') is None:
continue
fileName = part.get_filename()
if bool(fileName):
filePath = os.path.join(attachment_dir, fileName)
with open(filePath, 'wb') as f:
f.write(part.get_payload(decode=True))
return str(filePath)
Resolved:
The problem is that I shouldn't use root directory since it requires permission. I've changed it to home directory instead.
attachment_dir = '/home/dev_thomas_yang/folder_name/folder_name'
For people who needs to check their home direction, simply run this script.
from pathlib import Path
home= str(Path.home())
print(home)
Thanks Triplee for the patience to breakdown my issue despite my sloppy ways of presenting it!
The easiest fix hands down is to change the code so it doesn't try to write to /root. Have it write to the invoking user's home directory instead.
Your question doesn't show the relevant parts of the code, but just change attachment_dir so it's not an absolute path. Maybe separately take care of creating the directory if it doesn't already exist.
import pathlib
# ...
attachment_dir = pathlib.Path("cron/whatever/attachments").mkdir(parents=True, exist_ok=True)
# ...
for loop in circumstances:
get_attachments(something)
A better design altogether would be to have get_attachments accept the directory name as a parameter, so you can make this configurable from the code which calls it. Global variables are a nuisance and cause hard-to-debug problems because they hide information which is important for understanding the code, and tricky to change when you try to debug that code and don't know which parts of the code depend on the old value.
I expect a directory to be created and then a file to be opened within it for writing to when I execute my code below in Python 2.6.6,
import subprocess
def create_output_dir(work_dir):
output_dir = '/work/m/maxwell9/some_name5/'
subprocess.Popen(['mkdir', output_dir])
return output_dir
if __name__ == '__main__':
work_dir = '/work/m/maxwell9/'
output_dir = create_output_dir(work_dir)
#output_dir = '/work/m/maxwell9/some_name5/'
filename = output_dir + 'bt.sh'
with open(filename, 'w') as script:
print('there')
but instead I get the error,
Traceback (most recent call last):
File "slurm_test.py", line 13, in <module>
with open(filename, 'w') as script:
IOError: [Errno 2] No such file or directory: '/work/m/maxwell9/some_name5/bt.sh'
If I run the script, I can then see that the directory is created. If I then uncomment the line,
#output_dir = '/work/m/maxwell9/some_name5/'
and comment the line,
output_dir = create_output_dir(work_dir)
then the file is output fine. So there is something about creating the folder and then writing to it in the same script that is causing an error.
subprocess.Popen starts up an external process but doesn't wait for it to complete unless you tell it to (e.g. by calling .wait on the returned Popen instance). Most likely, mkdir is in the process of creating a directory while open(filename, 'w') attempts to create a file in that directory. This is an example of a "race condition".
The solution is to .wait on the open process (as noted above), or you can use one of the convenience wrappers subprocess.check_output, subprocess.check_call or (even better), you can avoid subprocess entirely by using os.mkdir or os.makedirs.
You could use the os library instead of subprocess, which makes for a more straightforward implementation. Try swapping out your create_output_dir function with this:
import os
def create_output_dir(work_dir):
try:
os.makedirs(work_dir)
except OSError:
pass
return work_dir
I have a python script that automates Erdas image processing software based on parameters gathered by the script from the image metadata. The python script gathers these parameters into list which gets written to a batch file, and then runs the batch file, which tells the Erdas software to process the images. When I run this outside of the IIS directory, the script works perfectly. However, when I run the script within the IIS directory, the script partially runs, but gives me an error message when it tries to run the batch file. Just an FYI, I've searched everywhere for a solution to this problem. I've tried changing folder and file permissions and IIS manager settings with no luck. Here is the error message:
Traceback (most recent call last):
File "python_page.py", line 1018, in <module>
main()
File "python_page.py", line 1002, in main
getAtmCorrectBatchFile(AtmCorrectBatchlist)
File "python_page.py", line 912, in getAtmCorrectBatchFile
f = open(basePath + '/atm_batch_process.bat', 'w')
IOError: [Errno 13] Permission denied: 'C:/inetpub/wwwroot/cgi-bin/atm_batch_process.bat'
Below is the code where the error is occurring. What I'm most confused about is that the error is located within the getAtmCorrectBatchFile() function that creates the batchfile, but the error arises when the script tries to call the RunAtmCorrectBatch() function listed directly below it. The script has successfully created the batch file and saved it in the cgi-bin directory at this point, so why would the error be within the function that creates the batch file instead of the one that tries to run the batch file?
Edit: I can right click the batch file within the cgi-bin directory and select run, which successfully outputs the processed images, just for some reason this doesn't work when the python script tries to run the batch file.
def getAtmCorrectBatchFile(batchlist):
f = open(basePath + '/atm_batch_process.bat', 'w') #line 912
for i in range (0, len(batchlist)):
f.write(str(batchlist[i]).replace('\\', '/') +'\n')
f.close()
def RunAtmCorrectBatch():
try:
from subprocess import Popen
p = Popen("/atm_batch_process.bat", cwd=basePath)
stdout, stderr = p.communicate()
except WindowsError:
pass
Opening a file available in present working directory's temp folder in python
I tried
pwdir=os.getcwd()
tempdir=pwdir+"/temp/test.txt"
f=open(tempdir,'r+')
When I print the path of tempdir, it is showing up correctly and also the contents of file are also read.
When I try to combine this operation from an Applescript, which calls this python script. I get an error like this
f=open(pwdir1,'r+')
IOError: [Errno 2] No such file or directory: '//temp/test.txt'" number 1
EDIT:
I am using Shell script from Applescript to call this pythonscript
do shell script "/Users/mymac/Documents/'Microsoft User Data'/test.py"
EDIT:
Python Code:
tempdir = os.path.join(os.getcwd(),'temp','htmlinput.html')
print tempdir
with open(tempdir) as f:
html=f.read()
Python output from terminal:(works perfectly fine)
/Users/mymac/Documents/Microsoft User Data/Outlook Script Menu Items/temp/htmlinput.html
I am also able to see the file contents.
Applescript Code:
do shell script "/Users/mymac/Documents/'Microsoft User Data'/'Outlook Script Menu Items'/new.py"
Applescript Error:
error "Microsoft Outlook got an error: Traceback (most recent call last):
File \"/Users/mymac/Documents/Microsoft User Data/Outlook Script Menu Items/new.py\", line 12, in <module>
with open(tempdir) as f:
IOError: [Errno 2] No such file or directory: '/temp/htmlinput.html'" number 1
I don't know Applescript -- or OS X in general. It looks like the script is being run from the root folder, and os.getcwd() returns '/'. The directory of the script itself is sys.path[0] or the dirname of the current module -- dirname(__file__) -- if it's a single script instead of a package. Try one of the following
import os, sys
tempdir = os.path.join(sys.path[0], 'temp', 'temp.txt')
or
import os
tempdir = os.path.join(os.path.dirname(__file__), 'temp', 'temp.txt')
The double slash is your problem. The right way to join file and path names in Python is with os.path.join. Try:
tempdir = os.path.join(os.getcwd(), 'temp', 'test.txt')
Also, you should probably be doing:
with open(tempdir) as f:
which will make sure tempdir gets closed even if you have an error.
Edit:
We need to see what tempdir is when the script is invoked by the AppleScript, not when it is invoked from the terminal. If you do
tempdir = os.path.join(os.getcwd(),'temp','htmlinput.html')
with open('/Users/mymac/Documents/temp.txt', 'w') as fwrite:
fwrite.write(tempdir)
What exactly ends up in the file /Users/mymac/Documents/temp.txt?