I wrote a Python script which calls subprocess which will create a file:
subprocess.run(["youtube-dl.exe", "-o " + fileName, link])
After that I'd like to move my file to a different directory:
shutil.copy("C:\\Users\\***\\Desktop\\m2.m4a", "C:\\Users\\***\\Desktop\\Songs\\m2.m4a")
But that gives me the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\***\Desktop\m2.m4a'
But the file definitely exists. I tried to move a file which has not just been altered, and it works.
In case Windows needs time to create the file properly, I paused the code after the file creation and before the shutil.move-command:
time.sleep(4)
But it still doesn't work. Is it maybe because subprocess is still running and shutil can't access the file at the same time? Or because Python doesn't know that a new file has just been created? Thanks for your help!
Related
Issue: Unable to save file in directory (/root/Notion/Image) when using Cron schedule
This is what my code is trying to do:
Check email
Download image attachment
Store in a directory - root/Notion/Image
Retrieve file path
The script is working when I run it manually in Google Cloud terminal. The problem is when I try to schedule it on Cron, it's unable to access the folder to save the file locally.
This is the error when the script failed and require permission:
Traceback (most recent call last):
File "Notion/test.py", line 121, in <module>
path = get_attachments(email.message_from_bytes(msg[0][1]))
File "Notion/test.py", line 47, in get_attachments
with open(filePath, 'wb') as f:
PermissionError: [Errno 13] Permission denied: '/root/Notion/Image/3.jpeg'
This is the code to retrieve attachment from email
def get_attachments(msg):
for part in msg.walk():
if part.get_content_maintype()=='multipart':
continue
if part.get('Content-Disposition') is None:
continue
fileName = part.get_filename()
if bool(fileName):
filePath = os.path.join(attachment_dir, fileName)
with open(filePath, 'wb') as f:
f.write(part.get_payload(decode=True))
return str(filePath)
Resolved:
The problem is that I shouldn't use root directory since it requires permission. I've changed it to home directory instead.
attachment_dir = '/home/dev_thomas_yang/folder_name/folder_name'
For people who needs to check their home direction, simply run this script.
from pathlib import Path
home= str(Path.home())
print(home)
Thanks Triplee for the patience to breakdown my issue despite my sloppy ways of presenting it!
The easiest fix hands down is to change the code so it doesn't try to write to /root. Have it write to the invoking user's home directory instead.
Your question doesn't show the relevant parts of the code, but just change attachment_dir so it's not an absolute path. Maybe separately take care of creating the directory if it doesn't already exist.
import pathlib
# ...
attachment_dir = pathlib.Path("cron/whatever/attachments").mkdir(parents=True, exist_ok=True)
# ...
for loop in circumstances:
get_attachments(something)
A better design altogether would be to have get_attachments accept the directory name as a parameter, so you can make this configurable from the code which calls it. Global variables are a nuisance and cause hard-to-debug problems because they hide information which is important for understanding the code, and tricky to change when you try to debug that code and don't know which parts of the code depend on the old value.
Relatively new Python user here, so please bear with me. As part of a program I am working on, I need to have Python loop through folders in a SharePoint site, copy a file in each folder, and paste the copies into a local folder. I am attempting to use shutil.copy, but I'm getting an error ("OSError: [Errno 22] Invalid argument"). I do not get any errors if I try to copy files from a local drive, leading me to believe SharePoint is the issue.
After doing some research, it appears that I need to install some type of module to work with SharePoint. Is this accurate? Even if I normally just use Windows Explorer to navigate/edit files on the SharePoint site?
The other issue is that my work still uses SharePoint 2007, so I think that may limit my options.
Any help is much appreciated. Thanks!
import xlrd
import shutil
import os
#This is the location of the file on the SharePoint
sp_file = "https://...test.xlsx"
#This is the desired destination locally.
sp_comp_file = "C:\\Users\\555555\\Desktop\\Python Test\\test.xlsx"
shutil.copy(sp_file, sp_comp_file)
Error:
Traceback (most recent call last):
File "C:\Users\555555\Desktop\Python\teststack.py", line 9, in <module>
shutil.copy(sp_file, sp_comp_file)
File "C:\Users\555555\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 235, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\Users\555555\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 114, in copyfile
with open(src, 'rb') as fsrc:
OSError: [Errno 22] Invalid argument: 'https://...test.xlsx'
It might be late now, but try to remove the "https:" from your variable called sp_file such as:
sp_file = "//...test.xlsx"
I'm doing the same kind of copy/paste on a SP site and without the "https:" it works for me.
Not sure it will work, because I don't get the same error than you if I add "https:" in my program.
I want to copy some pictures from on directory to another directory, and here is my code:
import os.path
import shutil
def copyFile(sourceDir,targetDir):
for files in os.listdir(sourceDir):
sourceFile=os.path.join(sourceDir,files)
if os.path.isfile(sourceFile) and sourceFile.find('.jpg')>0:
shutil.copy(sourceFile,targetDir)
for i in os.listdir('/Users/liuchong/Desktop/LFW/new'):
copyFile(i,'/Users/liuchong/Desktop/LFW/lfw')
But when I run it ,the terminal tells me thatOSError: [Errno 20] Not a directory: '.DS_Store'
I know 'DS_dstore' is a hidden file in Mac, but how can I solve this bug?
It is worth to mention what is the general meaning of the "Not a directory" error (Errno 20). It means that you are trying to operate with a path which sub-path is a real file and not a directory. That is it is a malformed, incorrect path.
Example: file.txt/test.txt where file.txt is an existing true file (not a directory).
Python will return this error for every I/O operation with such a file:
$ python -c 'import os; os.path.getsize("file.txt/test.txt");'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.7/genericpath.py", line 57, in getsize
return os.stat(filename).st_size
OSError: [Errno 20] Not a directory: 'file.txt/test.txt'
The OS will also return this error:
$ stat file.txt/test.txt
stat: cannot stat 'file.txt/test.txt': Not a directory
Every time you have this error it means that you concatenate file+file somewhere in your program.
This error is Unix OS specific. On Windows you should get "File not found" for such malformed path.
Your logic seems badly broken. You iterate through all the files in the directory, passing each one to copyFile. But inside that function, you again try to iterate through each file in the "directory" passed to the function: except that you're not passing only directories to the function, you're passing each file found in the original directory.
It's not clear what you are trying to do, but I think you need to remove one of those calls to listdir and the associated loop.
When you do os.listdir('/Users/liuchong/Desktop/LFW/lfw'), it returns both directories and files. You have to check the entry for directory before passing it as an argument.
def copyFile(sourceDir,targetDir):
for files in os.listdir(sourceDir):
sourceFile=os.path.join(sourceDir,files)
if os.path.isfile(sourceFile) and sourceFile.find('.jpg')>0:
shutil.copy(sourceFile,targetDir)
for i in os.listdir('/Users/liuchong/Desktop/LFW/lfw'):
if os.path.isdir(i):
i = os.path.join( '/Users/liuchong/Desktop/LFW/lfw', i)
copyFile(i,'/Users/liuchong/Desktop/LFW/lfw')
os.listdir('/Users/liuchong/Desktop/LFW/lfw')
returns a list which contains the names of entries in the directory that is given by the user. List is based on random choices.
To address this issue (I know I am late) here's a python 3 compatible code:-
import time
import os
from shutil import copyfile
def Copyfile(in_path, out_path, quantity = None):
filelist = os.listdir(in_path)
counter = 0
print("Copying to %r directory. Please wait... " % out_path)
start = time.time()
for i in range(quantity):
copyfile(filelist[i], out_path+filelist[i])
counter+=1
sys.stdout.write("\rTotal images copied : %r" % counter)
sys.stdout.flush()
end = time.time()
print("This move operation took %r seconds to run." %(end - start))
I expect a directory to be created and then a file to be opened within it for writing to when I execute my code below in Python 2.6.6,
import subprocess
def create_output_dir(work_dir):
output_dir = '/work/m/maxwell9/some_name5/'
subprocess.Popen(['mkdir', output_dir])
return output_dir
if __name__ == '__main__':
work_dir = '/work/m/maxwell9/'
output_dir = create_output_dir(work_dir)
#output_dir = '/work/m/maxwell9/some_name5/'
filename = output_dir + 'bt.sh'
with open(filename, 'w') as script:
print('there')
but instead I get the error,
Traceback (most recent call last):
File "slurm_test.py", line 13, in <module>
with open(filename, 'w') as script:
IOError: [Errno 2] No such file or directory: '/work/m/maxwell9/some_name5/bt.sh'
If I run the script, I can then see that the directory is created. If I then uncomment the line,
#output_dir = '/work/m/maxwell9/some_name5/'
and comment the line,
output_dir = create_output_dir(work_dir)
then the file is output fine. So there is something about creating the folder and then writing to it in the same script that is causing an error.
subprocess.Popen starts up an external process but doesn't wait for it to complete unless you tell it to (e.g. by calling .wait on the returned Popen instance). Most likely, mkdir is in the process of creating a directory while open(filename, 'w') attempts to create a file in that directory. This is an example of a "race condition".
The solution is to .wait on the open process (as noted above), or you can use one of the convenience wrappers subprocess.check_output, subprocess.check_call or (even better), you can avoid subprocess entirely by using os.mkdir or os.makedirs.
You could use the os library instead of subprocess, which makes for a more straightforward implementation. Try swapping out your create_output_dir function with this:
import os
def create_output_dir(work_dir):
try:
os.makedirs(work_dir)
except OSError:
pass
return work_dir
Opening a file available in present working directory's temp folder in python
I tried
pwdir=os.getcwd()
tempdir=pwdir+"/temp/test.txt"
f=open(tempdir,'r+')
When I print the path of tempdir, it is showing up correctly and also the contents of file are also read.
When I try to combine this operation from an Applescript, which calls this python script. I get an error like this
f=open(pwdir1,'r+')
IOError: [Errno 2] No such file or directory: '//temp/test.txt'" number 1
EDIT:
I am using Shell script from Applescript to call this pythonscript
do shell script "/Users/mymac/Documents/'Microsoft User Data'/test.py"
EDIT:
Python Code:
tempdir = os.path.join(os.getcwd(),'temp','htmlinput.html')
print tempdir
with open(tempdir) as f:
html=f.read()
Python output from terminal:(works perfectly fine)
/Users/mymac/Documents/Microsoft User Data/Outlook Script Menu Items/temp/htmlinput.html
I am also able to see the file contents.
Applescript Code:
do shell script "/Users/mymac/Documents/'Microsoft User Data'/'Outlook Script Menu Items'/new.py"
Applescript Error:
error "Microsoft Outlook got an error: Traceback (most recent call last):
File \"/Users/mymac/Documents/Microsoft User Data/Outlook Script Menu Items/new.py\", line 12, in <module>
with open(tempdir) as f:
IOError: [Errno 2] No such file or directory: '/temp/htmlinput.html'" number 1
I don't know Applescript -- or OS X in general. It looks like the script is being run from the root folder, and os.getcwd() returns '/'. The directory of the script itself is sys.path[0] or the dirname of the current module -- dirname(__file__) -- if it's a single script instead of a package. Try one of the following
import os, sys
tempdir = os.path.join(sys.path[0], 'temp', 'temp.txt')
or
import os
tempdir = os.path.join(os.path.dirname(__file__), 'temp', 'temp.txt')
The double slash is your problem. The right way to join file and path names in Python is with os.path.join. Try:
tempdir = os.path.join(os.getcwd(), 'temp', 'test.txt')
Also, you should probably be doing:
with open(tempdir) as f:
which will make sure tempdir gets closed even if you have an error.
Edit:
We need to see what tempdir is when the script is invoked by the AppleScript, not when it is invoked from the terminal. If you do
tempdir = os.path.join(os.getcwd(),'temp','htmlinput.html')
with open('/Users/mymac/Documents/temp.txt', 'w') as fwrite:
fwrite.write(tempdir)
What exactly ends up in the file /Users/mymac/Documents/temp.txt?