Develop Reference

Permission denied when trying to copy excel / word files that were created during a running while loop

I'm working on a task where I'm supposed to synchronize 2 files without using libraries that implement folder synchronization and this is what I've been able to do so far. This works as intended for all .txt files and folders at the moment and even for Excel and Word files as long as they are already in the folder before running the code.
While the code is running, a text file can be created and the script handles it without problems, but whenever a Word or Excel type file is created in the src folder, I get this error:
Traceback (most recent call last):
File "c:\Users\zaire\Desktop\hello.py", line 55, in <module>
shutil.copytree(source_folder, destination_folder, dirs_exist_ok=True)
File "C:\Users\zaire\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 560, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\zaire\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 514, in _copytree
raise Error(errors)
shutil.Error: [('D:\\src\\Nový List Microsoft Excelu.xlsx', 'D:\\dst\\Nový List Microsoft Excelu.xlsx', "[Errno 13] Permission denied: 'D:\\\\src\\\\Nový List Microsoft Excelu.xlsx'")]
Here is the code:
import shutil
import os
source_folder = "D:\src\\"
destination_folder = "D:\dst\\"
log_file = "D:\logfile.txt"
starting_source_folder = os.listdir(source_folder)
while True:
# check if a file or folder has been created by checking if starting_source_folder differs from current source_folder
if starting_source_folder != os.listdir(source_folder):
# check if a file has been removed from the source_folder by comparing it to starting_source_folder
for i in starting_source_folder:
if i not in os.listdir(source_folder):
starting_source_folder.remove(i)
print(i , 'was removed by user')
# check if a file has been created in the source_folder by comparing it to starting_source_folder
for i in os.listdir(source_folder):
if i not in starting_source_folder:
print('created', i)
starting_source_folder.append(i)
else: pass
# if there is no new created/removed file or folder, proceed with copying to dst
# print what files are gonna be copied
for file_name in os.listdir(source_folder):
if file_name not in os.listdir(destination_folder):
print('copied', file_name)
shutil.copytree(source_folder, destination_folder, dirs_exist_ok=True)
# remove files in destination_folder that are not in source_folder
for file_name2 in os.listdir(destination_folder):
if file_name2 not in os.listdir(source_folder):
if os.path.isfile(destination_folder + file_name2):
os.remove(destination_folder + file_name2)
print('removedd', file_name2)
else:
shutil.rmtree(destination_folder + file_name2)
print('removed folder', file_name2)
I've tried googling around for a solution for this permission error, but the answer to it seems to be to avoid using the shutil.copy function on directories, which I'm not doing in my code as I'm using shutil.copytree to copy the whole directory. When I run the code again, the Excel/Word file that had been created before and had caused the error gets instantly copied into the dst folder and the code keeps running until I try to create another Word/Excel type file (it could be more types but these are the types I've tested so far).
I've managed to get it to work a couple of times where I created 1-2 excel files when the code was running and it didn't break it, but the moment I created another one the code broke again. I'm not sure why it worked at times and now it doesn't as I haven't made changes to the code that should cause this behavior to change. I'm also having trouble otherwise googling this permission error because the problem occurs when the code is running, but if the while loop is gone and the code runs just once it works fine even for Excel and Word type files (it copies them from src to dst folder with no error), but I need the code to be periodically synchronizing the files. I'm aware that the code could be done way better but I am just a beginner.

Python script to create m3u files inside folders

I'm trying to run this specific python script to create .m3u files inside subfolders but it's not working and give me an error, any help is welcome.
The m3u file is a simple tracklist of the subfolder content with specified extensions and named after the subfolder, like this (extensions .aaa and .bbb are just examples):
Folder 1 (contains 'File 1.aaa', 'File 2.aaa', etc)
Folder 1.m3u generated inside Folder 1 with this list
File 1.aaa
File 2.aaa
Folder 2 (contains 'File 1.bbb', 'File 2.bbb', etc)
Folder 2.m3u generated inside Folder 2 with this list
File 1.bbb
File 2.bbb
Here is the script called makem3u.py (not mine, I don't know much about python):
#!/usr/bin/python
"""This script will create an m3u file with a tracklist of each .aaa or .bbb
found in a subdirectory, then name the m3u after the subdirectory. This helps
with multiple disks for emulation.
"""
import os
import os.path
EXT = ['.aaa', '.bbb']
cwd = os.getcwd()
dirs = os.listdir(cwd)
for d in dirs:
contents = os.listdir(os.path.join(cwd, d))
disks = [
os.path.join(d, f) for f in os.listdir(os.path.join(cwd, d))
if os.path.splitext(f)[-1] in EXT
]
if disks:
with open(os.path.join(cwd, '{d}.m3u'.format(d=d)), 'wb') as m3u:
m3u.writelines(['{disk}\n'.format(disk=disk) for disk in disks])
I get this error when I try to run it:
Traceback (most recent call last):
File "makem3u.py", line 16, in <module>
contents = os.listdir(os.path.join(cwd, d))
NotADirectoryError: [WinError 267] The directory name is invalid: 'path\\to\\file\\makem3u.py'
makem3u.py is inside a folder with the subfolders mentioned
Windows 10, Python 3.8.5, python is installed properly and the PATH is in enviroment variables and I can run other scripts just fine
What I'm doing wrong and how can I fix this? Is there a non-python alternative like create a .bat file to do that? Sorry for so many questions, I'm a noob in these things. Thank you in advance!
Also, is there a way to batch zip all the files in the subfolders (the generated m3u + original files) and name each zip after that subfolder? This is an extra, but would be helpful if possible

Python Directory Path

I am trying to write a python script to use the linux command wc to input the amount of lines in a file. I am iterating through a directory inputted by the user. However, whenever I get the absolute path of a file in the directory, it skips the directory it is in. So, the path isn't right and when I call wc on it, it doesn't work because it is trying to find the file in the directory above. I have 2 test text files in a directory called "testdirectory" which is located directly under "projectdirectory".
Script file:
import subprocess
import os
directory = raw_input("Enter directory name: ")
for root,dirs,files in os.walk(os.path.abspath(directory)):
for file in files:
file = os.path.abspath(file)
print(path) #Checking to see the path
subprocess.call(['wc','l',file])
This is what I get when running the program:
joe#joe-VirtualBox:~/Documents/projectdirectory$ python project.py
Enter directory name: testdirectory
/home/joe/Documents/projectdirectory/file2
wc: /home/joe/Documents/projectdirectory/file2: No such file or directory
/home/joe/Documents/projectdirectory/file1
wc: /home/joe/Documents/projectdirectory/file1: No such file or directory
I don't know why the path isn't /home/joe/Documents/projectdirectory/testdirectory/file2 since that is where the file is located.

You're using the output of os.walk wrong.
abspath is related to your program's current working directory, whereas your files are in the directory as specified by root. So you want to use
file = os.path.join(root, file)

Your issue is in the use of os.path.abspath(). All that this function does is appends the current working directory onto whatever the argument to the function is. You also need to have a - before the l option for wc. I think this fix might help you:
import os
directory = input("Enter directory name: ")
full_dir_path = os.path.abspath(directory)
for root,dirs,files in os.walk(full_dir_path):
for file in files:
full_file_path = os.path.join(root, file)
print(full_file_path) #Checking to see the path
subprocess.call(['wc','-l',full_file_path])

Trying to pull files from mapped network drive Python

I am trying to pull folder names from a network drive in Python. For example, if I have a mapped drive U:, I want to go through and grab all the folders in folder "example" from the path \emily\hello\example.
This is what I have tried so far.
import os
os.chdir('U:')
all_subdirs = [d for d in os.listdir('.') if os.path.isdir(d)]
for dirs in all_subdirs:
dir = os.path.join('\emily\hello\example', dirs)
os.chdir(dir)
current = os.getcwd()
new = str(current).split("\")[4]
print(new)
this causes many errors, and I am having an issue with the syntax of a shared drive on the network vs a folder locally on my computer.
I want to see them in a list so i can read line by line and compare this list to another list.
ps. i don't want the files in the folders, just the folder names in the folder
thanks~!
error message is C:\Users\212582086\AppData\Local\Continuum\Anaconda3\python.exe "C:/Users/212582086/Desktop/Vendor sort/main"
Traceback (most recent call last):
File "C:/Users/212582086/Desktop/Vendor sort/main", line 6, in
os.chdir(dir)
FileNotFoundError: [WinError 3] The system cannot find the path specified: '\emily\hello\example\Software Archive'
Process finished with exit code 1

Shutil.copy IO Error2 when directory exists

I'm encountering a troublesome problem with my code and I've been unable to figure it out. Basically I am copying files from a local directory on my computer to a Dropbox folder that acts as a project repository for me and some other folks.
I keep hitting an IO Error when executing the shutil.copy line. Errno 2, N osuch file or directory. However the directory and file both exist. When I change the directory to a different/test location (test_dir in my code), the code runs perfectly fine. Any insights would be greatly appreciated.
import os, os.path
import re
import shutil
import sys
#File Location
directory_list = "path where files are located"
#Dropbox base directory:
dropbox = "path to dropbox directory"
test_dir = "path to test directory on my local machine"
sed_files = os.listdir(directory_list)
for i in sed_files:
#print i.split("BBB")[0]
#df
copy_dir = re.sub(r'XXX',r'_',i.split("BBB")[0])
copy_dir = re.sub(r'ZZZ',r'/',copy_dir)
copy_dir = dropbox + copy_dir + "/FIXED/"
if not os.path.exists(copy_dir):
os.makedirs(copy_dir)
shutil.copy(directory_list+i,copy_dir)
#print directory_list+i
#os.rename(copy_dir+i,copy_dir+i.split("BBB")[1])
Traceback error is:
Traceback (most recent call last):
File "copy_SE_files.py", line 25, in <module> shutil.copy(direcotry_list+i,copydir)
File "C:\Python27\ArcGIS10.1\lib\shutil.py", line 116, in copy copyfile(src,dst)
File "C:\Python27\ArcGIS10.1\lib\shutil.py", line 82, in copyfile with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: 'C:/Users/myusername/Dropbox/NASA_HyspIRI_Project/Field_Data/Spectra/CVARS/April2014/LemonTrees/04172014_SE_LemonTreeCanopy/SE_Files/SpectraZZZCVARSZZZApril2014ZZZLemonTreesZZZZ04172014XXXSEXXXLemonTreeCanopyZZZSEXXXFilesBBBCVARS_na_LemonTrees_Bareground1_4deg_Refl_00355.sed'
Problem solved thanks to the keen eyes of stack overflow. Modified the line to read:
shutil.copy(directory_list+i,'\\\\?\\'+os.path.abspath(copy_dir))

You're failing because the combined length of the path is greater than Window's MAX_PATH limit. C:/Users/myusername/Dropbox/NASA_HyspIRI_Project/Field_Data/Spectra/CVARS/April2014/LemonTrees/04172014_SE_LemonTreeCanopy/SE_Files/SpectraZZZCVARSZZZApril2014ZZZLemonTreesZZZZ04172014XXXSEXXXLemonTreeCanopyZZZSEXXXFilesBBBCVARS_na_LemonTrees_Bareground1_4deg_Refl_00355.sed is 274 characters long, and without going to some trouble, most Windows file manipulation APIs won't work properly with a path longer than MAX_PATH (which is 260, and one of them is reserved for the NUL terminator).
Assuming Python uses the correct APIs, you can make it work with the extended path prefix, \\?\ (and it may require you to use backslashes rather than forward slashes in your path; I'm not clear on that; read the docs).

The first thing that jumped out at me was this line:
shutil.copy(directory_list+i,copy_dir)
Consider changing it to
shutil.copy(os.path.join(directory_list,i),copy_dir)
IOW, use os.path.join() when concatenating file paths.

One work around is:
try:
shutil.copy(src, dest)
except:
try:
shutil.copy(src, "\\\\?\\" + dest)
#If Long Path as per Maximum Path limitation Windows
except:
self.failed_TC=True
print("Failed to move the script "+os.path.basename(src)+" to "+dest)