Develop Reference

Python backup to zip file

I'm trying to make python backup script that zip all files from my synology folder but I have problem with files that is opened when script running. If file is opened script return error and stop zipping other files.
from datetime import datetime
from pathlib import Path
import zipfile
OBJECT_TO_BACKUP = r"\\192.168.xx.xx\folder" # The file or directory to backup
BACKUP_DIRECTORY = 'C:/backupCloud/folder/' # The location to store the backups in
MAX_BACKUP_AMOUNT = 5 # The maximum amount of backups to have in BACKUP_DIRECTORY
object_to_backup_path = Path(OBJECT_TO_BACKUP)
backup_directory_path = Path(BACKUP_DIRECTORY)
assert object_to_backup_path.exists() # Validate the object we are about to backup exists before we continue
# Validate the backup directory exists and create if required
backup_directory_path.mkdir(parents=True, exist_ok=True)
# Get the amount of past backup zips in the backup directory already
existing_backups = [
x for x in backup_directory_path.iterdir()
if x.is_file() and x.suffix == '.zip' and x.name.startswith('backup-')
]
# Enforce max backups and delete oldest if there will be too many after the new backup
oldest_to_newest_backup_by_name = list(sorted(existing_backups, key=lambda f: f.name))
while len(oldest_to_newest_backup_by_name) >= MAX_BACKUP_AMOUNT: # >= because we will have another soon
backup_to_delete = oldest_to_newest_backup_by_name.pop(0)
backup_to_delete.unlink()
# Create zip file (for both file and folder options)
backup_file_name = f'backup-{datetime.now().strftime("%Y%m%d%H%M%S")}-{object_to_backup_path.name}.zip'
zip_file = zipfile.ZipFile(str(backup_directory_path / backup_file_name), mode='w')
if object_to_backup_path.is_file():
# If the object to write is a file, write the file
zip_file.write(
object_to_backup_path.absolute(),
arcname=object_to_backup_path.name,
compress_type=zipfile.ZIP_DEFLATED
)
elif object_to_backup_path.is_dir():
# If the object to write is a directory, write all the files
for file in object_to_backup_path.glob('**/*'):
if file.is_file():
zip_file.write(
file.absolute(),
arcname=str(file.relative_to(object_to_backup_path)),
compress_type=zipfile.ZIP_DEFLATED
)
# Close the created zip file
zip_file.close()
Is there any possibility to zip opened files or maybe skip that files and continue with other files ?

Copy and rename pictures based on xml nodes

I'm trying to copy all pictures from one directory (also including subdirectories) to another target directory. Whenever the exact picture name is found in one of the xml files the tool should grap all information (attributes in the parent and child nodes) and create subdirectories based on those node informations, also it should rename the picture file.
The part when it extracts all the information from the nodes is already done.
from bs4 import BeautifulSoup as bs
path_xml = r"path\file.xml"
content = []
with open(res, "r") as file:
content = file.readlines()
content = "".join(content)
def get_filename(_content):
bs_content = bs(_content, "html.parser")
# some code
picture_path = f'{pm_1}{pm_2}\{pm_3}\{pm_4}\{pm_5}_{pm_6}_{pm_7}\{pm_8}\{pm_9}.jpg'
get_filename(content)
So in the end I get a string value with the directory path and the file name I want.
Now I struggle with opening all xml files in one directory instead of just opening one file. I tryed this:
import os
dir_xml = r"path"
res = []
for path in os.listdir(dir_xml):
if os.path.isfile(os.path.join(dir_xml, path)):
res.append(path)
with open(res, "r") as file:
content = file.readlines()
but it gives me this error: TypeError: expected str, bytes or os.PathLike object, not list
How can i read through all xml files instead of just one? I have hundreds of xml files so that will take a wile :D
And another question: How can i create directories base on string?
Lets say the value of picture_path is AB\C\D\E_F_G\H\I.jpg
I would need another directory path for the destination of the created folders and a function that somehow creates folders based on that string. How can I do that?

To read all XML files in a directory, you can modify your code as follows:
import os
dir_xml = r"path"
for path in os.listdir(dir_xml):
if path.endswith(".xml"):
with open(os.path.join(dir_xml, path), "r") as file:
content = file.readlines()
content = "".join(content)
get_filename(content)
This code uses the os.listdir() function to get a list of all files in the directory specified by dir_xml. It then uses a for loop to iterate over the list of files, checking if each file ends with the .xml extension. If it does, it opens the file, reads its content, and passes it to the get_filename function.
To create directories based on a string, you can use the os.makedirs function. For example:
import os
picture_path = r'AB\C\D\E_F_G\H\I.jpg'
dest_path = r'path_to_destination'
os.makedirs(os.path.join(dest_path, os.path.dirname(picture_path)), exist_ok=True)
In this code, os.path.join is used to combine the dest_path and the directory portion of picture_path into a full path. os.path.dirname is used to extract the directory portion of picture_path. The os.makedirs function is then used to create the directories specified by the path, and the exist_ok argument is set to True to allow the function to succeed even if the directories already exist.
Finally, you can use the shutil library to copy the picture file to the destination and rename it, like this:
import shutil
src_file = os.path.join(src_path, picture_path)
dst_file = os.path.join(dest_path, picture_path)
shutil.copy(src_file, dst_file)
Here, src_file is the full path to the source picture file and dst_file is the full path to the destination. The shutil.copy function is then used to copy the file from the source to the destination.

You can use os.walk() for recursive search of files:
import os
dir_xml = r"path"
for root, dirs, files in os.walk(dir_xml): #topdown=False
for names in files:
if ".xml" in names:
print(f"file path: {root}\n XML-Files: {names}")
with open(names, 'r') as file:
content = file.readlines()

Skip list of filenames from txt file with os.walk

I would like to upload files that users dump into a shared folder to an FTP site. Only certain files must be uploaded based on a pattern in the filename, and that works. I would like to avoid uploading files that have been uploaded in the past. A simple solution would be to move the files to a subdirectory once uploaded, but users whish for the files to remain where they are.
I was thinking of writing a filename to a text file when each iteration of the loops makes an update. Populating the text file works.
Excluding directories with os.walk is mentioned in many articles and I can get that to work fine, but excluding a list of filenames seems to be a bit more obscure
This is what I have so far:
import ftplib
import os
import os.path
import fnmatch
## set local path variables
dir = 'c:/Temp'
hist_path = 'C:/Temp/hist.txt'
pattern = '*SomePattern*'
## make the ftp connection and set appropriate working directory
ftp = ftplib.FTP('ftp.someserver.com')
ftp.login('someuser', 'somepassword')
ftp.cwd('somedirectory')
## make a list of previously uploaded files
hist_list = open(hist_path, 'r')
hist_content = hist_list.read()
# print(hist_content)
## loop through the files and upload them to the FTP as above
for root, dirs, files in os.walk(dir):
for fname in fnmatch.filter(files, pattern): # this filters for filenames that include the pattern
## upload each file to the ftp
os.chdir(dir)
full_fname = os.path.join(root, fname)
ftp.storbinary('STOR ' + fname, open(full_fname, 'rb'))
## add an entry for each file into the historical uploads log
f = open(hist_path, 'a')
f.write(fname + '\n')
f.close()
Any help would be appreciated

UploadWriteFailed(reason=WriteError('disallowed_name', None)

I'm trying to upload a whole folder to dropbox but only the files get uploaded. Should I create a folder programatically or can I solve the folder-uploading so simple? Thanks
import os
import dropbox
access_token = '***********************'
dbx = dropbox.Dropbox(access_token)
dropbox_destination = '/live'
local_directory = 'C:/Users/xoxo/Desktop/man'
for root, dirs, files in os.walk(local_directory):
for filename in files:
local_path = root + '/' + filename
print("local_path", local_path)
relative_path = os.path.relpath(local_path, local_directory)
dropbox_path = dropbox_destination + '/' + relative_path
# upload the file
with open(local_path, 'rb') as f:
dbx.files_upload(f.read(), dropbox_path)
error:
dropbox.exceptions.ApiError: ApiError('xxf84e5axxf86', UploadError('path', UploadWriteFailed(reason=WriteError('disallowed_name', None), upload_session_id='xxxxxxxxxxx')))

[Cross-linking for reference: https://www.dropboxforum.com/t5/API-support/UploadWriteFailed-reason-WriteError-disallowed-name-None/td-p/245765 ]
There are a few things to note here:
In your sample, you're only iterating over files, so you won't get dirs uploaded/created.
The /2/files/upload endpoint only accepts file uploads, not folders. If you want to create folders, use /2/files/create_folder_v2. You don't need to explicitly create folders for any parent folders in the path for files you upload via /2/files/upload though. Those will be automatically created with the upload.
Per the /2/files/upload documentation, disallowed_name means:
Dropbox will not save the file or folder because of its name.
So, it's likely you're getting this error because you're trying to upload an ignored filed, e.g., ".DS_STORE". You can find more information on those in this help article under "Ignored files".

Creating folders by checking numbers at beginning of a string with python

I have a list of strings which contains files like below
filename = [ '000101 FL - Project Title Page.DOC',
'014200 FL - References.DOC',
'095446 FL - Fabric-Wrapped Ceiling Panels.DOC',
'142113 FL - ELECTRIC TRACTION FREIGHT ELEVATORS.DOC']
I want to check if a folder with a name consisting of Div + first two numbers in each string exist, such as Div00, Div01, Div09, Div14 in this case. If not I would like to create this folder. Then store the name of the file in this folder.
In pseudocode I believe it would be similar to
for file in filenames
if 'Div' + file[0][0] not a folder
make folder 'Div' + file[0][0]
add file to folder
else
add file to folder Div + file[0][0]
There will be multiple files starting with the same two numbers this is why I want to sort them into folder.
Let me know if you need any clarification.

Use os.mkdir to create a directory and shutil.copy2 to copy a file,
import os
import shutil
filenames = [ '000101 FL - Project Title Page.DOC']
for filename in filenames:
folder = 'Div' + filename[:2] # 'Div00'
# Create the folder if doesn't exist
if not os.path.exists(folder):
os.makedirs(folder)
# Copy the file to `folder`
if os.path.isfile(filename):
shutil.copy2(filename, folder) # metadata is copied as well

You can just check if there is a folder and make it if it does not exist
if not os.path.exists(dirName):
os.makedirs(dirName)

Try something like this:
import os
import shutil
for file in filenames
dir_name = "Div%s" % file[0:2]
if not os.path.isdir(dir_name)
os.makedirs(dir_name)
shutil.copy(file, dir_name)