Develop Reference

Skip list of filenames from txt file with os.walk

I would like to upload files that users dump into a shared folder to an FTP site. Only certain files must be uploaded based on a pattern in the filename, and that works. I would like to avoid uploading files that have been uploaded in the past. A simple solution would be to move the files to a subdirectory once uploaded, but users whish for the files to remain where they are.
I was thinking of writing a filename to a text file when each iteration of the loops makes an update. Populating the text file works.
Excluding directories with os.walk is mentioned in many articles and I can get that to work fine, but excluding a list of filenames seems to be a bit more obscure
This is what I have so far:
import ftplib
import os
import os.path
import fnmatch
## set local path variables
dir = 'c:/Temp'
hist_path = 'C:/Temp/hist.txt'
pattern = '*SomePattern*'
## make the ftp connection and set appropriate working directory
ftp = ftplib.FTP('ftp.someserver.com')
ftp.login('someuser', 'somepassword')
ftp.cwd('somedirectory')
## make a list of previously uploaded files
hist_list = open(hist_path, 'r')
hist_content = hist_list.read()
# print(hist_content)
## loop through the files and upload them to the FTP as above
for root, dirs, files in os.walk(dir):
for fname in fnmatch.filter(files, pattern): # this filters for filenames that include the pattern
## upload each file to the ftp
os.chdir(dir)
full_fname = os.path.join(root, fname)
ftp.storbinary('STOR ' + fname, open(full_fname, 'rb'))
## add an entry for each file into the historical uploads log
f = open(hist_path, 'a')
f.write(fname + '\n')
f.close()
Any help would be appreciated

Can i have two different sets of app configurations?

I have made a flask server REST API that does an upload to a specific folder.
The upload must comply to a specific file extension, and must be in a specific folder.
So i made these two app.configs:
app.config['UPLOAD_EXTENSIONS'] = ['.stp', '.step']
app.config['UPLOAD_PATH'] = 'uploads'
However, i want to make a new route, for a new upload of a specific file extension to another folder.
So can i have two sets of app.config['UPLOAD_EXTENSIONS'] and app.config['UPLOAD_PATH']?
One set will be for extension1 in folder1 and the other set for extension2 in folder2.

Try using the extension Flask-Uploads .
Or, proceeding from the file format, form a subdirectory in your UPLOAD_PATH.
import os
def select_directory(filename: str) -> str:
file_format = filename.split('.')[1]
your_path1 = 'path1'
your_path2 = 'path2'
if file_format in ('your format1', 'your format2'):
full_path = os.path.join(app.config['UPLOAD_FOLDER'], your_path1, filename)
else:
full_path = os.path.join(app.config['UPLOAD_FOLDER'], your_path2, filename)
return full_path

Python - how to change directory

I am doing a school assignment where I have to take input from a user and save it to a text file.
My file structure will be something like:
- Customer register
- Customer ID
- .txt files 1-5
It can be saved in the python folder and I can make the folders like this:
os.makedirs("Customer register/Customer ID")
My question is, how do I set the path the text files are to be stored in, in the directory when I don't know the directory? So that no matter where the program is run it is saved in the "Customer ID" folder I create (but on the computer the program is run on)?
Also, how do I make this work on both windows and mac?
I also want to program to be able to be executed several times, and check if the folder is there and save to the "Customer ID" folder if it already exists. Is there a way to do that?
EDIT:
This is the code I am trying to use:
try:
dirs = os.makedirs("Folder")
path = os.getcwd()
os.chdir(path + "/Folder")
print (os.getcwd())
except:
if os.path.exists:
path = os.getcwd()
unique_filename = str(uuid.uuid4())
customerpath = os.getcwd()
os.chdir(customerpath + "/Folder/" + unique_filename)
I am able to create a folder and change the directory (everything in "try" works as I want).
When this folder is created I want to create a second folder with a random generated folder name (used for saving customer files). I can't get this to work in the same way.
Error:
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\Users\48736\PycharmProjects\tina/Folder/979b9026-b2f6-4526-a17a-3b53384f60c4'
EDIT 2:
try:
os.makedirs("Folder")
path = os.getcwd()
os.chdir(path + "/Folder")
print (os.getcwd())
except:
if os.path.exists:
path = os.getcwd()
os.chdir(os.path.join(path, 'Folder'))
print(os.getcwd())
def userId(folderid):
try:
if not os.path.exists(folderid):
os.makedirs(folderid)
except:
if os.path.exists(folderid):
os.chdir(path + "/Folder/" + folderid)
userId(str(uuid.uuid4()))
print(os.getcwd())
So I can now create a folder, change directory to the folder I have created and create a new folder with a unique filename within that folder.
But I can't change the directory again to the folder with the unique filename.
Any suggestions?
I have tried:
os.chdir(path + "/Folder/" + folderid)
os.chdir(path, 'Folder', folderid)
os.chdir(os.path.join(path, 'Folder', folderid))
But is still just stays in: C:\Users\47896\PycharmProjects\tina\Folder

You can use relative paths in your create directory command, i.e.
os.makedirs("./Customer register/Customer ID")
to create folder in project root (=where the primary caller is located) or
os.makedirs("../Customer register/Customer ID") in parent directory.
You can, of course, traverse the files tree as you need.
For specific options mentioned in your question, please, see makedirs documentation at Python 3 docs

here is solution
import os
import shutil
import uuid
path_on_system = os.getcwd() # directory where you want to save data
path = r'Folder' # your working directory
dir_path = os.path.join(path_on_system, path)
if not os.path.exists(dir_path):
os.makedirs(dir_path)
file_name = str(uuid.uuid4()) # file which you have created
if os.path.exists(file_name) and os.path.exists(dir_path):
shutil.move(file_name,os.path.join(dir_path,file_name))
else:
print(" {} does not exist".format(file_name))

Specifying a partial path for os.path.join

I have a python script that creates a PDF and saves it in a subfoler of the folder where the script is saved. I have the following that saves the file to the subfolder:
outfilename = "Test" + ".pdf" #in real code there is a var that holds the name of the file
outfiledir = 'C:/Users/JohnDoe/Desktop/dev/PARENTFOLDER/SUBFOLDER/' #parent folder is where the script is - subfolder is where the PDFs get saved to
outfilepath = os.path.join(outfiledir, outfilename)
Is there a way I can save the PDFs to the subfolder without having to specify the full path? Lets say I wanted yto make this script an exe that multiple computers could use, how would I display the path so that the PDFs are just saved in the subfoler?
Thanks!

Try it:
import os
dir_name = os.path.dirname(os.path.abspath(__file__)) + '/subdir'
path = os.path.join(dir_name, 'filename')

Uploading multiple files via FTP based on filename

Using Python 3.4
I am generating files that are will look like 'Report_XXXXXX.xlsx' with X being unique customer numbers. I have a server with folders that are named 'XXXXXX.CustomerName'. I am trying to loop through each file/report and upload it to the matching folder based on the customer no. I have something that is working in my small test environment but I don't think it is working how I want. It uploads the files, but I am trying to catch anytime it doesn't find a match. Currently it fails my IF statment for every file. I think I am looping too many times or over the wrong items.
import os
import ftplib
creds = [line.rstrip('\n') for line in open('C:\\folder\\credentials.txt')]
ftp = ftplib.FTP_TLS("server.com")
ftp.login(creds[0], creds[1])
ftp.prot_p()
src_dir = 'C:\\Reports\\'
src_files = os.listdir('C:\\Reports\\')
for folder_name in ftp.nlst():
for file_name in src_files:
if folder_name[0:6] == file_name[7:-5]:
ftp.cwd('/'+folder_name)
open_file = open(src_dir+file_name, 'rb')
ftp.storbinary('STOR '+file_name, open_file)
open_file.close()
else:
print('Folder ' + folder_name + ' Not Found')
ftp.quit()
So for example the source directory has 3 files: 'Report_100002.xlsx, Report_100003.xlsx, Report_100007.xlsx' And the server has matching folders and a few extra folders. The files upload, and the output looks like so:
Folder 100000.CustomerName Not Found
Folder 100000.CustomerName Not Found
Folder 100000.CustomerName Not Found
Folder 100002.CustomerName Not Found
Folder 100002.CustomerName Not Found
Folder 100003.CustomerName Not Found
Folder 100003.CustomerName Not Found
Folder 100007.CustomerName Not Found
Folder 100007.CustomerName Not Found
I am trying to get to a state where I can properly log each item and whether it was a success, what folder it landed in, etc...

In your inner for loop you compare all 3 file names in src_dir with folder_name, but maximally only one satisfies the condition in your if statement. So the other 2 or 3 files that don't match cause the output you are seeing, for every folder on the ftp server. You could use a flag to keep track of whether a match was found and print your output based on that flag.
Another thing is that you should start iterating over src_files and then find matching folder names by iterating over ftp.nlist() (you are interested in source files that don't have a matching folder, not the other way around). So something like this (assuming a source file is allowed to end up in multiple folders):
....
folder_names = ftp.nlst()
for file_name in src_files:
folder_found = False
for folder_name in folder_names:
if folder_name[0:6] == file_name[7:-5]:
folder_found = True
ftp.cwd('/'+folder_name)
open_file = open(src_dir+file_name, 'rb')
ftp.storbinary('STOR '+file_name, open_file)
open_file.close()
if not folder_found:
print('No destination folder found for ' + file_name)
ftp.quit()
(the folder_names = ftp.nlst() is there so you don't repeatedly list the directories on the server)