I have a python web form with two options - File upload and textarea. I need to take the values from each and pass them to another command-line program. I can easily pass the file name with file upload options, but I am not sure how to pass the value of the textarea.
I think what I need to do is:
Generate a unique file name
Create a temporary file with that name in the working directory
Save the values passed from textarea into the temporary file
Execute the commandline program from inside my python module and pass it the name of the temporary file
I am not sure how to generate a unique file name. Can anybody give me some tips on how to generate a unique file name? Any algorithms, suggestions, and lines of code are appreciated.
Thanks for your concern
I didn't think your question was very clear, but if all you need is a unique file name...
import uuid
unique_filename = str(uuid.uuid4())
If you want to make temporary files in Python, there's a module called tempfile in Python's standard libraries. If you want to launch other programs to operate on the file, use tempfile.mkstemp() to create files, and os.fdopen() to access the file descriptors that mkstemp() gives you.
Incidentally, you say you're running commands from a Python program? You should almost certainly be using the subprocess module.
So you can quite merrily write code that looks like:
import subprocess
import tempfile
import os
(fd, filename) = tempfile.mkstemp()
try:
tfile = os.fdopen(fd, "w")
tfile.write("Hello, world!\n")
tfile.close()
subprocess.Popen(["/bin/cat", filename]).wait()
finally:
os.remove(filename)
Running that, you should find that the cat command worked perfectly well, but the temporary file was deleted in the finally block. Be aware that you have to delete the temporary file that mkstemp() returns yourself - the library has no way of knowing when you're done with it!
(Edit: I had presumed that NamedTemporaryFile did exactly what you're after, but that might not be so convenient - the file gets deleted immediately when the temp file object is closed, and having other processes open the file before you've closed it won't work on some platforms, notably Windows. Sorry, fail on my part.)
The uuid module would be a good choice, I prefer to use uuid.uuid4().hex as random filename because it will return a hex string without dashes.
import uuid
filename = uuid.uuid4().hex
The outputs should like this:
>>> import uuid
>>> uuid.uuid()
UUID('20818854-3564-415c-9edc-9262fbb54c82')
>>> str(uuid.uuid4())
'f705a69a-8e98-442b-bd2e-9de010132dc4'
>>> uuid.uuid4().hex
'5ad02dfb08a04d889e3aa9545985e304' # <-- this one
Maybe you need unique temporary file?
import tempfile
f = tempfile.NamedTemporaryFile(mode='w+b', delete=False)
print f.name
f.close()
f is opened file. delete=False means do not delete file after closing.
If you need control over the name of the file, there are optional prefix=... and suffix=... arguments that take strings. See https://docs.python.org/3/library/tempfile.html.
You can use the datetime module
import datetime
uniq_filename = str(datetime.datetime.now().date()) + '_' + str(datetime.datetime.now().time()).replace(':', '.')
Note that:
I am using replace since the colons are not allowed in filenames in many operating systems.
That's it, this will give you a unique filename every single time.
In case you need short unique IDs as your filename, try shortuuid, shortuuid uses lowercase and uppercase letters and digits, and removing similar-looking characters such as l, 1, I, O and 0.
>>> import shortuuid
>>> shortuuid.uuid()
'Tw8VgM47kSS5iX2m8NExNa'
>>> len(ui)
22
compared to
>>> import uuid
>>> unique_filename = str(uuid.uuid4())
>>> len(unique_filename)
36
>>> unique_filename
'2d303ad1-79a1-4c1a-81f3-beea761b5fdf'
I came across this question, and I will add my solution for those who may be looking for something similar. My approach was just to make a random file name from ascii characters. It will be unique with a good probability.
from random import sample
from string import digits, ascii_uppercase, ascii_lowercase
from tempfile import gettempdir
from os import path
def rand_fname(suffix, length=8):
chars = ascii_lowercase + ascii_uppercase + digits
fname = path.join(gettempdir(), 'tmp-'
+ ''.join(sample(chars, length)) + suffix)
return fname if not path.exists(fname) \
else rand_fname(suffix, length)
This can be done using the unique function in ufp.path module.
import ufp.path
ufp.path.unique('./test.ext')
if current path exists 'test.ext' file. ufp.path.unique function return './test (d1).ext'.
To create a unique file path if its exist, use random package to generate a new string name for file. You may refer below code for same.
import os
import random
import string
def getUniquePath(folder, filename):
path = os.path.join(folder, filename)
while os.path.exists(path):
path = path.split('.')[0] + ''.join(random.choice(string.ascii_lowercase) for i in range(10)) + '.' + path.split('.')[1]
return path
Now you can use this path to create file accordingly.
Related
I'm kinda new to python and I feel like the answer to this is so simple but I have no idea what the answer is. I'm trying to move files from one place to another but I don't want to have to change my code every time I wanna move that file so I just want to get user input from the terminal.
import shutil
loop = True
while loop:
a = input()
shutil.move("/home/Path/a", "/home/Path/Pictures")
What do I have to put around the a so that it doesn't read it as part of the string?
This should do what you want. the os.path.join() will combine the string value in a, that you get from input with the first part of the path you have provided. You should use os.path.join() as this will form paths in a way that is system independent.
import shutil
import os
loop = True
while loop:
a = input()
shutil.move(os.path.join("/home/Path/", a), "/home/Path/Pictures")
Output:
>>> a = input()
test.txt
>>> path = os.path.join("/home/Path/", a)
>>> path
'/home/Path/test.txt'
You can also use "/home/Path/{0}".format(a) which will swap the value of a with {0}, or you can do do "/home/Path/{0}" + str(a) which will also do what you want.
Edited to account for Question in comment:
This will work if your directory doesn't have any sub-directories. it may still work if there are directories and files in there but I didn't test that.
import shutil
import os
files = os.listdir("/home/Path/")
for file in files:
shutil.move(os.path.join("/home/Path/", file), "/home/Path/Pictures")
one solution
a = 'test.csv'
path = '/home/Path/{}'.format(a)
>>> path
/home/Path/test.csv
I know about os.path.isfile(fname), but now I need to search if a file exists that is named FILEnTEST.txt where n could be any positive integer (so it could be FILE1TEST.txt or FILE9876TEST.txt)
I guess a solution to this could involve substrings that the filename starts/ends with OR one that involves somehow calling os.path.isfile('FILE' + n + 'TEST.txt') and replacing n with any number, but I don't know how to approach either solution.
You would need to write your own filtering system, by getting all the files in a directory and then matching them to a regex string and seeing if they fail the test or not:
import re
pattern = re.compile("FILE\d+TEST.txt")
dir = "/test/"
for filepath in os.listdir(dir):
if pattern.match(filepath):
#do stuff with matching file
I'm not near a machine with Python installed on it to test the code, but it should be something along those lines.
You can use a regular expression:
/FILE\d+TEST.txt/
Example: regexr.com.
Then you can use said regular expression and iterate through all of the files in a directory.
import re
import os
filename_re = 'FILE\d+TEST.txt'
for filename in os.listdir(directory):
if re.search(filename_re, filename):
# this file has the form FILEnTEST.txt
# do what you want with it now
You can also do it as such:
import os
import re
if len([file for file in os.listdir(directory) if re.search('regex', file)]):
# there's at least 1 such file
I am trying to copy and rename some PDFs with absolute paths.
ie: c:\users\andrew\pdf\p.pdf gets copied to c:\users\pdf\ORGp.pdf
Leaving two files in the directory p.pdf and ORGp.pdf
I've been working on this issue for the past hour and I can't seem to nail it.
Is there a more pythonic way to do it then splitting the string into a list and rejoining them after adding ORG on the last element?
Using python 2.7 on windows 8.
Your question is a bit ambiguous, but I will try to answer it anyway.
This is a python code sample that will copy under the new names, all files under a particular folder, specified at the beginning of the script:
import os
import shutil
folder_name = "c:\\users\\andrew\\pdf"
for root_folder, _, file_names in os.walk(folder_name):
for file_n in file_names:
new_name = os.path.join(root_folder, "ORG" + file_n)
old_name = os.path.join(root_folder, file_n)
print "We will copy at ", new_name, old_name
shutil.copyfile(old_name, new_name)
This code will copy and rename a list of absolute file paths:
import os
import shutil
files_to_rename = ["c:\\users\\andrew\\pdf\\p.pdf", "c:\\users\\andrew\\pdf2\\p2.pdf"]
for file_full_path in files_to_rename:
folder_n, file_n = os.path.split(file_full_path)
new_name = os.path.join(folder_n, "ORG" + file_n)
print "We will copy at ", new_name, file_full_path
shutil.copyfile(file_full_path, new_name)
I testing this script on Mac OS, with Python 2.7.7, but I think it should work nicely also on Windows.
You can try
import os
.......some logic.....
os.rename(filename, newfilename)
Splitting the string into a list and rejoining (after removing 'andrew' from the list and prefixing 'ORG' to the last element) is quite Pythonic. It's an explicit and obvious way to do it.
You can use the standard str and list methods to do it. However, there are various dedicated file path manipulation functions in the os.path module which you should become familiar with, but the str and list methods are fine when you are sure that all the file names you're processing are sane. os.path also has other useful file-related functions: you can check if a file exists, whether it's a file or a directory, get a file's timestamps, etc.
To actually copy the file once you've generated the new name, use shutil.copyfile(). You may also wish to check first that the file doesn't already exist using os.path.exists(). Unfortunately, some metadata gets lost in this process, eg file owners, as mentioned in the warning in the shutil docs.
This is what I ended up doing to do the rename. I'm not sure how pythonic it is, but it works.
split=fle.split('\\')
print split
pdf=split[len(split)-1]
pdf='ORG%s' % pdf
print pdf
del split[len(split)-1]
split.append(pdf)
fle1 = '\\'.join(split)
try:
shutil.copy(fle, fle1)
except:
print('failed copy')
return''
Though Windows is case insensitive, it does preserve case in filenames. In Python, is there any way to get a filename with case as it is stored on the file system?
E.g., in a Python program I have filename = "texas.txt", but want to know that it's actually stored "TEXAS.txt" on the file system, even if this is inconsequential for various file operations.
Here's the simplest way to do it:
>>> import win32api
>>> win32api.GetLongPathName(win32api.GetShortPathName('texas.txt')))
'TEXAS.txt'
I had problems with special characters with the win32api solution above. For unicode filenames you need to use:
win32api.GetLongPathNameW(win32api.GetShortPathName(path))
This one is standard library only and converts all path parts (except drive letter):
def casedpath(path):
r = glob.glob(re.sub(r'([^:/\\])(?=[/\\]|$)|\[', r'[\g<0>]', path))
return r and r[0] or path
And this one handles UNC paths in addition:
def casedpath_unc(path):
unc, p = os.path.splitunc(path)
r = glob.glob(unc + re.sub(r'([^:/\\])(?=[/\\]|$)|\[', r'[\g<0>]', p))
return r and r[0] or path
Note: It is somewhat slower than the file system dependent Win API "GetShortPathName" method, but works platform & file system independent and also when short filename generation is switched off on Windows volumes (fsutil.exe 8dot3name query C:). The latter is recommended at least for performance critical file systems when no 16bit apps rely anymore on that:
fsutil.exe behavior set disable8dot3 1
>>> import os
>>> os.listdir("./")
['FiLeNaMe.txt']
Does this answer your question?
and if you want to recurse directories
import os
path=os.path.join("c:\\","path")
for r,d,f in os.walk(path):
for file in f:
if file.lower() == "texas.txt":
print "Found: ",os.path.join( r , file )
You could use:
import os
a = os.listdir('mydirpath')
b = [f.lower() for f in a]
try:
i = b.index('texas.txt')
print a[i]
except ValueError:
print('File not found in this directory')
This of course assumes that your search string 'texas.txt' is in lowercase. If it isn't you'll have to convert it to lowercase first.
Here is the code example. Basically output.csv needs to remove any drive letter A:-Y: and replace it with Z: I tried to do this with a list (not complete yet) but it generates the error: TypeError: expected a character buffer object
#!/usr/bin/python
import os.path
import os
import shutil
import csv
import re
# Create the videos directory in the current directory
# If the directory exists ignore it.
#
# Moves all files with the .wmv extenstion to the
# videos folder for file structure
#
#Crawl the videos directory then change to videos directory
# create the videos.csv file in the videos directory
# replace any drive letter A:-Y: with Z:
def createCSV():
directory = "videos"
if not os.path.isdir("." + directory + "/"):
os.mkdir("./" + directory + "/")
for file in os.listdir("./"):
if os.path.splitext(file)[1] == ".wmv":
shutil.move(file, os.path.join("videos", file))
listDirectory = os.listdir("videos")
os.chdir(directory)
f = open("videos.csv", "w")
f.writelines(os.path.join(os.getcwd(), f + '\n') for f in listDirectory)
f = open('videos.csv', 'r')
w = open('output.csv', 'w')
f_cont = f.readlines()
for line in f_cont:
regex = re.compile("\b[GHI]:")
re.sub(regex, "Z:", line)
w.write(line)
f.close()
createCSV()
EDIT:
I think my flow/logic is wrong, the output.csv file that gets created still G: in the .csv it was not renamed to Z:\ from the re.sub line.
I can see you use some pythonic snippets, with smart uses of path.join and a commented code. This can get even better, let's rewrite a few things so we can solve your drive letters issue, and gain a more pythonic code on the way :
#!/usr/bin/env python
# -*- coding= UTF-8 -*-
# Firstly, modules can be documented using docstring, so drop the comments
"""
Create the videos directory in the current directory
If the directory exists ignore it.
Moves all files with the .wmv extension to the
videos folder for file structure
Crawl the videos directory then change to videos directory
create the videos.csv file in the videos directory
create output.csv replace any drive letter A:-Y: with Z:
"""
# not useful to import os and os.path as the second is contain in the first one
import os
import shutil
import csv
# import glob, it will be handy
import glob
import ntpath # this is to split the drive
# don't really need to use a function
# Here, don't bother checking if the directory exists
# and you don't need add any slash either
directory = "videos"
ext = "*.wmv"
try :
os.mkdir(directory)
except OSError :
pass
listDirectory = [] # creating a buffer so no need to list the dir twice
for file in glob.glob(ext): # much easier this way, isn't it ?
shutil.move(file, os.path.join(directory, file)) # good catch for shutil :-)
listDirectory.append(file)
os.chdir(directory)
# you've smartly imported the csv module, so let's use it !
f = open("videos.csv", "w")
vid_csv = csv.writer(f)
w = open('output.csv', 'w')
out_csv = csv.writer(w)
# let's do everything in one loop
for file in listDirectory :
file_path = os.path.abspath(file)
# Python includes functions to deal with drive letters :-D
# I use ntpath because I am under linux but you can use
# normal os.path functions on windows with the same names
file_path_with_new_letter = ntpath.join("Z:", ntpath.splitdrive(file_path)[1])
# let's write the csv, using tuples
vid_csv.writerow((file_path, ))
out_csv.writerow((file_path_with_new_letter, ))
It seems like the problem is in the loop at the bottom of your code. The string's replace method doesn't receive a list as its first arguments, but another string. You need to loop through your removeDrives list and call line.remove with every item in that list.
You could use
for driveletter in removedrives:
line = line.replace(driveletter, 'Z:')
thereby iterating over your list and replacing one of the possible drive letters after the other. As abyx wrote, replace expects a string, not a list, so you need this extra step.
Or use a regular expression like
import re
regex = re.compile(r"\b[FGH]:")
re.sub(regex, "Z:", line)
Additional bonus: Regex can check that it's really a drive letter and not, for example, a part of something bigger like OH: hydrogen group.
Apart from that, I suggest you use os.path's own path manipulation functions instead of trying to implement them yourself.
And of course, if you do anything further with the CSV file, take a look at the csv module.
A commentator above has already mentioned that you should close all the files you've opened. Or use with with statement:
with open("videos.csv", "w") as f:
do_stuff()