In my code, I want to execute import filename for all the files that are present in a directory. I have a file abc.py under workspace.
I am currently doing the following :
for filename in os.listdir(homedir/workspace)
exec "import " + filename
filename = eval(filename + '.function(variable)')
The problem is that instead of doing import abc, it is doing import abc.py, and then showing the error no module named py
How can I resolve this?
Thanks in advance!
You can use os.path.splitext
os.path.splitext(filename)[0]
Of the returned two element array, the first element is just the filename, the second element is the extension. To be extra safe, you could double check that you did indeed grab a .py file:
if os.path.splitext(filename)[1] == '.py':
# do stuff
To run .function(variable) for all Python files in a directory:
import os
import sys
from importlib import import_module
dirpath = 'homedir/workspace'
sys.path.append(dirpath)
for filename in os.listdir(path)
module_name, ext = os.path.splitext(filename)
if ext == '.py':
try:
result = import_module(module_name).function(variable)
except Exception as e:
sys.stderr.write('error: %s: %s\n' % (filename, e))
continue
else:
print('%s: %s' % (filename, result))
See also Building a minimal plugin architecture in Python.
You can use glob:
import glob
import os
os.chdir("/mydir")
for file in glob.glob("*.py"):
# do something with file
Read this to make sure You really want to use exec
Related
I tried to make a program which delete all of the empty files ( whose size is zero ). Then, i run the program by dragging the script file in "command prompt" and run it .
However, no empty files had deleted (but i have some of them).
Please help me to find the error in my code.
import os
a = os.listdir('C:\\Python27')
for folder in a :
sizes = os.stat('C:\\Python27')
b = sizes.st_size
s = folder
if b == 0 :
remove('C:\\Python27\s')
You're assigning the values iterator os.listdir returns to folder and yet you aren't using it at all in os.stat or os.remove, but instead you are passing to them fixed values that you don't need.
You should do something like this:
import os
dir = 'C:\\Python27'
for file_name in os.listdir(dir):
file_path = os.path.join(dir, file_name)
if os.stat(file_path).st_size == 0:
os.remove(file_path)
You can delete something like the following code and you need to add some exception handling. I have used a test folder name to demonstrate.
import os
import sys
dir = 'c:/temp/testfolder'
for root, dirs, files in os.walk(dir):
for file in files:
fname = os.path.join(root, file)
try:
if os.path.getsize(fname) == 0:
print("Removing file %s" %(fname))
os.remove(fname)
except:
print("error: unable to remove 0 byte file")
raise
I have this code that will let the user choose which file he wants to update by passing an argument in the command line, and then it do some more things but I have not included that here:
import sys
import os
from sys import argv
path = "/home/Desktop/python/test"
files = os.walk( path )
filename = argv[1]
if filename in files:
inputFile = open(filename, 'r')
else:
print "no match found"
sys.exit()
inputFile.close()
When I run the script it keeps giving me "no match found" but im pretty sure the file is there. I cant see what Im doing wrong
os.walk() returns a generator, one that produces tuples with (root, directories, files) values for each iteration.
You can't use that generator to test for a single file, not with a simple in membership test.
You'll also need to re-instate the whole path; you can't just open an unclassified filename without the directory it lives in. Just use a for loop here, and break once you found it. The else suite on a for loop only executes when you did not use break (e.g. the file was not found):
path = "/home/Desktop/python/test"
filename = argv[1]
for root, directories, files in os.walk(path):
if filename in files:
full_path = os.path.join(root, filename)
break
else:
print "no match found"
sys.exit()
with open(full_path) as input_file:
# do something with the file
I added a with statement to handle the lifetime of the file object; once the with block is exited the file is automatically closed for you.
Alternatively, you may use following code snippet.
import os.path
filename = argv[1]
path = "/home/Desktop/python/test/"
if os.path.isfile(path + filename):
inputFile = open(path + filename, "r")
else:
print "File Not Found"
I have a specific problem in python. Below is my folder structure.
dstfolder/slave1/slave
I want the contents of 'slave' folder to be moved to 'slave1' (parent folder). Once moved,
'slave' folder should be deleted. shutil.move seems to be not helping.
Please let me know how to do it ?
Example using the os and shutil modules:
from os.path import join
from os import listdir, rmdir
from shutil import move
root = 'dstfolder/slave1'
for filename in listdir(join(root, 'slave')):
move(join(root, 'slave', filename), join(root, filename))
rmdir(join(root, 'slave'))
I needed something a little more generic, i.e. move all the files from all the [sub]+folders into the root folder.
For example start with:
root_folder
|----test1.txt
|----1
|----test2.txt
|----2
|----test3.txt
And end up with:
root_folder
|----test1.txt
|----test2.txt
|----test3.txt
A quick recursive function does the trick:
import os, shutil, sys
def move_to_root_folder(root_path, cur_path):
for filename in os.listdir(cur_path):
if os.path.isfile(os.path.join(cur_path, filename)):
shutil.move(os.path.join(cur_path, filename), os.path.join(root_path, filename))
elif os.path.isdir(os.path.join(cur_path, filename)):
move_to_root_folder(root_path, os.path.join(cur_path, filename))
else:
sys.exit("Should never reach here.")
# remove empty folders
if cur_path != root_path:
os.rmdir(cur_path)
You will usually call it with the same argument for root_path and cur_path, e.g. move_to_root_folder(os.getcwd(),os.getcwd()) if you want to try it in the python environment.
The problem might be with the path you specified in the shutil.move function
Try this code
import os
import shutil
for r,d,f in os.walk("slave1"):
for files in f:
filepath = os.path.join(os.getcwd(),"slave1","slave", files)
destpath = os.path.join(os.getcwd(),"slave1")
shutil.copy(filepath,destpath)
shutil.rmtree(os.path.join(os.getcwd(),"slave1","slave"))
Paste it into a .py file in the dstfolder. I.e. slave1 and this file should remain side by side. and then run it. worked for me
Use this if the files have same names, new file names will have folder names joined by '_'
import shutil
import os
source = 'path to folder'
def recursive_copy(path):
for f in sorted(os.listdir(os.path.join(os.getcwd(), path))):
file = os.path.join(path, f)
if os.path.isfile(file):
temp = os.path.split(path)
f_name = '_'.join(temp)
file_name = f_name + '_' + f
shutil.move(file, file_name)
else:
recursive_copy(file)
recursive_copy(source)
Maybe you could get into the dictionary slave, and then
exec system('mv .........')
It will work won't it?
I'm working on a certain program where I need to do different things depending on the extension of the file. Could I just use this?
if m == *.mp3
...
elif m == *.flac
...
Assuming m is a string, you can use endswith:
if m.endswith('.mp3'):
...
elif m.endswith('.flac'):
...
To be case-insensitive, and to eliminate a potentially large else-if chain:
m.lower().endswith(('.png', '.jpg', '.jpeg'))
os.path provides many functions for manipulating paths/filenames. (docs)
os.path.splitext takes a path and splits the file extension from the end of it.
import os
filepaths = ["/folder/soundfile.mp3", "folder1/folder/soundfile.flac"]
for fp in filepaths:
# Split the extension from the path and normalise it to lowercase.
ext = os.path.splitext(fp)[-1].lower()
# Now we can simply use == to check for equality, no need for wildcards.
if ext == ".mp3":
print fp, "is an mp3!"
elif ext == ".flac":
print fp, "is a flac file!"
else:
print fp, "is an unknown file format."
Gives:
/folder/soundfile.mp3 is an mp3!
folder1/folder/soundfile.flac is a flac file!
Use pathlib From Python3.4 onwards.
from pathlib import Path
Path('my_file.mp3').suffix == '.mp3'
If you are working with folders that contain periods, you can perform an extra check using
Path('your_folder.mp3').is_file() and Path('your_folder.mp3').suffix == '.mp3'
to ensure that a folder with a .mp3 suffix is not interpreted to be an mp3 file.
Look at module fnmatch. That will do what you're trying to do.
import fnmatch
import os
for file in os.listdir('.'):
if fnmatch.fnmatch(file, '*.txt'):
print file
or perhaps:
from glob import glob
...
for files in glob('path/*.mp3'):
do something
for files in glob('path/*.flac'):
do something else
one easy way could be:
import os
if os.path.splitext(file)[1] == ".mp3":
# do something
os.path.splitext(file) will return a tuple with two values (the filename without extension + just the extension). The second index ([1]) will therefor give you just the extension. The cool thing is, that this way you can also access the filename pretty easily, if needed!
An old thread, but may help future readers...
I would avoid using .lower() on filenames if for no other reason than to make your code more platform independent. (linux is case sensistive, .lower() on a filename will surely corrupt your logic eventually ...or worse, an important file!)
Why not use re? (Although to be even more robust, you should check the magic file header of each file...
How to check type of files without extensions in python? )
import re
def checkext(fname):
if re.search('\.mp3$',fname,flags=re.IGNORECASE):
return('mp3')
if re.search('\.flac$',fname,flags=re.IGNORECASE):
return('flac')
return('skip')
flist = ['myfile.mp3', 'myfile.MP3','myfile.mP3','myfile.mp4','myfile.flack','myfile.FLAC',
'myfile.Mov','myfile.fLaC']
for f in flist:
print "{} ==> {}".format(f,checkext(f))
Output:
myfile.mp3 ==> mp3
myfile.MP3 ==> mp3
myfile.mP3 ==> mp3
myfile.mp4 ==> skip
myfile.flack ==> skip
myfile.FLAC ==> flac
myfile.Mov ==> skip
myfile.fLaC ==> flac
You should make sure the "file" isn't actually a folder before checking the extension. Some of the answers above don't account for folder names with periods. (folder.mp3 is a valid folder name).
Checking the extension of a file:
import os
file_path = "C:/folder/file.mp3"
if os.path.isfile(file_path):
file_extension = os.path.splitext(file_path)[1]
if file_extension.lower() == ".mp3":
print("It's an mp3")
if file_extension.lower() == ".flac":
print("It's a flac")
Output:
It's an mp3
Checking the extension of all files in a folder:
import os
directory = "C:/folder"
for file in os.listdir(directory):
file_path = os.path.join(directory, file)
if os.path.isfile(file_path):
file_extension = os.path.splitext(file_path)[1]
print(file, "ends in", file_extension)
Output:
abc.txt ends in .txt
file.mp3 ends in .mp3
song.flac ends in .flac
Comparing file extension against multiple types:
import os
file_path = "C:/folder/file.mp3"
if os.path.isfile(file_path):
file_extension = os.path.splitext(file_path)[1]
if file_extension.lower() in {'.mp3', '.flac', '.ogg'}:
print("It's a music file")
elif file_extension.lower() in {'.jpg', '.jpeg', '.png'}:
print("It's an image file")
Output:
It's a music file
import os
source = ['test_sound.flac','ts.mp3']
for files in source:
fileName,fileExtension = os.path.splitext(files)
print fileExtension # Print File Extensions
print fileName # It print file name
#!/usr/bin/python
import shutil, os
source = ['test_sound.flac','ts.mp3']
for files in source:
fileName,fileExtension = os.path.splitext(files)
if fileExtension==".flac" :
print 'This file is flac file %s' %files
elif fileExtension==".mp3":
print 'This file is mp3 file %s' %files
else:
print 'Format is not valid'
if (file.split(".")[1] == "mp3"):
print "its mp3"
elif (file.split(".")[1] == "flac"):
print "its flac"
else:
print "not compat"
If your file is uploaded then
import os
file= request.FILES['your_file_name'] #Your input file_name for your_file_name
ext = os.path.splitext(file.name)[-1].lower()
if ext=='.mp3':
#do something
elif ext=='.xls' or '.xlsx' or '.csv':
#do something
else:
#The uploaded file is not the required format
file='test.xlsx'
if file.endswith('.csv'):
print('file is CSV')
elif file.endswith('.xlsx'):
print('file is excel')
else:
print('none of them')
I'm surprised none of the answers proposed the use of the pathlib library.
Of course, its use is situational but when it comes to file handling or stats pathlib is gold.
Here's a snippet:
import pathlib
def get_parts(p: str or pathlib.Path) -> None:
p_ = pathlib.Path(p).expanduser().resolve()
print(p_)
print(f"file name: {p_.name}")
print(f"file extension: {p_.suffix}")
print(f"file extensions: {p_.suffixes}\n")
if __name__ == '__main__':
file_path = 'conf/conf.yml'
arch_file_path = 'export/lib.tar.gz'
get_parts(p=file_path)
get_parts(p=arch_file_path)
and the output:
/Users/hamster/temp/src/pro1/conf/conf.yml
file name: conf.yml
file extension: .yml
file extensions: ['.yml']
/Users/hamster/temp/src/pro1/conf/lib.tar.gz
file name: lib.tar.gz
file extension: .gz
file extensions: ['.tar', '.gz']
I want to delete all files with the extension .bak in a directory. How can I do that in Python?
Via os.listdir and os.remove:
import os
filelist = [ f for f in os.listdir(mydir) if f.endswith(".bak") ]
for f in filelist:
os.remove(os.path.join(mydir, f))
Using only a single loop:
for f in os.listdir(mydir):
if not f.endswith(".bak"):
continue
os.remove(os.path.join(mydir, f))
Or via glob.glob:
import glob, os, os.path
filelist = glob.glob(os.path.join(mydir, "*.bak"))
for f in filelist:
os.remove(f)
Be sure to be in the correct directory, eventually using os.chdir.
Use os.chdir to change directory .
Use glob.glob to generate a list of file names which end it '.bak'. The elements of the list are just strings.
Then you could use os.unlink to remove the files. (PS. os.unlink and os.remove are synonyms for the same function.)
#!/usr/bin/env python
import glob
import os
directory='/path/to/dir'
os.chdir(directory)
files=glob.glob('*.bak')
for filename in files:
os.unlink(filename)
In Python 3.5, os.scandir is better if you need to check for file attributes or type - see os.DirEntry for properties of the object that's returned by the function.
import os
for file in os.scandir(path):
if file.name.endswith(".bak"):
os.unlink(file.path)
This also doesn't require changing directories since each DirEntry already includes the full path to the file.
you can create a function. Add maxdepth as you like for traversing subdirectories.
def findNremove(path,pattern,maxdepth=1):
cpath=path.count(os.sep)
for r,d,f in os.walk(path):
if r.count(os.sep) - cpath <maxdepth:
for files in f:
if files.endswith(pattern):
try:
print "Removing %s" % (os.path.join(r,files))
#os.remove(os.path.join(r,files))
except Exception,e:
print e
else:
print "%s removed" % (os.path.join(r,files))
path=os.path.join("/home","dir1","dir2")
findNremove(path,".bak")
First glob them, then unlink.
I realize this is old; however, here would be how to do so using just the os module...
def purgedir(parent):
for root, dirs, files in os.walk(parent):
for item in files:
# Delete subordinate files
filespec = os.path.join(root, item)
if filespec.endswith('.bak'):
os.unlink(filespec)
for item in dirs:
# Recursively perform this operation for subordinate directories
purgedir(os.path.join(root, item))
For one line solution (Both Windows and Linux) ;
import glob,os
for file in glob.glob("<your_path>/*.bak"): print(file," this will be deleted")
if input("continue ?") == "Y":
for file in glob.glob("<your_path>/*.bak"): os.remove(file)
On Linux and macOS you can run simple command to the shell:
subprocess.run('rm /tmp/*.bak', shell=True)