Develop Reference

Need help creating a txt file in a folder which, is in the same directory

I'm pretty new to python. I have a homwork problem where we need to analyze corpora and then compare them. We also have to save the files as a .txt file after is has been processed with an attribute, the size.
So I need to create a .txt file in a seperate folder called trigram-models.
This folder is in the same directory as my python file. I think i have to use the os module but i'm not sure how.
Here is my code:
from langdetect import read_trigrams, trigram_table, write_trigrams
import os
def make_profiles(datafolder, profilefolder, size):
filelist = []
for file in os.listdir('./training'):
filelist.append(file)
print(filelist)
for file in filelist:
filen = "./training/"+file
print("fi", filen)
maketable = trigram_table(filen, size)
readdata = read_trigrams(filen)
#print("re", readdata)
splitname = str(file).split('-')
newname = splitname[0] + "." + str(size) + '.txt'
endtable = write_trigrams(readdata, newname)
return (endtable)
make_profiles("./training", "./trigram-models", 20)

To create a directory, I would use the following format which relies on a try / catch and prevents an error if the directory already exists:
dirName = 'tempDir'
try:
# Create target Directory
os.mkdir(dirName)
print("Directory " , dirName , " Created ")
except FileExistsError:
print("Directory " , dirName , " already exists")
To change your directory you can use the following:
os.chdir(directoryLocation)
I recommend reading chapter 8 in automating the boring stuff with python.
I hope this helps. If you have any questions please don't hesitate to ask.

First of all, be sure to indent all the code in your method for it to be appropriately enclosed.
You are also passing relative paths (datafolder, profilefolder) for your folders as method arguments, so you should use them inside the method instead.
Lastly, to create a file in a folder, I would recommend using the following algorithm:
file_path = '/'.join(profilefolder, newname)
with open(file_path, 'w') as ouf:
ouf.write(endtable)
You will probably need to replace "endtable" with a string representation of your data.
Hope it helps.

As to clarify on toti08's answer, you should replace os.absdir with os.path.absdir.
filelist = [os.path.abspath(f) for f in os.listdir(data_folder)]
instead of
filelist = [os.abspath(f) for f in os.listdir(data_folder)]

Your function is not using the argument profileFolder, where you specify the name of the output directory. So first of all you should use this information for creating a folder before processing your files.
So first thing would be to create this output directory.
Second is to save your files there, and to do that you need to append the file name to the output directory. Something like this:
def make_profiles(data_folder, output_folder, size):
filelist = []
for file in os.listdir(data_folder):
filelist.append(file)
# Create output folder
if not os.path.exists(output_folder):
os.mkdir(output_folder)
for file in filelist:
filen = "./training/"+file
#print("fi", filen)
splitname = str(file).split('-')
# Create new file by appending name to output_folder
newname = os.path.join(output_folder, splitname[0] + "." + str(size) + '.txt')
return (endtable)
make_profiles(./training, './trigram-models', 20)
Note that you can also specify the relative folder name (i.e. "trigram-models" only) and then create the output directory by appending this name to the current path:
output_folder = os.path.join(os.getcwd(), output_folder)
Also (not related to the question) this piece of code could be optimized:
filelist = []
for file in os.listdir(data_folder):
filelist.append(file)
os.listdir already returns a list, so you could directly write:
filelist = os.listdir(data_folder)
But since you're interested in the absolute path of each file you could better do:
filelist = [os.path.abspath(f) for f in os.listdir(data_folder)]
where you basically take each file returned by os.listdir and you append its absolute path to your file list. Doing this you could avoid the line filen = "./training/"+file.
So in the end your code should look something like this:
def make_profiles(data_folder, output_folder, size):
filelist = [os.abspath(f) for f in os.listdir(data_folder)]
# Create output folder
if not os.path.exists(output_folder):
os.mkdir(output_folder)
for file in filelist:
splitname = str(file).split('-')
# [...add other pieces of code]
# Create new file by appending name to output_folder
newname = os.path.join(output_folder, splitname[0] + "." + str(size) + '.txt')
# [...add other pieces of code]
return (endtable)
make_profiles(./training, './trigram-models', 20)

Python remove files from folder which are not in list

I'm looking for help in below code where I can remove the files from the folder which not available in the given csv file.
I read the input file in the pandas' data frame and convert it into the list then
reading the fileName from the folder and comparing the fileName with the available file in the folder and if it exists continue if not remove. but it is removing all the files including the not matching files.
I only want to remove the files which are not present in the file I'm reading using pandas data frame.
import os
import pandas as pd
path = "Adwords/"
flist = pd.read_csv('C:/mediaops/mapping/adword/file_name.csv')
file_name = flist['fileName'].tolist()
for filename in os.listdir(path):
print(filename)
if filename == file_name:
continue
elif filename != file_name:
os.remove(filename)

for filename in os.listdir(path):
print(filename)
if filename not in file_name:
os.remove(filename)

In your original solution, you are trying to do filename == file_name and filename != file_name, but you cannot do that.
See filename is a string and file_name is a list, and you cannot use == to compare them, you need to use membership operators like in and not in, like if filename not in file_name: which I did in my answer below
(Thanks to Tobias's Answer)
Now since that is out of the window, now you can iterate through all files using os.listdir, then use os.remove to remove the necessary files, in addition using os.path.join to get the full path of the file!
import os
#List all files in path
for filename in os.listdir(path):
#If file is not present in list
if filename not in file_name:
#Get full path of file and remove it
full_file_path = os.path.join(path, filename)
os.remove(full_file_path)

The problem is that file_name is a list if string, whereas filename is a single string, so the check filename != file_name will always be true and the file thus always be removed. Instead, use in and not in to check whether the string is (not) in the list of strings. Also, using a set would be faster. Also, those variable names are really confusing.
set_of_files = set(file_name)
for filename in os.listdir(path):
if filename not in set_of_files:
os.remove(filename)
Also, as noted in Devesh's answer, you may have to join the filename to the path in order to be able to actually remove the file.

when I implemented these answers it deleted all files in the dir, not in my list. So I wrote one for any weary traveler that may need this script. User needs to add in the path for where their files are and make a csvfile with the basename of the files that they want to keep. you can also add in the extention of the files that you want to look at if they happen to all the same.
The process is making the csv into a list based on each element in the first column and then checking to see if the files in the current dir are present in the list. If they are not then remove.
import os
import csv
import argparse
import sys
import pathlib
data_path = path = "/path/to/your/dir"
csv_guide = "filenamestokeep.csv"
csv_path = os.path.join(data_path, csv_guide)
ext = "input.your.extention.of.files.to.look.at.as.ext, like .txt"
with open(csv_path, 'r') as csvfile:
good_files = []
for n in csv.reader(csvfile):
if len(n) > 0: good_files.append(n[0])
print(good_files)
all_files = os.listdir(data_path)
for filename in all_files:
if filename.endswith(ext) and filename not in good_files:
print(filename)
full_file_path = os.path.join(data_path, filename)
print("File to delete: {} ".format(filename))
os.remove(full_file_path)
else:
print(f"Ignored -- {filename}")

Get file path without file name

Given a file path
/path/to/some/file.jpg
How would I get
/path/to/some
I'm currently doing
fullpath = '/path/to/some/file.jpg'
filepath = '/'.join(fullpath.split('/')[:-1])
But I think it is open to errors

With os.path.split:
dirname, fname = os.path.split(fullpath)
Per the docs:
Split the pathname path into a pair, (head, tail) where tail is the
last pathname component and head is everything leading up to that. The
tail part will never contain a slash; if path ends in a slash, tail
will be empty. If there is no slash in path, head will be empty.
os.path is always the module suitable for the platform that the code is running on.

Please try this
fullpath = '/path/to/some/file.jpg'
import os
os.path.dirname(fullpath)

Using pathlib you can get the path without the file name using the .parent attribute:
from pathlib import Path
fullpath = Path("/path/to/some/file.jpg")
filepath = str(fullpath.parent) # /path/to/some/
This handles both UNIX and Windows paths correctly.

With String rfind method.
fullpath = '/path/to/some/file.jpg'
index = fullpath.rfind('/')
fullpath[0:index]

Renaming multiple files in a directory using Python

I'm trying to rename multiple files in a directory using this Python script:
import os
path = '/Users/myName/Desktop/directory'
files = os.listdir(path)
i = 1
for file in files:
os.rename(file, str(i)+'.jpg')
i = i+1
When I run this script, I get the following error:
Traceback (most recent call last):
File "rename.py", line 7, in <module>
os.rename(file, str(i)+'.jpg')
OSError: [Errno 2] No such file or directory
Why is that? How can I solve this issue?
Thanks.

You are not giving the whole path while renaming, do it like this:
import os
path = '/Users/myName/Desktop/directory'
files = os.listdir(path)
for index, file in enumerate(files):
os.rename(os.path.join(path, file), os.path.join(path, ''.join([str(index), '.jpg'])))
Edit: Thanks to tavo, The first solution would move the file to the current directory, fixed that.

You have to make this path as a current working directory first.
simple enough.
rest of the code has no errors.
to make it current working directory:
os.chdir(path)

import os
from os import path
import shutil
Source_Path = 'E:\Binayak\deep_learning\Datasets\Class_2'
Destination = 'E:\Binayak\deep_learning\Datasets\Class_2_Dest'
#dst_folder = os.mkdir(Destination)
def main():
for count, filename in enumerate(os.listdir(Source_Path)):
dst = "Class_2_" + str(count) + ".jpg"
# rename all the files
os.rename(os.path.join(Source_Path, filename), os.path.join(Destination, dst))
# Driver Code
if __name__ == '__main__':
main()

As per #daniel's comment, os.listdir() returns just the filenames and not the full path of the file. Use os.path.join(path, file) to get the full path and rename that.
import os
path = 'C:\\Users\\Admin\\Desktop\\Jayesh'
files = os.listdir(path)
for file in files:
os.rename(os.path.join(path, file), os.path.join(path, 'xyz_' + file + '.csv'))

Just playing with the accepted answer define the path variable and list:
path = "/Your/path/to/folder/"
files = os.listdir(path)
and then loop over that list:
for index, file in enumerate(files):
#print (file)
os.rename(path+file, path +'file_' + str(index)+ '.jpg')
or loop over same way with one line as python list comprehension :
[os.rename(path+file, path +'jog_' + str(index)+ '.jpg') for index, file in enumerate(files)]
I think the first is more readable, in the second the first part of the loop is just the second part of the list comprehension

If your files are renaming in random manner then you have to sort the files in the directory first. The given code first sort then rename the files.
import os
import re
path = 'target_folder_directory'
files = os.listdir(path)
files.sort(key=lambda var:[int(x) if x.isdigit() else x for x in re.findall(r'[^0-9]|[0-9]+', var)])
for i, file in enumerate(files):
os.rename(path + file, path + "{}".format(i)+".jpg")

I wrote a quick and flexible script for renaming files, if you want a working solution without reinventing the wheel.
It renames files in the current directory by passing replacement functions.
Each function specifies a change you want done to all the matching file names. The code will determine the changes that will be done, and displays the differences it would generate using colors, and asks for confirmation to perform the changes.
You can find the source code here, and place it in the folder of which you want to rename files https://gist.github.com/aljgom/81e8e4ca9584b481523271b8725448b8
It works in pycharm, I haven't tested it in other consoles
The interaction will look something like this, after defining a few replacement functions
when it's running the first one, it would show all the differences from the files matching in the directory, and you can confirm to make the replacements or no, like this

This works for me and by increasing the index by 1 we can number the dataset.
import os
path = '/Users/myName/Desktop/directory'
files = os.listdir(path)
index=1
for index, file in enumerate(files):
os.rename(os.path.join(path, file),os.path.join(path,''.join([str(index),'.jpg'])))
index = index+1
But if your current image name start with a number this will not work.

How to rename a file using Python

I want to change a.txt to b.kml.

Use os.rename:
import os
os.rename('a.txt', 'b.kml')
Usage:
os.rename('from.extension.whatever','to.another.extension')

File may be inside a directory, in that case specify the path:
import os
old_file = os.path.join("directory", "a.txt")
new_file = os.path.join("directory", "b.kml")
os.rename(old_file, new_file)

As of Python 3.4 one can use the pathlib module to solve this.
If you happen to be on an older version, you can use the backported version found here
Let's assume you are not in the root path (just to add a bit of difficulty to it) you want to rename, and have to provide a full path, we can look at this:
some_path = 'a/b/c/the_file.extension'
So, you can take your path and create a Path object out of it:
from pathlib import Path
p = Path(some_path)
Just to provide some information around this object we have now, we can extract things out of it. For example, if for whatever reason we want to rename the file by modifying the filename from the_file to the_file_1, then we can get the filename part:
name_without_extension = p.stem
And still hold the extension in hand as well:
ext = p.suffix
We can perform our modification with a simple string manipulation:
Python 3.6 and greater make use of f-strings!
new_file_name = f"{name_without_extension}_1"
Otherwise:
new_file_name = "{}_{}".format(name_without_extension, 1)
And now we can perform our rename by calling the rename method on the path object we created and appending the ext to complete the proper rename structure we want:
p.rename(Path(p.parent, new_file_name + ext))
More shortly to showcase its simplicity:
Python 3.6+:
from pathlib import Path
p = Path(some_path)
p.rename(Path(p.parent, f"{p.stem}_1_{p.suffix}"))
Versions less than Python 3.6 use the string format method instead:
from pathlib import Path
p = Path(some_path)
p.rename(Path(p.parent, "{}_{}_{}".format(p.stem, 1, p.suffix))

import shutil
shutil.move('a.txt', 'b.kml')
This will work to rename or move a file.

os.rename(old, new)
This is found in the Python docs: http://docs.python.org/library/os.html

As of Python version 3.3 and later, it is generally preferred to use os.replace instead of os.rename so FileExistsError is not raised if the destination file already exists.
assert os.path.isfile('old.txt')
assert os.path.isfile('new.txt')
os.rename('old.txt', 'new.txt')
# Raises FileExistsError
os.replace('old.txt', 'new.txt')
# Does not raise exception
assert not os.path.isfile('old.txt')
assert os.path.isfile('new.txt')
See the documentation.

Use os.rename. But you have to pass full path of both files to the function. If I have a file a.txt on my desktop so I will do and also I have to give full of renamed file too.
os.rename('C:\\Users\\Desktop\\a.txt', 'C:\\Users\\Desktop\\b.kml')

One important point to note here, we should check if any files exists with the new filename.
suppose if b.kml file exists then renaming other file with the same filename leads to deletion of existing b.kml.
import os
if not os.path.exists('b.kml'):
os.rename('a.txt','b.kml')

import os
# Set the path
path = 'a\\b\\c'
# save current working directory
saved_cwd = os.getcwd()
# change your cwd to the directory which contains files
os.chdir(path)
os.rename('a.txt', 'b.klm')
# moving back to the directory you were in
os.chdir(saved_cwd)

Using the Pathlib library's Path.rename instead of os.rename:
import pathlib
original_path = pathlib.Path('a.txt')
new_path = original_path.rename('b.kml')

Here is an example using pathlib only without touching os which changes the names of all files in a directory, based on a string replace operation without using also string concatenation:
from pathlib import Path
path = Path('/talend/studio/plugins/org.talend.designer.components.bigdata_7.3.1.20200214_1052\components/tMongoDB44Connection')
for p in path.glob("tMongoDBConnection*"):
new_name = p.name.replace("tMongoDBConnection", "tMongoDB44Connection")
new_name = p.parent/new_name
p.rename(new_name)

import shutil
import os
files = os.listdir("./pics/")
for key in range(0, len(files)):
print files[key]
shutil.move("./pics/" + files[key],"./pics/img" + str(key) + ".jpeg")
This should do it. python 3+

How to change the first letter of filename in a directory:
import os
path = "/"
for file in os.listdir(path):
os.rename(path + file, path + file.lower().capitalize())
then = os.listdir(path)
print(then)

If you are Using Windows and you want to rename your 1000s of files in a folder then:
You can use the below code. (Python3)
import os
path = os.chdir(input("Enter the path of the Your Image Folder : ")) #Here put the path of your folder where your images are stored
image_name = input("Enter your Image name : ") #Here, enter the name you want your images to have
i = 0
for file in os.listdir(path):
new_file_name = image_name+"_" + str(i) + ".jpg" #here you can change the extention of your renmamed file.
os.rename(file,new_file_name)
i = i + 1
input("Renamed all Images!!")

os.chdir(r"D:\Folder1\Folder2")
os.rename(src,dst)
#src and dst should be inside Folder2

import os
import re
from pathlib import Path
for f in os.listdir(training_data_dir2):
for file in os.listdir( training_data_dir2 + '/' + f):
oldfile= Path(training_data_dir2 + '/' + f + '/' + file)
newfile = Path(training_data_dir2 + '/' + f + '/' + file[49:])
p=oldfile
p.rename(newfile)

You can use os.system to invoke terminal to accomplish the task:
os.system('mv oldfile newfile')