Strange behaviour with os.join() - python

I've noticed odd behaviour with Pythons os.join(). in that I'm adding a year and filename to a path. Here's my code.
#!/usr/bin/env python
import os
#------------------------------------------------
def file_walk(root, ext):
# Walk file with me, Laura Palmer!
fList = []
for current, dirs, files in os.walk(root):
for file in files:
fname = os.path.join(current, file) # this works fine, yeah!
src = os.path.isfile(fname)
if src:
if fname.endswith(ext):
fList.append(fname)
return fList
myFolder = r"d:\temp\test"
myExt = ".html"
myYear = "2019"
allfiles = file_walk(myFolder, myExt)
for theFile in allfiles:
sourceFile = theFile
destinFile = os.path.join(myFolder, myYear, theFile)
print sourceFile
print destinFile
print
myFile = "bookmarks_06_05_2019.html"
print os.path.join(myFolder, myYear, myFile)
# EoF
As strings, they work fine (see last line), but as paths, not so well :(
Output I'm getting from print destinFile
d:\temp\test\bookmarks_01_26_2018.html
d:\temp\test\bookmarks_05_06_2014.html
d:\temp\test\bookmarks_06_05_2019.html
I'm expecting the follow:
d:\temp\test\2019\bookmarks_01_26_2018.html
d:\temp\test\2019\bookmarks_05_06_2014.html
d:\temp\test\2019\bookmarks_06_05_2019.html
Can anyone point me in the right direction of where I'm going wrong?

theFile is an absolute file path. If all you want from it is the base name, use:
destinFile = os.path.join(myFolder, myYear, os.path.basename(theFile))
Note that os.path.join returns the last absolute argument with any relative arguments after that one combined in a path. This is why the result didn't have the 2019 component.

Related

python file name change

I am trying to change file names like below:
the 000000 are the same number.
000000_ABC.png --->000000+1_ABC.png
000000_DEF.png --->000000+2_DEF.png
000000_GHI.png --->000000+3_GHI.png
000000_JKL.png --->000000+4_JKL.png
In order to do so, I wrote code like below.
img_files = os.listdir(PATH_TO_PNG_FILES)
for img_file, i in zip(img_files, range(len(img_files))):
new_img_file = img_file.replace("_", "+"+str(i)+"_")
os.rename(path + img_file, path + new_img_file)
There are more than just four files and more of similar lines.
The problem is that immediately after running pycharm, it successfully produces the desired results, but after I run another page related to the result directories, the results continue to be changed like below even after the process finished. I do not understand why.
000000+1+1_ABC.png
000000+2+2_DEF.png
000000+3+3_GHI.png
000000+4+4_JKL.png
or
otherwise "+unexpected number"
This is because the other directory may already contain file in the format of "000000+1_ABC.png" and your script is changing _ to "+1_" resulting in "000000+1+1_ABC.png". To solve this you can add a if statement to check it should not contain "+" symbol.
img_files = os.listdir(path inside of which the png files are saved)
for img_file, i in zip(img_files, range(len(img_files))):
if not ("+" in img_file):
new_img_file = img_file.replace("_", "+"+str(i)+"_")
os.rename(path + img_file, path + new_img_file)
A simple and naive way would be to add a verification to check whether there is a '+' in the filename. If you have other files which may contain a +, you may have to check for a stricter pattern.
I made a YouTube video https://youtu.be/K9jhAPZLZLc on how to rename multiple files like the one you have assuming all your files are in the same directory.
To answer your question. assuming all image files are in the same folder.
path = 'C:\\Users\\USER\\Desktop\\rename_images\\images\\' # path to your images
files = os.listdir(path)
for count, filename in enumerate(files):
# Get the file extension
file, file_extension = os.path.splitext(filename)
# check if the current file is a folder or not
full_path = f'{path}{filename}'
if os.path.isdir(full_path):
print('This is a directory')
elif os.path.isfile(full_path):
print('This is a normal file')
# Rename
if not '+' in file:
try:
file_split = file.split('_')
zeros = file_split[0]
alpha = file_split[-1]
current_file_name = os.path.join(path, filename)
new_file_name = os.path.join(path, ''.join([f'{zeros}+{count}_{alpha}', file_extension]))
os.rename(current_file_name, new_file_name)
except:
pass
else:
pass
else:
print('This is a special file')
I would imagine that the problem comes from modifying the name insted of overwriting.
import os
DIR_PATH = 'files'
def rename_files(dir_name):
img_files = os.listdir(dir_name)
for i in range(len(img_files)):
file_name = img_files[i].split('_')[-1]
file_name = '000000+{0}_{1}'.format(i, file_name)
os.rename(
os.path.join(dir_name, img_files[i]),
os.path.join(dir_name, file_name)
)
if __name__ == '__main__':
rename_files(DIR_PATH)

Need to upload sub-dirs and their contents, not just files in current dir

A script was supplied to me in order to upload files to a cloud bucket. You input the dir where the files you want to upload are and bingo bango, done.
What needs to happen is that there are additional sub dirs with their own files in them that I would like to transfer as well based on the input of the root dir. They would need to retain their tree structure relative to the root dir input.
Using the current code I get a write error/access denied fail. I know this is because the for loop is using os.listdir which can't parse the extra sub dirs and files but I'm not sure how to modify.
I attempted to get all the information I needed using os.walk and parsing that out. I verified with some print tests that it was looking in the right place for everything. However I hit a wall when I got this error when running the script:
folder\folder\lib\ntpath.py", line 76, in join
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not list
I understand that something is being generated as a list when it shouldn't be but I'm not sure how to go about this...
This is the original script provided to me below. I have added the variable at the top just to be a little less abstract.
local_directory_path = 'C:\folder\folder\sync\FROM_LOCAL_UPLOAD'
def upload_folder_to_cloud(self, mount_id, local_directory_path):
''' This method will list every file at the local_directory_path and then for each,
it will call the api method athera.sync.upload_file for every file in your local directory
'''
_, destination_folder = os.path.split(local_directory_path)
if not destination_folder:
self.logger.error("Make sure the provided 'local_directory_path' does not end with a '/' or a '\\'")
sys.exit(2)
destination_folder = destination_folder + "/"
self.logger.info("Folder = {}".format(destination_folder))
for filename in os.listdir(local_directory_path):
destination_path = destination_folder + filename
filepath = os.path.join(local_directory_path, filename)
with open(filepath, "rb") as f:
_, err = self.client.upload_file(self.group_id, mount_id, f, destination_path=destination_path,)
if err != None:
self.logger.error(err)
sys.exit(4)
return destination_folder
This is what I modified it to as a test:
for root, dirs, files in os.walk(local_directory_path):
srcFile = (os.path.join(files))
srcRoot = (os.path.join(root))
rootSplit = os.path.normpath(srcRoot).split(os.path.sep)
srcDirs = '/'.join(rootSplit[4:])
src = str('fixLocalFolder') + '/' + str(srcDirs) +'/'+ (files)
dst = str(srcDirs) + '/' + (files)
destination_folder = str(srcRoot) + "/"
destination_path = str(destination_folder) + str(srcFile)
filepath = os.path.join((str(srcDirs), str(srcFile)))
with open(filepath, "rb") as f:
_, err = self.client.upload_file(
self.group_id,
mount_id,
f,
destination_path=destination_path,
)
if err != None:
self.logger.error(err)
sys.exit(4)
return destination_folder
I do not code for a living so I am sure I am not going about this the right way. I apologize for any code atrocities in advance. Thank you!
I do see some issues in that code, even without testing it. Something like the following might work for that loop. (Note! Untested!).
for root, dirs, files in os.walk(local_directory_path):
# Iterate through files in the currently processed directory
for current_file in files:
# Full path to file
src_file = os.path.join(root, current_file)
# Get the sub-path relative the original root.
sub_path = os.path.relpath(root, start=destination_folder)
# Get the destination path
destination_path = os.path.join(sub_path, current_file)
with open(src_file, "rb") as f:
_, err = self.client.upload_file(
self.group_id,
mount_id,
f,
destination_path=destination_path,
)
if err != None:
self.logger.error(err)
sys.exit(4)
I believe your central problem was misunderstanding what os.walk gives you. It gives you listing of each directory (and subdirectory), one after another.
Thus the values of one iterations might look like (when listing /mydir):
# First iteration:
root = "/mydir"
dirs = ["subdir", ...]
files = ["something.doc", "something else.txt"]
# Second iteration:
root = "/mydir/subdir"
dirs = ["sub-sub-dir1", ...]
files = ["file1.txt", "file2.txt", ...]

How to rename a bunch of files using python?

I am a complete beginner in python. I need to rename a bunch of files with dates in the name. The names all look like:
front 7.25.16
left 7.25.16
right 7.25.16
I would like them to start with the date rather then front, left, or right, so that front 7.25.16 becomes 7.25.16 front.
I have tried using regular expressions and os.walk and I have run into troubles with both. Right now I am just trying to print the file names to prove os.walk is working. Right now my code looks like this:
import re, shutil, os
K = re.compile(r"(\d+.\d+.\d+)")
RE_Date = K.search("front 7.25.16")
for root, dirs, filenames in os.walk("path"):
for filename in filenames:
print ("the filename is: " + filename)
print ("")
Any advice would be greatly appreciated.
Check this example to rename file as per your need.
import os
filenames = ["front 7.25.16.jpg", "left 7.25.16.jpg", "right 7.25.16.jpg"]
for file_name in filenames:
x = file_name.split(' ')[0]
y = file_name.split(' ')[1]
new_name = '{} {}{}'.format(os.path.splitext(y)[0], x, os.path.splitext(y)[-1])
print new_name
output:
7.25.16 front.jpg
7.25.16 left.jpg
7.25.16 right.jpg
In your code your can use os.rename for rename files
import os
for root, dirs, filenames in os.walk("path"):
for file_name in filenames:
x = file_name.split(' ')[0]
y = file_name.split(' ')[1]
new_name = '{} {}{}'.format(os.path.splitext(y)[0], x, os.path.splitext(y)[-1])
file_path = os.path.join(root, file_name)
new_path = os.path.join(root, new_name)
os.rename(file_name, new_path)

Python: Read multiple files and move them to a directory according to their content

I am quite new to python, but I would like to use it for the following tasks:
read all files in a directory
look for a specific character in all lines of the files
if this character is present only once in the file copy the file in a specific directory.
I tried the following code:
#! /usr/bin/python
import glob
import shutil
path = '/xxxx/Dir/*.txt'
files=glob.glob(path)
for file in files:
f=open(file)
f.read()
total = 0
for line in f:
if "*TPR_4*" in line:
total_line = total + 1
if total_line == 1:
shutil.copy(f, 'xxxx/Test/')
f.close()
However, it is not working.
Any suggestion?
shutil.copy() takes file names as arguments not open files. You should change your call:
shutil.copy(file, 'xxxx/Test/')
Also: file is a poor name choice. It's a built-in function's name.
The logic is not quite correct, also you are mixing up total and total_line and shutil.copy takes the name, not the object as an argument. And note that the if .. in line does not use globbing syntax, i.e. to search for TPR_4, use 'TPR_4', not '*TPR_4*'. Try the following:
#! /usr/bin/python
import glob
import shutil
path = '/xxxx/Dir/*.txt'
files=glob.glob(path)
for file in files:
f=open(file)
total = 0
for line in f:
if "TPR_4" in line:
total += 1
if total > 1:
break # no need to go through the file any further
f.close()
if total == 1:
shutil.copy(file, 'xxxx/Test/')
I wrote some code for your question, maybe it's good for you.
import os, shutil
dir_path = '/Users/Bob/Projects/Demo'
some_char = 'abc'
dest_dir = "/Users/Bob/tmp"
for root, dirs, files in os.walk(dir_path):
for _file in files:
file_path = os.path.join(root, _file)
copy = False
with open(file_path, 'r') as f:
while True:
line = f.readline()
if not line:
break
if str(line).find(some_char) > -1:
copy = True
break
if copy:
shutil.copy(file_path, dest_dir)
print file_path, ' copy...'

Renaming file in same directory using Python

So I'm trying to iterate through a list of files that are within a subfolder named eachjpgfile and change the file from doc to the subfolder eachjpgfile mantaining the file's name but when I do this it adds the file to directory before eachjpgfile rather than keeping it in it. Looking at the code below, can you see why is it doing this and how can I keep it in the eachjpgfile directory?
Here is the code:
for eachjpgfile in filelist:
os.chdir(eachjpgfile)
newdirectorypath = os.curdir
list_of_files = os.listdir(newdirectorypath)
for eachfile in list_of_files:
onlyfilename = os.path.splitext(eachfile)[0]
if onlyfilename == 'doc':
newjpgfilename = eachfile.replace(onlyfilename,eachjpgfile)
os.rename(eachfile, newjpgfilename)
There is a lot of weird stuff going on in here, but I think the one that's causing your particular issue is using 'eachjpgfile' in 'eachfile.replace'.
From what I can tell, the 'eachjpgfile' you're passing in is a full-path, so you're replacing 'doc' in the filename with '/full/path/to/eachjpgfile', which puts it parallel to the 'eachjpgfile' directory regardless of your current working directory.
You could add a line to split the path/file names prior to the replace:
for eachjpgfile in filelist:
os.chdir(eachjpgfile)
newdirectorypath = os.curdir
list_of_files = os.listdir(newdirectorypath)
for eachfile in list_of_files:
onlyfilename = os.path.splitext(eachfile)[0]
if onlyfilename == 'doc':
root, pathName= os.path.split(eachjpgfile) #split out dir name
newjpgfilename = eachfile.replace(onlyfilename,pathName)
os.rename(eachfile, newjpgfilename)
which is a very dirty fix for a very dirty script. :)
try this:
import os
path = '.'
recursive = False # do not descent into subdirs
for root,dirs,files in os.walk( path ) :
for name in files :
new_name = name.replace( 'aaa', 'bbb' )
if name != new_name :
print name, "->", new_name
os.rename( os.path.join( root, name),
os.path.join( root, new_name ) )
if not recursive :
break

Categories

Resources