Develop Reference

python slice a string and use .find to get last three letters to determine the type of file

what i am asking is how can i correct my if statement, essentially i have a file attachment called 'fileName' and i am trying to get the last 3 letters from that file to determine if that type of file is in my config (csv, txt).
valid_filename = myconfig file (csv, txt)
def load_file():
try:
# get file from read email and assign a directory path (attachments/file)
for fileName in os.listdir(config['files']['folder_path']):
# validate the file extension per config
# if valid_filename: else send email failure
valid_filename = config['valid_files']['valid']
if fileName[-3:].find(valid_filename):
file_path = os.path.join(config['files']['folder_path'], fileName)
# open file path and read it as csv file using csv.reader
with open(file_path, "r") as csv_file:
csvReader = csv.reader(csv_file, delimiter=',')
first_row = True
let me know if i can clarify anything better

Try pathlib, for example
Assuming the config file is in the form:
[valid_files]
valid = .csv, .txt
[files]
forder_path = .
# other imports
import pathlib
def load_file():
valid_suffixes = [e.strip() for e in config['valid_files']['valid'].split(",")]
folder_path = config['files']['folder_path']
for fileName in os.listdir(folder_path):
if pathlib.Path(filename).suffix in valid_suffixes:
file_path = os.path.join(folder_path, fileName)
with open(file_path, "r") as csv_file:
...

find() method returns -1 if the string you're searching for is not found. To check if the element exists in the string, check if find returns -1.

Is there a way to read and write on multiple text files in a folder using Python?

I am trying to retrieve several text files from a folder. Afterwards, I am trying to read all of the files within the directory to which I then append a blank line at the top of each file.
However, once I run the program it does not execute what I desire. This is the code:
import os
folderPath = "./textFiles"
def myFilesAddEmptyLine():
for file in os.listdir(folderPath):
if file.endswith(".txt"):
with open(file, "r+") as myFile:
# print(myFile)
# ^ This returns "<_io.TextIOWrapper name='test.txt' mode='r+' encoding='cp1252'>" in the console.
fileContent = myFile.read()
myFile.seek(0, 0)
myFile.write("\n" + fileContent)
myFilesAddEmptyLine()
On the other hand, if I read the file directly without trying to automate the process using os, it executes what I am trying to achieve flawlessly. Therefore, the following piece of code opens the file and appends a blank line at the top of the file.
def myFilesAddEmptyLine():
with open("test.txt", "r+") as myFile:
fileContent = myFile.read()
myFile.seek(0, 0)
myFile.write("\n" + fileContent)
myFilesAddEmptyLine()
Could anyone kindly outline what the issue with the first piece of code is? Thanks in advance!

As user #asylumax pointed out in the comments, this:
import os
folderPath = "./textFiles"
def myFilesAddEmptyLine():
for file in os.listdir(folderPath):
if file.endswith(".txt"):
with open(file, "r+") as myFile:
fileContent = myFile.read()
myFile.seek(0, 0)
myFile.write("\n" + fileContent)
myFilesAddEmptyLine()
Needed to be changed to this:
import os
folderPath = "./textFiles"
def myFilesAddEmptyLine():
for file in os.listdir(folderPath):
if file.endswith(".txt"):
with open(os.path.join(folderPath, file), "r+") as myFile: #This is the line that needed changing.
fileContent = myFile.read()
myFile.seek(0, 0)
myFile.write("\n" + fileContent)
print(myFile)
myFilesAddEmptyLine()

Python wont write text file

I write:
f = open("Textfile.txt","w")
f.write("This is a text file")
f.close()
But when i open the text file nothing has been written, does anybody know why?

using it in a with statement should handle the closing of the file for you. Does this solve the problem?
with open('Textfile.txt','w') as f:
f.write('This is a text file')

To save a file at the desired location use the full path.
with open('/home/me/project/testfile.txt', 'w') as outfile:
outfile.write('content')
If you want to save your file in the same directory as your script you can use __file__.
import os.path
path_of_script = os.path.dirname(__file__)
with open(os.path.join(path_of_script, 'testfile.txt', 'w') as outfile:
outfile.write('content')

Error when trying to read and write multiple files

I modified the code based on the comments from experts in this thread. Now the script reads and writes all the individual files. The script reiterates, highlight and write the output. The current issue is, after highlighting the last instance of the search item, the script removes all the remaining contents after the last search instance in the output of each file.
Here is the modified code:
import os
import sys
import re
source = raw_input("Enter the source files path:")
listfiles = os.listdir(source)
for f in listfiles:
filepath = source+'\\'+f
infile = open(filepath, 'r+')
source_content = infile.read()
color = ('red')
regex = re.compile(r"(\b be \b)|(\b by \b)|(\b user \b)|(\bmay\b)|(\bmight\b)|(\bwill\b)|(\b's\b)|(\bdon't\b)|(\bdoesn't\b)|(\bwon't\b)|(\bsupport\b)|(\bcan't\b)|(\bkill\b)|(\betc\b)|(\b NA \b)|(\bfollow\b)|(\bhang\b)|(\bbelow\b)", re.I)
i = 0; output = ""
for m in regex.finditer(source_content):
output += "".join([source_content[i:m.start()],
"<strong><span style='color:%s'>" % color[0:],
source_content[m.start():m.end()],
"</span></strong>"])
i = m.end()
outfile = open(filepath, 'w+')
outfile.seek(0)
outfile.write(output)
print "\nProcess Completed!\n"
infile.close()
outfile.close()
raw_input()

The error message tells you what the error is:
No such file or directory: 'sample1.html'
Make sure the file exists. Or do a try statement to give it a default behavior.

The reason why you get that error is because the python script doesn't have any knowledge about where the files are located that you want to open.
You have to provide the file path to open it as I have done below. I have simply concatenated the source file path+'\\'+filename and saved the result in a variable named as filepath. Now simply use this variable to open a file in open().
import os
import sys
source = raw_input("Enter the source files path:")
listfiles = os.listdir(source)
for f in listfiles:
filepath = source+'\\'+f # This is the file path
infile = open(filepath, 'r')
Also there are couple of other problems with your code, if you want to open the file for both reading and writing then you have to use r+ mode. More over in case of Windows if you open a file using r+ mode then you may have to use file.seek() before file.write() to avoid an other issue. You can read the reason for using the file.seek() here.

Replace and overwrite instead of appending

I have the following code:
import re
#open the xml file for reading:
file = open('path/test.xml','r+')
#convert to string:
data = file.read()
file.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>",data))
file.close()
where I'd like to replace the old content that's in the file with the new content. However, when I execute my code, the file "test.xml" is appended, i.e. I have the old content follwed by the new "replaced" content. What can I do in order to delete the old stuff and only keep the new?

You need seek to the beginning of the file before writing and then use file.truncate() if you want to do inplace replace:
import re
myfile = "path/test.xml"
with open(myfile, "r+") as f:
data = f.read()
f.seek(0)
f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>", r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", data))
f.truncate()
The other way is to read the file then open it again with open(myfile, 'w'):
with open(myfile, "r") as f:
data = f.read()
with open(myfile, "w") as f:
f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>", r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", data))
Neither truncate nor open(..., 'w') will change the inode number of the file (I tested twice, once with Ubuntu 12.04 NFS and once with ext4).
By the way, this is not really related to Python. The interpreter calls the corresponding low level API. The method truncate() works the same in the C programming language: See http://man7.org/linux/man-pages/man2/truncate.2.html

file='path/test.xml'
with open(file, 'w') as filetowrite:
filetowrite.write('new content')
Open the file in 'w' mode, you will be able to replace its current text save the file with new contents.

Using truncate(), the solution could be
import re
#open the xml file for reading:
with open('path/test.xml','r+') as f:
#convert to string:
data = f.read()
f.seek(0)
f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>",data))
f.truncate()

import os#must import this library
if os.path.exists('TwitterDB.csv'):
os.remove('TwitterDB.csv') #this deletes the file
else:
print("The file does not exist")#add this to prevent errors
I had a similar problem, and instead of overwriting my existing file using the different 'modes', I just deleted the file before using it again, so that it would be as if I was appending to a new file on each run of my code.

See from How to Replace String in File works in a simple way and is an answer that works with replace
fin = open("data.txt", "rt")
fout = open("out.txt", "wt")
for line in fin:
fout.write(line.replace('pyton', 'python'))
fin.close()
fout.close()

in my case the following code did the trick
with open("output.json", "w+") as outfile: #using w+ mode to create file if it not exists. and overwrite the existing content
json.dump(result_plot, outfile)

Using python3 pathlib library:
import re
from pathlib import Path
import shutil
shutil.copy2("/tmp/test.xml", "/tmp/test.xml.bak") # create backup
filepath = Path("/tmp/test.xml")
content = filepath.read_text()
filepath.write_text(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", content))
Similar method using different approach to backups:
from pathlib import Path
filepath = Path("/tmp/test.xml")
filepath.rename(filepath.with_suffix('.bak')) # different approach to backups
content = filepath.read_text()
filepath.write_text(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", content))