I'm having a problem reading a logfile I'm creating in another method.
The logfile has file-paths of unzipped files in it (unpacking is in another function). The logfile looks kinda like this
/home/usr/Downloads/outdir/Code/XXX.something
/home/usr/Downloads/outdir/Code/YYY.something
/home/usr/Downloads/outdir/Code/AAA.something
#staticmethod
def iterate_log(path):
results = str()
for dirpath, dirnames, files in os.walk(path):
for name in files:
results += '%s\n' % os.path.join(dirpath, name)
with open(os.path.join(EXTRACT_PATH_, LOGNAME_), 'w') as logfile:
logfile.write(results)
return os.path.join(EXTRACT_PATH_, LOGNAME_)
#staticmethod
def read_received_files(from_file):
with open(from_file, 'r') as data:
data = data.readlines()
for lines in data:
# to reduce confusion I would use
# for line in data:
read_files = open(lines)
print(read_files)
return read_files
Now I want to read the logfile line by line (the parameter from_files in the second method is the return value of the first method right now) and open those files and return them (for using them elsewhere).
readlines() and read() are both giving me errors so far
readlines() = [Errno2] No sucht file or directory: '/../../../logfile.log\n
read() = IsADirectoryError: [Errno21]
whole traceback:
Traceback (most recent call last):
File "files_received_test.py", line 13, in <module>
main()
File "files_received_test.py", line 9, in main
j = Filesystem.execute_code(FILENAME_)
File "/home/usr/PycharmProjects/Projektgruppe/code/src/filesystem_hasher.py", line 16, in execute_code
Filesystem.read_received_files(create_log)
File "/home/usr/PycharmProjects/Projektgruppe/code/src/filesystem_hasher.py", line 54, in read_received_files
read_files = open(lines)
FileNotFoundError: [Errno 2] No such file or directory: '/home/usr/Downloads/outdir/unpacked_files.log\n'
You just need to strip the newline character so change
read_files = open(lines)
to
read_files = open(lines.strip())
as an observation - critical to read each character in an error message - the message was telling you that there was no file named
'/home/usr/Downloads/outdir/unpacked_files.log\n'
so it is useful to then try to understand why the \n showed up - this did not match your expectation of the file name so you should wonder why the file has that name
Related
I have looked at How to open a list of files in Python This problem similar but not covered.
path = "C:\\test\\test5\\"
files = os.listdir(path)
fileNames = []
for f in files:
fileNames.append(f)
for fileName in fileNames:
pathFileName = path + fileName
print(f"This is the path: {pathFileName}")
fin = open(pathFileName, 'rt')
texts = []
with open(fileName) as file_in:
# read file text lines into an array
for text in file_in:
texts.append(text)
for text in texts:
print(text)
The file aaaatest.txt is in C:\test\test5 The output is:
This is the path: C:\test\test5\aaaatest.txt
Traceback (most recent call last):
File "c:\Users\david\source\repos\python-street-spell\diffLibFieldFix.py", line 30, in <module>
with open(fileName) as file_in:
FileNotFoundError: [Errno 2] No such file or directory: 'aaaatest.txt'
So here's the point. If I take a copy of aaaatest.txt (leaving original where it is) and put it in the current working directory. Running the script again I get:
This is the path: C:\test\test5\aaaatest.txt
A triple AAA test
This is the path: C:\test\test5\AALTONEN-ALLAN_PENCARROW_PAGE_1.txt
Traceback (most recent call last):
File "c:\Users\david\source\repos\python-street-spell\diffLibFieldFix.py", line 30, in <module>
with open(fileName) as file_in:
FileNotFoundError: [Errno 2] No such file or directory: 'AALTONEN-ALLAN_PENCARROW_PAGE_1.txt'
The file aaaatest.txt is opened and the single line of text, contained in it, is outputted. Following this an attempt is made to open the next file of C:\test\test5 where the same error occurs again.
Seems to me that while the path is saying C:\test\test5 the file is only being read from the cwd?
I am trying to open a txt file for reading with this code:-
type_comments = [] #Declare an empty list
with open ('society6comments.txt', 'rt') as in_file: #Open file for reading of text data.
for line in in_file: #For each line of text store in a string variable named "line", and
type_comments.append(line.rstrip('\n')) #add that line to our list of lines.
Error:-
Error - Traceback (most recent call last):
File "c:/Users/sultan/python/society6/society6_promotion.py", line 6, in <module>
with open ('society6comments.txt', 'rt') as in_file:
FileNotFoundError: [Errno 2] No such file or directory: 'society6comments.txt'
I already have a file name with 'society6comments.txt' in the same directory has my script so why is it showing error?
The fact that the text file is in the same directory as your program does not make that directory the current working directory. Put the full path to the file in your open() call.
You can use os.path.dirname(__file__) to obtain the directory name of the script, and then join the file name you want:
import os
with open (os.path.join(os.path.dirname(os.path.abspath(__file__)), 'society6comments.txt'), 'rt') as in_file:
I'm trying to use glob.glob to provide support for more than one filetype. The code I have is supposed to take files with the extensions '.pdf', '.xls', and '.xlsx' residing in the directory '/mnt/Test' and execute the code below after files matching have been found.
When I replace the existing for loop with just
for filename in glob.glob("*.xlsx"):
print filename
It works just fine.
When attempting to run the following code:
def main():
os.chdir("/mnt/Test")
extensions = ("*.xls", ".xlsx", ".pdf")
filename = []
for files in extensions:
filename.extend(glob.glob(files))
print filename
sys.stdout.flush()
doc_id, version = doc_placeholder(filename)
print 'doc_id:', doc_id, 'version:', version
workspace_upload(doc_id, version, filename)
print "%s has been found. Preparing next phase..." % filename
ftp_connection.cwd(remote_path)
fh = open(filename, 'rb')
ftp_connection.storbinary('STOR %s' % timestr + '_' + filename, fh)
fh.close()
send_email(filename)
I run across the following error:
Report /mnt/Test/fileTest.xlsx has been added.
[]
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/dist- packages/watchdog/observers/api.py", line 199, in run
self.dispatch_events(self.event_queue, self.timeout)
File "/usr/local/lib/python2.7/dist- packages/watchdog/observers/api.py", line 368, in dispatch_events
handler.dispatch(event)
File "/usr/local/lib/python2.7/dist-packages/watchdog/events.py", line 330, in dispatch
_method_map[event_type](event)
File "observe.py", line 14, in on_created
fero.main()
File "/home/tesuser/project-a/testing.py", line 129, in main
doc_id, version = doc_placeholder(filename)
File "/home/testuser/project-a/testing.py", line 58, in doc_placeholder
payload = {'documents':[{'document':{'name':os.path.splitext(filename)[0],'parentId':parent_id()}}]}
File "/usr/lib/python2.7/posixpath.py", line 105, in splitext
return genericpath._splitext(p, sep, altsep, extsep)
File "/usr/lib/python2.7/genericpath.py", line 91, in _splitext
sepIndex = p.rfind(sep)
AttributeError: 'list' object has no attribute 'rfind'
How can I edit the code above to achieve what I need?
Thanks in advance, everyone. Appreciate the help.
doc_placeholder includes this snippet, os.path.splitext(filename). Assuming filename is the list you passed in you've given a list to os.path.splittext when it is expecting a string.
Fix this by iterating over each filename instead of trying to process the entire list at once.
def main():
os.chdir("/mnt/Test")
extensions = ("*.xls", "*.xlsx", "*.pdf")
filenames = [] # made 'filename' plural to indicate it's a list
# building list of filenames moved to separate loop
for files in extensions:
filenames.extend(glob.glob(files))
# iterate over filenames
for filename in filenames:
print filename
sys.stdout.flush()
doc_id, version = doc_placeholder(filename)
print 'doc_id:', doc_id, 'version:', version
workspace_upload(doc_id, version, filename)
print "%s has been found. Preparing next phase..." % filename
ftp_connection.cwd(remote_path)
fh = open(filename, 'rb')
ftp_connection.storbinary('STOR %s' % timestr + '_' + filename, fh)
fh.close()
send_email(filename)
I have a folder full of files in the following format:
temp0.txt
temp1.txt
temp3.txt
.
..
temp999.txt
...
The second line of each of these files contains the string I want to rename each file to respectively. To be clear, if "temp0.txt" contains "textfile0" in the second line, I want "temp0.txt" to be renamed to "textfile0.txt". Similarly, if "temp999.txt" contains "textfile123" in the second line, I want "temp999.txt" to be renamed to "textfile123.txt".
The following is what I have so far, but it doesn't work.
import os, linecache
for filename in os.listdir("."):
with open(filename) as openfile:
firstline = linecache.getline(openfile, 2)
os.rename(filename, firstline.strip()+".txt")
Any help would be greatly appreciated!
The error I receive is as follows:
Traceback (most recent call last):
File "rename_ZINC.py", line 5, in <module>
firstline = linecache.getline(openfile, 2)
File "/usr/lib64/python2.7/linecache.py", line 14, in getline
lines = getlines(filename, module_globals)
File "/usr/lib64/python2.7/linecache.py", line 40, in getlines
return updatecache(filename, module_globals)
File "/usr/lib64/python2.7/linecache.py", line 75, in updatecache
if not filename or (filename.startswith('<') and filename.endswith('>')):
AttributeError: 'file' object has no attribute 'startswith'
Try using the builtin openfile.readline() instead of linecache to get the necessary line.
Just to tell you where you are going wrong.
linecache requires a filename as the first argument (as a string) , not the complete file obect. From documentation -
linecache.getline(filename, lineno[, module_globals])
Get line lineno from file named filename. This function will never raise an exception — it will return '' on errors (the terminating newline character will be included for lines that are found).
So you should not open the file and then pass in the file object, instead you should have directly used the filename . Example -
for filename in os.listdir("."):
secondline = linecache.getline(filename , 2)
os.rename(filename, secondline.strip()+".txt")
Try using a simpler method
import os,re
def changeName(filename):
with open(filename, "r") as f:
line = next(f)
secondline = next(f)
if secondline == "textfile" + str(re.search(r'\d+', filename).group()):
#re.search() gets the first integer in the filename
os.rename(filename, secondline + ".txt")
for root, dirs, files in os.walk("Directory"):
for file in files:
file = os.path.join(root, file)
changeName(file)
I have files on the linux server and their file names are broken because of the middle \r character. I couldn't download those files by using WinScp or Filezilla on windows.
Moreover I couldn't rename or process them properly in python.
On command
files = os.listdir("2014/")
I got this list set.
['16963_6_iris2570_20150110_052515\r_172518.gpx', '29174_3_Sunnam0223_20150114_0 10833\r_130835.gpx', '35767_3_samsi2_20150117_035045\r_155047.gpx', '36581_4_kix ing_20150117_045424\r_165425.gpx', '33383_4_rnrghk10kr_20150117_101618\r_101619. gpx']
On command:
file1 = files[0]
output: _172518.gpxs2570_20150110_052515
Then I try to replace \r
file2 = files[0].replace('\r', '')
output: 16963_6_iris2570_20150110_052515_172518.gpx
That's good but when I try to rename:
os.rename("2014/"+file1, "2014/"+file2)
f = open(file2, "r")
data = f.readlines()
f.close()
output:
Traceback (most recent call last):
File "test.py", line 25, in <module>
f = open(file2, "r")
IOError: [Errno 2] No such file or directory: '29174_3_Sunnam0223_20150114_010833_130835.gpx'
Did you try:
f = open("2014/"+file2, "r")
In your example code above, you included the 2014 folder name in your rename, but not in your open call.