new python2.7 user here. I've searched for similar queries and I can't quite see what I'm doing wrong. I have a short script to read though all the files in a directory, and reading each one in turn, write them to a single master file.
My code is below; I can see two things going wrong at this time (although I get no error messages), 1 - that it only appears to open the first file in the list its supposed to be looping through (so my guess is I've an error using glob?), and 2 - although the print(str) statements display to the console nps, the output file never gets written too.
I've double checked that the file exists, it is empty and I'm passing the correct path & filename in when I call the function.
Any help is much appreciated.
#!/usr/bin/env python
import glob
import sys
filestoberecognised=sys.argv[1]
outputfile=sys.argv[2]
filecontents=glob.glob(filestoberecognised)
with open(outputfile,'w+') as f:
for i, row in enumerate(filecontents):
print(row) # this correctly prints to console
f.write(row+'\n') # this should write the filename of the filestoberecognised to the outputfile
with open(row,'r') as labfile:
for j, line in enumerate(labfile): # this should write words in label file
f.write('%s'%(line))
print('%s'%(line))
labfile.close() # ensures each file looped through is closed
f.write('\n.\n')
f.flush()
f.close()
Related
I have a file in my python folder called data.txt and i have another file read.py trying to read text from data.txt but when i change something in data.txt my read doesn't show anything new i put
Something else i tried wasn't working and i found something that read, but when i changed it to something that was actually meaningful it didn't print the new text.
Can someone explain why it doesn't refresh, or what i need to do to fix it?
with open("data.txt") as f:
file_content = f.read().rstrip("\n")
print(file_content)
First and foremost, strings are immutable in Python - once you use file.read(), that returned object cannot change.
That being said, you must re-read the file at any given point the file contents may change.
For example
read.py
def get_contents(filepath):
with open(filepath) as f:
return f.read().rstrip("\n")
main.py
from read import get_contents
import time
print(get_contents("data.txt"))
time.sleep(30)
# .. change file somehow
print(get_contents("data.txt"))
Now, you could setup an infinite loop that watches the file's last modification timestamp from the OS, then always have the latest changes, but that seems like a waste of resources unless you have a specific need for that (e.g. tailing a log file), however there are arguably better tools for that
It was unclear from your question if you do the read once or multiple times. So here are steps to do:
Make sure you call the read function repeatedly with a certain interval
Check if you actually save file after modification
Make sure there are no file usage conflicts
So here is a description of each step:
When you read a file the way you shared it gets closed, meaning it is read only once, you need to read it multiple times if you want to see changes, so make it with some kind of interval in another thread or async or whatever suits your application best.
This step is obvious, remember to hit ctrl+c
It may happen that a single file is being accessed by multiple processes, for example your editor and the script, now to prevent errors try the following code:
def read_file(file_name: str):
while True:
try:
with open(file_name) as f:
return f.read().rstrip("\n")
except IOError:
pass
I have a folder with several csv-files in it. I have to change the filename of every file with a string that I find in the file. So I tried the script below. It looks like it is working until I try to rename the file.
What did I try:
First I didn't have the file.close() line in the program, but did didn't fix the problem
I added a line print(file.closed) to see if the file was actually closed
I tried to get the os.rename out of the indented 'with' block. But I keep getting the same error
I tried to get the os.rename out of any block. But then I get a Winerror 123, where it sais that the filename , directoryname etc. is incorrect.
I also read the questions WindowsError 32 while trying to os.rename and Windows Error: 32 when trying to rename file in python.
I understood that maybe I had to close the file with f.close since this is the handler, but that didn't work as well.
The code that I tried:
for f in glob.glob("/path/*.csv"):
with open(f, "r") as file:
#read the lines in the csv-file
data = file.read()
#search the lines that have been read for a pattern and save that in "search"
search = re.findall("some_pattern", data)
#The result was a list. With this line I tried to change it into a string
file.close()
Listtostring = ''.join([str(elem) for elem in search])
#I only want to use a part of the match in the new file name
name = Listtostring.replace("part_of_string", "")
os.rename(f,f+name)
I hope somebody can give me some tips and explain what I am doing wrong. Pretty new to Python, so if you can give me some insight in my mistakes, than it's appreciated!
Thank you for your comments and time. It seemed that one of the files that was opened was still busy in some process and therefore the code didn’t work. I first closed all the applications that were running, but that didn’t work. After that I restarted the computer and the script worked fine!
I am a new python learner and now i have entered into file handling.
I tried solution for my problem but failed, so posting my question. before duplication please consider my question.
I tried to create a file, it worked.
writing in the file also worked.
But when i tried to read the text or values in the file, it returns empty.
I use command line terminal to work with python and running in Ubuntu OS.
The coding which I have tried is given below. The file is created in the desired location and the written text is also present.
f0=open("filehandling.txt","wb")
f0.write("my second attempt")
s=f0.read(10);
print s
I also tried with wb+, r+. But it just returns as empty
edit 1:
I have attached the coding below. I entered one by one in command line
fo = open("samp.txt", "wb")
fo.write( "Text is here\n");
fo.close()
fo = open("samp.txt", "r+")
str = fo.read(10);
print "Read String is : ", str
fo.close()
First of all if you open with wb flag then the file will be only in writeable mode. If you want to both read and write then you need wb+ flag. If you don't want the file to be truncated each time then you need rb+.
Now files are streams with pointers pointing at a certain location inside the file. If you write
f0.write("my second attempt")
then the pointer points at the [pointer before writing] (in your case the begining of the file, i.e. 0) plus [length of written bytes] (in your case 17, which is the end of the file). In order to read whole file you have to move that pointer back to the begining and then read:
f0.seek(0)
data = f0.read()
I have a python script used to edit a text file. Firstly, the first line of the text file is removed. After that, a line is added to the end of the text file.
I noticed a weird phenomenon, but I cannot explain the reason of this behaviour:
This script works as expected (removes the first line and adds a line at the end of the file):
import fileinput
# remove first line of text file
i = 0
for line in fileinput.input('test.txt', inplace=True):
i += 1
if i != 1:
print line.strip()
# add a line at the end of the file
f = open('test.txt', 'a+') # <= line that is moved
f.write('test5')
f.close()
But in the following script, as the text file is opened before removing, the removal occurs but the content isn't added (with the write() method):
import fileinput
# file opened before removing
f = open('test.txt', 'a+') # <= line that is moved
# remove first line of text file
i = 0
for line in fileinput.input('test.txt', inplace=True):
i += 1
if i != 1:
print line.strip()
# add a line at the end of the file
f.write('test5')
f.close()
Note that in the second example, open() is placed a the beginning, whereas in the first it is called after removing the last line of the text file.
What's the explanation of the behaviour?
When using fileinput with the inplace parameter, the modified content is saved in a backup file. The backup file is renamed to the original file when the output file is closed. In your example, you do not close the fileinput file explicitly, relying on the self-triggered closing, which is not documented and might be unreliable.
The behaviour you describe in the first example is best explained if we assume that opening the same file again triggers fileinput.close(). In your second example, the renaming only happens after f.close() is executed, thus overwriting the other changes (adding "test5").
So apparently you should explicitly call fileinput.close() in order to have full control over when your changes are written to disk. (It is generally recommended to release external resources explicitly as soon as they are not needed anymore.)
EDIT:
After more profound testing, this is what I think is happening in your second example:
You open a stream with mode a+ to the text file and bind it to the variable f.
You use fileinput to alter the same file. Under the hood, a new file is created, which is afterwards renamed to what the file was called originally. Note: this doesn't actually change the original file – rather, the original file is made inaccessible, as its former name now points to a new file.
However, the stream f still points to the original file (which has no file name anymore). You can still write to this file and close it properly, but you cannot see it anymore (since it has no filename anymore).
Please note that I'm not an expert in this kind of low-level operations; the details might be wrong and the terminology certainly is. Also, the behaviour might be different across OS and Python implementations. However, it might still help you understand why things go different from what you expected.
In conclusion I'd say that you shouldn't be doing what you do in your second example. Why don't you just read the file into memory, alter it, and then write it back to disk? Actual in-place (on-disk) altering of files is no fun in Python, as it's too high-level a language.
This is similar or identical to csv writer not closing file but I'm not 100% sure why my behaviour is different.
def LoadCSV:
with open('test.csv', 'r') as csvfile:
targetReader = csv.reader(csvfile, delimiter=',')
for row in targetReader:
...
then finally in the function
csvfile.close()
This opens the test.csv file in the same direction as the script. Desired behaviour is for when the script has done what it's doing to the rows in the function, it renames the sheet to test.[timestamp] to archive it and watches the directory for a new sheet to arrive.
Later down the code;
os.rename('test.csv', "test." + time.strftime("%x") )
Gives an error that the file can't be renamed because a process is still using it. How do I close this file once I'm done? csvfile.close() doesn't raise an exception, and if I step through my code in interactive mode I can see that csvfile is a "closed file object." What even is that? Surely an open file is an object but a closed one isn't, how do I make my code forget this even exists so I can then do IO on the file?
NOT FOR POINTS.
Code is not valid anyway, since your function name is wrong. If that was not intentional, better edit it or to produce a pseudo-replica of your code, rather than have us guess what the issue is.
To iterate, the issues with your code:
def LoadCSV is not valid. def LoadCSV() is. Proof in following screenshot. Notice how the lack of () is showing syntax error markers.
Fixing (1) above, your next problem is using csvfile.close(). If the code is properly written, once the code is out of the scope of with, the file is closed automatically. Even if the renaming part of the code is inside the function, it shouldn't pose any problems.
Final word of warning -- using the format string %x will produce date-strings like 08/25/14, depending on locale. Obviously, this is erroneous, as a / is invalid in filenames in Windows (try renaming a file manually with this). Better to be very explicit and just use %m%d%y instead.
Finally, see the running code on my end. If your code is not structured like this, then other errors we cannot guess might arise.
Result as follows after running:
Code for reference:
import csv
import os
import time
def LoadCSV():
with open("test.csv", "r") as csvfile:
targetReader = csv.reader(csvfile, delimiter=",")
for row in targetReader:
print row
new_name = "test.%s.csv" % time.strftime("%m%d%y")
print new_name
os.rename("test.csv", new_name)
LoadCSV()
Note that on my end, there is nothing that watches my file. Antivirus is on, and no multithreading obviously is enabled. Check if one of your other scripts concurrently watches this file for changes. It's better if instead of watching the file, the file is sent as an argument post-renaming to this other function instead, as this might be the reason why it's "being used". On the one hand, and this is untested on my side, possibly better to copy the file with a new name rather than rename it.
Hope this helps.
When you are using a with block you do not need to close the file, it should be released outside the scope. If you want python to "forget" the entire filehandle you could delete it with del csvfile. But since you are using with you should not delete the variable inside the scope.
Try without the with scope instead:
csvfile = open('test.csv','r')
targetReader = csv.reader(csvfile, delimiter=',')
for row in targetReader:
....
csvfile.close()
del targetReader
os.rename('test.csv','test.'+time.strftime('%x'))
It might be the csv reader that still access the file when you are using a with block.