so I had an exercise asking to write to a file (using newlines) and then open it in reading mode. I did exactly that and the console outputs the right result. What happened was that I tried to do it within the 'w' mode code block and the console outputs nothing.
For example:
with open('test.txt', 'w') as wf:
nicknames = ["Big Tuna", "Prison Mike", "Booster Seat"]
wf.write('\n'.join(nicknames))
with open('test.txt', 'r') as rf:
print(rf.read())
I understand that the program only closes the file after the with statement, but I need clarification on my understanding of what's happening here.
So, what I understood so far is that the program first creates the test.txt file (test.txt did not exist in my file path) and then proceeds to convert and write the given contents into the test.txt file. After that, the nested with tries to open a file named test.txt in reading mode, but the console will not output anything because the program is trying to open a file that is already opened, that's why it cannot read into an already opened file.
Please correct me if I'm misunderstood the above because I am unsure whether or not I've understood it correctly, thank you.
That’s not what’s happening. Unix systems, at least, will happily let you open a file multiple times.
However, Python’s IO is buffered by default. You need to flush the data you’ve written out to the file before you can read the data from it. See https://docs.python.org/3/library/io.html#io.IOBase.flush for more information about this. (Summary: put wf.flush() after the wf.write(…) call and before attempting to read from it.
Related
I am attempting to output a new txt file but it come up blank. I am doing this
my_file = open("something.txt","w")
#and then
my_file.write("hello")
Right after this line it just says 5 and then no text comes up in the file
What am I doing wrong?
You must close the file before the write is flushed. If I open an interpreter and then enter:
my_file = open('something.txt', 'w')
my_file.write('hello')
and then open the file in a text program, there is no text.
If I then issue:
my_file.close()
Voila! Text!
If you just want to flush once and keep writing, you can do that too:
my_file.flush()
my_file.write('\nhello again') # file still says 'hello'
my_file.flush() # now it says 'hello again' on the next line
By the way, if you happen to read the beautiful, wonderful documentation for file.write, which is only 2 lines long, you would have your answer (emphasis mine):
Write a string to the file. There is no return value. Due to buffering, the string may not actually show up in the file until the flush() or close() method is called.
If you don't want to care about closing file, use with:
with open("something.txt","w") as f:
f.write('hello')
Then python will take care of closing the file for you automatically.
As Two-Bit Alchemist pointed out, the file has to be closed. The python file writer uses a buffer (BufferedIOBase I think), meaning it collects a certain number of bytes before writing them to disk in bulk. This is done to save overhead when a lot of write operations are performed on a single file.
Also: When working with files, try using a with-environment to make sure your file is closed after you are done writing/reading:
with open("somefile.txt", "w") as myfile:
myfile.write("42")
# when you reach this point, i.e. leave the with-environment,
# the file is closed automatically.
The python file writer uses a buffer (BufferedIOBase I think), meaning
it collects a certain number of bytes before writing them to disk in
bulk. This is done to save overhead when a lot of write operations are
performed on a single file. Ref #m00am
Your code is also okk. Just add a statement for close file, then work correctly.
my_file = open("fin.txt","w")
#and then
my_file.write("hello")
my_file.close()
I am a beginner, writing a python script in which I need it to create a file that I can write information to. However, I am having problems getting it to create a new, not previously existing file.
for example, I have:
file = open(coordinates.kml, 'w')
which it proceeds to tell me:
nameerror: name 'coordinates' is not defined.
Of course it isn't defined, I'm trying to make that file.
Everything I read on creating a new file says to take this route, but it simply will not allow me. What am I doing wrong?
I even tried to flat out define it...
file = coordinates.kml
file_open = open(file, 'w')
... and essentially got the same result.
You need to pass coordinates.kml as a string, so place them in quotes (single or double is fine).
file = open("coordinates.kml", "w")
In addition to the above answer,
If you want to create a file in the same path, then no problem or else you need to specify the path as well in the quotes.
But surely opening a file with read permission will throw an error as you are trying to access an nonexistent file.
To be future proof and independent of the platforms you can read and write files in binaries. For example if this is Python on Windows, there could be some alternations done to the end of line. Hence reading and writing in Binary mode should help, using switches "rb" and "wb"
file = open("coordinates.kml", "wb")
And also remember to close the file session, else can throw errors while re running the script.
in a py module, I write:
outFile = open(fileName, mode='w')
if A:
outFile.write(...)
if B:
outFile.write(...)
and in these lines, I didn't use flush or close method.
Then after these lines, I want to check whether this "outFile" object is empty or not. How can I do with it?
There are a few problems with your code.
You can't .write to a file that you opened with 'r'. You need to open(fileName, 'w').
If A or B then you've certainly written to the file, so it's not empty!
Barring those. you can get the length of a file with
os.stat(outFile.fileno())
EDIT: I'll explain what flush does. Python is often used to do quite large amounts of file reads and writes, which can be slow. It is thus tweaked to make them as fast as possible. One way that is does so is to "buffer" such writes and then do them all in one big block: when you write a small string, Python will remember it but won't actually write it to the file until it thinks it should.
This means that if you want to tell whether you have written data to the file by inspecting the file, you have to tell Python to write all the data it's remembering first, or else you might not see it. flush is the command to write all the buffered data.
Of course, if you ask Python whether it's written anything to the file, say by inspecting the position in the file (.tell()), then it will know about the buffering.
If you've already written to the file, you can use .tell() to check if the current file position is nonzero:
>>> handle = open('/tmp/file.txt', 'w')
>>> handle.write('foo')
>>> handle.tell()
3
This won't work if you .seek() back to the beginning of the file.
You can use os.stat to get file info:
import os
fileSize = os.stat(fileName).st_size
with open("filename.txt", "r+") as f:
if f.read():
# file isn't empty
f.write("something")
# uncomment this line if you want to delete everything else in the file
# f.truncate()
else:
# file is empty
f.write("somethingelse")
"r+" mode always you to read & write.
"with" will automatically close file
I'm trying to use a subprocess to write the output to a data file, and then parse through it in order to check for some data in it. However, when I need to do the reading through the file's lines, I always get a blank file unless I close the file and then reopen it. While it works, I just don't like having to do this and I want to know why it happens. Is it an issue with subprocess, or another intricacy of the file mode?
dumpFile=open(filename,"w+")
dump = subprocess.Popen(dumpPars,stdout=dumpFile)
dump.wait()
At this point, if I try to read the file, I get nothing. However, it works fine by doing these commands after:
dumpFile.close()
dumpFile=open(filename,"r")
The with statement automatically closes the file after the block ends:
with open(filename, "w+") as dumpFile:
dump = subprocess.Popen(dumpPars, stdout=dumpFile)
dump.wait()
with open(filename, "r") as dumpFile:
# dumpFile reading code goes here
You probably need to seek back to the beginning of the file, otherwise the file pointer will be at the end of the file when you try to read it:
dumpFile.seek(0)
However, if you don't need to actually store dumpFile, it's probably better to do something like:
dump = = subprocess.Popen(dumpPars,stdout=subprocess.PIPE)
stdoutdata,_ = dump.communicate() #now parse stdoutdata
unless your command produces large volumes of data.
If you want to read what you've already written, either close and reopen the file, or "rewind" it - seek to offset 0.
If you want to read the file while it is being written, you can do so (don't even need to write it to disk), see this other question Capture output from a program
I'm running into a problem that I haven't seen anyone on StackOverflow encounter or even google for that matter.
My main goal is to be able to replace occurences of a string in the file with another string. Is there a way there a way to be able to acess all of the lines in the file.
The problem is that when I try to read in a large text file (1-2 gb) of text, python only reads a subset of it.
For example, I'll do a really simply command such as:
newfile = open("newfile.txt","w")
f = open("filename.txt","r")
for line in f:
replaced = line.replace("string1", "string2")
newfile.write(replaced)
And it only writes the first 382 mb of the original file. Has anyone encountered this problem previously?
I tried a few different solutions such as using:
import fileinput
for i, line in enumerate(fileinput.input("filename.txt", inplace=1)
sys.stdout.write(line.replace("string1", "string2")
But it has the same effect. Nor does reading the file in chunks such as using
f.read(10000)
I've narrowed it down to mostly likely being a reading in problem and not a writing problem because it happens for simply printing out lines. I know that there are more lines. When I open it in a full text editor such as Vim, I can see what the last line should be, and it is not the last line that python prints.
Can anyone offer any advice or things to try?
I'm currently using a 32-bit version of Windows XP with 3.25 gb of ram, and running Python 2.7
Try:
f = open("filename.txt", "rb")
On Windows, rb means open file in binary mode. According to the docs, text mode vs. binary mode only has an impact on end-of-line characters. But (if I remember correctly) I believe opening files in text mode on Windows also does something with EOF (hex 1A).
You can also specify the mode when using fileinput:
fileinput.input("filename.txt", inplace=1, mode="rb")
Are you sure the problem is with reading and not with writing out?
Do you close the file that is written to, either explicitly newfile.close() or using the with construct?
Not closing the output file is often the source of such problems when buffering is going on somewhere. If that's the case in your setting too, closing should fix your initial solutions.
If you use the file like this:
with open("filename.txt") as f:
for line in f:
newfile.write(line.replace("string1", "string2"))
It should only read into memory one line at a time, unless you keep a reference to that line in memory.
After each line is read it will be up to pythons garbage collector to get rid of it. Give this a try and see if it works for you :)
Found to solution thanks to Gareth Latty. Using an iterator:
def read_in_chunks(file, chunk_size=1000):
while True:
data = file.read(chunk_size)
if not data: break
yield data
This answer was posted as an edit to the question Python Does Not Read Entire Text File by the OP user1297872 under CC BY-SA 3.0.