I am trying to write the lists into a file. My code writes all the list except last list. I don't know why. Can someone please take a look at my code and let me know what I am doing wrong?
complete_test=[['apple','ball'],['test','test1'],['apple','testing']]
counter = 1
for i in complete_test:
r=open("testing"+str(counter)+".txt",'w')
for j in i:
r.write(j+'\n')
counter=counter +1
Thank you.
You need to call r.close().
This doesn't happen if you run your code as a Python file, but it's reproducible in the interpreter, and it happens for this reason:
All of the changes to a file are buffered, rather than executed immediately. CPython will close files when there are no longer any valid references to them, such as when the only variable referencing them is overwritten on each iteration of your loop. (And when they are closed, all of the buffered changes are flushed—written out.) On the final iteration, you never close the file because the variable r sticks around. You can verify this because calling exit() in the interpreter closes the file and causes the changes to be written.
This is a motivating example for context managers and with statements, as in inspectorG4dget's answer. They handle the opening and closing of the files for you. Use that code, rather than actually calling r.close(), and understand that this is what's going when you do it.
Here's a much cleaner way of doing the same thing:
complete_test=[['apple','ball'],['test','test1'],['apple','testing']]
for i,sub in enumerate(complete_list, 1): # `enumerate` gives the index of each element in the list as well
with open("testing{}".format(i), 'w') as outfile: # no need to worry about opening and closing, if you use `with`
outfile.write('\n'.join(sub)) # no need to write in a loop
outfile.write('\n')
Related
This is probably a dumb question, but I'm new to programming and I have a recursive function set up that I'm trying to figure out. For any print function in Python, is it necessarily true that lines are printed in the order that they are written in the script OR for larger outputs, is it possible that smaller length outputs can get printed first in the console even though the print statement is later in the code (maybe due to some memory lag)?
Example:
def test_print():
#don't run this, but was meant for scale. Is there any chance the 1 would print before the list of lists?
print([[i for i in range(10000)] for j in range(10000)])
print(1)
Print statements pile output into stdout in the order the code was written. Top to bottom. It isn't possible any other way because that's the way the code is interpreted. Memory lag doesn't play any role here because the output to your console is a line for line rendition of the data that was piled into stdout. And the order the data was written to it can't change, so you'll maintain chronology. Of course, you can always play around with the how the print function itself works. But I wouldn't recommend tampering with standard library functions.
As said above, print() function is executed in the order which they are in your code. But you yourself can change the order in which you want it executed, after all you have every right to instruct the code to do whatever you want.
You'll always get the same order in the output as the order you execute print() functions in Python.
I am using seek function to extract new lines in an updated file. My code looks like this:
read_data=open('path-to-myfile','r')
read_data.seek(0,2)
while True:
time.sleep(sometime)
new_data=read_data.readlines()
do something with new_data
myfile is a csv file that will be constantly updated
The problem is that usually after several loops inside the while, new_data return nothing. It can be different loop numbers. While I checked myfile, it is still updating..... So any problem I have on my code ? Or is there any other way to do this ?
Any help appreciated !!
You have two programs accessing the same file on disk? If that is the case, then the resource may be locking. I set up an example script that writes to a file, and another file that reads for changes based on the code you provided.
So in one instance of python:
import time
while True:
time.sleep(2)
with open('test.txt','a') as read_data:
read_data.seek(0,2)
read_data.write("bibbity boopity\n")
And in another instance of python
import time
read_data=open('test.txt','r')
read_data.seek(0,2)
while True:
time.sleep(1)
new_data=read_data.readlines()
print(new_data)
In this case, the resource is updating slower than its being read, so changes printed by the bottom prog will be blank. But if I speed up the changes per second, well I still see them. But there are some instances where not all the updates are seen.
You may want to use asynchronous file reading to catch all the changes. Python 3 asyncio library doesn't support async file read/write, but curio does.
See also this question
I've made a map editor in Python2.7.9 for a small project and I'm looking for ways to preserve the data I edit in the event of some unhandled exception. My editor already has a method for saving out data, and my current solution is to have the main loop wrapped in a try..finally block, similar to this example:
import os, datetime #..and others.
if __name__ == '__main__':
DataMgr = DataManager() # initializes the editor.
save_note = None
try:
MainLoop() # unsurprisingly, this calls the main loop.
except Exception as e: # I am of the impression this will catch every type of exception.
save_note = "Exception dump: %s : %s." % (type(e).__name__, e) # A memo appended to the comments in the save file.
finally:
exception_fp = DataMgr.cwd + "dump_%s.kmap" % str(datetime.datetime.now())
DataMgr.saveFile(exception_fp, memo = save_note) # saves out to a dump file using a familiar method with a note outlining what happened.
This seems like the best way to make sure that, no matter what happens, an attempt is made to preserve the editor's current state (to the extent that saveFile() is equipped to do so) in the event that it should crash. But I wonder if encapsulating my entire main loop in a try block is actually safe and efficient and good form. Is it? Are there risks or problems? Is there a better or more conventional way?
Wrapping the main loop in a try...finally block is the accepted pattern when you need something to happen no matter what. In some cases it's logging and continuing, in others it's saving everything possible and quitting.
So you're code is fine.
If your file isn't that big, I would suggest maybe reading the entire input file into memory, closing the file, then doing your data processing on the copy in memory, this will solve any problems you have with not corrupting your data at the cost of potentially slowing down your runtime.
Alternatively, take a look at the atexit python module. This allows you to register a function(s) for a automatic callback function when the program exits.
That being said what you have should work reasonably well.
In my code, I write a file to my hard disk. After that, I need to import the generated file and then continue processing it.
for i in xrange(10):
filename=generateFile()
# takes some time, I wish to freeze the program here
# and continue once the file is ready in the system
file=importFile(filename)
processFile(file)
If I run the code snippet in one go, most likely file=importFile(filename) will complain that that file does not exist, since the generation takes some time.
I used to manually run filename=generateFile() and wait before running file=importFile(filename).
Now that I'm using a for loop, I'm searching for an automatic way.
You could use time.sleep and I would expect that if you are loading a module this way you would need to reload rather than import after the first import.
However, unless the file is very large why not just generate the string and then eval or exec it?
Note that since your file generation function is not being invoked in a thread it should be blocking and will only return when it thinks it has finished writing - possibly you can improve things by ensuring that the file writer ends with outfile.flush() then outfile.close() but on some OSs there may still be a time when the file is not actually available.
for i in xrange(10):
(filename, is_finished)=generateFile()
while is_finished:
file=importFile(filename)
processFile(file)
continue;
I think you should use a flag to test if the file is generate.
This code works fine for me. Appends data at the end.
def writeFile(dataFile, nameFile):
fob = open(nameFile,'a+')
fob.write("%s\n"%dataFile)
fob.close()
But the problem is when I close the program and later run again I found that all the previous data were lost. Process is started to write from the start and there is no data in the file.
But during the run it perfectly add a line at the end of file.
I can't understand the problem. Please some one help.
NB: I am using Ubuntu-10.04 with python 2.6
There is nothing wrong with the code you posted here... I tend to agree with other the comments that this file is probably being overwritten elsewhere in your code.
The only suggestion I can think of to test this explicitly (if your use case can tolerate it) is to throw in an exit() statement at the end of the function and then open the file externally (aka in gedit) and see if the last change took.
Alternatively to the exit, you could run the program in the terminal and include a call to pdb at the end of this function which would interrupt the program without killing it:
import pdb; pdb.set_trace()
You will then have to hit c to continue the program each time this runs.
If that checks out, do a search for other places this file might be getting opened.