My code currently writes a dictionary which contains scores for a class to a CSV file. This part is correctly done by the program and the scores are wrote to file, however the latest dictionary written to file is not printed. For example, after the code has been ran once, it will not be printed however once the code has been ran for a second time, the first bit of data is printed however the new data isn't. Can someone tell me where I am going wrong?
SortedScores = sorted(Class10x1.items(), key = lambda t: t[0], reverse = True) #this sorts the scores in alphabetical order and by the highest score
FileWriter = csv.writer(open('10x1 Class Score.csv', 'a+'))
FileWriter.writerow(SortedScores) #the sorted scores are written to file
print "Okay here are your scores!\n"
I am guessing the problem is here somewhere however I cannot quite pinpoint what or where it is. I have tried to solve this by changing the mode of the file when it is read back in to r, r+ and rb, however all have the same consequence.
ReadFile = csv.reader(open("10x1 Class Score.csv", "r")) #this opens the file using csv.reader in read mode
for row in ReadFile:
print row
return
from Input output- python docs:
It is good practice to use the with keyword when dealing with file objects. This has the advantage that the file is properly closed after its suite finishes, even if an exception is raised on the way. It is also much shorter than writing equivalent try-finally blocks:
>>> with open('workfile', 'r') as f:
... read_data = f.read()
>>> f.closed
True
File objects have some additional methods, such as isatty() and truncate() which are less frequently used; consult the Library Reference for a complete guide to file objects.
I'm not sure why they bury that so far in the documentation since it is really useful and a very common beginner mistake:
SortedScores = sorted(Class10x1.items(), key = lambda t: t[0], reverse = True) #this sorts the scores in alphabetical order and by the highest score
with open('10x1 Class Score.csv', 'a+') as file:
FileWriter = csv.writer(file)
FileWriter.writerow(SortedScores) #the sorted scores are written to file
print "Okay here are your scores!\n"
this will close the file for you even if an error is raised which prevents many possibilities of loss of data
the reason it did not appear to write to the file is because when you do .write_row() it doesn't immediately write to the hard drive, only to a buffer which is occasionally emptied into the file on hard drive, although with only one write statement it has no need to empty.
Remember to close the file after operation, otherwise the data will not be saved properly.
Try to use with keyword so that Python will handle the closure for you:
import csv
with open('10x1 Class Score.csv', 'a+') as f:
csv_writer = csv.writer(f)
# write something into the file
...
# when the above block is done, file will be automatically closed
# so that the file is saved properly
Related
I have a code in python that creates and compares the content of two text files and tells me when they are different:
data=str(information)
#this creates the first file, the one used as a control group of sorts
f=open("text1.txt", "w+")
f.write(data)
f.close()
while True:
#the other file keeps updating, so it's inside a loop
data2=str(newinfo)
f=open("text2.txt", "w+")
f.write(data2)
f.close()
#I'm guessing the error is probably here
read = str(open("text1.txt", "r"))
read2 = str(open("text2.txt", "r"))
if read2 != read:
Notifica = True
break
Both data and data2 are the html from a website I'm reading with BeautifulSoup, that part is working.
However the program keeps thinking the the two text files are different even when they are exactly the same. I think I'm doing this the wrong way, any help?
the open() function returns an object, not the content of the file.
you are comparing two references to different file objects.
you should read the contents of the file and then compare it.
you should do:
read = open("text1.txt", "r").read()
read2 = open("text2.txt", "r").read()
Since HTML file can contain a Huge number of characters your code can use lot of resources. A good way (might be the most use) is to use MD5 Hash.
The MD5 algorithm is deprecated for cryptographic purpose but should be enough for your aim.
I am trying to read a file and I am printing the content out in alphabetical order. it is only reading 6 items from the file and no more. I am not asking someone to do it for me , if someone can just guide me in the direction of what I am doing wrong here is my code so far.
infile = open("unsorted_fruits.txt", "r")
outfile=open("sorted_fruits.txt","w")
fruit=infile.read(50).split()
#outfile.write(fruit)
fruit.sort()
for numbers in range(len(fruit)):
print(fruit[numbers])
infile.close()
outfile.close()
Try .read() to set fruit to the entire file split.
Also, try using context managers so you don't have to call close() by yourself:
with open("unsorted_fruits.txt", "r") as infile:
#... do your stuff
#close will be called automatically
If you're concerned about massive files, then consider other questions that read large files in batches: Lazy Method for Reading Big File in Python?
In the code sample you provided outfile is not used (commented out) but you can use nested context managers.
I'm doing an exercise from a book, and can't figure this out; this isn't homework, I'm trying to teach myself, and I have been trying anything that I can think of to make this work!
I need to read lines from a text file, and then write the lines into a new file, in reversed order.
My problem is, I can get it into reversed order (I'm assuming that it means last line of the 'r' file is printed first in the 'w' file), but it also prints the words reversed, making it unreadable.
Is there a way to possibly write the lines in reversed order, but the lines stay readable?
Here's what I have:
filetoread = 'yolo.txt'
file = open(filetoread, 'r')
words = file.read()
wrf = 'yip.txt'
wfile = open(wrf, 'w')
for lines in reversed(list(words)):
print wfile.write(lines)
You may want to note that it's usually easier and safer to use the with statement. That way you don't have to worry about opening a file and forgetting to close it. Plus, if an exception occurs inside the with block, the file is still closed automatically; you'd have to use a try...finally statement to do this without with.
Here's your example using with:
with open('yolo.txt', 'r') as readf, open('yip.txt', 'w') as writef:
for line in reversed(readf.readlines()):
writef.write(line)
you mean something like
wfile.write("".join(reversed(list(open("some_file.txt")))))
This is similar or identical to csv writer not closing file but I'm not 100% sure why my behaviour is different.
def LoadCSV:
with open('test.csv', 'r') as csvfile:
targetReader = csv.reader(csvfile, delimiter=',')
for row in targetReader:
...
then finally in the function
csvfile.close()
This opens the test.csv file in the same direction as the script. Desired behaviour is for when the script has done what it's doing to the rows in the function, it renames the sheet to test.[timestamp] to archive it and watches the directory for a new sheet to arrive.
Later down the code;
os.rename('test.csv', "test." + time.strftime("%x") )
Gives an error that the file can't be renamed because a process is still using it. How do I close this file once I'm done? csvfile.close() doesn't raise an exception, and if I step through my code in interactive mode I can see that csvfile is a "closed file object." What even is that? Surely an open file is an object but a closed one isn't, how do I make my code forget this even exists so I can then do IO on the file?
NOT FOR POINTS.
Code is not valid anyway, since your function name is wrong. If that was not intentional, better edit it or to produce a pseudo-replica of your code, rather than have us guess what the issue is.
To iterate, the issues with your code:
def LoadCSV is not valid. def LoadCSV() is. Proof in following screenshot. Notice how the lack of () is showing syntax error markers.
Fixing (1) above, your next problem is using csvfile.close(). If the code is properly written, once the code is out of the scope of with, the file is closed automatically. Even if the renaming part of the code is inside the function, it shouldn't pose any problems.
Final word of warning -- using the format string %x will produce date-strings like 08/25/14, depending on locale. Obviously, this is erroneous, as a / is invalid in filenames in Windows (try renaming a file manually with this). Better to be very explicit and just use %m%d%y instead.
Finally, see the running code on my end. If your code is not structured like this, then other errors we cannot guess might arise.
Result as follows after running:
Code for reference:
import csv
import os
import time
def LoadCSV():
with open("test.csv", "r") as csvfile:
targetReader = csv.reader(csvfile, delimiter=",")
for row in targetReader:
print row
new_name = "test.%s.csv" % time.strftime("%m%d%y")
print new_name
os.rename("test.csv", new_name)
LoadCSV()
Note that on my end, there is nothing that watches my file. Antivirus is on, and no multithreading obviously is enabled. Check if one of your other scripts concurrently watches this file for changes. It's better if instead of watching the file, the file is sent as an argument post-renaming to this other function instead, as this might be the reason why it's "being used". On the one hand, and this is untested on my side, possibly better to copy the file with a new name rather than rename it.
Hope this helps.
When you are using a with block you do not need to close the file, it should be released outside the scope. If you want python to "forget" the entire filehandle you could delete it with del csvfile. But since you are using with you should not delete the variable inside the scope.
Try without the with scope instead:
csvfile = open('test.csv','r')
targetReader = csv.reader(csvfile, delimiter=',')
for row in targetReader:
....
csvfile.close()
del targetReader
os.rename('test.csv','test.'+time.strftime('%x'))
It might be the csv reader that still access the file when you are using a with block.
Started Python a week ago and I have some questions to ask about reading and writing to the same files. I've gone through some tutorials online but I am still confused about it. I can understand simple read and write files.
openFile = open("filepath", "r")
readFile = openFile.read()
print readFile
openFile = open("filepath", "a")
appendFile = openFile.write("\nTest 123")
openFile.close()
But, if I try the following I get a bunch of unknown text in the text file I am writing to. Can anyone explain why I am getting such errors and why I cannot use the same openFile object the way shown below.
# I get an error when I use the codes below:
openFile = open("filepath", "r+")
writeFile = openFile.write("Test abc")
readFile = openFile.read()
print readFile
openFile.close()
I will try to clarify my problems. In the example above, openFile is the object used to open file. I have no problems if I want write to it the first time. If I want to use the same openFile to read files or append something to it. It doesn't happen or an error is given. I have to declare the same/different open file object before I can perform another read/write action to the same file.
#I have no problems if I do this:
openFile = open("filepath", "r+")
writeFile = openFile.write("Test abc")
openFile2 = open("filepath", "r+")
readFile = openFile2.read()
print readFile
openFile.close()
I will be grateful if anyone can tell me what I did wrong here or is it just a Pythong thing. I am using Python 2.7. Thanks!
Updated Response:
This seems like a bug specific to Windows - http://bugs.python.org/issue1521491.
Quoting from the workaround explained at http://mail.python.org/pipermail/python-bugs-list/2005-August/029886.html
the effect of mixing reads with writes on a file open for update is
entirely undefined unless a file-positioning operation occurs between
them (for example, a seek()). I can't guess what
you expect to happen, but seems most likely that what you
intend could be obtained reliably by inserting
fp.seek(fp.tell())
between read() and your write().
My original response demonstrates how reading/writing on the same file opened for appending works. It is apparently not true if you are using Windows.
Original Response:
In 'r+' mode, using write method will write the string object to the file based on where the pointer is. In your case, it will append the string "Test abc" to the start of the file. See an example below:
>>> f=open("a","r+")
>>> f.read()
'Test abc\nfasdfafasdfa\nsdfgsd\n'
>>> f.write("foooooooooooooo")
>>> f.close()
>>> f=open("a","r+")
>>> f.read()
'Test abc\nfasdfafasdfa\nsdfgsd\nfoooooooooooooo'
The string "foooooooooooooo" got appended at the end of the file since the pointer was already at the end of the file.
Are you on a system that differentiates between binary and text files? You might want to use 'rb+' as a mode in that case.
Append 'b' to the mode to open the file in binary mode, on systems
that differentiate between binary and text files; on systems that
don’t have this distinction, adding the 'b' has no effect.
http://docs.python.org/2/library/functions.html#open
Every open file has an implicit pointer which indicates where data will be read and written. Normally this defaults to the start of the file, but if you use a mode of a (append) then it defaults to the end of the file. It's also worth noting that the w mode will truncate your file (i.e. delete all the contents) even if you add + to the mode.
Whenever you read or write N characters, the read/write pointer will move forward that amount within the file. I find it helps to think of this like an old cassette tape, if you remember those. So, if you executed the following code:
fd = open("testfile.txt", "w+")
fd.write("This is a test file.\n")
fd.close()
fd = open("testfile.txt", "r+")
print fd.read(4)
fd.write(" IS")
fd.close()
... It should end up printing This and then leaving the file content as This IS a test file.. This is because the initial read(4) returns the first 4 characters of the file, because the pointer is at the start of the file. It leaves the pointer at the space character just after This, so the following write(" IS") overwrites the next three characters with a space (the same as is already there) followed by IS, replacing the existing is.
You can use the seek() method of the file to jump to a specific point. After the example above, if you executed the following:
fd = open("testfile.txt", "r+")
fd.seek(10)
fd.write("TEST")
fd.close()
... Then you'll find that the file now contains This IS a TEST file..
All this applies on Unix systems, and you can test those examples to make sure. However, I've had problems mixing read() and write() on Windows systems. For example, when I execute that first example on my Windows machine then it correctly prints This, but when I check the file afterwards the write() has been completely ignored. However, the second example (using seek()) seems to work fine on Windows.
In summary, if you want to read/write from the middle of a file in Windows I'd suggest always using an explicit seek() instead of relying on the position of the read/write pointer. If you're doing only reads or only writes then it's pretty safe.
One final point - if you're specifying paths on Windows as literal strings, remember to escape your backslashes:
fd = open("C:\\Users\\johndoe\\Desktop\\testfile.txt", "r+")
Or you can use raw strings by putting an r at the start:
fd = open(r"C:\Users\johndoe\Desktop\testfile.txt", "r+")
Or the most portable option is to use os.path.join():
fd = open(os.path.join("C:\\", "Users", "johndoe", "Desktop", "testfile.txt"), "r+")
You can find more information about file IO in the official Python docs.
Reading and Writing happens where the current file pointer is and it advances with each read/write.
In your particular case, writing to the openFile, causes the file-pointer to point to the end of file. Trying to read from the end would result EOF.
You need to reset the file pointer, to point to the beginning of the file before through seek(0) before reading from it
You can read, modify and save to the same file in python but you have actually to replace the whole content in file, and to call before updating file content:
# set the pointer to the beginning of the file in order to rewrite the content
edit_file.seek(0)
I needed a function to go through all subdirectories of folder and edit content of the files based on some criteria, if it helps:
new_file_content = ""
for directories, subdirectories, files in os.walk(folder_path):
for file_name in files:
file_path = os.path.join(directories, file_name)
# open file for reading and writing
with io.open(file_path, "r+", encoding="utf-8") as edit_file:
for current_line in edit_file:
if condition in current_line:
# update current line
current_line = current_line.replace('john', 'jack')
new_file_content += current_line
# set the pointer to the beginning of the file in order to rewrite the content
edit_file.seek(0)
# delete actual file content
edit_file.truncate()
# rewrite updated file content
edit_file.write(new_file_content)
# empties new content in order to set for next iteration
new_file_content = ""
edit_file.close()