how can I clear a complete csv file with python. Most forum entries that cover the issue of deleting row/columns basically say, write the stuff you want to keep into a new file. I need to completely clear a file - how can I do that?
Basically you want to truncate the file, this can be any file. In this case it's a csv file so:
filename = "filewithcontents.csv"
# opening the file with w+ mode truncates the file
f = open(filename, "w+")
f.close()
Your question is rather strange, but I'll interpret it literally. Clearing a file is not the same as deleting it.
You want to open a file object to the CSV file, and then truncate the file, bringing it to zero length.
f = open("filename.csv", "w")
f.truncate()
f.close()
If you want to delete it instead, that's just a os filesystem call:
import os
os.remove("filename.csv")
The Python csv module is only for reading and writing whole CSV files but not for manipulating them. If you need to filter data from file then you have to read it, create a new csv file and write the filtered rows back to new file.
Related
I am trying to create a binary file (called textsnew) and then append two (previously created) binary files to it. When I print the resulting (textsnew), it only shows the first file appended to it, not the second one. I do however see that the size of the new file (textsnew) is the sum of the two appended files. Maybe Im opening it incorrectly? This is my code
with open("/path/textsnew", "ab") as myfile, open("/path/names", "rb") as file2:
myfile.write(file2.read())
with open("/path/textsnew", "ab") as myfile, open("/path/namesthree", "rb") as file2:
myfile.write(file2.read())
this code is for reading the file:
import pickle
infile1 = open('/path/textsnew','rb')
names1 = pickle.load(infile1)
print (names1)
Open the new file, write its data.
Then, while the new file is still open (in append mode), open the second file, read its data and immediately write that data to the first file.
Then repeat the procedure for the third file.
Everything in binary, of course, although it will work just as well with text files. Linux/Macos/*nix don't even really care.
This also assume that the built-in I/O buffer size will read the full file contents in one go, as in your question. Otherwise, you would need to create a loop around the read/write parts.
with open('/path/textsnew', 'ab') as fpout:
fpout.write(data)
with open('/path/names', 'rb') as fpin:
fpout.write(fpin.read())
with open('/path/namesthree', 'rb') as fpin:
fpout.write(fpin.read())
I have a csv file that consists of 3 elements per list. I'm trying to make the first element of each list a nested list of string elements that I can easily search through. I can't get my code to print out my results, and therefore I don't know if I've actually correctly rewritten the csv file.
with open('trump_tweets_proto.csv', 'w+') as file:
contents = csv.reader(file)
contents2 = csv.writer(file)
for row in contents:
for info in row:
contents2.writerow(row[0].split())
print(row[0])
You can't have a reader and writer object open on the same file at the same time. Or if you can, you certainly can't use both simultaneously and not mess up your file.
Open the file for reading (and be sure to look at the csv module documentation on how to open a csv file correctly, you're missing the newline parameter and probably should specify an encoding as well).
Open another (temporary) file for writing, and then copy the temp file over the old file. Or do everyting in memory with the first file, then open the file for writing and write the new data directly.
I'm attempting to read a CSV file and then write the read CSV into another CSV file.
Here is my code so far:
import csv
with open ("mastertable.csv") as file:
for row in file:
print row
with open("table.csv", "w") as f:
f.write(file)
I eventually want to read a CSV file write to a new CSV with appended data.
I get this error when I try to run it.
Traceback (most recent call last):
File "readlines.py", line 8, in <module>
f.write(file)
TypeError: expected a character buffer object
From what I understood it seems that I have to close the file, but I thought with automatically closed it?
I'm not sure why I can write a string to text but I can't simply write a CSV to another CSV almost like just making a copy by iterating over it.
To read in a CSV and write to a different one, you might do something like this:
with open("table.csv", "w") as f:
with open ("mastertable.csv") as file:
for row in file:
f.write(row)
But I would only do that if the rows needed to be edited while transcribed. For the described use case, you can simply copy it with shutil before hand then opening it to append to it. This method will be much faster, not to mention far more readable.
The with operator will handle file closing for you, and will close the file when you leave that block of code (given by the indentation level)
It looks like you intend to make use of the Python csv module. The following should be a good starting point for what you are trying to acheive:
import csv
with open("mastertable.csv", "r") as file_input, open("table.csv", "wb") as file_output:
csv_input = csv.reader(file_input)
csv_output = csv.writer(file_output)
for cols in csv_input:
cols.append("more data")
csv_output.writerow(cols)
This will read mastertable.csv file in a line at a time as a list of columns. I append an extra column, and then write each line to table.csv.
Note, when you leave the scope of a with statement, the file is automatically closed.
The file variable is not really actual file data but it is a refernce pointer which is used to read data. When you do the following:
with open ("mastertable.csv") as file:
for row in file:
print row
file pointer get closed automatically. The write method expects a character buffer or a string as the input not a file pointer.
If you just want to copy data, you can do something like this:
data = ""
with open ("mastertable.csv","r") as file:
data = file.read()
with open ("table.csv","a") as file:
file.write(data)`
Can we write on (Here it should be editting infact) & read the same csv file at the same time in Python using the csv library?
Can a csv file be opened for editing & appending?
If so, how?
Short answer: no
Long answer: it depends
Appending data is perfectly possible using the CSV writer. Just open the file in append "a" mode:
with file("data.csv", "a" as fh:
w = csvwriter(fh):
w.writerow(...)
Editing a CSV file is not that simple as you will need to insert and remove parts of the file unless the columns you are editing are fixed length. The csv module has no builtin method for this.
You can open the original file, remove (or rename the original file) and open a new file with the same name:
with file("data.csv", "r") as rfh:
os.remove("data.csv"):
r = csvreader(rfh)
with file("data.csv", "w") as wfh:
w = csvwriter(wfh)
# ... read from r and write to w
Under Linux the original file will stay available for reading until the point when it is closed, so you do not need to rename it beforehand. I'm not too familiar with windows so you might need to rename the original file before creating the new file and remove the old file after closing it.
Another important bit: You can read and write from/to the same file without any troubles if your writing is limited to appending data.
with file("data.csv", "r") as rfh, file("data.csv", "a") as wfh:
r = csvreader(rfh)
w = csvwriter(wfh)
# you can read using r and append using a
Just be careful - your reader will be able to read the lines that you have just written using the writer. Be careful that you do not end up in an infinite loop causing a very big file.
I have the following code which is intended to remove specific lines of a file. When I run it, it prints the two filenames that live in the directory, then deletes all information in them. What am I doing wrong? I'm using Python 3.2 under Windows.
import os
files = [file for file in os.listdir() if file.split(".")[-1] == "txt"]
for file in files:
print(file)
input = open(file,"r")
output = open(file,"w")
for line in input:
print(line)
# if line is good, write it to output
input.close()
output.close()
open(file, 'w') wipes the file. To prevent that, open it in r+ mode (read+write/don't wipe), then read it all at once, filter the lines, and write them back out again. Something like
with open(file, "r+") as f:
lines = f.readlines() # read entire file into memory
f.seek(0) # go back to the beginning of the file
f.writelines(filter(good, lines)) # dump the filtered lines back
f.truncate() # wipe the remains of the old file
I've assumed that good is a function telling whether a line should be kept.
If your file fits in memory, the easiest solution is to open the file for reading, read its contents to memory, close the file, open it for writing and write the filtered output back:
with open(file_name) as f:
lines = list(f)
# filter lines
with open(file_name, "w") as f: # This removes the file contents
f.writelines(lines)
Since you are not intermangling read and write operations, the advanced file modes like "r+" are unnecessary here, and only compicate things.
If the file does not fit into memory, the usual approach is to write the output to a new, temporary file, and move it back to the original file name after processing is finished.
One way is to use the fileinput stdlib module. Then you don't have to worry about open/closing and file modes etc...
import fileinput
from contextlib import closing
import os
fnames = [fname for fname in os.listdir() if fname.split(".")[-1] == "txt"] # use splitext
with closing(fileinput.input(fnames, inplace=True)) as fin:
for line in fin:
# some condition
if 'z' not in line: # your condition here
print line, # suppress new line but adjust for py3 - print(line, eol='') ?
When using inplace=True - the fileinput redirects stdout to be to the file currently opened. A backup of the file with a default '.bak' extension is created which may come in useful if needed.
jon#minerva:~$ cat testtext.txt
one
two
three
four
five
six
seven
eight
nine
ten
After running the above with a condition of not line.startswith('t'):
jon#minerva:~$ cat testtext.txt
one
four
five
six
seven
eight
nine
You're deleting everything when you open the file to write to it. You can't have an open read and write to a file at the same time. Use open(file,"r+") instead, and then save all the lines to another variable before writing anything.
You should not open the same file for reading and writing at the same time.
"w" means create a empty for writing. If the file already exists, its data will be deleted.
So you can use a different file name for writing.