I am trying to add a new row to my old CSV file. Basically, it gets updated each time I run the Python script.
Right now I am storing the old CSV rows values in a list and then deleting the CSV file and creating it again with the new list value.
I wanted to know are there any better ways of doing this.
with open('document.csv','a') as fd:
fd.write(myCsvRow)
Opening a file with the 'a' parameter allows you to append to the end of the file instead of simply overwriting the existing content. Try that.
I prefer this solution using the csv module from the standard library and the with statement to avoid leaving the file open.
The key point is using 'a' for appending when you open the file.
import csv
fields=['first','second','third']
with open(r'name', 'a') as f:
writer = csv.writer(f)
writer.writerow(fields)
If you are using Python 2.7 you may experience superfluous new lines in Windows. You can try to avoid them using 'ab' instead of 'a' this will, however, cause you TypeError: a bytes-like object is required, not 'str' in python and CSV in Python 3.6. Adding the newline='', as Natacha suggests, will cause you a backward incompatibility between Python 2 and 3.
Based in the answer of #G M and paying attention to the #John La Rooy's warning, I was able to append a new row opening the file in 'a'mode.
Even in windows, in order to avoid the newline problem, you must declare it as newline=''.
Now you can open the file in 'a'mode (without the b).
import csv
with open(r'names.csv', 'a', newline='') as csvfile:
fieldnames = ['This','aNew']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writerow({'This':'is', 'aNew':'Row'})
I didn't try with the regular writer (without the Dict), but I think that it'll be ok too.
If you use pandas, you can append your dataframes to an existing CSV file this way:
df.to_csv('log.csv', mode='a', index=False, header=False)
With mode='a' we ensure that we append, rather than overwrite, and with header=False we ensure that we append only the values of df rows, rather than header + values.
Are you opening the file with mode of 'a' instead of 'w'?
See Reading and Writing Files in the python docs
7.2. Reading and Writing Files
open() returns a file object, and is most commonly used with two arguments: open(filename, mode).
>>> f = open('workfile', 'w')
>>> print f <open file 'workfile', mode 'w' at 80a0960>
The first argument is a string containing the filename. The second argument is
another string containing a few characters describing the way in which
the file will be used. mode can be 'r' when the file will only be
read, 'w' for only writing (an existing file with the same name will
be erased), and 'a' opens the file for appending; any data written to
the file is automatically added to the end. 'r+' opens the file for
both reading and writing. The mode argument is optional; 'r' will be
assumed if it’s omitted.
On Windows, 'b' appended to the mode opens the file in binary mode, so
there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows
makes a distinction between text and binary files; the end-of-line
characters in text files are automatically altered slightly when data
is read or written. This behind-the-scenes modification to file data
is fine for ASCII text files, but it’ll corrupt binary data like that
in JPEG or EXE files. Be very careful to use binary mode when reading
and writing such files. On Unix, it doesn’t hurt to append a 'b' to
the mode, so you can use it platform-independently for all binary
files.
If the file exists and contains data, then it is possible to generate the fieldname parameter for csv.DictWriter automatically:
# read header automatically
with open(myFile, "r") as f:
reader = csv.reader(f)
for header in reader:
break
# add row to CSV file
with open(myFile, "a", newline='') as f:
writer = csv.DictWriter(f, fieldnames=header)
writer.writerow(myDict)
I use the following approach to append a new line in a .csv file:
pose_x = 1
pose_y = 2
with open('path-to-your-csv-file.csv', mode='a') as file_:
file_.write("{},{}".format(pose_x, pose_y))
file_.write("\n") # Next line.
[NOTE]:
mode='a' is append mode.
# I like using the codecs opening in a with
field_names = ['latitude', 'longitude', 'date', 'user', 'text']
with codecs.open(filename,"ab", encoding='utf-8') as logfile:
logger = csv.DictWriter(logfile, fieldnames=field_names)
logger.writeheader()
# some more code stuff
for video in aList:
video_result = {}
video_result['date'] = video['snippet']['publishedAt']
video_result['user'] = video['id']
video_result['text'] = video['snippet']['description'].encode('utf8')
logger.writerow(video_result)
I want to overwrite the data in a CSV file when my code runs a second time. I've been using the a+ mode when opening the CSV file and w for writing. However, the new data is getting appended to the existing file instead of overwriting it. How do I overwrite the file?
Here's my code:
with open(r'C:\Users\Desktop\news.csv', 'a+', encoding='utf-8-sig') as file:
writer = csv.writer(file, delimiter=',')
if file.tell()==0:
writer.writerow(['title', 'news', 'img-url'])
if writer.writerow != 0:
writer.writerow([title,news,img])
return writer
Call truncate to clear the file before writing to it.
I have a csv file that consists of 3 elements per list. I'm trying to make the first element of each list a nested list of string elements that I can easily search through. I can't get my code to print out my results, and therefore I don't know if I've actually correctly rewritten the csv file.
with open('trump_tweets_proto.csv', 'w+') as file:
contents = csv.reader(file)
contents2 = csv.writer(file)
for row in contents:
for info in row:
contents2.writerow(row[0].split())
print(row[0])
You can't have a reader and writer object open on the same file at the same time. Or if you can, you certainly can't use both simultaneously and not mess up your file.
Open the file for reading (and be sure to look at the csv module documentation on how to open a csv file correctly, you're missing the newline parameter and probably should specify an encoding as well).
Open another (temporary) file for writing, and then copy the temp file over the old file. Or do everyting in memory with the first file, then open the file for writing and write the new data directly.
I'm attempting to read a CSV file and then write the read CSV into another CSV file.
Here is my code so far:
import csv
with open ("mastertable.csv") as file:
for row in file:
print row
with open("table.csv", "w") as f:
f.write(file)
I eventually want to read a CSV file write to a new CSV with appended data.
I get this error when I try to run it.
Traceback (most recent call last):
File "readlines.py", line 8, in <module>
f.write(file)
TypeError: expected a character buffer object
From what I understood it seems that I have to close the file, but I thought with automatically closed it?
I'm not sure why I can write a string to text but I can't simply write a CSV to another CSV almost like just making a copy by iterating over it.
To read in a CSV and write to a different one, you might do something like this:
with open("table.csv", "w") as f:
with open ("mastertable.csv") as file:
for row in file:
f.write(row)
But I would only do that if the rows needed to be edited while transcribed. For the described use case, you can simply copy it with shutil before hand then opening it to append to it. This method will be much faster, not to mention far more readable.
The with operator will handle file closing for you, and will close the file when you leave that block of code (given by the indentation level)
It looks like you intend to make use of the Python csv module. The following should be a good starting point for what you are trying to acheive:
import csv
with open("mastertable.csv", "r") as file_input, open("table.csv", "wb") as file_output:
csv_input = csv.reader(file_input)
csv_output = csv.writer(file_output)
for cols in csv_input:
cols.append("more data")
csv_output.writerow(cols)
This will read mastertable.csv file in a line at a time as a list of columns. I append an extra column, and then write each line to table.csv.
Note, when you leave the scope of a with statement, the file is automatically closed.
The file variable is not really actual file data but it is a refernce pointer which is used to read data. When you do the following:
with open ("mastertable.csv") as file:
for row in file:
print row
file pointer get closed automatically. The write method expects a character buffer or a string as the input not a file pointer.
If you just want to copy data, you can do something like this:
data = ""
with open ("mastertable.csv","r") as file:
data = file.read()
with open ("table.csv","a") as file:
file.write(data)`
how can I clear a complete csv file with python. Most forum entries that cover the issue of deleting row/columns basically say, write the stuff you want to keep into a new file. I need to completely clear a file - how can I do that?
Basically you want to truncate the file, this can be any file. In this case it's a csv file so:
filename = "filewithcontents.csv"
# opening the file with w+ mode truncates the file
f = open(filename, "w+")
f.close()
Your question is rather strange, but I'll interpret it literally. Clearing a file is not the same as deleting it.
You want to open a file object to the CSV file, and then truncate the file, bringing it to zero length.
f = open("filename.csv", "w")
f.truncate()
f.close()
If you want to delete it instead, that's just a os filesystem call:
import os
os.remove("filename.csv")
The Python csv module is only for reading and writing whole CSV files but not for manipulating them. If you need to filter data from file then you have to read it, create a new csv file and write the filtered rows back to new file.