This question already has answers here:
How do I append to a file?
(13 answers)
Closed 6 years ago.
I'm developing an application to write lines to a CSV. However, when I run the application a second time, the line that was already written is overwritten with the new line. How can I make it so that the writer writes to the next blank line, not one that already has data in it? I haven't found anything on how to do this. Here's my code below:
listsof = [1, 2, 3, 4]
with open('C:/Users/Family3/Downloads/weather.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(listsof)
Try this instead (note the 'a' for append):
with open('C:/Users/Family3/Downloads/weather.csv', 'a', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(listsof)
The 'a' opens the file for appending and does not erase the already existing file with the same name.
It is because, whenever you open your csv file, you never take into account that you might already have data in your file. When you load your csvfile into your program, you should append your new data to the file instead of just writing to it.
Related
This question already has answers here:
Difference between modes a, a+, w, w+, and r+ in built-in open function?
(9 answers)
Read file from line 2 or skip header row
(8 answers)
Closed 7 months ago.
my csv re-writes new data instead of appending to existing data, so I want to add more data to existing data. And also, how can I skip header when looping through my csv file?
my_file = open("data/food_and_nutrition.csv",'w', encoding='UTF8')
writer = csv.writer(my_file)
By using r+ mode, this will allow you to read and append new data to existing data. to skip the header line in your file, do this next(my_file)
The code below should help you through.
with open("data/food_and_nutrition.csv" 'r+', encoding='UTF8', newline='') as my_file:
writer = csv.writer(my_file)
csv_reader = csv.reader(my_file)
# Skip CSV heading
next(csv_reader)
for line in csv_reader:
if new_data not in line:
# To write
writer.writerow(new_data)
This question already has answers here:
CSV in Python adding an extra carriage return, on Windows
(6 answers)
Closed 1 year ago.
This code below works fine, but the data pasted via csv lib onto a CSV file leaves a row blank for every entry and I'm struggling to see why.
import praw
import configReddit
import csv
reddit = praw.Reddit(
client_id=configReddit.client_id,
client_secret=configReddit.client_secret,
password=configReddit.password,
user_agent=configReddit.user_agent,
username=configReddit.username,
)
with open('blockchainstable.csv', 'w', encoding="utf-8") as csvfile:
comment_writer = csv.writer(csvfile)
for comment in reddit.subreddit("CryptoCurrency").stream.comments():
print(comment.body)
comment_writer.writerow([comment.body])
Add , newline='' as in the example in the docs. E.g.,
with open('blockchainstable.csv', 'w', encoding="utf-8", newline='') as csvfile:
...
This question already has answers here:
CSV file written with Python has blank lines between each row
(11 answers)
Closed 2 years ago.
I am trying to clean a csv file that I downloaded from google sheets. But my output when I write it to a new csv file has the rows separated
code:
with open(output_file, 'w') as new_file:
writer = csv.writer(new_file)
writer.writerow(["Name", "Id", "Agent", "WinLoss", "Fees",
"Rakeback", "Rebate", "Net"])
for row in reader:
fees = row["Total"]
total_rebate = rebate(row["Winning+fees"], rebate_rate)
winloss = get_winnings(row["Winning+fees"], row["Total"])
rakeback = calculate_rakeback(fees)
net = calculate_net(winloss, total_rebate, rakeback)
writer.writerow([row["Player Name"], row["Player ID"], row["Agent Name"], winloss,
fees, rakeback, total_rebate, net])
new_file.close()
screenshot of output
The problem is that the standard windows line seperator \r\n is written to the csv file in text mode. But then the Python runtime replaces the \n with \r\n so that in the end the file containt \r\r\n at every line break, hence the extra line.
To solve this problem set the lineterminator attribute when creating the writer:
writer = csv.writer(new_file, lineterminator='\n')
You can read more about the problem in this thread.
Edit
As noted by Mark Tolonen the recommended way to go about this would be to add newline='' in the open statement, so with open(output_file, 'w', newline='') as new_file:
Also see the csv documentation for more info.
I'm using the csv module in Python 3.8 to read and modify data in .csv files on macOS.
Python seems to change all of the EOL characters in my original .csv files.
This behaviour is not desired, because it makes it impossible for me to keep track of data changes.
All of the lines have '^M' appended to them, (which is '\r', a.k.a. the carriage return character).
The result is that, in Git, all of the lines are marked as changed.
When reading the original .csv file in binary mode, Python tells me that original EOL character is '\r\n'.
So I try to use this EOL character when writing to .csv:
def file_to_rows(path):
rows = []
with open(path) as csv_file:
row_reader = csv.reader(
csv_file,
delimiter=';',
quotechar='|',
quoting=csv.QUOTE_MINIMAL)
for row in row_reader:
rows.append(row)
return rows
def rows_to_file(rows, path):
with open(path, 'w', endline='\r\n') as csvfile:
rowswriter = csv.writer(
csvfile,
delimiter=';',
quotechar='|',
quoting=csv.QUOTE_MINIMAL)
for row in rows:
rowswriter.writerow(row)
# Running this function on a file should show NO changes in Git.
def csv_pass_through(path):
rows = file_to_rows(path)
rows_to_file(rows, path)
But git diff still shows me an '^M' has been added to all lines.
So it seems like Python is adding one carriage return character too many.
So, how does one read/write .csv data transparently (i.e. WITHOUT implicitly changing anything)?
martineau's comment was right. You can override the default '\r\n' in the writer constructor, like so:
def rows_to_file(rows, path):
with open(path, 'w') as csvfile:
rowswriter = csv.writer(
csvfile,
delimiter=';',
lineterminator='\n',
quotechar='|',
quoting=csv.QUOTE_MINIMAL)
for row in rows:
rowswriter.writerow(row)
I am currently trying to write a csv file in python. The format is as following:
1; 2.51; 12
123; 2.414; 142
EDIT: I already get the above format in my CSV, so the python code seems ok. It appears to be an excel issue which is olved by changing the settigs as #chucksmash mentioned.
However, when I try to open the generated csv file with excel, it doesn't recognize decimal separators. 2.414 is treated as 2414 in excel.
csvfile = open('C:/Users/SUUSER/JRITraffic/Data/data.csv', 'wb')
writer = csv.writer(csvfile, delimiter=";")
writer.writerow(some_array_with_floats)
Did you check that the csv file is generated correctly as you want? Also, try to specify the delimeter character that your using for the csv file when you import/open your file. In this case, it is a semicolon.
For python 3, I think your above code will also run into a TypeError, which may be part of the problem.
I just made a modification with your open method to be 'w' instead of 'wb' since the array has float and not binary data. This seemed to generate the result that you were looking for.
csvfile = open('C:/Users/SUUSER/JRITraffic/Data/data.csv', 'w')
An ugly solution, if you really want to use ; as the separator:
import csv
import os
with open('a.csv', 'wb') as csvfile:
csvfile.write('sep=;'+ os.linesep) # new line
writer = csv.writer(csvfile, delimiter=";")
writer.writerow([1, 2.51, 12])
writer.writerow([123, 2.414, 142])
This will produce:
sep=;
1;2.51;12
123;2.414;142
which is recognized fine by Excel.
I personally would go with , as the separator in which case you do not need the first line, so you can basically:
import csv
with open('a.csv', 'wb') as csvfile:
writer = csv.writer(csvfile) # default delimiter is `,`
writer.writerow([1, 2.51, 12])
writer.writerow([123, 2.414, 142])
And excel will recognize what is going on.
A way to do this is to specify dialect=csv.excel in the writer. For example:
a = [[1, 2.51, 12],[123, 2.414, 142]]
csvfile = open('data.csv', 'wb')
writer = csv.writer(csvfile, delimiter=";", dialect=csv.excel)
writer.writerows(a)
csvfile.close()
Unless Excel is already configured to use semicolon as its default delimiter, it will be necessary to import data.csv using Data/FromText and specify semicolon as the delimiter in the Text Import Wizard step 2 screen.
Very little documentation is provided for the Dialect class at csv.Dialect. More information about it is at Dialects in the PyMOTW's "csv – Comma-separated value files" article on the Python csv module. More information about csv.writer() is available at https://docs.python.org/2/library/csv.html#csv.writer.