This question already has answers here:
CSV file written with Python has blank lines between each row
(11 answers)
Closed 2 years ago.
I am trying to clean a csv file that I downloaded from google sheets. But my output when I write it to a new csv file has the rows separated
code:
with open(output_file, 'w') as new_file:
writer = csv.writer(new_file)
writer.writerow(["Name", "Id", "Agent", "WinLoss", "Fees",
"Rakeback", "Rebate", "Net"])
for row in reader:
fees = row["Total"]
total_rebate = rebate(row["Winning+fees"], rebate_rate)
winloss = get_winnings(row["Winning+fees"], row["Total"])
rakeback = calculate_rakeback(fees)
net = calculate_net(winloss, total_rebate, rakeback)
writer.writerow([row["Player Name"], row["Player ID"], row["Agent Name"], winloss,
fees, rakeback, total_rebate, net])
new_file.close()
screenshot of output
The problem is that the standard windows line seperator \r\n is written to the csv file in text mode. But then the Python runtime replaces the \n with \r\n so that in the end the file containt \r\r\n at every line break, hence the extra line.
To solve this problem set the lineterminator attribute when creating the writer:
writer = csv.writer(new_file, lineterminator='\n')
You can read more about the problem in this thread.
Edit
As noted by Mark Tolonen the recommended way to go about this would be to add newline='' in the open statement, so with open(output_file, 'w', newline='') as new_file:
Also see the csv documentation for more info.
Related
This question already has answers here:
Difference between modes a, a+, w, w+, and r+ in built-in open function?
(9 answers)
Read file from line 2 or skip header row
(8 answers)
Closed 7 months ago.
my csv re-writes new data instead of appending to existing data, so I want to add more data to existing data. And also, how can I skip header when looping through my csv file?
my_file = open("data/food_and_nutrition.csv",'w', encoding='UTF8')
writer = csv.writer(my_file)
By using r+ mode, this will allow you to read and append new data to existing data. to skip the header line in your file, do this next(my_file)
The code below should help you through.
with open("data/food_and_nutrition.csv" 'r+', encoding='UTF8', newline='') as my_file:
writer = csv.writer(my_file)
csv_reader = csv.reader(my_file)
# Skip CSV heading
next(csv_reader)
for line in csv_reader:
if new_data not in line:
# To write
writer.writerow(new_data)
This question already has answers here:
CSV in Python adding an extra carriage return, on Windows
(6 answers)
Closed 1 year ago.
This code below works fine, but the data pasted via csv lib onto a CSV file leaves a row blank for every entry and I'm struggling to see why.
import praw
import configReddit
import csv
reddit = praw.Reddit(
client_id=configReddit.client_id,
client_secret=configReddit.client_secret,
password=configReddit.password,
user_agent=configReddit.user_agent,
username=configReddit.username,
)
with open('blockchainstable.csv', 'w', encoding="utf-8") as csvfile:
comment_writer = csv.writer(csvfile)
for comment in reddit.subreddit("CryptoCurrency").stream.comments():
print(comment.body)
comment_writer.writerow([comment.body])
Add , newline='' as in the example in the docs. E.g.,
with open('blockchainstable.csv', 'w', encoding="utf-8", newline='') as csvfile:
...
I tried to write output file as a CSV file but getting either an error or not the expected result. I am using Python 3.5.2 and 2.7 also.
Getting error in Python 3.5:
wr.writerow(var)
TypeError: a bytes-like object is required, not 'str'
and
In Python 2.7, I am getting all column result in one column.
Expected Result:
An output file same format as the input file.
Code:
import csv
f1 = open("input_1.csv", "r")
resultFile = open("out.csv", "wb")
wr = csv.writer(resultFile, quotechar=',')
def sort_duplicates(f1):
for i in range(0, len(f1)):
f1.insert(f1.index(f1[i])+1, f1[i])
f1.pop(i+1)
for var in f1:
#print (var)
wr.writerow([var])
If I am using resultFile = open("out.csv", "w"), I get one row extra in the output file.
If I am using above code, getting one row and column extra.
On Python 3, csv requires that you open the file in text mode, not binary mode. Drop the b from your file mode. You should really use newline='' too:
resultFile = open("out.csv", "w", newline='')
Better still, use the file object as a context manager to ensure it is closed automatically:
with open("input_1.csv", "r") as f1, \
open("out.csv", "w", newline='') as resultFile:
wr = csv.writer(resultFile, dialect='excel')
for var in f1:
wr.writerow([var.rstrip('\n')])
I've also stripped the lines from f1 (just to remove the newline) and put the line in a list; csv.writer.writerow wants a sequence with columns, not a single string.
Quoting the csv.writer() documentation:
If csvfile is a file object, it should be opened with newline='' [1]. [...] All other non-string data are stringified with str() before being written.
[1] If newline='' is not specified, newlines embedded inside quoted fields will not be interpreted correctly, and on platforms that use \r\n linendings on write an extra \r will be added. It should always be safe to specify newline='', since the csv module does its own (universal) newline handling.
Others have answered that you should open the output file in text mode when using Python 3, i.e.
with open('out.csv', 'w', newline='') as resultFile:
...
But you also need to parse the incoming CSV data. As it is your code reads each line of the input CSV file as a single string. Then, without splitting that line into its constituent fields, it passes the string to the CSV writer. As a result, the csv.writer will treat the string as a sequence and output each character , including any terminating new line character, as a separate field. For example, if your input CSV file contains:
1,2,3,4
Your output file would be written like this:
1,",",2,",",3,",",4,"
"
You should change the for loop to this:
for row in csv.reader(f1):
# process the row
wr.writerow(row)
Now the input CSV file will be parsed into fields and row will contain a list of strings - one for each field. For the previous example, row would be:
for row in csv.reader(f1):
print(row)
['1', '2', '3', '4']
And when that list is passed to the csv.writer the output to the file will be:
1,2,3,4
Putting all of that together you get this code:
import csv
with open('input_1.csv') as f1, open('out.csv', 'w', newline='') as resultFile:
wr = csv.writer(resultFile, dialect='excel')
for row in csv.reader(f1):
wr.writerow(row)
open file without b mode
b mode open your file as binary
you can open file as w
open_file = open("filename.csv", "w")
You are opening the input file in normal read mode but the output file is opened in binary mode, correct way
resultFile = open("out.csv", "w")
As shown above if you replace "wb" with "w" it will work.
I am currently trying to write a csv file in python. The format is as following:
1; 2.51; 12
123; 2.414; 142
EDIT: I already get the above format in my CSV, so the python code seems ok. It appears to be an excel issue which is olved by changing the settigs as #chucksmash mentioned.
However, when I try to open the generated csv file with excel, it doesn't recognize decimal separators. 2.414 is treated as 2414 in excel.
csvfile = open('C:/Users/SUUSER/JRITraffic/Data/data.csv', 'wb')
writer = csv.writer(csvfile, delimiter=";")
writer.writerow(some_array_with_floats)
Did you check that the csv file is generated correctly as you want? Also, try to specify the delimeter character that your using for the csv file when you import/open your file. In this case, it is a semicolon.
For python 3, I think your above code will also run into a TypeError, which may be part of the problem.
I just made a modification with your open method to be 'w' instead of 'wb' since the array has float and not binary data. This seemed to generate the result that you were looking for.
csvfile = open('C:/Users/SUUSER/JRITraffic/Data/data.csv', 'w')
An ugly solution, if you really want to use ; as the separator:
import csv
import os
with open('a.csv', 'wb') as csvfile:
csvfile.write('sep=;'+ os.linesep) # new line
writer = csv.writer(csvfile, delimiter=";")
writer.writerow([1, 2.51, 12])
writer.writerow([123, 2.414, 142])
This will produce:
sep=;
1;2.51;12
123;2.414;142
which is recognized fine by Excel.
I personally would go with , as the separator in which case you do not need the first line, so you can basically:
import csv
with open('a.csv', 'wb') as csvfile:
writer = csv.writer(csvfile) # default delimiter is `,`
writer.writerow([1, 2.51, 12])
writer.writerow([123, 2.414, 142])
And excel will recognize what is going on.
A way to do this is to specify dialect=csv.excel in the writer. For example:
a = [[1, 2.51, 12],[123, 2.414, 142]]
csvfile = open('data.csv', 'wb')
writer = csv.writer(csvfile, delimiter=";", dialect=csv.excel)
writer.writerows(a)
csvfile.close()
Unless Excel is already configured to use semicolon as its default delimiter, it will be necessary to import data.csv using Data/FromText and specify semicolon as the delimiter in the Text Import Wizard step 2 screen.
Very little documentation is provided for the Dialect class at csv.Dialect. More information about it is at Dialects in the PyMOTW's "csv – Comma-separated value files" article on the Python csv module. More information about csv.writer() is available at https://docs.python.org/2/library/csv.html#csv.writer.
This question already has answers here:
How do I append to a file?
(13 answers)
Closed 6 years ago.
I'm developing an application to write lines to a CSV. However, when I run the application a second time, the line that was already written is overwritten with the new line. How can I make it so that the writer writes to the next blank line, not one that already has data in it? I haven't found anything on how to do this. Here's my code below:
listsof = [1, 2, 3, 4]
with open('C:/Users/Family3/Downloads/weather.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(listsof)
Try this instead (note the 'a' for append):
with open('C:/Users/Family3/Downloads/weather.csv', 'a', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(listsof)
The 'a' opens the file for appending and does not erase the already existing file with the same name.
It is because, whenever you open your csv file, you never take into account that you might already have data in your file. When you load your csvfile into your program, you should append your new data to the file instead of just writing to it.