How to specify "," as 3-character delimiter string in .csv? - python

How to specify (when reading a file) and assign (in output) this 3-character string"," as .csv delimiter? For example: "col1","col2","col3","col4"
The code where it needs to be used for reading and output:
import csv
with open('a.csv', 'r') as infile, open('reordered.csv', 'a') as outfile:
fieldnames = ['B', 'C', 'A', 'D']
writer = csv.DictWriter(outfile, delimiter=',', fieldnames=fieldnames)
writer.writeheader()
for row in csv.DictReader(infile):
writer.writerow(row)
I tried to look it up (https://docs.python.org/2.0/ref/strings.html), etc. but it is not clear.
UPDATE: As pointed out by others, this may not be "," delimiter, but a , delimiter and "values". Regardless, my values contain commas ,, so the "," pattern between columns helps to maintain column structure.


You'll want to define the quote character, setting the delimiter will not get you the desired results.
import csv
with open('a.csv', 'r') as f:
reader = csv.reader(f, delimiter=',', quotechar='"')
for row in reader:
print row
Edit: As for changes to your code, it would look somewhat like this:
with open('a.csv', 'r') as infile, open('reordered.csv', 'a') as outfile:
fieldnamesout = ['B', 'C', 'A', 'D']
fieldnamesin = ['A', 'B', 'C', 'D']
reader = csv.DictReader(infile, delimiter=',', quotechar='"', fieldnames=fieldnamesin)
writer = csv.DictWriter(outfile, delimiter=',', quotechar='"', fieldnames=fieldnamesout, quoting=csv.QUOTE_ALL)
for row in reader:
writer.writerow(row)
Note the quoting=csv.QUOTE_ALL, which instructs the writer to quote all fields according to the quotechar, this may or may not be what you want, other options would be quoting=csv.QUOTE_NONNUMERIC.


Seems like your values are string literals. I don't know if the cvs module can handle this out of the box. It is not a parsing problem it is a data processing problem.
Your values are in the quote chars. So either you truncate them when processing your data field_value.replace('"', '') and add them back later on '"{}"'.format(field_value) or you straight operate on the data fields leaving the quote char in place.

Related

Trying to append csv into another csv AS A ROW but I am getting this AttributeError: '_io.TextIOWrapper' object has no attribute 'writerows'

i am trying to append the content in one of my CSV as a ROW to another csv but i am getting this attribute error...I am unsure how to fix it. I think the issue is with writer.writerows(row) but I don't what i should change it to for .writerows(row) to work
This is my below code for appending the first csv to the second csv.
with open(csv1', 'r', encoding='utf8') as reader, open(csv2', 'a', encoding='utf8') as writer:
for row in reader:
writer.writerows(row)

Use write() instead because writerows() is belong to csv.writer, not normal io. However, if you want to append at the end of the file, you need to make sure that the last row contain new line (i.e., \n) already.
with open('test1.csv', 'r', encoding='utf8') as reader:
with open('test2.csv', 'a', encoding='utf8') as writer:
writer.write("\n") # no need if the last row have new line already
for line in reader:
writer.write(line)
Or, if you want to use csv, you can use writerows() as shown in code below:
import csv
with open('test1.csv', 'r', encoding='utf8') as reader:
with open('test2.csv', 'a', encoding='utf8') as writer:
csv_reader = csv.reader(reader)
csv_writer = csv.writer(writer)
csv_writer.writerow([]) # again, no need if the last row have new line already
csv_writer.writerows(csv_reader)

Trying to take info from a CSV file, rearrange the columns and then write the new output to a new CSV file in Python

I have a set of data in a CSV file that basically requires re-ordering and the re-ordered data writing to a new CSV file.
The data looks like this to start
Communit,Equtions,8000,707757,2024.96,0,99
Annlins,EXSES,5063,536835,71.26,0,99
K ad,EXPSES,3028,40360,37.31,0,99
Harr White,EXSES,1644,10634264,85.55,0,99
Emge,Equutions,89250,68895,93.53,0,99
HMC,120PE249,83210,12039,1651.86,0,99
7 columns of data separated by a comma. To make it a bit more readable I shall focus on the first line.
So it starts like - Communit,Equtions,8000,707757,2024.96,0,99
And needs to end up like - Communit,8000,707757,2024.96,Equtions,99
My current code can print it to the screen but I'm struggling to get it to write to a file
import csv
with open('C:\\Impexp\\Input\\02B-210722.csv') as f:
reader = csv.reader(f, delimiter=',')
for row in reader:
print(",".join([row[0], row[2], row[3], row[4], row[1], row[6]]))
I did try changing the sys.stdout to a file but that wouldn't work.
I'm a bit rusty with my coding as I mainly use SQL and spreadsheets are my primary focus and this is my first time dabbling with Python.
Any help appreciated, have tried looking at other posts to try and cobble together a solution that fits my problem, but so far no joy.

You can use csv.writer() to write patched data into another CSV:
import csv
with open(r'C:\Impexp\Input\02B-210722.csv') as i_f, \
open(r'C:\Impexp\Input\02B-210722_patched.csv', 'w', newline='') as o_f:
reader = csv.reader(i_f)
writer = csv.writer(o_f)
for row in reader:
row[4] = row.pop(1)
writer.writerow(row)
If you want to modify existing file without creating new (which I don't recommend you to do without backups) you can open file in r+ mode, read all rows into list and rewrite same file:
import csv
with open(r'C:\Impexp\Input\02B-210722.csv', 'r+', newline='') as f:
reader = csv.reader(f)
rows = []
for row in reader:
row[4] = row.pop(1)
rows.append(row)
f.seek(0)
writer = csv.writer(f)
writer.writerows(rows)
f.truncate()

You can use pandas to rearrange the columns:
import pandas as pd
df = pd.read_csv('data.csv')
new_cols = ['A', 'C', 'D', 'E', 'F', 'B', 'G']
df = df[new_cols]
df.to_csv('new_data.csv', index=False)
data.csv
A,B,C,D,E,F,G
Communit,Equtions,8000,707757,2024.96,0,99
Annlins,EXSES,5063,536835,71.26,0,99
K ad,EXPSES,3028,40360,37.31,0,99
Harr White,EXSES,1644,10634264,85.55,0,99
Emge,Equutions,89250,68895,93.53,0,99
HMC,120PE249,83210,12039,1651.86,0,99
new_data.csv
A,C,D,E,F,B,G
Communit,8000,707757,2024.96,0,Equtions,99
Annlins,5063,536835,71.26,0,EXSES,99
K ad,3028,40360,37.31,0,EXPSES,99
Harr White,1644,10634264,85.55,0,EXSES,99
Emge,89250,68895,93.53,0,Equutions,99
HMC,83210,12039,1651.86,0,120PE249,99
Or, using the csv module:
import csv
with open('data.csv') as f, open('new_data.csv', 'w', newline='') as g:
reader = csv.reader(f, delimiter=',')
writer = csv.writer(g, delimiter=',')
for row in reader:
writer.writerow([row[0], row[2], row[3], row[4], row[1], row[6]])
data.csv
Communit,Equtions,8000,707757,2024.96,0,99
Annlins,EXSES,5063,536835,71.26,0,99
K ad,EXPSES,3028,40360,37.31,0,99
Harr White,EXSES,1644,10634264,85.55,0,99
Emge,Equutions,89250,68895,93.53,0,99
HMC,120PE249,83210,12039,1651.86,0,99
new_data.csv
Communit,8000,707757,2024.96,Equtions,99
Annlins,5063,536835,71.26,EXSES,99
K ad,3028,40360,37.31,EXPSES,99
Harr White,1644,10634264,85.55,EXSES,99
Emge,89250,68895,93.53,Equutions,99
HMC,83210,12039,1651.86,120PE249,99

Writing to csv has empty rows

I have this script:
import csv
import unicodedata
with open('output.csv', 'a', encoding='cp1252') as csvfile:
writer = csv.writer(csvfile)
with open('input.csv', 'r', encoding='cp1252') as csvfile:
for row in csv.reader(csvfile):
name_array = u''.join([c for c in unicodedata.normalize('NFKD', row[0].lower()) if (c.isalnum() or c.isspace()) if not unicodedata.combining(c)]).split()
writer.writerow(name_array)
which would create a name breakdown from a csv list of names. It works fine but the output has empty rows between successful name breakdowns.
Sample input.csv:
"Lastname, Firstname Secondname"
"Lastname1 Lastname2, Firstname1"
Sample output.csv:
lastname,firstname,secondname
##### empty row ####
lastname1,lastname2,firstname1
How do I remove the empty row?

In your csv.writer, specify a keyword argument for lineterminator='\n', which should eliminate the extra empty line.

Python - Can't read dictionary values from CSV file

I have two variations of CSV files. One of them uses double quotes, the other one doesn't.
A: "shipment_id","status","to_name","to_address_1" etc
B: shipment_id,status,to_name,to_address_1 etc
How can read the CSV and print out the value for shipment_id regardless of which type of CSV is submitted?
My code doesn't seem to work when the CSV doesn't use double quotes.
with open(file_location) as f_obj:
reader = csv.DictReader(f_obj, delimiter=',')
for line in reader:
print(line['shipment_id'])

Try this:
with open(file_location) as f_obj:
f_obj = f_obj.read().replace('"','').splitlines()
reader = csv.DictReader(f_obj, delimiter=',')
for line in reader:
print(line['shipment_id'])
.replace('"', '') will work if it has double quotes, and it will do nothing if it doesn't.
Let me know if it works :)

Based on what I think the .csv file should look like and experience from pandas read_csv, decided to give my input as follow
example of the test.csv file
"1233",No,N,C
9999,OK,C,N
example of the test1.csv file
"321",ok,P,A
980,No,A,G
"1980","No",A,"G"
Code with specified fieldnames for test.csv with print(line['shipment_id']):
import csv
with open('test.csv') as f_obj:
reader = csv.DictReader(f_obj, delimiter=',', fieldnames=['shipment_id','status','to_name','to_address_1'])
for line in reader:
print(line['shipment_id'])
output:
1233
9999
Code with specified fieldnames for test1.csv with print(line['shipment_id']):
with open('test1.csv') as f_obj:
reader_ddQ = csv.DictReader(f_obj, delimiter=',', fieldnames=['shipment_id','status','to_name','to_address_1'])
for line in reader_ddQ:
print(line['shipment_id'])
output:
321
980
1980
Code with specified fieldnames for test1.csv with print(line):
with open('test1.csv') as f_obj:
reader = csv.DictReader(f_obj, delimiter=',', fieldnames=['shipment_id','status','to_name','to_address_1'])
for line in reader:
print(line)
output:
OrderedDict([('shipment_id', '321'), ('status', 'ok'), ('to_name', 'P'), ('to_address_1', 'A')])
OrderedDict([('shipment_id', '980'), ('status', 'No'), ('to_name', 'A'), ('to_address_1', 'G')])
OrderedDict([('shipment_id', '1980'), ('status', 'No'), ('to_name', 'A'), ('to_address_1', 'G')])
Source for the csv.DictReader

You should be able to use quotechar as parameter:
reader = csv.DictReader(f_obj, delimiter=',', quotechar='"')
(or maybe '\"' - I don't know how Python handles this).
This should work on both versions of your data.
If DictReader doen't support the quotechar parameter, try to use it on csv.reader directly.

Remove double quotes from csv row

I have two files: src.csv and dst.csv. The code below reads the second row from src.csv and appends it to dst.csv. The issue is the output in dst.csv is contained within double quotes ("").
Expected result:
10, 5, 5, 10, 1
Output:
"10, 5, 5, 10, 1"
I have tried using quoting=csv.QUOTE_NONE, escapechar=' ' in csv.writer and it does remove the quotes though the output now contains a blank space after each csv value.
Here is my code:
import csv
with open('src.csv', 'r') as src, open('dst.csv', 'a', newline='') as dst:
wr = csv.writer(dst, dialect='excel', delimiter=',', quoting=csv.QUOTE_NONE, escapechar=' ')
next(src)
for row in src:
wr.writerow([row.rstrip('\n')])
Any suggestions?

You don't split the source file rows into columns so you just ended up writing a 1 column csv. Use a reader instead:
import csv
with open('src.csv', 'r') as src, open('dst.csv', 'a', newline='') as dst:
wr = csv.writer(dst, dialect='excel', delimiter=',', quoting=csv.QUOTE_NONE, escapechar=' ')
next(src)
reader = csv.reader(src)
for row in reader:
wr.writerow(row)

I think you have to use csv.reader() to read row as list of number - now you read row as one string and csv.writer has to add "" because you have , in this string.

Categories

Resources