Develop Reference

Change a text file to csv (multiple rows)

I have a txt file that reads as:
interface 0/1
no data
no data
no data
no data
no data
interface 0/2
no data
etc...
I would like to have it output to a csv with a format of:
interface 0/1 | no data | no data | no data | no data | no data
interface 0/2 | no data | no data | no data | no data | no data
I have tried using the csv module and write row with no good results. Any help would be appreciated.

Looks like you want to group 6 lines into a row and write those, eg:
import csv
from itertools import islice
with open('input.txt') as fin, open('output.csv', 'w') as fout:
pipe_out = csv.writer(fout, delimiter='|')
rows = iter(lambda: list(islice(fin, 6)), [])
pipe_out.writerows(rows)

Here is a simple, slightly different answer that uses python 2.7 only (no csv)
f=open("data.txt", 'r') # original data
f1=open("output_data.txt", 'w') # this is to write the output
newline=""
for line in f:
if "interface" not in line:
newline=newline + r" | " + line.strip()
else:
# print newline # check this
f1.write(newline + '\n')
newline=line.strip()
# print newline # just a visual check
f1.write(newline)

You can also try without the csv module:
f = open("file.txt", "r")
of = open("file.csv", "w")
data = f.readlines()
checkpoint = ""
for line in data:
line = line.strip("\n")
if "interface" in line:
of.write(checkpoint + line)
checkpoint = "\n"
else:
of.write(" | " + line)
f.close()
of.close()

CSV module is usually not necessary. This script will do just fine.
f1 = open('file1.txt', 'r')
f2 = open('file2.csv', 'w+')
lines = f1.read()
for line in lines.split('\n'):
if line == "":
continue
elif line.startswith("interface"):
f2.write("\n"+line+",")
else:
f2.write(line+",")

I've done it like this:
content = ''
with open('text_file.txt', 'r') as txt_file:
content = txt_file.read()
blocks = content.split('interface')
csv_content = ''
for block in blocks:
if block != '':
csv_content += 'interface %s\n' % ' | '.join(block.splitlines())
with open('csv_file.csv', 'w') as csv_file:
csv_file.write(csv_content)

It can be by collecting a row's data from each line in the file and outputting it as a csv row whenever a line starting with 'interface' is encountered:
import csv
input_filename = 'interface.txt'
output_filename = 'interface.csv'
with open(input_filename) as infile, \
open(output_filename, 'w', newline='') as outfile:
writer = csv.writer(outfile, delimiter='|')
row = []
for line in (line.rstrip() for line in infile):
if not line.startswith('interface'):
row.append(line)
else: # new group
if row:
writer.writerow(row) # output last group
row = [line]
writer.writerow(row) # output last row
Contents of interface.csv file afterwards:
interface 0/1|no data|no data|no data|no data|no data
interface 0/2|no data|no data|no data|no data|no data

Missing headers in csv output file (python)

I am trying to output a csv file, but the problem is, the headers are gone, and I tried looking at my code line by line but I don't know what's wrong with my code..
My sample data is :
ABC.csv (assuming there are multiple data in it so I also add the code on how to remove it)
KeyID,GeneralID
145258,KL456
145259,BG486
145260,HJ789
145261,KL456
145259,BG486
145259,BG486
My code:
import csv
import fileinput
from collections import Counter
file_path_1 = "ABC.csv"
key_id = []
general_id = []
total_general_id = []
with open(file_path_1, 'rU') as f:
reader = csv.reader(f)
header = next(reader)
lines = [line for line in reader]
counts = Counter([l[1] for l in lines])
new_lines = [l + [str(counts[l[1])] for l in lines]
with open(file_path_1, 'wb') as f:
writer = csv.writer(f)
writer.writerow(header + ['Total_GeneralID'])
writer.writerows(new_lines)
with open(file_path_1, 'rU') as f:
reader = csv.DictReader(f)
for row in reader:
key_id.append(row['KeyID'])
general_id.append(row['GeneralID'])
total_general_id.append(['Total_GeneralID'])
New_List = [[] for _ in range(len(key_id))]
for attr in range(len(key_id)):
New_List[attr].append(key_id[attr])
New_List[attr].append(general_id[attr])
New_List[attr].append(total_general_id[attr])
with open('result_id_with_total.csv', 'wb+') as newfile:
header = ['KEY ID', 'GENERAL ID' , 'TOTAL GENERAL ID']
wr = csv.writer(newfile, delimiter=',', quoting = csv.QUOTE_MINIMAL)
wr.writerow(header) #I already add the headers but it won't work.
for item in New_List:
if item not in newfile:
wr.writerow(item)
Unfortunately, my output would be like this(result_id_with_total.csv);
145258,KL456,2
145259,BG486,1
145260,HJ789,1
145261,KL456,2
What I am trying to achieve;
KEY ID,GENERAL ID,TOTAL GENERAL ID
145258,KL456,2
145259,BG486,1
145260,HJ789,1
145261,KL456,2
My main problem in this code:
wr.writerow(header)
won't work.

This is to do with opening a file with wb+ (write bytes). Because when you write a file in bytes mode you need to pass to it an array of bytes and not strings.
I get this error in the console when I run it:
TypeError: a bytes-like object is required, not 'str'
Try changing wb+ to just w, this does the trick.
with open('result_id_with_total.csv', 'w') as newfile:
header = ['KEY ID', 'GENERAL ID' , 'TOTAL GENERAL ID']
wr = csv.writer(newfile, delimiter=',', quoting = csv.QUOTE_MINIMAL)

csv value modification for certain cells on odd rows on a particular column

Hi I'm trying to finish this small piece of code for modifying csv files, I've got this far with some help:
edit... some more info.
Basically what I’m looking to do is make some small changes to the csv file depending on the project and parent issue in JIRA. Python will then make the changes to the csv file before it is then read into JIRA - that’s the second part of the program I’ve not even really looked at yet.
I’m only looking to change the BOX-123 type cells and leave the blank ones blank.
But the idea of the program is that I can use it to make some small changes to a template which will then automatically create some issues in JIRA.
import os
import csv
project = 'Dudgeon'
parent = 'BOX-111'
rows = (1,1007)
current = os.getcwd()
filename = 'test.csv'
filepath = os.path.join(os.getcwd(), filename)
#print(current)
#print(filename)
print(filepath)
with open(filepath, 'r') as csvfile:
readCSV = csv.reader(csvfile)
next(readCSV, None)
for row in readCSV:
print(row[16])
row_count =sum(1 for row in readCSV)
print(row_count)
with open(filepath, 'r') as infile, open('out.csv', 'w') as outfile:
outfile.write(infile.readline()) # write out the 1st line
for line in infile:
cols = line.strip().split(',')
cols[16] = project
outfile.write(','.join(cols) + '\n')
with open('out.csv', 'r') as infile, open('out1.csv', 'w') as outfile:
for row in infile:
if row % 2 != 0:
cols [15] = parent
outfile.write()
Any help really appreciated.

You want to use the row's index when comparing to 0. Use enumerate():
with open('out.csv', 'r') as infile, open('out1.csv', 'w') as outfile:
for rowidx,row in enumerate(infile):
cols = row.strip().split(',')
if rowidx % 2 != 0:
cols[15] = parent
outfile.write(cols)
You really should be using the csv module here, though. Untested but should get you started.
with open('out.csv', 'r') as infile, open('out1.csv', 'w') as outfile:
reader = csv.reader(infile)
writer = csv.writer(outfile)
for rowidx,row in enumerate(reader):
if rowidx % 2 != 0:
row[15] = parent
writer.write_row(row)

A friend helped me last night and this is what they came up with:
with open(filepath, 'r') as infile, open('out.csv', 'w') as outfile:
outfile.write(infile.readline()) # write out the 1st line
for line in infile:
cols = line.strip().split(',')
cols[16] = project
outfile.write(','.join(cols) + '\n')
with open('out.csv', 'r') as infile, open('out1.csv', 'w') as outfile:
outfile.write(infile.readline()) # write out the 1st line
lineCounter = 0
for line in infile:
lineCounter += 1
cols = line.strip().split(',')
if lineCounter % 2 != 0:
cols[15] = parent
outfile.write(','.join(cols) + '\n')

How to export data from csv as a list(python 3)

I have a list like this(python 3)
my_list = [["xxx","moon",150],["wordq","pop",3]]
and i save it on a csv using this code
import csv
myfile = open("pppp.csv", 'wb')
with open("pppp.csv", "w", newline='') as myfile:
wr = csv.writer(myfile, quoting=csv.QUOTE_NONE)
wr.writerows(list_of_DVDsuppliers)
now i need to export this csv in to my program as a list and change the data .
please help me ?

Just convert the data you get from reader() to a list:
data = csv.reader(open('example.csv','r'))
data = list(data)
print data

Unless you have a reason why you are using newline='', you can skip that and below code works with python 2.7,
import csv
my_list = [["xxx","moon",150],["wordq","pop",3]]
myfile = open("pppp.csv", 'wb')
with open("pppp.csv", "w") as myfile:
wr = csv.writer(myfile, quoting=csv.QUOTE_NONE)
wr.writerows(my_list)
data = csv.reader(open('pppp.csv','r'))
for row in data:
print row

csv writer not closing file

im reading a csv file and then writing a new one:
import csv
with open('thefile.csv', 'rb') as f:
data = list(csv.reader(f))
import collections
counter = collections.defaultdict(int)
for row in data:
counter[row[11]] += 1
writer = csv.writer(open('/pythonwork/thefile_subset1.csv', 'w'))
for row in data:
if counter[row[11]] >= 500:
writer.writerow(row)
for some reason i cannot get the csv.writer to close the file. when i open the file it opens it as READ ONLY because it says that is still open.
how do i close thefile_subset1.csv after i am done with it?

with open('/pythonwork/thefile_subset1.csv', 'w') as outfile:
writer = csv.writer(outfile)
for row in data:
if counter[row[11]] >= 500:
writer.writerow(row)

You can break out the open command into its own variable, so that you can close it later.
f = open('/pythonwork/thefile_subset1.csv', 'w')
writer = csv.writer(f)
f.close()
csv.writer throws a ValueError if you try to write to a closed file.

close the file, not the csv writer. To do this, you'll need to open the file first before instantiating your writer rather than keeping it all in one line.
import csv
import collections
with open('thefile.csv', 'rb') as f:
data = list(csv.reader(f))
counter = collections.defaultdict(int)
for row in data:
counter[row[11]] += 1
f.close() # good idea to close if you're done with it
fSubset = open('/pythonwork/thefile_subset1.csv', 'w')
writer = csv.writer(fSubset)
for row in data:
if counter[row[11]] >= 500:
writer.writerow(row)
fSubset.close()
Also, I would suggest keeping your imports at the top of the script and closing the first file when you're done with it.

Force the writer to clean up:
del writer

Look at the difference:
with open('thefile.csv', 'rb') as f:
data = list(csv.reader(f))
vs:
writer = csv.writer(open('/pythonwork/thefile_subset1.csv', 'w'))