Python, data rearrange - python

I have data that look like below
id,Stage1Pow1,Stage1Pow2,Stage2Pow1,Stage2Pow2
A,1.0,1.5,1.1,1.4
B,0.9,1.2,0.9,1.1
C,1.0,1.0,0.8,0.8
how do I convert them into below format, in order to store and further analyze by stage.
id,StageNo,Pow1,Pow2
A,s1,1.0,1.5
A,s2,1.1,1.4
B,s1,0.9,1.2
B,s2,0.9,1.1
C,s1,1.0,1.0
C,s2,0.8,0.8
I'm very new to Python.

Let 'raw' be your input file name, and 'output' be your output file name.
import csv
data = []
with open('raw', 'r') as f:
csvreader = csv.reader(f, delimiter=',')
for row in csvreader:
data.append(row)
with open('output', 'w') as f:
csvwriter = csv.writer(f)
csvwriter.writerow(['id', 'StageNo', 'Pow1', 'Pow2'])
for row in data[1:]:
csvwriter.writerow([row[0], 's1', row[1], row[2]])
csvwriter.writerow([row[0], 's2', row[3], row[4]])
more info on https://docs.python.org/2/library/csv.html

You can use csv modules csv.DictReader() and csv.DictWriter()
import csv
with open('input.csv','r') as file:
reader = csv.DictReader(file)
with open('output.csv','w') as file_output:
writer = csv.DictWriter(file_output,fieldnames=['id','StageNo','Pow1','Pow2'])
writer.writeheader()
for line in reader:
line_split1 = {'id':line['id'],'StageNo':'s1','Pow1':line['Stage1Pow1'],'Pow2':line['Stage1Pow2']}
line_split2 = {'id':line['id'],'StageNo':'s2','Pow1':line['Stage2Pow1'],'Pow2':line['Stage2Pow2']}
writer.writerows([line_split1,line_split2])
Output:
id,StageNo,Pow1,Pow2
A,s1,1.0,1.5
A,s2,1.1,1.4
B,s1,0.9,1.2
B,s2,0.9,1.1
C,s1,1.0,1.0
C,s2,0.8,0.8

Related

How to rewrite only one field (line[n]) in csv but in same time be able to work with data?

I have a csv file with three lines and three columns here.
This is the csv file
At first I want to print all the lines.
Subsequently, for each of them, program check whether it is written in the second field(index 1) USA. If so, program will take the price from the third field and multiply it by two.
Now I need to rewrite this doubled price instead of 2000 (in line with the USA)
import csv
with open('countries.csv', 'r') as source:
reader = csv.reader(source)
writer = csv.writer(source)
for line in reader:
print(*line, sep=';')
with open('countries.csv', 'r') as source:
reader = csv.reader(source)
for line in reader:
if line[2] == "USA":
actual_price = int(line[2])
print(actual_price)
new_price = int(actual_price) * 2
print(new_price)
Someone has already advised me to use the creation of a new file.
But this causes problems when I want to work with the data in the file first.
import csv
import os
with open('countries.csv', mode='r') as oldfile, open(
'countries.tmp', mode='w', newline='') as newfile:
# define a reader and a writer
reader = csv.reader(oldfile, delimiter=';', quotechar='"')
writer = csv.writer(newfile, delimiter=';', quotechar='"',
quoting=csv.QUOTE_MINIMAL)
for line in reader:
print(*line, sep=';')
# copy everything changing the third field
for line in reader:
if line[2] == "USA":
actual_price = int(line[2])
print(actual_price)
new_price = int(actual_price) * 2
print(new_price)
for row in reader:
writer.writerow([row[0], row[1], ,new_price])
# ok, time to rename the file
os.replace('countries.tmp', 'countries.csv')
Thank you for answer
You are changing new_price at every iteration of your for loop. You should therefore be writing the row within the loop where you change the value:
with open('countries.csv', mode='r') as oldfile, open('countries.tmp', mode='w', newline='') as newfile:
reader = csv.reader(oldfile, delimiter=';', quotechar='"')
writer = csv.writer(newfile, delimiter=';', quotechar='"', quoting=csv.QUOTE_MINIMAL)
for row in reader:
price = int(row[2])
if row[1] == "USA":
price = price*2
writer.writerow([row[0], row[1], price])
os.replace('countries.tmp', 'countries.csv')

How to print output to a csv file?

I need help in writing code to print output to csv file. The code is posted below. Appreciate your help.
import csv
result = {}
with open('data.csv', 'rb') as csvfile:
csvreader = csv.reader(csvfile, delimiter=',', quotechar='"')
for row in csvreader:
if row[0] in result:
result[row[0]].append(row[1])
else:
result[row[0]] = [row[1]]
print(result)
Use csv.writer to write the rows in result to an output stream:
with open('output.csv', 'w') as csvfile:
csvwriter = csv.writer(csvfile)
for row in result.items():
csvwriter.writerow(row)

CSV created with Python is blank

This question was answered/resolved in the comments section.
I am a noob to Python and I wrote the code below thinking it would copy all the rows with "NY" as the state in the state field/column to a new csv file called "Output.csv".
import csv
f = open(r'C:\Users..\input.csv', 'r')
reader = csv.DictReader(f, delimiter=',')
output = open("C:...\Output.csv",'w')
fieldnames = ['firstScan', 'FinalScan', 'City', 'State', 'cld', 'daydiff']
writer = csv.DictWriter(output, fieldnames=fieldnames, delimiter=',')
for row in reader:
if row['State'] == 'NY':
writer.writerow(row)
Everything runs fine but the output csv is completely blank. The first tab is named "Output" but the sheet is blank. If I have it output to txt, that is blank as well. Any suggestions?
Try this instead:
import csv
with open('C:/Users/felasniper/Desktop/input.csv') as f:
reader = csv.DictReader(f, delimiter=',')
output = open("C:/Users/felasniper/Desktop/Output.csv", 'w')
fieldnames = ['firstScan', 'FinalScan', 'City', 'State', 'cld', 'daydiff']
writer = csv.DictWriter(output, fieldnames=fieldnames, delimiter=',')
for row in reader:
if row['State'] == 'NY':
writer.writerow(row)

Adding a column in csv using python

I have hundreds of .csv files with 40 rows and 34 columns each. I want to add a column at position 26 and column 26-34 should shift to make space for the new one. First row of the file is empty and second row has the titles and rest have the values. The new column should have a title in row two and rest of the rows can be zero.
Please help me with this code in python.
import csv
infilename = r'C:\Users\Sulabh Kumra\Desktop\input.csv'
outfilename = r'C:\Users\Sulabh Kumra\Desktop\output.csv'
with open(infilename, 'rb') as fp_in, open(outfilename, 'wb') as fp_out:
reader = csv.reader(fp_in, delimiter=",")
headers = next(reader) # read first row
writer = csv.writer(fp_out, delimiter=",")
writer.writerow(headers)
for row in reader:
row.append(row[2])
writer.writerow(row)
Inserting into a python list is pretty easy: some_list[2:2] = ['stuff','to','insert']
So your code would look like the following:
import csv
infilename = r'C:\Users\Sulabh Kumra\Desktop\input.csv'
outfilename = r'C:\Users\Sulabh Kumra\Desktop\output.csv'
with open(infilename, 'rb') as fp_in, open(outfilename, 'wb') as fp_out:
reader = csv.reader(fp_in, delimiter=",")
writer = csv.writer(fp_out, delimiter=",")
blank_line = next(reader)
writer.writerow(blank_line)
headers = next(reader) # read title row
headers[26:26] = ['New Label']
writer.writerow(headers)
for row in reader:
row[26:26] = [0]
writer.writerow(row)

Taking data from text file and writing it as a .csv file in python

EDIT: Thanks for the answers guys, got what I needed!!
Basically I am trying to take what I have stored in my textfile and I am trying to write that into a .csv file. In my file are tweets that I have stored and I am trying to have one tweet in each cell in my .csv file.
Right now it is only taking one tweet and creating a .csv file with it and I need it to take all of them. Any help is greatly appreciated. Here is what I have so far.
with open('reddit.txt', 'rb') as f:
reader = csv.reader(f, delimiter=':', quoting = csv.QUOTE_NONE)
for row in reader:
print row
cr = csv.writer(open('reddit.csv', 'wb'))
cr.writerow(row)
You'll need to create the writer outside of the loop:
with open('reddit.txt', 'rb') as input_file:
reader = csv.reader(input_file, delimiter=':', quoting = csv.QUOTE_NONE)
with open('reddit.csv', 'wb') as output_file:
writer = csv.writer(output_file)
for row in reader:
writer.writerow(row)
Although here it might be cleaner to open the files without with:
input_file = open('reddit.txt', 'rb')
output_file = open('reddit.csv', 'wb')
reader = csv.reader(input_file, delimiter=':', quoting=csv.QUOTE_NONE)
writer = csv.writer(output_file)
for row in reader:
writer.writerow(row)
input_file.close()
output_file.close()
Or you can still use with and just have a really long line:
with open('reddit.txt', 'rb') as input_file, open('reddit.csv', 'wb') as output_file:
reader = csv.reader(input_file, delimiter=':', quoting = csv.QUOTE_NONE)
writer = csv.writer(output_file)
for row in reader:
writer.writerow(row)
The line cr = csv.writer(open('reddit.csv', 'wb')) is inside the for loop. You need to open the file just once, place this line after
reader = csv.reader(f, delimiter=':', quoting = csv.QUOTE_NONE)
Then write to it as you did in each loop iteration.

Categories

Resources