I'm trying to write data from a list of lists to a csv file. This is a simplified version of what I have
class Point(object):
def __init__(self, weight, height):
self.weight = weight
self.height = height
def get_BMI(self):
return (self.weight * self.height) / 42 # this is not how you calculate BMI but let's say
myList = [[Point(30, 183)],[Point(63, 153)]]
Because of the way the data is set up, I store the points in a nested loop. If I wanted to access the first point object’s BMI, I would type
myList[0][0].get_BMI()
I want to write each point's BMI to a CSV (delimited by a comma). How do I do that?
Here's how I thought but it isn't exactly straight forward:
import csv
with open('output.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
writer.writerows(myList)
It doesn't return any error however it doesn't actually create the CSV file either. Also I want to write the values in myList[i][j].get_BMI() to file. I don't have a problem with permissions because I use Spyder (python IDE) as root. Right now I'm just running the script through the Spyder console but it should still work and output the CSV file.
writerows expects a list of list of strings or numbers. You should start by creating a list with the BMI values so that they can get written into your csv file:
import csv
BMIs = [[point.get_BMI() for point in points] for points in myList]
with open('output.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
writer.writerows(BMIs)
You can do this with writerows, as it expects a list of rows - each row should be formatted as per the dialect parameter to csv.writer, which can be ignored in this case without fear of any backfiring.
So writerows can take a structure that looks like myList. The problem is that you need to access all the points and grab their BMIs (this can be done in a list comprehension)
To illustrate how writerows can be used (and to add a number to each point, so that all your rows don't have just one entry (which would be frustrating), I added the call to enumerate.
Therefore, you no longer need complex loops or any such. Enjoy:
myList = [[Point(30, 183)],[Point(63, 153)]]
with open('output.csv', 'w') as outfile:
writer = csv.writer(outfile)
writer.writerows(enumerate(p.get_BMI() for p in itertools.chain.from_iterable(myList)))
There are three issues:
The nested lists must be flattened. To accomplish this, use itertools.chain.from_iterable.
The row data for the CSV must be customized. To accomplish this, use list comprehensions.
output.csv is not being created. I suspect that the output.csv is being created but being placed in an unexpected location. You could try hardcoding a full path for testing to see if this is the case.
Here is code that demonstrates #1 and #2:
import csv
from itertools import chain
with open('output.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
flattenedList = chain.from_iterable(myList)
writer.writerows((pt.weight, pt.height, pt.get_BMI()) for pt in flattenedList)
myList = [[Point(30, 183)],[Point(63, 153)]]
with open('output.csv', 'w') as outfile:
writer = csv.writer(outfile,delimiter=',')
for i in myList:
writer.writerow([i])
Related
In order to work out some asymptotic behavior on the topic of twin prime conjecture, I am required to take a raw file(.csv or .txt) and convert that data into a list in python where I could reach by pointing its index number.
That is, I have a big(~10 million numbers) list of prime numbers in .csv file, lets say that this is that list:
2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61,67,71,73,79,83
I am and trying to produce the following
[2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61,67,71,73,79,83]
in order to examine, ay the third element in the list, which is 5.
The approach I am taking is the following:
import sys
import csv
# The csv file might contain very huge fields, therefore increase the field_size_limit:
csv.field_size_limit(sys.maxsize)
with open('primes1.csv') as csvfile:
reader = csv.reader(csvfile, delimiter=' ')
output = []
for i in reader:
output.append(i)
Then, if printing,
for rows in output:
print(rows)
I am getting
['2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61,67,71,73,79,83'].
How does one resolve this? Thank you very much.
Maybe this:
with open("primes1.csv", "r") as f:
lst = [int(i) for i in f.read().split(",")]
You don't need to use the csv reader for that (like the other answer showed) but if you want to, you could do it like this, reading just the first row.
Your code is iterating rows and adding them to the output list, but you need to iterate columns just in the first row. The next(reader) call returns just the first row.
with open('test.csv','r') as csvFile:
reader = csv.reader(csvFile, delimiter=',')
output = [int(i) for i in next(reader)]
# alternate approach
# output = [int(i) for i in csvFile.read().strip().split(',')]
print(output)
For the last part of a Python assignment, I need to iterate through the lists of lists and print to the console the rows with category 'Hardware'. This is the csv file:
Hardware,Hammer,10,10.99
Hardware,Wrench,12,5.75
Food,Beans,32,1.99
Paper,Plates,100,2.59
The following is the code for the last part, which simply opens the file to be read and passed into a list:
def read_text():
with open("products.csv", "r", newline="") as file:
reader = csv.reader(file)
temp_prod = list(reader)
I'm having an issue with coding the right for loop to pull out the 'Hardware' rows. Help would be appreciated! Thanks.
When you are stuck, try to do a lot of printing to see what is going on. You were in a good point. temp_prod gave you a list of lists. You have to iterate through this list of list and ask if the first item is equal to your search criteria.
You code should look like that:
def read_text(my_file):
import csv
with open(my_file, "r", newline="") as file:
reader = csv.reader(file)
temp_prod = list(reader)
for row in temp_prod:
if row[0] == "Hardware":
print(row)
Call the function that way:
read_text("filepath.csv")
Please note that this function (as you requested) doesn't return anything, it only prints.
I have a list filled with values. I want to export the list to a .csv file.
I want the values to be on top of each other, yet right now they get exported next to each other.
e.g. I want it to be like:
1
2
3
not
1 2 3
How do I have to change my code?
with open('C:(...)file.csv', "w", newline='\n') as csfile:
wr = csv.writer(csfile, delimiter =';')
wr.writerow([item[0] for item in sntlst])
It is important to receive item[0] of my list.
Change the delimiter argument to '\n':
with open('C:(...)file.csv', "w", newline='\n') as csfile:
wr = csv.writer(csfile, delimiter='\n')
wr.writerow([item[0] for item in sntlst])
The newline='\n' isn't even necessary, so you can do without it:
with open('C:(...)file.csv', "w") as csfile:
wr = csv.writer(csfile, delimiter='\n')
wr.writerow([item[0] for item in sntlst])
The answer you have accepted (by #Ann Zen) may work, but that's due to a fluke because technically it's abusing the use of delimiter keyword argument with respect to the sample shown in the documentation of opening files for a csv.writer — and because of that it might now work properly on all platforms.
So, that said, in my opinion the code shown below would be a proper and better way to do things. It makes each item in your list a "row" of one item in the CSV file that's being created.
import csv
sntlst = [1, 2, 3]
output_file_path = 'list_file.csv'
with open(output_file_path, "w", newline='') as csvfile:
wr = csv.writer(csvfile)
wr.writerows([item] for item in sntlst)
Sorry if this has already been answered before; the searches I have done have not been helpful.
I have a file that stores data as such:
name,number
(Although perhaps not relevant to the question, I will have to add entries to this file. I know how to do this.)
My question is for the pythonic(?) way of analyzing the data and sorting it in ascending order. So if the file was:
alex,30
bob,20
and I have to add the entry
carol, 25
The file should be rewritten as
bob,20
carol,25
alex,30
My first attempt was to store the entire file as a string (by read()) and then split by lines to get a list of strings, procedurally split those strings by a comma, and then create a new list of scores then sort that, but this doesn't seem right and fails because I don't have a way to go "back" once I have the order of scores.
I am unable to use libraries for this program.
Edit:
My first attempt I did not test because all it manages to do is sort a list of the scores; I don't know of a way to get the "entries" back.
file = open("scores.txt" , "r")
data = file.read()
list_data = data.split()
data.append([name,score])
for i in range(len(list_data)):
list_scores = list_scores.append(list_data[i][1])
list_scores = sorted(list_scores)
As you can see, this gives me an ascending list of scores, but I do not know where to go from here in order to sort the list of name, score entries.
You will just have to write the sorted entries back to some file, using some basic string formatting:
with open('scores.txt') as f_in, open('file_out.txt', 'w') as f_out:
entries = [(x, int(y)) for x, y in (line.strip().split(',') for line in f_in)]
entries.append(('carol', 25))
entries.sort(key=lambda e: e[1])
for x, y in entries:
f_out.write('{},{}\n'.format(x, y))
I'm going to assume you're capable of putting your data into a .csv file in the following format:
Name,Number
John,20
Jane,25
Then you can use csv.DictReader to read this into a dictionary with something like as shown in the listed example:
with(open('name_age.csv', 'w') as csvfile:
reader = csv.DictReader(csvfile)
and write to it using
with(open('name_age.csv') as csvfile:
writer = csv.DictWriter(csvfile)
writer.writerow({'Name':'Carol','Number':25})
You can then sort it using python's built-in operator as shown here
this a function that will take a filename and sort it for you
def sort_file(filename):
f = open(filename, 'r')
text = f.read()
f.close()
lines = [i.split(',') for i in text.splitlines()]
lines.sort(key=lambda x: x[1])
lines = [', '.join(i) for i in lines]
text = '\n'.join(lines)
f = open(filename, 'w')
f.write(text)
f.close()
I have a csv file that has each line formatted with the line name followed by 11 pieces of data. Here is an example of a line.
CW1,0,-0.38,2.04,1.34,0.76,1.07,0.98,0.81,0.92,0.70,0.64
There are 12 lines in total, each with a unique name and data.
What I would like to do is extract the first cell from each line and use that to name the corresponding data, either as a variable equal to a list containing that line's data, or maybe as a dictionary, with the first cell being the key.
I am new to working with inputting files, so the farthest I have gotten is to read the file in using the stock solution in the documentation
import csv
path = r'data.csv'
with open(path,'rb') as csvFile:
reader = csv.reader(csvFile,delimiter=' ')
for row in reader:
print(row[0])
I am failing to figure out how to assign each row to a new variable, especially when I am not sure what the variable names will be (this is because the csv file will be created by a user other than myself).
The destination for this data is a tool that I have written. It accepts lists as input such as...
CW1 = [0,-0.38,2.04,1.34,0.76,1.07,0.98,0.81,0.92,0.70,0.64]
so this would be the ideal end solution. If it is easier, and considered better to have the output of the file read be in another format, I can certainly re-write my tool to work with that data type.
As Scironic said in their answer, it is best to use a dict for this sort of thing.
However, be aware that dict objects do not have any "order" - the order of the rows will be lost if you use one. If this is a problem, you can use an OrderedDict instead (which is just what it sounds like: a dict that "remembers" the order of its contents):
import csv
from collections import OrderedDict as od
data = od() # ordered dict object remembers the order in the csv file
with open(path,'rb') as csvFile:
reader = csv.reader(csvFile, delimiter = ' ')
for row in reader:
data[row[0]] = row[1:] # Slice the row up into 0 (first item) and 1: (remaining)
Now if you go looping through your data object, the contents will be in the same order as in the csv file:
for d in data.values():
myspecialtool(*d)
You need to use a dict for these kinds of things (dynamic variables):
import csv
path = r'data.csv'
data = {}
with open(path,'rb') as csvFile:
reader = csv.reader(csvFile,delimiter=' ')
for row in reader:
data[row[0]] = row[1:]
dicts are especially useful for dynamic variables and are the best method to store things like this. to access you just need to use:
data['CW1']
This solution also means that if you add any extra rows in with new names, you won't have to change anything.
If you are desperate to have the variable names in the global namespace and not within a dict, use exec (N.B. IF ANY OF THIS USES INPUT FROM OUTSIDE SOURCES, USING EXEC/EVAL CAN BE HIGHLY DANGEROUS (rm * level) SO MAKE SURE ALL INPUT IS CONTROLLED AND UNDERSTOOD BY YOURSELF).
with open(path,'rb') as csvFile:
reader = csv.reader(csvFile,delimiter=' ')
for row in reader:
exec("{} = {}".format(row[0], row[1:])
In python, you can use slicing: row[1:] will contain the row, except the first element, so you could do:
>>> d={}
>>> with open("f") as f:
... c = csv.reader(f, delimiter=',')
... for r in c:
... d[r[0]]=map(int,r[1:])
...
>>> d
{'var1': [1, 3, 1], 'var2': [3, 0, -1]}
Regarding variable variables, check How do I do variable variables in Python? or How to get a variable name as a string in Python?. I would stick to dictionary though.
An alternative to using the proper csv library could be as follows:
path = r'data.csv'
csvRows = open(path, "r").readlines()
dataRows = [[float(col) for col in row.rstrip("\n").split(",")[1:]] for row in csvRows]
for dataRow in dataRows: # Where dataRow is a list of numbers
print dataRow
You could then call your function where the print statement is.
This reads the whole file in and produces a list of lines with trailing newlines. It then removes each newline and splits each row into a list of strings. It skips the initial column and calls float() for each entry. Resulting in a list of lists. It depends how important the first column is?