Related
I have a 2d list saved in a text file that looks like this (showing the first 2 entries):
('9b7dad', "text", 'http://imgur.com/gallery/SPdGm27', '1', 'A', 5)
('2b6ebj', 'text2', 'https://i.redd.it/lzft358csdi21.jpg', '1', 'B', 6)
How should this be loaded into a list? (so for example list[0][0] = '9b7dad', list[1][1] = 'text2' etc)
You could try this:
f = open(<your file path>)
result = [
[g.replace("'", "")
for g in l.strip('()\n').replace(' ', '').replace('"', '').split(',')]
for l in f.readlines()]
f.close()
Given a text file with each line in the form you've shown:
('9b7dad', "text", 'http://imgur.com/gallery/SPdGm27', '1', 'A', 5)
('2b6ebj', 'text2', 'https://i.redd.it/lzft358csdi21.jpg', '1', 'B', 6)
You can use Pandas which offers a more straightforward way to handle/manipulate different data types.
Import pandas and read in the file, here called 'stack.txt':
import pandas as pd
data = pd.read_csv('stack.txt', sep=",", header=None)
Returns only the list of list:
alist = data.values.tolist()
Print to check:
print(alist)
[['9b7dad', 'text', 'http://imgur.com/gallery/SPdGm27', '1', 'A', 5],
['2b6ebj', 'text2', 'https://i.redd.it/lzft358csdi21.jpg', '1', 'B', 6]]
If need to process columns:
for i in range(len(data.columns)):
if i == 0:
data[i] = data[i].map(lambda x: str(x)[1:])
data[i] = data[i].map(lambda x: str(x)[1:-1])
if i == 5:
data[i] = data[i].map(lambda x: str(x)[:-1])
data[i] = data[i].astype(int)
if 0 < i < 5:
data[i] = data[i].map(lambda x: str(x)[2:-1])
#!/usr/bin/env python
import sys
myList = []
for line in sys.stdin:
elems = line.strip('()\n').replace(' ', '').split(',')
elems = [x.strip('\'\"') for x in elems]
myList.append(elems)
print(myList[0][0])
print(myList[1][1])
To use:
$ python ./load.py < someText.txt
9b7dad
text2
Use int(), float(), or str() to coerce fields in elems to certain types, as needed. Use a try..except block to catch malformed input.
import ast
with open(file_name) as f:
content = f.readlines()
content = [list(ast.literal_eval(x)) for x in content]
How to read files:
In Python, how do I read a file line-by-line into a list?
More about eval:
Convert string representation of list to list
try this (converting tuple to list):
my_list = []
my_list.append(list('9b7dad', "text", 'http://imgur.com/gallery/SPdGm27', '1', 'A', 5))
my_list.append(list('2b6ebj', 'text2', 'https://i.redd.it/lzft358csdi21.jpg', '1', 'B', 6))
result is a list of lists i.e. a 2 dimensional list. You can easily modify the code to fetch a line at a time in a for loop and append it to the list. Consider using split(',') if it is a comma separated list instead of a tuple e.g.
mylist = []
with open(filename, 'r') as my_file:
for text in my_file.readlines()
my_list.append(text.split(','))
This question already has answers here:
How do I read and write CSV files with Python?
(7 answers)
Closed 4 years ago.
I want to save the options from a file in a list like:
options = [
["ala", "bala", "ma", "sc"],
["fg", "ada", "aas","asd"],
]
This options are for a quiz. For questions I was able to take the text line by line from the file, but for options this is not working:
with open('Options.txt', 'r') as k:
options = k.readlines()
What should I do?
You can store each set of options as 1 line, comma-separated:
# create a file
filename = "Options.txt"
with open(filename, 'w') as file:
file.write("a,b,c,d\n")
file.write("4,7,8,9\n")
file.write("27,k,l,pp\n")
Your file would look like:
a,b,c,d
4,7,8,9
27,k,l,pp
You can read in this file by:
# read file in
options =[]
with open(filename, 'r') as file:
for line in file:
options.append(line.strip().split(","))
print(options)
Output:
[['a', 'b', 'c', 'd'], ['4', '7', '8', '9'], ['27', 'k', 'l', 'pp']]
If your Options.txt file has data as follows:
ala bala ma sc
fg ada aas asd
---
--- so on
Do this:
with open('Options.txt', 'r') as k:
lst = []
file_line = k.readline()
while file_line:
file_line_lst = list(file_line.strip().split())
lst.append(file_line_lst)
file_line = k.readline()
The issue your facing here is this is more easily handled by pulling the contents in as a pair of lists data structures rather than a string object.
The simplest way is to rename your options.txt file options.py and assuming this code runs in the same directory, it could look like this.
In [4]: import options as o
In [5]: gennie = ((o.options[0]+o.options[1]))
In [6]: type(gennie)
Out[6]: list
In [7]: gennie
Out[7]: ['ala', 'bala', 'ma', 'sc', 'fg', 'ada', 'aas', 'asd']
In [8]: g = iter(gennie)
In [9]: while True:
...: try:
...: print(g.__next__())
...: except StopIteration:
...: break
...:
Output:
ala
bala
ma
sc
fg
ada
aas
asd
I have a recursive function that reads a list of scout records from a file, and adds then in order of their ID's to a list box. The function is called with addScouts(1) The function is below:
def addScouts(self,I):
i = I
with open(fileName,"r") as f:
lines = f.readlines()
for line in lines:
if str(line.split(",")[3])[:-1] == str(i):
self.scoutList.insert(END,line[:-1])
i += 1
return self.addScouts(i)
return
My issue is that my file ID's are ordered 1,2,4,5 as at some point I removed the scout with ID of 3. However, when I run the function to re-order the scouts in the list box (the function above), it only lists the scouts up to and including ID 3. This is because when i = 3, none of the items in the file are equal to 3, so the function reaches the end and returns before it gets a chance to check the remaining records.
File contents:
Kris,Rice,17,1
Olly,Fallows,17,2
Olivia,Bird,17,4
Louis,Martin,18,5
Any idea's how to fix this?
Just sort on the last column:
sorted(f,key=lambda x: int(x.split(",")[-1]))
You can use bisect to find where to put the new data to keep the data ordered after it is sorted once:
from bisect import bisect
import csv
with open("foo.txt") as f:
r = list(csv.reader(f))
keys = [int(row[-1]) for row in r]
new = ["foo","bar","12","3"]
ind = bisect(keys, int(new[-1]))
r.insert(ind,new)
print(r)
Output:
[['Kris', 'Rice', '17', '1'], ['Olly', 'Fallows', '17', '2'], ['foo', 'bar', '12', '3'], ['Olivia', 'Bird', '17', '4'], ['Louis', 'Martin', '18', '5']]
A simpler way is to check for the first row that has a higher id, if none are higher just append to the end:
import csv
with open("foo.txt") as f:
r = list(csv.reader(f))
new = ["foo","bar","12","3"]
key = int(new[-1])
ind = None
for i, row in enumerate(r):
if int(row[-1]) >= key:
ind = i
break
r.insert(ind, new) if ind is not None else r.append(new)
print(r)
Output:
[['Kris', 'Rice', '17', '1'], ['Olly', 'Fallows', '17', '2'], ['foo', 'bar', '12', '3'], ['Olivia', 'Bird', '17', '4'], ['Louis', 'Martin', '18', '5']
To always keep that file in order when adding a new value we just need to write to a temp file, writing the line in the correct place and then replace the original with the updated file:
import csv
from tempfile import NamedTemporaryFile
from shutil import move
with open("foo.csv") as f, NamedTemporaryFile(dir=".", delete=False) as temp:
r = csv.reader(f)
wr = csv.writer(temp)
new = ["foo", "bar", "12", "3"]
key, ind = int(new[-1]), None
for i, row in enumerate(r):
if int(row[-1]) >= key:
wr.writerow(new)
wr.writerow(row)
wr.writerows(r)
break
wr.writerow(row)
else:
wr.writerow(new)
move(temp.name, "foo.csv")
foo.csv after will have the data in order:
Kris,Rice,17,1
Olly,Fallows,17,2
foo,bar,12,3
Olivia,Bird,17,4
Louis,Martin,18,5
You can check if your list has the same length as your file and if not, you run addScouts again, and if true, you end. Like this:
def addScouts(self,I):
i = I
with open(fileName,"r") as f:
lines = f.readlines()
for line in lines:
if str(line.split(",")[3])[:-1] == str(i):
self.scoutList.insert(END,line[:-1])
i += 1
return self.addScouts(i)
if len(scoutList) < len(lines):
return self.addScouts(i+1)
else:
return
Okay so I have a list called names and there are other words in the list but this is the result of names[0]
Chen,David,M,334791530,,11Z,,16770712,,,,,,00015956753,
Chen,Peter,M,321564726,,11B,,19979810,,,,,,00012446698,
Chung,Rowan,M,32355988,,11T,,17890708,,,,,,00012127821,
Chung,Kyle,M,387638355,,10U,,19970317,,,,,,00015604870,
Fan,Mark,M,34217543,,10U,,19707713,,,,,,00015799079,
How do I split names[0] so that it comes out with just the last name, first name, and gender?
Here's the rest of my code:
file = open('CASS.txt', 'r')
f = file.readlines()
file.close()
for line in f:
if line.find('ICS3M105')>=0:
names = line.split()
for name in names[0]:
if name in range(0,1):
print(names)
for line in f:
names = line.split()
print names[0].split(',')[0:3]
with open('CASS.txt', 'r') as f:
for line in f:
name_last, name_first, gender = line.split(',')[0:3]
Or using the csv module which will may be more reliable for upcoming tasks
import csv
with open('CASS.txt', 'r') as f:
for row in csv.reader(f):
name_last, name_first, gender = row[0:3]
>>> s = """Chen,David,M,334791530,,11Z,,16770712,,,,,,00015956753,
Chen,Peter,M,321564726,,11B,,19979810,,,,,,00012446698,
Chung,Rowan,M,32355988,,11T,,17890708,,,,,,00012127821,
Chung,Kyle,M,387638355,,10U,,19970317,,,,,,00015604870,
Fan,Mark,M,34217543,,10U,,19707713,,,,,,00015799079,"""
You can use a list comprehension to split on commas, then use slicing to index element [0] to [2] (inclusive) of the split operation.
>>> [i.split(',')[:3] for i in s.split('\n')]
[['Chen', 'David', 'M'],
['Chen', 'Peter', 'M'],
['Chung', 'Rowan', 'M'],
['Chung', 'Kyle', 'M'],
['Fan', 'Mark', 'M']]
I have a function, table(), that reads a csv file and returns the rows in individual lists. I want to map these lists to create a dictionary, with field headers being the keys and the underlying rows being the values. I cannot seem to do this however. When I try to call only the first element within the list of lists I created from the function ( l) in the command prompt, it returns all the lists up to 'N', the first letter in the word 'None', despite me breaking (return) if reader is None. When I do the same with a sys.stdout to a text file, it does the same, but the 'N' is replaced with <type 'list'>. Does anyone know what I'm doing wrong, and how I can go about creating a dictionary (or a list of columns, for that matter) from a CSV file given my table() function?
import csv
def table():
with open('C:\\Python27\\test.csv', 'rb') as f:
reader = csv.reader(f)
for row in reader:
print row
if reader is None:
return
l = list(str(table()))
keys = l[0]
print keys
Output text file:
['Field1', 'Field2', 'Field3', 'Field4']
['a', '1', 'I', 'access']
['b', '2', 'II', 'accessing\n']
['c', '3', 'III', 'accommodation']
['d', '4', 'IIII', 'basically']
<type 'list'>
More pythonically
def table():
with open('C:\\Python27\\test.csv', 'rb') as f:
reader = csv.reader(f)
for row in reader:
yield row
You don't actually return anything from the table function. Try it like this:
def table():
with open('C:\\Python27\\test.csv', 'rb') as f:
lines = []
reader = csv.reader(f)
if reader:
for row in reader:
lines.append(row)
return lines
else:
return []