I'm trying to remove a None value from a csv file I have. I have converted blank values to None values in the first part of the below code but in the last part when I envoke filter It prints the column_list but the None values remain also. I need to remove them so I can work out max/min values of each which doesn't appear to work with them in the list?
with (inFile) as f:
_= next(f)
list_of_lists = [[float(i) if i.strip() != '' else None for i in line.split(',')[2:]]
for line in f]
inFile.close()
log_GDP_list = [item[0] for item in list_of_lists]
social_support_list = [item[1] for item in list_of_lists]
healthy_birth_list = [item[2] for item in list_of_lists]
freedom_choices_list = [item[3] for item in list_of_lists]
generosity_list = [item[4] for item in list_of_lists]
confidence_gov_list = [item[5] for item in list_of_lists]
column_list = []
column_list.append([log_GDP_list, social_support_list, healthy_birth_list, freedom_choices_list, generosity_list, confidence_gov_list])
res = list(filter(None, column_list))
print(res)
Also, when running the filter on just one of the row lists (such as log_GDP_list) it removes the None values but I still get an error saying I can't run max() or min() on floats (all values were converted from strings to floats in the first bit of the code).
You currently have something like this
l = [
float(i) if i.strip() != '' else None
for i in line.split(',')[2:]
]
what you want is this:
l = [
float(i)
for i in line.split(',')[2:]
if i.strip()
]
This way, when i.strip() evaluates to False, the item wont be added to the resulting list at all.
Related
I have a list of list elements that I'm modifying. And after that modification I want to put them into a list that have the same structure as the original one:
list_data_type = [
[['void1']], [['uint8']], [['uint8'], ['uint32']], [['void2']], [['void3']], [['void4']], [['void5']]
]
Firstly I check for elements that have more than one element. So in this case that would be element with index number = 2. Then I change it into a string, strip it from brackets [] and " " marks and convert it to a list. Then I take other elements and do the same thing. After conversion I want to create a new list with those elements, but without unnecessary symbols. So my desired output would look like this:
list_data_converted = [
['void1'], ['uint8'], ['uint8', 'uint32'], ['void2'], ['void3'], ['void4'], ['void5']
]
Conversion works and I can print out elements, but I have a problem with appending them to a list. My code saves only last value from original list:
def Convert(string):
li = list(string.split(" "))
return li
for element in list_data_type:
if type(element) == list:
print("element is a:", element, type(element))
if len(element) > 1:
id_of_el = list_data_type.index(element)
el_str = str(element).replace('[', '').replace("'", '').replace("'", '').replace(']', '').replace(',', '')
el_con = Convert(el_str)
elif len(element <= 1):
elements_w_1_el = element
list_el = []
for i in range(len(elements_w_1_el)):
el_str_2 = str(element).replace('[', '').replace("'", '').replace("'", '').replace(']', '').replace(',', '')
list_el.append(elements_w_1_el[i])
And my out instead looking like "list_data_converted", has only one element - ['void5']. How do I fix that?
Converting a list to a string to flatten it is a very... cumbersome approach.
Try simple list-comprehension:
list_data_type = [[v[0] for v in l] for l in list_data_type]
Type casting the list into a string and then replacing the characters and then again converting the string into list might be bad way to achieve what you're doing.
Try this :
def flatten(lst):
if lst == []:
return lst
if isinstance(lst[0], list):
return flatten(lst[0]) + flatten(lst[1:])
return lst[:1] + flatten(lst[1:])
list_data_converted = [flatten(element) for element in list_data_type]
This actually flattens any list item inside list_data_type and keep them in a single list. This should work with any depth of list inside list.
Output print(list_data_converted) would give the following :
[
['void1'], ['uint8'], ['uint8', 'uint32'], ['void2'], ['void3'], ['void4'], ['void5']
]
I have the list_of_lists and I need to get the string that contains 'height' in the sublists and if there is no height at all I need to get 'nvt' for the whole sublist.
I have tried the following:
list_of_lists = [['width=9','length=3'],['width=6','length=4','height=4']]
_lists = []
for list in list_of_lists:
list1 = []
for st in list:
if ("height" ) in st:
list1.append(st)
else:
list1.append('nvt')
_lists.append(list1)
OUT = _lists
the result I need to have is :
_lists = ['nvt', 'height=4']
what I'm getting is:
_lists = [['nvt','nvt'],['nvt','nvt','height=4']]
This is a good case for implementing a for/else construct as follows:
list_of_lists = [['width=9','length=3'],['width=6','length=4','height=4']]
result = []
for e in list_of_lists:
for ss in e:
if ss.startswith('height'):
result.append(ss)
break
else:
result.append('nvt')
print(result)
Output:
['nvt', 'height=4']
Note:
This could probably be done with a list comprehension but I think this is more obvious and probably has no significant difference in terms of performance
This should work, you can assign height variable to first value in the sublist where s.startswith("height") is True, and if nothing matches this filter, you can assign height to 'nvt'.
_lists = []
for sublist in list_of_lists:
height = next(filter(lambda s: s.startswith("height"), sublist), 'nvt')
_lists.append(height)
And if you wish to be crazy, you can use list comprehension to reduce the code to the:
_lists = [next(filter(lambda s: s.startswith("height"), sublist), 'nvt') for sublist in list_of_lists]
Try this (Python 3.x):
import re
list_of_lists = [['width=9','length=3'],['width=6','length=4','height=4']]
_lists = []
r = re.compile("height=")
for li in list_of_lists:
match = list(filter(r.match, li))
if len(match) > 0:
_lists.extend(match)
else:
_lists.append('nvt')
OUT = _lists
print(OUT)
What is the best way to remove single quotes in a python list?
I have the following input:
['"A","B","C",False,False',
'"A","B","C",False,False',
'"A","B","C",False,False']
But i want the following output:
["A","B","C",False,False,
"A","B","C",False,False,
"A","B","C",False,False]
Your function (modified):
def f1(database, table, lst):
print(" processing mapping list...")
rows = []
for i in range(len(lst)):
sublist = []
sublist.append(database)
sublist.append(table)
sublist.append(lst[i][0].split(':')[0])
sublist.append(lst[i][1]["a"])
sublist.append(lst[i][1]["b"])
rows.append(sublist)
rows = [item for subl in rows for item in subl]
rows = json.dumps(rows)
return rows
I'm trying to create a big list that will contain lists of strings. I iterate over the input list of strings and create a temporary list.
Input:
['Mike','Angela','Bill','\n','Robert','Pam','\n',...]
My desired output:
[['Mike','Angela','Bill'],['Robert','Pam']...]
What i get:
[['Mike','Angela','Bill'],['Angela','Bill'],['Bill']...]
Code:
for i in range(0,len(temp)):
temporary = []
while(temp[i] != '\n' and i<len(temp)-1):
temporary.append(temp[i])
i+=1
bigList.append(temporary)
Use itertools.groupby
from itertools import groupby
names = ['Mike','Angela','Bill','\n','Robert','Pam']
[list(g) for k,g in groupby(names, lambda x:x=='\n') if not k]
#[['Mike', 'Angela', 'Bill'], ['Robert', 'Pam']]
Fixing your code, I'd recommend iterating over each element directly, appending to a nested list -
r = [[]]
for i in temp:
if i.strip():
r[-1].append(i)
else:
r.append([])
Note that if temp ends with a newline, r will have a trailing empty [] list. You can get rid of that though:
if not r[-1]:
del r[-1]
Another option would be using itertools.groupby, which the other answerer has already mentioned. Although, your method is more performant.
Your for loop was scanning over the temp array just fine, but the while loop on the inside was advancing that index. And then your while loop would reduce the index. This caused the repitition.
temp = ['mike','angela','bill','\n','robert','pam','\n','liz','anya','\n']
# !make sure to include this '\n' at the end of temp!
bigList = []
temporary = []
for i in range(0,len(temp)):
if(temp[i] != '\n'):
temporary.append(temp[i])
print(temporary)
else:
print(temporary)
bigList.append(temporary)
temporary = []
You could try:
a_list = ['Mike','Angela','Bill','\n','Robert','Pam','\n']
result = []
start = 0
end = 0
for indx, name in enumerate(a_list):
if name == '\n':
end = indx
sublist = a_list[start:end]
if sublist:
result.append(sublist)
start = indx + 1
>>> result
[['Mike', 'Angela', 'Bill'], ['Robert', 'Pam']]
My code
class getCol:
matrix = []
def __init__(self, file, delim=" "):
with open(file, 'rU') as f:
getCol.matrix = [filter(None, l.split(delim)) for l in f]
def __getitem__ (self, key):
column = []
for row in getCol.matrix:
try:
column.append(row[key])
except IndexError:
# pass
column.append("")
return column
list1 = getCol('/home/milenko/EDIs/site1/newst2.txt')[0]
list2 = getCol('/home/milenko/EDIs/site2/newst2.txt')[0]
list3 = getCol('/home/milenko/EDIs/site3/newst2.txt')[0]
list4 = getCol('/home/milenko/EDIs/site4/newst2.txt')[0]
list5 = getCol('/home/milenko/EDIs/site5/newst2.txt')[0]
list6 = getCol('/home/milenko/EDIs/site6/newst2.txt')[0]
list7 = getCol('/home/milenko/EDIs/site7/newst2.txt')[0]
list8 = getCol('/home/milenko/EDIs/site8/newst2.txt')[0]
list9 = getCol('/home/milenko/EDIs/site9/newst2.txt')[0]
list10 = getCol('/home/milenko/EDIs/site10/newst2.txt')[0]
list11 = getCol('/home/milenko/EDIs/site11/newst2.txt')[0]
list12 = getCol('/home/milenko/EDIs/site12/newst2.txt')[0]
list13 = getCol('/home/milenko/EDIs/site13/newst2.txt')[0]
list14 = getCol('/home/milenko/EDIs/site14/newst2.txt')[0]
list15 = getCol('/home/milenko/EDIs/site15/newst2.txt')[0]
list_of_lists = []
list_of_lists.append(list1)
list_of_lists.append(list2)
list_of_lists.append(list3)
list_of_lists.append(list4)
list_of_lists.append(list5)
list_of_lists.append(list6)
list_of_lists.append(list7)
list_of_lists.append(list8)
list_of_lists.append(list9)
list_of_lists.append(list10)
list_of_lists.append(list11)
list_of_lists.append(list12)
list_of_lists.append(list13)
list_of_lists.append(list14)
list_of_lists.append(list15)
result = []
# Loop the inner lists from list_of_lists, this will be list1, list2, list3...
for inner_list in list_of_lists:
# Loop each element of the inner lists
for element in inner_list:
# Make sure the element is not already in the result (this can also be done with sets)
if element not in result:
# Add the inner element to result
result.append(element)
# Sort the result
result = sorted(result)
print("\n".join(map(str, result)))
But problem is here
1.92413
10.15704
1026.00000
10260.00000
10672.43359
11.81549
1104.06055
114.21478
12.00000
12415.04102
1284.33289
13.74474
132.00000
132.86391
1376.00000
13760.00000
14442.18457
1494.04028
15.00000
I just want normal ordering from smallest to largest.How should I solve this?Is there any other alternative to sort?
It's sorting as strings. To sort as numbers, use the key argument:
result = sorted(result, key=float)
This converts each string to a float for sorting purposes, but leaves the
data as is.
Since you're assigning the result to the same identifier, you can also:
result.sort(key=float)
You need to convert the values in your list from strings to a numeric type like floats.