Develop Reference

how to break a python list into sublist by separator

i have a list like this [[],[],['a'],['b'],[],[],['c'],['d'],['m']] and I need to keep the empty list intact, and group the remaining in place, resulting in [[],[],['a','b'],[],[],['c','d','m']]
I tried something like this based on another question but it tends to group the empty array as well. Thanks for any advice
my_temp = []
my_final = []
for item in my_array:
if item != []:
my_temp.append(item)
else:
my_final.append(my_temp);
my_temp = []
print(my_final);

There are a couple of issues. First off, you're adding a list the temp list you're building up, instead of extending it. Also, there's also an edge cases if your source list doesn't end with an empty list.
One possible way to solve this is to add each item to the last list on the array if that last item has items in it and the target item isn't empty:
my_final = []
for item in my_array:
if len(my_final) > 0 and len(my_final[-1]) > 0 and len(item) > 0:
my_final[-1].extend(item)
else:
my_final.append(item)
print(my_final)

Not optimised, but works:
my_array = [[],[],['a'],['b'],[],[],['c'],['d'],['m']]
my_temp = []
my_final = []
for item in my_array:
if not item: # item is an empty list
if my_temp: # if we previously have collected items, we add them now to the final list and clear the temp list
my_final.append(my_temp)
my_temp = []
my_final.append(item) # add empty list
else:
my_temp += item # keep collecting non-empty items
if my_temp: # same logic as within the loop, add any remaining items to the final list
my_final.append(my_temp)
print(my_final)
Output:
[[], [], ['a', 'b'], [], [], ['c', 'd', 'm']]

Your problems include:
You confused append and extend when adding my_temp
You neglect to append the final accumulated list
When you hit an empty list, you forget to add it at all: you weren't grouping them, you were ignoring them.
Amended code:
my_array = [[],[],['a'],['b'],[],[],['c'],['d'],['m']]
my_temp = []
my_final = []
for item in my_array:
if item != []:
my_temp.extend(item)
else:
my_final.append(my_temp);
if my_temp: # Also append item
my_final.append(item)
my_temp = []
if my_temp:
my_final.append(my_temp)
print(my_final)

Solution with simple generator:
def groups(values):
group = []
for value in values:
if value == []:
if group != []:
yield group
group = []
yield value
else:
group.append(value[0])
if group != []:
yield group
l = [[],[],['a'],['b'],[],[],['c'],['d'],['m']]
print(list(groups(l)))
# [[], [], ['a', 'b'], [], [], ['c', 'd', 'm']]

Loop over each item in a row and compare with each item from another row then save the result in a new column_python

I want to loop in python, over each item from a row against other items from the correspondent row from another column.
If item is not present in the row of the second column then should append to the new list that will be converted in another column (this should also eliminate duplicates when appending through if i not in c).
The goal is to compare items from each row of a column against items from the correspondent row in another column and to save the unique values from the first column, in a new column same df.
df columns
This is just an example, I have much many items in each row
I tried using this code but nothing happened and conversion of the list into the column it's not correct from what I have tested
a= df['final_key_concat'].tolist()
b = df['attributes_tokenize'].tolist()
c = []
for i in df.values:
for i in a:
if i in a:
if i not in b:
if i not in c:
c.append(i)
print(c)
df['new'] = pd.Series(c)
Any help is more than needed, thanks in advance

So seeing as you have these two variables one way would be:
a= df['final_key_concat'].tolist()
b = df['attributes_tokenize'].tolist()
Try something like this:
new = {}
for index, items in enumerate(a):
for thing in items:
if thing not in b[index]:
if index in new:
new[index].append(thing)
else:
new[index] = [thing]
Then map the dictionary to the df.
df['new'] = df.index.map(new)
There are better ways to do it but this should work.

This should be what you want:
import pandas as pd
data = {'final_key_concat':[['Camiseta', 'Tecnica', 'hombre', 'barate'],
['deportivas', 'calcetin', 'hombres', 'deportivas', 'shoes']],
'attributes_tokenize':[['The', 'North', 'Face', 'manga'], ['deportivas',
'calcetin', 'shoes', 'North']]} #recreated from your image
df = pd.DataFrame(data)
a= df['final_key_concat'].tolist() #this generates a list of lists
b = df['attributes_tokenize'].tolist()#this also generates a list of lists
#Both list a and b need to be flattened so as to access their elements the way you want it
c = [itm for sblst in a for itm in sblst] #flatten list a using list comprehension
d = [itm for sblst in b for itm in sblst] #flatten list b using list comprehension
final_list = [itm for itm in c if itm not in d]#Sort elements common to both list c and d
print (final_list)
Result
['Camiseta', 'Tecnica', 'hombre', 'barate', 'hombres']

def parse_str_into_list(s):
if s.startswith('[') and s.endswith(']'):
return ' '.join(s.strip('[]').strip("'").split("', '"))
return s
def filter_restrict_words(row):
targets = parse_str_into_list(row[0]).split(' ', -1)
restricts = parse_str_into_list(row[1]).split(' ', -1)
print(restricts)
# start for loop each words
# use set type to save words or list if we need to keep words in order
words_to_keep = []
for word in targets:
# condition to keep eligible words
if word not in restricts and 3 < len(word) < 45 and word not in words_to_keep:
words_to_keep.append(word)
print(words_to_keep)
return ' '.join(words_to_keep)
df['FINAL_KEYWORDS'] = df[[col_target, col_restrict]].apply(lambda x: filter_restrict_words(x), axis=1)

Python can't remove None value from list of lists

I'm trying to remove a None value from a csv file I have. I have converted blank values to None values in the first part of the below code but in the last part when I envoke filter It prints the column_list but the None values remain also. I need to remove them so I can work out max/min values of each which doesn't appear to work with them in the list?
with (inFile) as f:
_= next(f)
list_of_lists = [[float(i) if i.strip() != '' else None for i in line.split(',')[2:]]
for line in f]
inFile.close()
log_GDP_list = [item[0] for item in list_of_lists]
social_support_list = [item[1] for item in list_of_lists]
healthy_birth_list = [item[2] for item in list_of_lists]
freedom_choices_list = [item[3] for item in list_of_lists]
generosity_list = [item[4] for item in list_of_lists]
confidence_gov_list = [item[5] for item in list_of_lists]
column_list = []
column_list.append([log_GDP_list, social_support_list, healthy_birth_list, freedom_choices_list, generosity_list, confidence_gov_list])
res = list(filter(None, column_list))
print(res)
Also, when running the filter on just one of the row lists (such as log_GDP_list) it removes the None values but I still get an error saying I can't run max() or min() on floats (all values were converted from strings to floats in the first bit of the code).

You currently have something like this
l = [
float(i) if i.strip() != '' else None
for i in line.split(',')[2:]
]
what you want is this:
l = [
float(i)
for i in line.split(',')[2:]
if i.strip()
]
This way, when i.strip() evaluates to False, the item wont be added to the resulting list at all.

While loop within for loop for list of lists

I'm trying to create a big list that will contain lists of strings. I iterate over the input list of strings and create a temporary list.
Input:
['Mike','Angela','Bill','\n','Robert','Pam','\n',...]
My desired output:
[['Mike','Angela','Bill'],['Robert','Pam']...]
What i get:
[['Mike','Angela','Bill'],['Angela','Bill'],['Bill']...]
Code:
for i in range(0,len(temp)):
temporary = []
while(temp[i] != '\n' and i<len(temp)-1):
temporary.append(temp[i])
i+=1
bigList.append(temporary)

Use itertools.groupby
from itertools import groupby
names = ['Mike','Angela','Bill','\n','Robert','Pam']
[list(g) for k,g in groupby(names, lambda x:x=='\n') if not k]
#[['Mike', 'Angela', 'Bill'], ['Robert', 'Pam']]

Fixing your code, I'd recommend iterating over each element directly, appending to a nested list -
r = [[]]
for i in temp:
if i.strip():
r[-1].append(i)
else:
r.append([])
Note that if temp ends with a newline, r will have a trailing empty [] list. You can get rid of that though:
if not r[-1]:
del r[-1]
Another option would be using itertools.groupby, which the other answerer has already mentioned. Although, your method is more performant.

Your for loop was scanning over the temp array just fine, but the while loop on the inside was advancing that index. And then your while loop would reduce the index. This caused the repitition.
temp = ['mike','angela','bill','\n','robert','pam','\n','liz','anya','\n']
# !make sure to include this '\n' at the end of temp!
bigList = []
temporary = []
for i in range(0,len(temp)):
if(temp[i] != '\n'):
temporary.append(temp[i])
print(temporary)
else:
print(temporary)
bigList.append(temporary)
temporary = []

You could try:
a_list = ['Mike','Angela','Bill','\n','Robert','Pam','\n']
result = []
start = 0
end = 0
for indx, name in enumerate(a_list):
if name == '\n':
end = indx
sublist = a_list[start:end]
if sublist:
result.append(sublist)
start = indx + 1
>>> result
[['Mike', 'Angela', 'Bill'], ['Robert', 'Pam']]

Group elements of a list based on repetition of values

I am really new to Python and I am having a issue figuring out the problem below.
I have a list like:
my_list = ['testOne:100', 'testTwo:88', 'testThree:76', 'testOne:78', 'testTwo:88', 'testOne:73', 'testTwo:66', 'testThree:90']
And I want to group the elements based on the occurrence of elements that start with 'testOne'.
Expected Result:
new_list=[['testOne:100', 'testTwo:88', 'testThree:76'], ['testOne:78', 'testTwo:88'], ['testOne:73', 'testTwo:66', 'testThree:90']]

Just start a new list at every testOne.
>>> new_list = []
>>> for item in my_list:
if item.startswith('testOne:'):
new_list.append([])
new_list[-1].append(item)
>>> new_list
[['testOne:100', 'testTwo:88', 'testThree:76'], ['testOne:78', 'testTwo:88'], ['testOne:73', 'testTwo:66', 'testThree:90']]

Not a cool one-liner, but this works also with more general labels:
result = [[]]
seen = set()
for entry in my_list:
test, val = entry.split(":")
if test in seen:
result.append([entry])
seen = {test}
else:
result[-1].append(entry)
seen.add(test)
Here, we are keeping track of the test labels we've already seen in a set and starting a new list whenever we encounter a label we've already seen in the same list.
Alternatively, assuming the lists always start with testOne, you could just start a new list whenever the label is testOne:
result = []
for entry in my_list:
test, val = entry.split(":")
if test == "testOne":
result.append([entry])
else:
result[-1].append(entry)

It'd be nice to have an easy one liner, but I think it'd end up looking a bit too complicated if I tried that. Here's what I came up with:
# Create a list of the starting indices:
ind = [i for i, e in enumerate(my_list) if e.split(':')[0] == 'testOne']
# Create a list of slices using pairs of indices:
new_list = [my_list[i:j] for (i, j) in zip(ind, ind[1:] + [None])]

Not very sophisticated but it works:
my_list = ['testOne:100', 'testTwo:88', 'testThree:76', 'testOne:78', 'testTwo:88', 'testOne:73', 'testTwo:66', 'testThree:90']
splitting_word = 'testOne'
new_list = list()
partial_list = list()
for item in my_list:
if item.startswith(splitting_word) and partial_list:
new_list.append(partial_list)
partial_list = list()
partial_list.append(item)
new_list.append(partial_list)

joining the list into a string with delimiter |
step1="|".join(my_list)
splitting the listing based on 'testOne'
step2=step1.split("testOne")
appending "testOne" to the list elements to get the result
new_list=[[i for i in str('testOne'+i).split("|") if len(i)>0] for i in step2[1:]]