The code I currently have:
lst = [[0, 0, 0, 0, 0, 1], [1, 2, 3, 4, 5, 4, 3, 0], [1, 4, 5, 6, 7, 0]]
for week in lst:
print(week)
for day in week:
if day == 0:
week.remove(day)
print(lst)
#[[0, 0, 1], [1, 2, 3, 4, 5, 4, 3], [1, 4, 5, 6, 7]]
I have hard time understanding why the zeroes in the first sublist are not being removed
Is this what you're looking for?
lst = [[0, 0, 0, 0, 0, 1], [1, 2, 3, 4, 5, 4, 3, 0], [1, 4, 5, 6, 7, 0]]
lst = [[j for j in x if not j == 0] for x in lst]
lst
output
[[1], [1, 2, 3, 4, 5, 4, 3], [1, 4, 5, 6, 7]]
The problem with your code is that you edit the list while working with it. For this reason every second zero in the first sublist is not even being checked, let alone removed.
Little improvement on your code saves indexes of zeroes and then deletes them.
lst = [[0, 0, 0, 0, 0, 1], [1, 2, 3, 4, 5, 4, 3, 0], [1, 4, 5, 6, 7, 0]]
for week in lst:
print(week)
for day in week:
ind = [week.index(day) for day in week if day == 0]
for i in ind:
week.pop(i)
print(lst)
Use list comprehension
lst = [[x for x in y if x] for y in lst]
Just replace remove by pop and it should work
From https://docs.python.org/3/tutorial/datastructures.html :
list.remove(x)
Remove the first item from the list whose value is equal to x. It raises a ValueError if there is no such item.
list.pop([i])
Remove the item at the given position in the list, and return it. If no index is specified, a.pop() removes and returns the last item in the list. (The square brackets around the i in the method signature denote that the parameter is optional, not that you should type square brackets at that position. You will see this notation frequently in the Python Library Reference.)
However i would try the following :
lst = [list(filter(lambda day: day != 0, week)) for week in lst]
or
lst = [[day for day in week if day != 0] for week in lst]
Sources :
Remove all occurrences of a value from a list?
https://www.w3schools.com/python/python_lists_comprehension.asp
I need to convert a string representation into a 2 dimensional list.
I am trying to read inputs from a text file. All inputs follow a standard format where each line represents a different variable. When I read the following line from the input file using f.readline():
[[0, 2, 1, 2, 3, 0], [2, 0, 3, 0, 1, 0], [2, 0, 0, 3, 3, 0], [0, 2, 3, 0, 0, 3], [1, 2, 3, 1, 0, 2], [2, 1, 0, 1, 3, 0]]
This line is read in as a string, but I need it converted to a 2 dimensional list. A constraint for this project is I cannot use any packages, only base python.
How do I do this?
Something like that should work:
text = "[[0, 2, 1, 2, 3, 0], [2, 0, 3, 0, 1, 0], [2, 0, 0, 3, 3, 0], [0, 2, 3, 0, 0, 3], [1, 2, 3, 1, 0, 2], [2, 1, 0, 1, 3, 0]]"
output = []
for sublist in text.split('], '):
sublist = sublist.replace('[', '').replace(']', '')
data = []
for number in sublist.split(', '):
data.append(int(number))
output.append(data)
print(output)
Using list comprehension:
text = "[[0, 2, 1, 2, 3, 0], [2, 0, 3, 0, 1, 0], [2, 0, 0, 3, 3, 0], [0, 2, 3, 0, 0, 3], [1, 2, 3, 1, 0, 2], [2, 1, 0, 1, 3, 0]]"
output = [[int(number) for number in sublist.replace('[', '').replace(']', '').split(', ')] for sublist in text.split('], ')]
print(output)
This short script should work:
string_lists = "[[0, 2, 1, 2, 3, 0], [2, 0, 3, 0, 1, 0], [2, 0, 0, 3, 3, 0], [0, 2, 3, 0, 0, 3], [1, 2, 3, 1, 0, 2], [2, 1, 0, 1, 3, 0]]"
parsed_list = string_lists[1:-2].split("], ") # remove last 2 brackets and then split into lists
for index, row in enumerate(parsed_list):
parsed_list[index] = row[1:].split(", ") # split sublists into individual elements
parsed_list[index] = [int(num) for num in parsed_list[index]] # cast each element to int
print(parsed_list)
Note that you don't have to create additional list like I did, you can just work on your starting string (just change "parsed_list" to "string_lists" everywhere in above script)
Let's see some pattern firstly. We have:
[ [a1, a2, ..., ap1], [b1, b2, ..., bp2], ..., [xx1, xx2, ..., xxpn] ].
we have to select some separator which will help us divide it into groups. I'd prob go for ], [ but you can choose even ],
so we have to remove [[ and ]] from our string to have clear pattern.
inp.strip('[]')
we split it into sublists
inp.strip('[]').split('], [')
now every sublist in format:
a1, a2, ..., ap1
we split it on , so we will get items.
sublist.split(', ')
We summarize it and take as one list comprehension:
list_from_string = [[int(item) for item in sublist.split(', ')] for sublist in inp.strip('[]').split('], [')]
What a fun problem! Found this simple solution that iterates the string only once:
output_list = list() # Contains the output
inner_list = None # Current "row"/"column"
current_value = "" # Keep track of the current value
for char in list_as_text[1:-1]:
if char == " ":
# Ignore spaces
continue
elif char.isdigit(): # Add `or char == "."` to support float values
# Found a digit, keep track of the current value
current_value += char
elif char == ",":
# Found a comma, save the value in inner_list
inner_list.append(int(current_value)) # Use `float(current_value)` to support float values
# Start a new current_vale
current_value = ""
elif char == "[":
# We start a new inner_list
inner_list = list()
elif char == "]":
# We completed an inner_list
output_list.append(inner_list)
Here is a step-by-step approach using split(), rstrip() and lstrip() methods of the string datatype:
row = '[[0, 2, 1, 2, 3, 0], [2, 0, 3, 0, 1, 0], [2, 0, 0, 3, 3, 0], [0, 2, 3, 0, 0, 3], [1, 2, 3, 1, 0, 2], [2, 1, 0, 1, 3, 0]]'
listOfStrings = [s.rstrip(' ,]') for s in row.split('[')]
listOfStrings = [s for s in listOfStrings if len(s) > 0]
listOfLists = [[int(n.lstrip(' ')) for n in s.split(',')] for s in listOfStrings]
print(listOfLists)
Line 1 creates a string containing comma-separated values for each list in the original with some extra empty strings.
Line 2 eliminates the empty strings.
Line 3 creates a list of lists of numbers.
I have a particular problem that I'm trying to solve and not having much luck.
It's quite difficult to explain fully but I have a list of lists that I'm generating through an intent detection algorithm I'm writing, I need to pair them together based on the first element of each of the sublists.
The list will be sorted so they will always be in order of lowest to highest, and essentially each number has a list of numbers to pair with. So if index[0][0] is 3, then pair with any 10, 11, 18, 19 or 20 - duplicating if necessary. And so on.
SUB_NOUNS = {
3: [10, 11, 18, 19, 20],
10: [22],
}
Here are some examples of my inputs and my desired outputs from them, explaining what I need to happen.
# For only 1 item leave unchanged
In: [[3, 1, 0]]
Out: [[3, 1, 0]]
# Second item is sub item of first item - so join.
In: [[3, 1, 0], [10, 2, 0]]
Out: [[3, 1, 0, 10, 2, 0]]
# Second and third item are sub items of first item, make two lists.
In: [[3, 2, 0], [10, 3, 0], [10, 4, 0]]
Out: [[3, 2, 0, 10, 3, 0], [3, 2, 0, 10, 4, 0]]
# Three items, no sub items - unchanged
In: [[3, 1, 0], [3, 2, 0], [3, 3, 0]]
Out: [[3, 1, 0], [3, 2, 0], [3, 3, 0]]
# Third item is sub item of second item which is sub item of first item - Join all
In: [[3, 2, 0], [10, 3, 0], [22, 12, 0]]
Out: [[3, 2, 0, 10, 3, 0, 22, 12, 0]]
# Third item is sub item of second and first item, make two lists.
In: [[3, 1, 0], [3, 2, 0], [18, 0, 0]]
Out: [[3, 1, 0, 18, 0, 0], [3, 2, 0, 18, 0, 0,]]
Hopefully I've explained this well enough, I'm not exactly looking for a perfect solution as there are other potential combinations that I haven't listed here but if someone could point me in the right direction I can probably figure it out from there.
Your question is a lot harder than it seems with a quick read!
That said, I have tried few things (cause it made me stubborn...) and I will present: (1) observations/questions, (2) a potential answer and (3) some comments on your original solution
Starting with a recap of requirements:
You have a list of lists
The sub-lists are ordered based on their first element
You have a map, SUB_NOUNS, which tells us which lists can be followed by which others based on each list's first element
You are looking for potential combinations
Observations (and maybe open questions):
In the OP you are showing a SUB_NOUNS example with 2 entries. I am assuming that more than two entries are also possible
In the example of [[3, 2, 0], [10, 3, 0], [22, 12, 0]] we see that you are looking for the longest possible combo, therefore:
Combinations can have 2+ levels (unbounded). This smells like "recursion" to me and that's how the proposed solution is
Altho [10, 3, 0], [22, 12, 0] (without the [3, 2, 0]) is a valid combination, you do not list it in the example output. This makes me assume that you do not want to reuse lists that you have already used in a longer combo as a starting point of another combo
In the example of [[3, 1, 0], [3, 2, 0], [18, 0, 0]], based on the desired output we see that the same list can participate in multiple combos, as long as the "prefix" (the lists before it) are different
Proposed Solution
As mentioned earlier, this is recursive and has two parts:
get_all_combos which gathers all the potential combinations for all the lists that can act as a starting point. This also controls list reuse (see later)
get_combos: The recursive part of the solution that forms all the combinations for a given starting list
(You can ignore or remove the logging, but is usually needed in recursion for debugging purposes...)
def get_all_combos(data, allow_reuse=False):
""" For each element in the data list, find its combinations recursively"""
all_combos = []
all_used_lists = set()
for item_num, item in enumerate(data):
# Do not start new combinations from lists
# that appear somewhere inside another combination
if item_num in all_used_lists and not allow_reuse:
continue
combos, used = get_combos(data, item_num)
if combos:
all_combos.extend(combos)
all_used_lists.update(used)
return all_combos or data
def get_combos(data, list_num=0):
"""
Return all combinations from list_num onwards and also the indexes
of the lists that were used to form those combos
"""
lg.debug(f"{list_num * 4 * ' '} list num: {list_num}")
combos = []
used_lists = set()
current_list = data[list_num]
current_first = current_list[0]
# Filter all allowed pairs
for pair_num, pair in enumerate(data[list_num + 1:], list_num + 1):
# Skip elements that cannot be combined (we could also use filter())
if pair[0] not in SUB_NOUNS[current_first]:
continue
lg.debug(f"{list_num * 4 * ' '} Pair_num {pair_num}")
# Get combos of this pair
subcombos, used = get_combos(data, pair_num)
lg.debug(f"{list_num * 4 * ' '} Subcombos {subcombos}")
# If there are no subcombinations, just combine the current list
# with its child one (pair)
if not subcombos:
combos.append(current_list + pair)
used_lists.update({list_num, pair_num})
lg.debug(f"{list_num * 4 * ' '} Inserting {combos[0]}")
continue
# Here we have sub-combos. For each one of them, merge it to the
# current combos
for combo in subcombos:
combos.append(current_list + combo)
used_lists.update(used | {list_num})
lg.debug(f"{list_num * 4 * ' '} Extending appends {combos[0]}")
lg.debug(f"{list_num * 4 * ' '} Combos {combos}")
lg.debug(f"{list_num * 4 * ' '} Used {used_lists}")
return combos, used_lists
Now, before we go to look at some output of the above code, lets take a quick look at your solution (inline comments):
def pair_lists(data):
pairs = []
# > Reversal can be potentially avoided here
for noun in reversed(data):
try:
# > Instead of itertool+filter false, you could just use filter()
# > with the reverse condition right?
result = itertools.filterfalse(lambda x: x[0] not in SUB_NOUNS[noun[0]], data)
for each in result:
paired = []
# > ^--- The following if is always false since paired is always []
if each not in paired:
paired.insert(0, each)
# > insert vs append depends on the reversal in the outer loop
paired.insert(0, noun)
pairs.append(list(itertools.chain(*paired)))
except KeyError:
pass
return pairs if pairs else data
A more compact version which should be 100% the same logic is the following:
def pair_lists_urban(data):
pairs = []
for noun in data:
try:
result = filter(lambda x: x[0] in SUB_NOUNS[noun[0]], data)
for each in result:
pairs.append(noun + each)
except KeyError:
pass
return pairs if pairs else data
Note that both the above versions have a contraint/issue: They look at only two levels of lists to form a combo (ie N and N+1). This will become apparent in the next section
Tests
So, lets run all the solutions so far with the example lists that you provided. The main script is:
import itertools
import logging as lg
import sys
SUB_NOUNS = {
3: [10, 11, 18, 19, 20],
10: [22],
}
# ... funcs for each solution
test_lists = [
[[3, 1, 0]],
[[3, 1, 0], [10, 2, 0]],
[[3, 2, 0], [10, 3, 0], [10, 4, 0]],
[[3, 1, 0], [3, 2, 0], [3, 3, 0]],
[[3, 2, 0], [10, 3, 0], [22, 12, 0]],
[[3, 1, 0], [3, 2, 0], [18, 0, 0]],
]
for l in test_lists:
print(f"\nIn: {l}")
print(f"Recur Out: {get_all_combos(l)}")
print(f"Iter. Out: {pair_lists(l)}")
print(f"Iter2 Out: {pair_lists_urban(l)}")
Looking at the output, we have (with comments):
# All good
In: [[3, 1, 0]]
Recur Out: [[3, 1, 0]]
Iter. Out: [[3, 1, 0]]
Iter2 Out: [[3, 1, 0]]
# All good
In: [[3, 1, 0], [10, 2, 0]]
Recur Out: [[3, 1, 0, 10, 2, 0]]
Iter. Out: [[3, 1, 0, 10, 2, 0]]
Iter2 Out: [[3, 1, 0, 10, 2, 0]]
# All good
In: [[3, 2, 0], [10, 3, 0], [10, 4, 0]]
Recur Out: [[3, 2, 0, 10, 3, 0], [3, 2, 0, 10, 4, 0]]
Iter. Out: [[3, 2, 0, 10, 3, 0], [3, 2, 0, 10, 4, 0]]
Iter2 Out: [[3, 2, 0, 10, 3, 0], [3, 2, 0, 10, 4, 0]]
# All good
In: [[3, 1, 0], [3, 2, 0], [3, 3, 0]]
Recur Out: [[3, 1, 0], [3, 2, 0], [3, 3, 0]]
Iter. Out: [[3, 1, 0], [3, 2, 0], [3, 3, 0]]
Iter2 Out: [[3, 1, 0], [3, 2, 0], [3, 3, 0]]
# ! some issues: Recursion outputs the desired result while the other
# two solutions do not. This is because as mentioned earlier, these
# are only looking at two levels for each pair and that is also why
# resulting combos have only 6 elements
In: [[3, 2, 0], [10, 3, 0], [22, 12, 0]]
Recur Out: [[3, 2, 0, 10, 3, 0, 22, 12, 0]]
Iter. Out: [[10, 3, 0, 22, 12, 0], [3, 2, 0, 10, 3, 0]]
Iter2 Out: [[3, 2, 0, 10, 3, 0], [10, 3, 0, 22, 12, 0]]
# Results look ok, one nit: combos are coming out in different order
# (some append/insert issue somewhere i 'd say)
In: [[3, 1, 0], [3, 2, 0], [18, 0, 0]]
Recur Out: [[3, 1, 0, 18, 0, 0], [3, 2, 0, 18, 0, 0]]
Iter. Out: [[3, 2, 0, 18, 0, 0], [3, 1, 0, 18, 0, 0]]
Iter2 Out: [[3, 1, 0, 18, 0, 0], [3, 2, 0, 18, 0, 0]]
Considerations
Focusing on the recursive solution, I am looking into few further cases:
What if "loops" appear in the SUB_NOUNS
What if we have some more SUB_NOUNS
How can we control list reuse in the proposed solution
For this I am using the following test:
SUB_NOUNS = {
3: [10, 11, 18, 19, 20],
10: [22, 30],
# Third level. Note that we can have 10->22->30 and 10->30
22: [30, 42],
# Unhandled case where a larger number can be followed by a smaller number
30: [1, 22],
}
test_list = [[3, 2, 0], [10, 3, 0], [22, 12, 0], [30, 0, 6], [42, 8, 9]]
print(f"\nIn: {test_list}")
print(f"Recur No Reuse: {get_all_combos(test_list)}")
print(f"Recur Reuse : {get_all_combos(test_list, True)}")
Starting with the easy one: loops are avoided in the recursive solution by always processing lists from left to right and only after the current list (data[list_num + 1:]). This way we never go back and avoid infinite recursion. However, this is also a limitation to be kept in mind and the reason that the SUB_NOUNS[30] above will have no effect (since 1 or 22 will never appear in the input after 30)
With more SUB_NOUNS you have more "branches" and potential combinations. The output is:
In: [[3, 2, 0], [10, 3, 0], [22, 12, 0], [30, 0, 6], [42, 8, 9]]
Recur No Reuse: [
[3, 2, 0, 10, 3, 0, 22, 12, 0, 30, 0, 6],
[3, 2, 0, 10, 3, 0, 22, 12, 0, 42, 8, 9],
[3, 2, 0, 10, 3, 0, 30, 0, 6],
]
Recur Reuse : [
[3, 2, 0, 10, 3, 0, 22, 12, 0, 30, 0, 6],
[3, 2, 0, 10, 3, 0, 22, 12, 0, 42, 8, 9],
[3, 2, 0, 10, 3, 0, 30, 0, 6],
[10, 3, 0, 22, 12, 0, 30, 0, 6],
[10, 3, 0, 22, 12, 0, 42, 8, 9],
[10, 3, 0, 30, 0, 6],
[22, 12, 0, 30, 0, 6],
[22, 12, 0, 42, 8, 9],
]
Notes - open-ended points:
You can see in the above example how things can branch out and how both 10->30 and 10->22->30 combinations are considered
You can also see double-branching: on the 10->22 path we have 10->22->30 and 10->22->42
Notice that in the second invocation we set allow_reuse=True in get_all_combos. This allows the code to return combos whose first element/list has already participated in another combination. Altho in your [[3, 2, 0], [10, 3, 0], [22, 12, 0]] example you do not seem to look for such combos, they are all "correct" and follow the rules of SUB_NOUNS
As you said in the post, this might not be the best or a perfect solution but I believe recursion allows you to do all sorts of things which would be really hard otherwise in this particular case. I found this problem fairly complex due to the numerous possibilities of input data, constraining it a bit and narrowing down the scope if possible (ex. input always has up to 3 sub-lists) would help simplify the solution.
Here is my solution with my method of correcting the error found in one of the example inputs, as #urban pointed out it's far from ideal, merely what I came up with after a few hours of pulling my hair out over this problem.
def pair_lists(self, data):
pairs = [] # Create an empty list to store the noun pairs
combined = False # Variable for checking if we need to join nouns
for noun in reversed(data): # Loop over the nouns in reverse
try:
result = itertools.filterfalse(lambda x: x[0] not in SUB_NOUNS[noun[0]],
data) # Filter out any that aren't sub_nouns
for each in result: # Loop the results
paired = [] # Make an empty list to pair them in to
if each not in paired: # If the current iteration isn't in paired
paired.insert(0, each) # insert it at index 0
paired.insert(0, noun) # insert the noun at the start
chained = list(itertools.chain(*paired)) # chain them together
for pair in pairs: # loop the pairs
if pair[:-3] == chained[3:]: # if there is an intersection
pairs.remove(pair) # remove the pair
pairs.insert(0, chained + pair[3:]) # then insert them joined
combined = True # set combined to true
if not combined: # if combined isn't true
pairs.append(chained) # append the chained pairs
except KeyError:
pass
return pairs if pairs else data
I am having a two dimensional list like:
[[1, 1, 0, 0], [1, 0, 1, 0], [0, 1, 1, 0], [1, 1, 1, 0], [1, 1, 1, 0], [0, 1, 1, 0], [1, 1, 1, 1]]
and I want to XOR the inner sub-lists with each other. So at some point of time with some combination I will be getting a sub list with all zeros like [0,0,0,0] and if I don't get with two sub lists I have to go for XORing of three sub-lists till I get again [0,0,0,0], if not then, have to go for four sub-lists.
The problem is I can do like picking up two lists and XOR each element then save it in separate sub-list and this works but each time I have to change my code so is there any way to XOR sub-list like [1, 1, 0, 0] ^ [1, 0, 1, 0] instead of doing like lis[i][j]^lis[i+1][j] so that I can manage my code recursively?
You could find shorter but I think this would work:
STOP=[0,0,0,0]
def nxor(l,res=None,p=1):
if not res: res= l[0]
if res == STOP:
return p-1 # or l[:p]
try:
return nxor(l,list(iter.imap (operator.xor, *list([res,l[p]]) )),p+1)
except:
return None # or res
print nxor (l)
It returns the position of the last sub-list that lead to [0,0,0,0]; it would be easy to return the sub-list instead, or the actual result (see comments).
If you want to test all the combinations, you could use itertools:
for i in xrange(2,len(l)):
for j in itertools.combinations(l,i):
print nxor(list(j))
What you could do is to convert the 2-D bit lists to a 1-D number list and then XOR them together.
a = [[1, 1, 0, 0], [1, 0, 1, 0], [0, 1, 1, 0], [1, 1, 1, 0], [1, 1, 1, 0], [0, 1, 1, 0], [1, 1, 1, 1]]
b = map(lambda x: int("".join(map(lambda y: str(y), x)),2), a)
b is now: [12, 10, 6, 14, 14, 6, 15]
Now you are working with a 1-D list which will make that easier.