My Python code generates a list everytime it loops:
list = np.genfromtxt('temp.txt', usecols=3, dtype=[('floatname','float')], skip_header=1)
But I want to save each one - I need a list of lists right?
So I tried:
list[i] = np.genfromtxt('temp.txt', usecols=3, dtype=[('floatname','float')], skip_header=1)
But Python now tells me that "list" is not defined. I'm not sure how I go about defining it. Also, is a list of lists the same as an array??
Thank you!
You want to create an empty list, then append the created list to it. This will give you the list of lists. Example:
>>> l = []
>>> l.append([1,2,3])
>>> l.append([4,5,6])
>>> l
[[1, 2, 3], [4, 5, 6]]
Create your list before your loop, else it will be created at each loop.
>>> list1 = []
>>> for i in range(10) :
... list1.append( range(i,10) )
...
>>> list1
[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [1, 2, 3, 4, 5, 6, 7, 8, 9], [2, 3, 4, 5, 6, 7, 8, 9], [3, 4, 5, 6, 7, 8, 9], [4, 5, 6, 7, 8, 9], [5, 6, 7, 8, 9], [6, 7, 8, 9], [7, 8, 9], [8, 9], [9]]
Use append method, eg:
lst = []
line = np.genfromtxt('temp.txt', usecols=3, dtype=[('floatname','float')], skip_header=1)
lst.append(line)
First of all do not use list as a variable name- that is a builtin function.
I'm not super clear of what you're asking (a little more context would help), but maybe this is helpful-
my_list = []
my_list.append(np.genfromtxt('temp.txt', usecols=3, dtype=[('floatname','float')], skip_header=1))
my_list.append(np.genfromtxt('temp2.txt', usecols=3, dtype=[('floatname','float')], skip_header=1))
That will create a list (a type of mutable array in python) called my_list with the output of the np.getfromtext() method in the first 2 indexes.
The first can be referenced with my_list[0] and the second with my_list[1]
Just came across the same issue today...
In order to create a list of lists you will have firstly to store your data, array, or other type of variable into a list. Then, create a new empty list and append to it the lists that you just created. At the end you should end up with a list of lists:
list_1=data_1.tolist()
list_2=data_2.tolist()
listoflists = []
listoflists.append(list_1)
listoflists.append(list_2)
Related
I have a 2D list = [[1, 8, 3], [4, 5, 6], [0, 5, 7]], and I want to delete columns in a loop.
For example, columns with index: 0(first) and 2(last) - - the result after deletions should be: [8, 5, 5].
There is a problem, because when I delete the 0th column, the size of the list is decreased to (0,1), and the 2nd index is out of scope.
What is the fastest method to delete columns in a loop without the out-of-scope problem?
For a better picture:
[[1, 8, 3],
[4, 5, 6],
[0, 5, 7]]
There is no such shortcut in python except for iterating over all the list items and removing those index values.
However, you can use pandas which is meant for some other purpose but will do the task.
import pandas as pd
s = [[1, 8, 3], [4, 5, 6], [0, 5, 7]]
df = pd.DataFrame(s,columns=['val1','val2','val3'])
li = df.drop('val1',axis=1).values.tolist()
now li will look like this
[[8, 3], [5, 6], [5, 7]]
You can use numpy like this:
import numpy as np
my_list = np.array([[1, 8, 3], [4, 5, 6], [0, 5, 7]])
new_list = my_list[:, 1].copy()
print(new_list)
Output:
>>> [8, 5, 5]
Also numpy.delete(your_list, index, axis) is do the same job:
new_list = np.delete(my_list,(0, 2), axis=1)
(0, 2) is the indices of the columns 0 and 2
axis=1 says numpy that (0, 2) are columns indices not rows.
if you want to delete rows 0 and 2 you can change axis=1 to axis=0
Output is a little different:
>>> array([[8],
[5],
[5]])
For a pure python approach:
my_list = [[1, 8, 3], [4, 5, 6], [0, 5, 7]]
new_list = [value[1] for value in my_list]
print(new_list)
Output:
>>> [8, 5, 5]
L is 2D list:
print(map(lambda x: x[1:], L))
data= [[1, 8, 3], [4, 5, 6], [0, 5, 7]]
index_to_remove=[0,2]
[list(x) for x in zip(*[d for i,d in enumerate(zip(*data)) if i not in index_to_remove])]
If I understood your question correctly, you want to keep the middle element (index 1) of each list,in that case I would suggest creating a new list. There could be other better ways, for sure. But you could try this, if this works for you:
twoD_list = [[1, 8, 3], [4, 5, 6], [0, 5, 7]]
def keep_col( twoD_list ,index_to_keep = 1):
final_list = []
for x in twoD_list:
final_list.append(x[index_to_keep])
return final_list
final_list = keep_col( twoD_list , 1)
Final output:
[8,5,5]
Assuming you always want only the second element and the inner lists always have at least two elements.
Pure python with list comprehension:
lst = [
[1, 8, 3],
[4, 5, 6],
[0, 5, 7],
]
filtered_lst = [
inner_element
for inner_lst in lst
for i, inner_element in enumerate(inner_lst)
if i == 1
]
print(filtered_lst)
# [8, 5, 5]
If you want you can the reassign the new list to the old variable:
lst = filtered_lst
The advantages of this method are:
no need to worry about the list being altered while you iterate it,
no need to import other libraries
list comprehension is built-in
list comprehension is often the fastest way to filter a list (see for example this article)
easier to read and maintain that other solutions (in my opinion).
Via itemgetter to extract the value at index 1.
from operator import itemgetter
my_list = [[1, 8, 3], [4, 5, 6], [0, 5, 7]]
result = list(map(itemgetter(1), my_list))
try this
my_list = [[1, 8, 3], [4, 5, 6], [0, 5, 7]]
filter_col=[0,2]
col_length=3
my_list=[[x[i] for i in range(col_length) if i not in filter_col] for x in my_list]
u do not want to directly mutate the list that you are working on
this performs a list comprehension to create a new list from the existing list
edit:
just saw u wanted only a flat list
assuming u only want one element for the list u can use
my_list=[x[1] for x in my_list]
I have a dictionaries inside a list like this:
sample_dict = [{1: [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10], \
[1, 2, 3, 4, 5], \
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]]}, \
{2: [[3, 4, 6, 7, 8, 9, 10, 11], [1, 2, 3, 6, 10], []]}]
Now, I would like to check the key 1's first value in the list with key 2's first value.
something like this,
Compare Values (first value of list of lists of key 1)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
with (first value of list of lists of key 2)
[3, 4, 6, 7, 8, 9, 10, 11]
If they are a match I would like to append it to a new list matching_list, if not I would to append non-matching values into another list non_matching_list.
This is what I tried so far,
matching_list = []
non_matching_list = []
for each_dict in sample_dict:
current_dict_values = []
for key, value_list in each_dict.items():
temp_dict_values = []
for value in value_list:
temp_dict_values.append(value)
.... don't know how to keep track of key 1's first list of lists values.
I was thinking of creating a temporary list to keep track of key 1 list values, but I am stuck and not sure how to proceed.
My final output should be like this:
matching_list = [[3,4,6,7,8,9,10], [1,2,3], []]
non_matching_list = [[1,2,5,11],[4,5,6,10],[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]]
How can I achieve my output? Any ideas would be great.
This can be achieved by converting lists to sets to make operations like symmetric_difference() and intersection() for your non_matching_list and matching_list respectively.
Here is one of the solutions:
matching_list, non_matching_list = [], []
for lists1, lists2 in zip(sample_dict[0].values(), sample_dict[1].values()):
for l1, l2 in zip(lists1, lists2):
matching_list.append(list(set(l1) & set(l2)))
non_matching_list.append(list(set(l1).symmetric_difference(set(l2))))
Note that using set(l1) & set(l2) is same as set(l1).intersection(set(l2)), so basically it's an intersection operation here.
I'm also using builtin zip() function to aggregate elements from each of the iterables ( both lists ).
I'm using Python 2.7 and have the following:
my_list = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
I'd like create a 1-d list where the elements are ordered by position in sublist and then order of sublist. So the correct output for the above list is:
[1, 4, 7, 2, 5, 8, 3, 6, 9]
Here's my (incorrect) attempt:
def reorder_and_flatten(my_list):
my_list = [item for sublist in my_list for item in sublist]
result_nums = []
for i in range(len(my_list)):
result_nums.extend(my_list[i::3])
return result_nums
result = reorder_and_flatten(my_list)
This flattens my 2-d list and gives me:
[1, 4, 7, 2, 5, 8, 3, 6, 9, 4, 7, 5, 8, 6, 9, 7, 8, 9]
The first half of this list is correct but the second isn't.
I'd also like my function to be able to handle only 2 sublists. For instance, if given:
[[1, 2, 3], [], [7, 8, 9]
the correct output is:
[1, 7, 2, 8, 3, 9]
Any thoughts?
Thanks!
You're attempting to flatten, and then reorder, which makes things a lot harder than reordering and then flattening.
First, for your initial problem, that's just "unzip", as explained in the docs for zip:
>>> my_list = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> list(zip(*my_list))
... [(1, 4, 7), (2, 5, 8), (3, 6, 9)]
(In Python 2.7, you could just write zip(…) here instead of list(zip(…)), but this way, the same demonstration works identically in both 2.x and 3.x.)
And then, you already know how to flatten that:
>>> [item for sublist in zip(*my_list) for item in sublist]
[1, 4, 7, 2, 5, 8, 3, 6, 9]
But things get a bit more complicated for your second case, where some of the lists may be empty (or maybe just shorter?).
There's no function that's like zip but skips over missing values. You can write one pretty easily. But instead… there is a function that's like zip but fills in missing values with None (or anything else you prefer), izip_longest. So, we can just use that, then filter out the None values as we flatten:
>>> my_list = [[1, 2, 3], [], [7, 8, 9]]
>>> from itertools import izip_longest
>>> list(izip_longest(*my_list))
[(1, None, 7), (2, None, 8), (3, None, 9)]
>>> [item for sublist in izip_longest(*my_list) for item in sublist if item is not None]
[1, 7, 2, 8, 3, 9]
(In Python 3, the function izip_longest is renamed zip_longest.)
It's worth noting that the roundrobin recipe, as covered by ShadowRanger's answer, is an even nicer solution to this problem, and even easier to use (just copy and paste it from the docs, or pip install more_itertools and use it from there). It is a bit harder to understand—but it's worth taking the time to understand it (and asking for help if you get stuck).
result = [l[i] for i in range(max(len(v) for v in my_list)) for l in my_list if l]
i.e.
my_list = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
[l[i] for i in range(max(len(v) for v in my_list)) for l in my_list if l]
# => [1, 4, 7, 2, 5, 8, 3, 6, 9]
my_list = [[1, 2, 3], [], [7, 8, 9]]
[l[i] for i in range(max(len(v) for v in my_list)) for l in my_list if l]
# => [1, 7, 2, 8, 3, 9]
The itertools module's recipes section provides a roundrobin recipe that would do exactly what you want. It produces a generator, but your expected behavior would be seen with:
# define roundrobin recipe here
from itertools import cycle, islice
def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
# Recipe credited to George Sakkis
pending = len(iterables)
nexts = cycle(iter(it).next for it in iterables)
while pending:
try:
for next in nexts:
yield next()
except StopIteration:
pending -= 1
nexts = cycle(islice(nexts, pending))
def reorder_and_flatten(my_list):
return list(roundrobin(*my_list))
Your original code's main issue is that it looped over for i in range(len(my_list)):, extending with my_list[i::3]. Problem is, this ends up duplicating elements from index 3 onwards (index 3 was already selected as the second element of the index 0 slice). There are lots of other small logic errors here, so it's much easier to reuse a recipe.
This will be fairly performant, and generalize better than most hand-rolled solutions (it will round robin correctly even if the sublists are of uneven length, and it doesn't require second pass filtering or special handling of any kind to allow None as a value like zip_longest does).
If you are happy to use a 3rd party library, you can use NumPy and np.ndarray.ravel:
import numpy as np
A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
res_a = A.ravel('F') # array([1, 4, 7, 2, 5, 8, 3, 6, 9])
For the case where you have one or more empty lists, you can use filter to remove empty lists:
B = np.array(list(filter(None, [[1, 2, 3], [], [7, 8, 9]])))
res_b = B.ravel('F') # array([1, 7, 2, 8, 3, 9])
Both solutions require non-empty sublists to contain the same number of items. If list conversion is necessary you can use, for example, res_a.tolist().
While these "black box" methods won't teach you much, they will be faster for large arrays than list-based operations. See also What are the advantages of NumPy over regular Python lists?
I have two lists of lists, of equal length, like this:
lstA = [[1,4,5,6],[4,5],[5,6],[],[],[],[],[]]
lstB = [[7,8],[4,5],[],[],[],[2,7,8],[7,8],[6,7]]
And I want to concatenate the sublists at each index position such that they make a single sublist, like this:
newlst = [[1,4,5,6,7,8],[4,5],[5,6],[],[],[2,7,8],[7,8],[6,7]]
Ideally, the new sublists will remove duplicates (like in newlst[1]). I converted the integers to strings, and attempted this:
for i in range(len(lstA)):
c = [item + item for item in strA[i], strB[i]]
but that adds each item from each list to itself before adding to the other list, resulting in something like this:
failedlst = [[["1","4","5","6","1","4","5","6"],["7","8","7","8"]],[["4","5","4","5"],["4","5","4","5"]]...etc]
And this still doesn't actually join the two sublists, just makes a new sublist of the two sublists. Any help would be greatly appeciated!
Making a list by concatenating items in parallel is very simple, using a list comprehension in combination with the zip function.
newlst = [x+y for x,y in zip(lstA, lstB)]
If you want to remove duplicates, you can use a set. If you then want to put the items back in order in a list, you can use sorted.
In combination, this:
newlst = [sorted(set(x+y)) for x,y in zip(lstA, lstB)]
You could use:
lstA = [[1,4,5,6],[4,5],[5,6],[],[],[],[],[]]
lstB = [[7,8],[4,5],[],[],[],[2,7,8],[7,8],[6,7]]
answer = []
for idx in range(len(lstA)):
answer.append(sorted(list(set(lstA[idx]+lstB[idx]))))
print(answer)
Output
[[1, 4, 5, 6, 7, 8], [4, 5], [5, 6], [], [], [2, 7, 8], [7, 8], [6, 7]]
Using zip and chain.from_iterable from the itertools module.
In [94]: from itertools import chain
In [95]: lstA = [[1,4,5,6],[4,5],[5,6],[],[],[],[],[]]
In [96]: lstB = [[7,8],[4,5],[],[],[],[2,7,8],[7,8],[6,7]]
In [97]: [list(set(chain.from_iterable(item))) for item in zip(lstA, lstB)]
Out[97]: [[1, 4, 5, 6, 7, 8], [4, 5], [5, 6], [], [], [8, 2, 7], [8, 7], [6, 7]]
If you want to sort the sublist then:
In [98]: [sorted(set(chain.from_iterable(item))) for item in zip(lstA, lstB)]
Out[98]: [[1, 4, 5, 6, 7, 8], [4, 4, 5, 5], [5, 6], [], [], [2, 7, 8], [7, 8], [6, 7]]
This question already has answers here:
(Python) adding a list to another without the brackets
(3 answers)
Closed 7 years ago.
Here I'm trying to merge this two lists, making one whit all items.
n = [[1, 2, 3], [4, 5, 6, 7, 8, 9]]
def flatten(n):
s=[]
for x in n:
s.append(x)
return s
print flatten(n)
I'm trying to have as a result
[1,2,3,4,5,6,7,8,9]
but I'm getting
[[1, 2, 3], [4, 5, 6, 7, 8, 9]]
I dont understand why, I think I'm clearly assigning each value to the list 's' in the for loop.
You're appending to the list. Each sublist is appended to the new list as its own item, exactly the way it was originally. You want to extend the list instead:
s.extend(x)
Use extend, instead of append
n = [[1, 2, 3], [4, 5, 6, 7, 8, 9]]
def flatten(n):
s=[]
for x in n:
s.extend(x)
return s
print flatten(n)
Best of luck.
You should be using list.extend, append is appending each sublist not adding just the contents. x is each sublist so just appending the sublist is obviously going to give you a list of lists again.
You can also use itertools.chain to flatten the list:
n = [[1, 2, 3], [4, 5, 6, 7, 8, 9]]
print(list(chain.from_iterable(n)))
Or use a list comp:
n = [[1, 2, 3], [4, 5, 6, 7, 8, 9]]
print([ele for sub in n for ele in sub])