Develop Reference

Add part of the list to another list in Python [duplicate]

This question already has answers here:
How do I concatenate two lists in Python?
(31 answers)
Closed 2 months ago.
I am trying to understand if it makes sense to take the content of a list and append it to another list.
I have the first list created through a loop function, that will get specific lines out of a file and will save them in a list.
Then a second list is used to save these lines, and start a new cycle over another file.
My idea was to get the list once that the for cycle is done, dump it into the second list, then start a new cycle, dump the content of the first list again into the second but appending it, so the second list will be the sum of all the smaller list files created in my loop. The list has to be appended only if certain conditions met.
It looks like something similar to this:
# This is done for each log in my directory, i have a loop running
for logs in mydir:
for line in mylog:
#...if the conditions are met
list1.append(line)
for item in list1:
if "string" in item: #if somewhere in the list1 i have a match for a string
list2.append(list1) # append every line in list1 to list2
del list1 [:] # delete the content of the list1
break
else:
del list1 [:] # delete the list content and start all over
Does this makes sense or should I go for a different route?
I need something efficient that would not take up too many cycles, since the list of logs is long and each text file is pretty big; so I thought that the lists would fit the purpose.

You probably want
list2.extend(list1)
instead of
list2.append(list1)
Here's the difference:
>>> a = [1, 2, 3]
>>> b = [4, 5, 6]
>>> c = [7, 8, 9]
>>> b.append(a)
>>> b
[4, 5, 6, [1, 2, 3]]
>>> c.extend(a)
>>> c
[7, 8, 9, 1, 2, 3]
Since list.extend() accepts an arbitrary iterable, you can also replace
for line in mylog:
list1.append(line)
by
list1.extend(mylog)

To recap on the previous answers. If you have a list with [0,1,2] and another one with [3,4,5] and you want to merge them, so it becomes [0,1,2,3,4,5], you can either use chaining or extending and should know the differences to use it wisely for your needs.
Extending a list
Using the list classes extend method, you can do a copy of the elements from one list onto another. However this will cause extra memory usage, which should be fine in most cases, but might cause problems if you want to be memory efficient.
a = [0,1,2]
b = [3,4,5]
a.extend(b)
>>[0,1,2,3,4,5]
Chaining a list
Contrary you can use itertools.chain to wire many lists, which will return a so called iterator that can be used to iterate over the lists. This is more memory efficient as it is not copying elements over but just pointing to the next list.
import itertools
a = [0,1,2]
b = [3,4,5]
c = itertools.chain(a, b)
Make an iterator that returns elements from the first iterable until it is exhausted, then proceeds to the next iterable, until all of the iterables are exhausted. Used for treating consecutive sequences as a single sequence.

Take a look at itertools.chain for a fast way to treat many small lists as a single big list (or at least as a single big iterable) without copying the smaller lists:
>>> import itertools
>>> p = ['a', 'b', 'c']
>>> q = ['d', 'e', 'f']
>>> r = ['g', 'h', 'i']
>>> for x in itertools.chain(p, q, r):
print x.upper()

You can also combine two lists (say a,b) using the '+' operator.
For example,
a = [1,2,3,4]
b = [4,5,6,7]
c = a + b
Output:
>>> c
[1, 2, 3, 4, 4, 5, 6, 7]

That seems fairly reasonable for what you're trying to do.
A slightly shorter version which leans on Python to do more of the heavy lifting might be:
for logs in mydir:
for line in mylog:
#...if the conditions are met
list1.append(line)
if any(True for line in list1 if "string" in line):
list2.extend(list1)
del list1
....
The (True for line in list1 if "string" in line) iterates over list and emits True whenever a match is found. any() uses short-circuit evaluation to return True as soon as the first True element is found. list2.extend() appends the contents of list1 to the end.

You can simply concatnate two lists, e.g:
list1 = [0, 1]
list2 = [2, 3]
list3 = list1 + list2
print(list3)
>> [0, 1, 2, 3]

Using the map() and reduce() built-in functions
def file_to_list(file):
#stuff to parse file to a list
return list
files = [...list of files...]
L = map(file_to_list, files)
flat_L = reduce(lambda x,y:x+y, L)
Minimal "for looping" and elegant coding pattern :)

you can use __add__ Magic method:
a = [1,2,3]
b = [4,5,6]
c = a.__add__(b)
Output:
>>> c
[1,2,3,4,5,6]

If we have list like below:
list = [2,2,3,4]
two ways to copy it into another list.
1.
x = [list] # x =[] x.append(list) same
print("length is {}".format(len(x)))
for i in x:
print(i)
length is 1
[2, 2, 3, 4]
2.
x = [l for l in list]
print("length is {}".format(len(x)))
for i in x:
print(i)
length is 4
2
2
3
4

Merge two lists that are generated by a python code

I know we can merge two lists by using something like final_list= list1 + list2 but if the lists are generated by a python code and they don't have a variable associated with them like list1 and list2, how can we merge them? Say, my code does something like print output to give:
[1,2,3,4]
[2,0,5,6]
I'd like to merge them so I can get unique values using set(final_list). But how do I get the final_list?
PS- My code can return multiple lists. It is not restricted to two.

def somefunc(param):
#does something
return alist,blist
my_alist,my_blist = somefunc(myparam)
print my_alist, my_blist
#prints both lists.
When you return multiple values from a function they are returned in a tuple. You can easily unpack the tuple

You can either modify the function which is generating output, or the harder way being you manually convert it into a string and then into a set.
list = []
strings_of_list = output.split('\n')
for string in strings_of_list:
values = string[1:-1].split(',')
for val in values:
list+=[int(val)]
set(list)

Assign a variable to a function. Taking the lists the function generated, join them together in another variable. Just make sure that your function returns the generated list, and doesn't just print it out.
# my_list_generator returns two values.
>>> a, b = my_list_generator()
>>> a
[1, 2, 3, 4]
>>> b
[2, 0, 5, 6]
>>> final_list = a + b
>>> final_list
[1, 2, 3, 4, 2, 0, 5, 6]
Cross all that out! Now that I know the function can return multiple objects, let do this (with a little list comprehension):
lists = [i for i in my_list_generator()]
# lists might look like [[1, 2, 3, 4], [2, 0, 5, 6]]
# And now use a for loop to get each value
final_list = []
for sublist in lists:
final_list.extend(sublist)
# final_list will look like [1,2,3,4,2,0,5,6]
Also, if you don't want duplicates, just do one more thing:
real_final_list = [i for i in final_list if i not in real_final_list]

If I understand correctly:
You have a function (let's call it listGen() for now) which returns some number of lists. Now, you want to put these list together into one big list, final_list.
You could do the following:
# listGen defined earlier
final_list = []
for i in listGen():
final_list += i
unique_values = set(final_list) # or whatever you wanted to do with it
Since listGen returns a tuple, we can loop over its contents, those being the lists you want to append to each other.

Python - finding value position in a list and outputting as a new list

Currently learning python in my course and I'm a little confused on this part, unfortunately my next prac isn't until next week so I thought I'd ask here.
We're supposed to write a function called find() that takes a list, searches for the value in the list and returns as a new list of the positions in the list the number was found. Eg:
list = 2, 3, 7, 2, 3, 3, 5
number = 3
the output would be: 0, 4, 5
The question requires us to use a loop but not to use built-in functions, slice expressions, list methods, or string methods unless specified, with the only ones specified for this part being: .append() and range().
We're given a file we're not allowed to edit:
import test_lists
list_A = ['q', 'w', 'e', 'r', 't', 'y']
print(test_lists.find(list_A, 'r'))
print(test_lists.find(list_A, 'b'))
From what I've attempted in a file named test_lists, the output gives <function find at 0x0000000003B89D90>:
def find(list1, listValue):
findList = 0
for x in range(findList):
findList.find(list1, value)
return find
anyone able to please push me in the right direction and explain it to me? We were told we could use .append() but I don't seem to understand how that would fit into this situation as from what I am aware append only adds to to the string. I feel like I'm pretty far off track.
Thanks

Using enumerate and list comprehension:
>>> lst = 2, 3, 7, 2, 3, 3, 5
>>> number = 3
>>> [i for i, x in enumerate(lst) if x == number]
[1, 4, 5]
>>> list_A = ['q', 'w', 'e', 'r', 't', 'y']
>>> [i for i, x in enumerate(list_A) if x == 'r']
[3]
BTW, don't use list as a variable name. It shadows builtin type/function list.
UPDATE
without using enumerate:
>>> lst = 2, 3, 7, 2, 3, 3, 5
>>> number = 3
>>> [i for i in range(len(lst)) if lst[i] == number]
[1, 4, 5]

Will this do ?
def find(list1, listValue):
found = []
for index in range(0,len(list1)):
if list1[index]==listValue:
found.append(index)
return found

I guess you can't use enumerate ?
Then, you you have to do the same thing by yourself : iterate over the indexes of the list and for each index, see if the element is the expected element. If it is, you had the index into your result list :
expected_element = 3
my_list = [...]
result_list = []
for i in range(len(my_list)):
if my_list[i] == expected_element:
result_list.append(i)
It's not very pythonic but if you have so much contraint, i think it's good
And if you can't use len(my_list), you can get the size by iterate over a first time :
def size(l):
s = 0
for _ in l:
s += 1
return s

A list comprehension would be the most Pythonic way to do this. But writing it explicitly for instructional purposes, here is another way to do it.
def find(number, numList):
indexMatches = [] # initialize your index array
for index in range(len(numList)): # iterate over the list of values
if number == numList[index]: # compare the number against the current value
indexMatches.append(index) # if they match, add it to the index array
return indexMatches # return the list of indexes where matches occured
find(3, [2,3,7,2,3,3,5])
Output
[1, 4, 5]

Using 'not' in a 'for-loop statement'

Why is not possible to use a not in a for statement?
Assuming that both object and list are iterable
If you can't do that is there another way around?
Here is an example, but "obviously" a syntax error:
tree = ["Wood", "Plank", "Apples", "Monkey"]
plant = ["Flower", "Plank", "Rose"]
for plant not in tree:
# Do something
pass
else:
# Do other stuff
pass

Here's one way, using sets and assuming that both objects and list are iterable:
for x in set(objects).difference(lst):
# do something
First of all, you should not call a variable list, that'll clash with a built-in name. Now the explanation: the expression set(objects).difference(lst) performs a set difference, for example:
lst = [1, 2, 3, 4]
objects = [1, 2, 5, 6]
set(objects).difference(lst)
=> set([5, 6])
As you can see, we found the elements in objects that are not in the list.

If objects and list are two lists, and you want to iterate over every element of objects that isn't in list, you want the following:
for object in objects:
if object not in list:
do_whatever_with(object)
This loops over everything in objects and only processes the ones that aren't in list. Note that this won't be very efficient; you could make a set out of list for efficient in checking:
s = set(list)
for object in objects:
if object not in s:
do_whatever_with(object)

It looks like you are confusing a couple of things. The for loop is used to iterate over sequences (lists, tuples, characters of a string, sets, etc). The not operator reverses boolean values. Some examples:
>>> items = ['s1', 's2', 's3']
>>> for item in items:
... print item
...
s1
s2
s3
>>> # Checking whether an item is in a list.
... print 's1' in items
True
>>> print 's4' in items
False
>>>
>>> # Negating
... print 's1' not in items
False
>>> print 's4' not in items
True

If you mean to iterate over a list except few:
original = ["a","b","c","d","e"]
to_exclude = ["b","e"]
for item [item for item in orginal if not item in to_exclude]: print item
Produces:
a
c
d

You may use list comprehension combined with inline if:
>>> lst = [1, 2, 3, 4]
>>> objects = [1, 2, 5, 6]
>>> [i for i in objects if i not in lst]
[5, 6]

And another way:
from itertools import ifilterfalse
for obj in ifilterfalse(set(to_exclude).__contains__, objects):
# do something

Here is a simple way to achieve what you want:
list_i_have = [1, 2, 4]
list_to_compare = [2, 4, 6, 7]
for l in list_i_have:
if l not in list_to_compare:
do_something()
else:
do_another_thing()
Foreach item in the list you have, you can have a exclude list to check it is inside of list_to_compare.
You can also achieve this with list comprehension:
["it is inside the list" if x in (3, 4, 5) else "it is not" for x in (1, 2, 3)]

Create an empty list with certain size in Python

How do I create an empty list that can hold 10 elements?
After that, I want to assign values in that list. For example:
xs = list()
for i in range(0, 9):
xs[i] = i
However, that gives IndexError: list assignment index out of range. Why?
Editor's note:
In Python, lists do not have a set capacity, but it is not possible to assign to elements that aren't already present. Answers here show code that creates a list with 10 "dummy" elements to replace later. However, most beginners encountering this problem really just want to build a list by adding elements to it. That should be done using the .append method, although there will often be problem-specific ways to create the list more directly. Please see Why does this iterative list-growing code give IndexError: list assignment index out of range? How can I repeatedly add elements to a list? for details.

You cannot assign to a list like xs[i] = value, unless the list already is initialized with at least i+1 elements. Instead, use xs.append(value) to add elements to the end of the list. (Though you could use the assignment notation if you were using a dictionary instead of a list.)
Creating an empty list:
>>> xs = [None] * 10
>>> xs
[None, None, None, None, None, None, None, None, None, None]
Assigning a value to an existing element of the above list:
>>> xs[1] = 5
>>> xs
[None, 5, None, None, None, None, None, None, None, None]
Keep in mind that something like xs[15] = 5 would still fail, as our list has only 10 elements.
range(x) creates a list from [0, 1, 2, ... x-1]
# 2.X only. Use list(range(10)) in 3.X.
>>> xs = range(10)
>>> xs
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Using a function to create a list:
>>> def display():
... xs = []
... for i in range(9): # This is just to tell you how to create a list.
... xs.append(i)
... return xs
...
>>> print display()
[0, 1, 2, 3, 4, 5, 6, 7, 8]
List comprehension (Using the squares because for range you don't need to do all this, you can just return range(0,9) ):
>>> def display():
... return [x**2 for x in range(9)]
...
>>> print display()
[0, 1, 4, 9, 16, 25, 36, 49, 64]

Try this instead:
lst = [None] * 10
The above will create a list of size 10, where each position is initialized to None. After that, you can add elements to it:
lst = [None] * 10
for i in range(10):
lst[i] = i
Admittedly, that's not the Pythonic way to do things. Better do this:
lst = []
for i in range(10):
lst.append(i)
Or even simpler, in Python 2.x you can do this to initialize a list with values from 0 to 9:
lst = range(10)
And in Python 3.x:
lst = list(range(10))

varunl's currently accepted answer
>>> l = [None] * 10
>>> l
[None, None, None, None, None, None, None, None, None, None]
Works well for non-reference types like numbers. Unfortunately if you want to create a list-of-lists you will run into referencing errors. Example in Python 2.7.6:
>>> a = [[]]*10
>>> a
[[], [], [], [], [], [], [], [], [], []]
>>> a[0].append(0)
>>> a
[[0], [0], [0], [0], [0], [0], [0], [0], [0], [0]]
>>>
As you can see, each element is pointing to the same list object. To get around this, you can create a method that will initialize each position to a different object reference.
def init_list_of_objects(size):
list_of_objects = list()
for i in range(0,size):
list_of_objects.append( list() ) #different object reference each time
return list_of_objects
>>> a = init_list_of_objects(10)
>>> a
[[], [], [], [], [], [], [], [], [], []]
>>> a[0].append(0)
>>> a
[[0], [], [], [], [], [], [], [], [], []]
>>>
There is likely a default, built-in python way of doing this (instead of writing a function), but I'm not sure what it is. Would be happy to be corrected!
Edit: It's [ [] for _ in range(10)]
Example :
>>> [ [random.random() for _ in range(2) ] for _ in range(5)]
>>> [[0.7528051908943816, 0.4325669600055032], [0.510983236521753, 0.7789949902294716], [0.09475179523690558, 0.30216475640534635], [0.3996890132468158, 0.6374322093017013], [0.3374204010027543, 0.4514925173253973]]

There are two "quick" methods:
x = length_of_your_list
a = [None]*x
# or
a = [None for _ in xrange(x)]
It appears that [None]*x is faster:
>>> from timeit import timeit
>>> timeit("[None]*100",number=10000)
0.023542165756225586
>>> timeit("[None for _ in xrange(100)]",number=10000)
0.07616496086120605
But if you are ok with a range (e.g. [0,1,2,3,...,x-1]), then range(x) might be fastest:
>>> timeit("range(100)",number=10000)
0.012513160705566406

You can .append(element) to the list, e.g.:
s1.append(i)
What you are currently trying to do is access an element (s1[i]) that does not exist.

I'm surprised nobody suggest this simple approach to creating a list of empty lists. This is an old thread, but just adding this for completeness. This will create a list of 10 empty lists
x = [[] for i in range(10)]

The accepted answer has some gotchas. For example:
>>> a = [{}] * 3
>>> a
[{}, {}, {}]
>>> a[0]['hello'] = 5
>>> a
[{'hello': 5}, {'hello': 5}, {'hello': 5}]
>>>
So each dictionary refers to the same object. Same holds true if you initialize with arrays or objects.
You could do this instead:
>>> b = [{} for i in range(0, 3)]
>>> b
[{}, {}, {}]
>>> b[0]['hello'] = 6
>>> b
[{'hello': 6}, {}, {}]
>>>

How do I create an empty list that can hold 10 elements?
All lists can hold as many elements as you like, subject only to the limit of available memory. The only "size" of a list that matters is the number of elements currently in it.
However, that gives IndexError: list assignment index out of range. Why?
The first time through the loop, i is equal to 0. Thus, we attempt xs[0] = 0. This does not work because there are currently 0 elements in the list, so 0 is not a valid index.
We cannot use indexing to write list elements that don't already exist - we can only overwrite existing ones. Instead, we should use the .append method:
xs = list();
for i in range(0, 9):
xs.append(i)
The next problem you will note is that your list will actually have only 9 elements, because the end point is skipped by the range function. (As side notes: [] works just as well as list(), the semicolon is unnecessary, and only one parameter is needed for range if you're starting from 0.) Addressing those issues gives:
xs = []
for i in range(10):
xs.append(i)
However, this is still missing the mark - range is not some magical keyword that's part of the language the way for (or, say, def) is.
In 2.x, range is a function, which directly returns the list that we already wanted:
xs = range(10) # 2.x specific!
# In 3.x, we don't get a list; we can do a lot of things with the
# result, but we can't e.g. append or replace elements.
In 3.x, range is a cleverly designed class, and range(10) creates an instance. To get the desired list, we can simply feed it to the list constructor:
xs = list(range(10)) # correct in 3.x, redundant in 2.x

One simple way to create a 2D matrix of size n using nested list comprehensions:
m = [[None for _ in range(n)] for _ in range(n)]

I'm a bit surprised that the easiest way to create an initialised list is not in any of these answers. Just use a generator in the list function:
list(range(9))

Another option is to use numpy for fixed size arrays (of pointers):
> pip install numpy
import numpy as np
a = np.empty(10, dtype=np.object)
a[1] = 2
a[5] = "john"
a[3] = []
If you just want numbers, you can do with numpy:
a = np.arange(10)

Here's my code for 2D list in python which would read no. of rows from the input :
empty = []
row = int(input())
for i in range(row):
temp = list(map(int, input().split()))
empty.append(temp)
for i in empty:
for j in i:
print(j, end=' ')
print('')

A list is always "iterable" and you can always add new elements to it:
insert: list.insert(indexPosition, value)
append: list.append(value)
extend: list.extend(value)
In your case, you had instantiated an empty list of length 0. Therefore, when you try to add any value to the list using the list index (i), it is referring to a location that does not exist. Therefore, you were getting the error "IndexError: list assignment index out of range".
You can try this instead:
s1 = list();
for i in range(0,9):
s1.append(i)
print (s1)
To create a list of size 10(let's say), you can first create an empty array, like np.empty(10) and then convert it to list using arrayName.tolist(). Alternately, you can chain them as well.
**`np.empty(10).tolist()`**

I came across this SO question while searching for a similar problem. I had to build a 2D array and then replace some elements of each list (in 2D array) with elements from a dict.
I then came across this SO question which helped me, maybe this will help other beginners to get around.
The key trick was to initialize the 2D array as an numpy array and then using array[i,j] instead of array[i][j].
For reference this is the piece of code where I had to use this :
nd_array = []
for i in range(30):
nd_array.append(np.zeros(shape = (32,1)))
new_array = []
for i in range(len(lines)):
new_array.append(nd_array)
new_array = np.asarray(new_array)
for i in range(len(lines)):
splits = lines[i].split(' ')
for j in range(len(splits)):
#print(new_array[i][j])
new_array[i,j] = final_embeddings[dictionary[str(splits[j])]-1].reshape(32,1)
Now I know we can use list comprehension but for simplicity sake I am using a nested for loop. Hope this helps others who come across this post.

Not technically a list but similar to a list in terms of functionality and it's a fixed length
from collections import deque
my_deque_size_10 = deque(maxlen=10)
If it's full, ie got 10 items then adding another item results in item #index 0 being discarded. FIFO..but you can also append in either direction.
Used in say
a rolling average of stats
piping a list through it aka sliding a window over a list until you get a match against another deque object.
If you need a list then when full just use list(deque object)

s1 = []
for i in range(11):
s1.append(i)
print s1
To create a list, just use these brackets: "[]"
To add something to a list, use list.append()

Make it more reusable as a function.
def createEmptyList(length,fill=None):
'''
return a (empty) list of a given length
Example:
print createEmptyList(3,-1)
>> [-1, -1, -1]
print createEmptyList(4)
>> [None, None, None, None]
'''
return [fill] * length

This code generates an array that contains 10 random numbers.
import random
numrand=[]
for i in range(0,10):
a = random.randint(1,50)
numrand.append(a)
print(a,i)
print(numrand)