Develop Reference

Pandas, adding multiple columns of list

I have a dataframe like this one
df = pd.DataFrame({'A' : [['a', 'b', 'c'], ['e', 'f', 'g','g']], 'B' : [['1', '4', 'a'], ['5', 'a']]})
I would like to create another column C that will be a column of list like the others but this one will be the "union" of the others
Something like this :
df = pd.DataFrame({'A' : [['a', 'b', 'c'], ['e', 'f', 'g', 'g']], 'B' : [['1', '4', 'a'], ['5', 'a']], 'C' : [['a', 'b', 'c', '1', '4', 'a'], ['e', 'f', 'g', 'g', '5', 'a']]})
But i have like hundreds of columns and C will be the "union" of these hundreds of columns i dont want to index on it like this :
df['C'] = df['A'] + df['B]
And i dont want to make a for loop because the dataframe i am manipulating are too big and i want something fast
Thank you for helping

As you have lists, you cannot vectorize the operation.
A list comprehension might be the fastest:
from itertools import chain
df['out'] = [list(chain.from_iterable(x[1:])) for x in df.itertuples()]
Example:
A B C out
0 [a, b, c] [1, 4, a] [x, y] [a, b, c, 1, 4, a, x, y]
1 [e, f, g, g] [5, a] [z] [e, f, g, g, 5, a, z]

As an alternative to #mozway 's answer, you could try something like this:
df = pd.DataFrame({'A': [['a', 'b', 'c'], ['e', 'f', 'g','g']], 'B' : [['1', '4', 'a'], ['5', 'a']]})
df['C'] = df.sum(axis=1).astype(str)
use 'astype' as required for list contents

you can use the apply method
df['C']=df.apply(lambda x: [' '.join(i) for i in list(x[df.columns.to_list()])], axis=1)

Multiple Lists into Single List based on Index

I'm new to python and I'm trying to merge three different lists into one list based on the index value as shown in the example below:
All three lists are of same size.
A=['ABC', 'PQR', 'MNO']
B=['X', 'Y', 'Z']
C=['1','2','3']***
The output that I wanted is
P=[['ABC', 'X', '1'],['PQR', 'Y', '2'],['MNO', 'Z', '3']]
Thanks in advance.

I usually do it with numpy, as it is a simple traspose, and works with as many lists as you throw at it:
import numpy as np
A = ['ABC', 'PQR', 'MNO']
B = ['X', 'Y', 'Z']
C = ['1', '2', '3']
lists = [A, B, C]
numpy_array = np.array(lists)
transpose = numpy_array.T
transpose_list = transpose.tolist()
print(transpose_list)

Here is the solution for you using the for loop with the range() function:
A=['ABC', 'PQR', 'MNO']
B=['X', 'Y', 'Z']
C=['1','2','3']
list1=[]
for i in range(len(A)):
list1.append([A[i],B[i],C[i]])
display(list1)
OUTPUT:
[['ABC', 'X', '1'], ['PQR', 'Y', '2'], ['MNO', 'Z', '3']]
Using for loop with the zip() function:
l=[]
for a,b,c in zip(A,B,C):
l.append([a,b,c])
display(l)
OUTPUT:
[['ABC', 'X', '1'], ['PQR', 'Y', '2'], ['MNO', 'Z', '3']]
You don't want to use for loop?
Then here is the map() function for you:
result = list(map(lambda a, b, c: [a,b,c] , A, B,C))
display(result)
OUTPUT:
[['ABC', 'X', '1'], ['PQR', 'Y', '2'], ['MNO', 'Z', '3']]

you can use list comprehension to get desired output;
a=[[x,y,z] for x,y,z in zip(A,B,C)]
print(a)

How do I check whether a variable was changed?

Say I have two lists
[['1', '2', '1', '3', '1', '3'], ['A', 'G', 'T', 'T', 'T', 'G']]
In this case each index matches the number on the left with the letter on the right, so 1 : A, and 2 : G and so on. I want to see if AT LEAST one number on the left changes mapping. So, I want to know if ANY number changes mapping. So if 1 : A changes to 1 : T, I would have True returned.

You can create a dictionary:
s = [['1', '2', '1', '3', '1', '3'], ['A', 'G', 'T', 'T', 'T', 'G']]
new_s = {b:a for a, b in zip(*s)}
final_vals = [a for a, b in new_s.items() if any(d == b for c, d in new_s.items() if c != a)]
Output:
['A', 'T']

Actually perform the assignments in a dictionary, stop whenever one changes an existing entry.
def check_overwrite(keys, values):
d = {}
for k,v in zip(keys, values):
if d.setdefault(k, v) != v:
return True
return False
print check_overwrite(['1', '2', '1', '3', '1', '3'], ['A', 'G', 'T', 'T', 'T', 'G'])

If you want to know if it's not only changed but what changed this (stolen from above) should help
>>> numbers = ['1', '2', '1', '3', '1', '3']
>>> letters = ['A', 'G', 'T', 'T', 'T', 'G']
>>> def check_overwrite(keys, values):
... d = {}
... overlap = {}
... for k,v in zip(keys, values):
... if d.setdefault(k, v) != v:
... overlap[k] = v
... return overlap
...
>>> check_overwrite(numbers, letters)
{'1': 'T', '3': 'G'}

Concatenate two lists using condition

I need to concatenate lists A and B using condition that two lists transform to list C, with the same length, but each i element doesn't change position and list is sorted. Else return empty list:
A = ['a', 'b', 'c']
B = ['x', 'y', 'z']
concatenated to: C = ['a', 'b', 'z'].
or
A = ['8', '2', '3']
B = ['1', '2', '0']
concatenated to: C = ['1', '2', '3'].
I absolutely have no idea how I can start solve my problem. Which algorithm I have to use?
UPD:
Another examples:
A = ['a', 'b', 'c']
B = ['x', 'x', 'x']
result is: ['a', 'b', 'x']
A = ['b', 'a']
B = ['x', 'c']
result must be: ['b', 'c'].
And result of concatenation of these two lists is an empty list:
A = ['e','a'],
B = ['f','b']

Do you just need to switch the last two elements in each list? If so, this is a simple way that lets you see what is going on.
lst1 = ['a', 'b', 'c']
lst2 = ['x', 'x', 'x']
lst3 = lst1
lst3[len(lst2)-1] = lst2[len(lst2)-1]
lst3
Out: ['a', 'b', 'x']
Else, explain the logic behind what you are trying to do a little more.

This can be done with a greedy algorithm, iterate over both list and take the minimum value that is greater from the previous item you had taken.
prev = None
A = ['a', 'b', 'c']
B = ['x', 'x', 'x']
n = len(A)
result = []
for i in range(n):
mn, mx = A[i],B[i]
if mx<mn:
mn,mx = mx,mn
if prev == None or mn >= prev:
prev = mn
result.append(prev)
elif mx >= prev:
prev = mx
result.append(prev)
else:
print "can't be done"
break
if len(result) == n:
print result

merge complex matrix with python

I have a complex matrix that looks like this:
[[ ['x', '1', '2', '3', '4'],
['y', '5', '6', '7', '8']],
[ ['x', 'a', 'b', 'c', 'd'],
['y', 'e', 'f', 'g', 'h'] ] ]
I want to turn it into this:
['x', '1a', '2b', '3c', '4d'],
['y', '5e', '6f', '7g', '8h']
I'm busting my head but not managing to achieve the result. Also, even though I only have two groups of nested 5-items long lists, in theory I want to solve this for an infinite number of groups of the same size.

You can use a dict here:
>>> from operator import add
>>> lis = [[ ['x', '1', '2', '3', '4'],
['y', '5', '6', '7', '8']],
[ ['x', 'a', 'b', 'c', 'd'],
['y', 'e', 'f', 'g', 'h'] ] ]
>>> dic = {}
for item in lis:
for x in item:
k, v = x[0], x[1:]
if k in dic:
dic[k] = map(add, dic[k], v)
else:
dic[k] = v
...
>>> dic
{'y': ['5e', '6f', '7g', '8h'], 'x': ['1a', '2b', '3c', '4d']}
#list of lists
>>> [[k] + v for k, v in dic.iteritems()]
[['y', '5e', '6f', '7g', '8h'], ['x', '1a', '2b', '3c', '4d']]
Another solution using zip, reduce and a list comprehension:
>>> from operator import add
>>> def func(x, y):
... return map(add, x, y[1:])
>>> [[item[0][0]] + reduce(func, item[1:], item[0][1:]) for item in zip(*lis)]
[['x', '1a', '2b', '3c', '4d'], ['y', '5e', '6f', '7g', '8h']]

Here's a "fun" solution. Since you did not provide any information about your array's structure, I assumed the easiest variant:
import numpy
a = numpy.array([[
['x', '1', '2', '3', '4'],
['y', '5', '6', '7', '8']],
[
['x', 'a', 'b', 'c', 'd'],
['y', 'e', 'f', 'g', 'h']]],
dtype=numpy.object)
res = a[0].copy()
for chunk in a[1:]:
res[:,1:] += chunk[:,1:]
print(res)