Develop Reference

creating a list of dictionaries from pandas dataframe

This is my df:
df = pd.DataFrame({'sym': ['a', 'b', 'c', 'x', 'y', 'z', 'q', 'w', 'e'],
'sym_t': ['tsla', 'msft', 'f', 'aapl', 'aa', 'gg', 'amd', 'ba', 'c']})
I want to separate this df into groups of three and create a list of dictionaries:
options = [{'value':'a b c', 'label':'tsla msft f'}, {'value':'x y z', 'label':'aapl aa gg'}, {'value':'q w e', 'label':'amd ba c'}]
How can I create that list? My original df has over 1000 rows.

Try groupby to concatenate the rows, then to_dict:
tmp = df.groupby(np.arange(len(df))//3).agg(' '.join)
tmp.columns = ['value', 'label']
tmp.to_dict(orient='records')
Output:
[{'value': 'a b c', 'label': 'tsla msft f'},
{'value': 'x y z', 'label': 'aapl aa gg'},
{'value': 'q w e', 'label': 'amd ba c'}]

Updating dictionaries of list from another dictionary python

I have two list of dictionaries: I am trying to compare test2 with test1 and update accordingly.
test1 = [{'names': ['a', 'b', 'c'],
'country': 'USA',
'state': 'Texas'},
{'names': ['d', 'e', 'f'],
'country': 'Australia',
'state': 'Melbourne'},
{'names': ['i', 'j', 'k'],
'country': 'canada',
'state': 'Toronto'},
{'names': ['l', 'm', 'n'],
'country': 'Austria',
'state': 'Burgenland'}]
test2 = [{'code': 4286,
'list_of_countries': ['USA',
'Australia',
'Colombia',
'Denmark',
'Greece',
'Iceland']},
{'code':4287,
'list_of_countries': ['Texas',
'Argentina',
'Austria',
'Bangladesh', 'canada']}]
Expected Output:
test2 = [{'names':['a', 'b', 'c', 'd', 'e', 'f'],
'country': ['USA', 'Australia'],
'state': ['Texas', 'Melbourne'],
'code':4286},
{'names':['i', 'j', 'k', 'l', 'm', 'n'],
'country': ['canada', 'Austria'],
'state': ['Toronto','Burgenland'],
'code':4287}]
Tried below snippet: By searching the test1 country in test2 list_of_countries:
for i in test1:
for j in test2:
a = []
if i.get('country') in j.get('list_of_countries'):
a.append({'country':i.get('country'), 'state':i.get('state'})
j.update(a)

You can transform test2 to a dictionary, associating each entry in list_of_countries with their proper key. Then, you can use this result for grouping:
test2 = [{'code': 4286, 'list_of_countries': ['USA', 'Australia', 'Colombia', 'Denmark', 'Greece', 'Iceland']}, {'code': 4287, 'list_of_countries': ['Texas', 'Argentina', 'Austria', 'Bangladesh', 'canada']}]
test1 = [{'names': ['a', 'b', 'c'], 'country': 'USA', 'state': 'Texas'}, {'names': ['d', 'e', 'f'], 'country': 'Australia', 'state': 'Melbourne'}, {'names': ['i', 'j', 'k'], 'country': 'canada', 'state': 'Toronto'}, {'names': ['l', 'm', 'n'], 'country': 'Austria', 'state': 'Burgenland'}]
d = {i:k['code'] for k in test2 for i in k['list_of_countries']}
Now, you can create a series of defaultdicts associated with the country code. By looping over the country/state dicts in test1, you can keep a running update of the states and countries that are associated with each code:
from collections import defaultdict
new_d = dict(zip(d.values(), [defaultdict(list) for _ in d]))
for i in test1:
for a, b in i.items():
new_d[d[i['country']]][a].append(b)
r = [{'code':a, **b, 'names':[j for k in b['names'] for j in k]} for a, b in new_d.items()]
The final list comprehension transforms new_d to your desired format, a list of dictionaries.
Output:
[{'code': 4286, 'names': ['a', 'b', 'c', 'd', 'e', 'f'], 'country': ['USA', 'Australia'], 'state': ['Texas', 'Melbourne']}, {'code': 4287, 'names': ['i', 'j', 'k', 'l', 'm', 'n'], 'country': ['canada', 'Austria'], 'state': ['Toronto', 'Burgenland']}]

Replacing the zeros with the previous value in "list of lists" in python

I have a list called "Names.
Names = [['Bassett', 'Richard', '1745-04-02', 'M', 'sen', 'DE', 'Anti-Administration', 1745], ['Bland', 'Theodorick', '1742-03-21', 'M', 'rep', 'VA', '', 1742], ['Burke', 'Aedanus', '1743-06-16', 'M', 'rep', 'SC', '', 0]]
In the names list if the year column (index 7) equals 0 then I want to replace it with the previous year value. For eg: the third list's year value is 0, I want to replace it with 1742.

You need to store the latest valid year and pass it on to the next stage. reduce passes a value from one stage to the next and since lists are passed by reference we can modify the lists in place.
Names = [['Bassett', 'Richard', '1745-04-02', 'M', 'sen', 'DE', 'Anti-Administration', 1745],
['Bland', 'Theodorick', '1742-03-21', 'M', 'rep', 'VA', '', 1742],
['Burke', 'Aedanus', '1743-06-16', 'M', 'rep', 'SC', '', 0]]
def fill_year(year, ns):
if ns[7] == 0:
ns[7] = year
return ns[7]
reduce(fill_year, Names, 0)
print Names
Apparently reduce is deprecated in python3.
Try:
year = 0
for ns in Names:
ns[7] = year if ns[7] == 0 else ns[7]
year = ns[7]

You can try this :
Names = [['Bassett', 'Richard', '1745-04-02', 'M', 'sen', 'DE', 'Anti-Administration', 1745],
['Bland', 'Theodorick', '1742-03-21', 'M', 'rep', 'VA', '', 1742],
['Burke', 'Aedanus', '1743-06-16', 'M', 'rep', 'SC', '', 0]]
for ele in range(len(Names)):
if Names[ele][7] == 0:
Names[ele][7] = (Names[ele-1][2].split('-'))[0]
print(Names)
Explanation:
Using for-loop and range(len()) itearte over the length number of times
for ele in range(len(Names)): #it will iterate over three times as len -> 3
Next check for the year value at index 7 if it equals to 0 then copy the previous year means the year present in previous iteration step means ele-1. For example if it is in ele(2nd iteration) then it will fetch the year from ele-1 (1st iteration).
Names[ele][7] = (Names[ele-1][2].split('-'))[0]
The year is combined in date format. To retrieve only the year use the split() to split the string using - as delimiter
'1742-03-21' -> [1742, 03, 21]
So the year is in index 0.
(Names[ele-1][2].split('-'))[0] -> we get year from here
Finally update the year of current ele to the one which we got.
Output:
[['Bassett', 'Richard', '1745-04-02', 'M', 'sen', 'DE', 'Anti-Administration', 1745], ['Bland', 'Theodorick', '1742-03-21', 'M', 'rep', 'VA', '', 1742], ['Burke', 'Aedanus', '1743-06-16', 'M', 'rep', 'SC', '', '1742']]

The Pandas package has a function for this and is useful for many other types of operations on tabular data.
If you are willing to use it you could solve this fill problem as follows:
import pandas as pd
df = pd.DataFrame(Names, columns=['A', 'B', 'C', 'D', 'E', 'F', 'G', 'Year'])
df['Year'] = df['Year'].replace({0: None}).fillna(method='ffill')
print(df)
Output:
A B C D E F G Year
0 Bassett Richard 1745-04-02 M sen DE Anti-Administration 1745
1 Bland Theodorick 1742-03-21 M rep VA 1742
2 Burke Aedanus 1743-06-16 M rep SC 1742
UPDATE:
As pointed out by #miradulo, Series.replace has a method argument so you can do the operation in one go as follows:
df['Year'] = df['Year'].replace(0, method='ffill')

You can use unpacking:
Names = [['Bassett', 'Richard', '1745-04-02', 'M', 'sen', 'DE', 'Anti-Administration', 1745], ['Bland', 'Theodorick', '1742-03-21', 'M', 'rep', 'VA', '', 1742], ['Burke', 'Aedanus', '1743-06-16', 'M', 'rep', 'SC', '', 0]]
new_names = [b+[Names[i-1][-1]] if not a else [*b, a] for i, [*b, a] in enumerate(Names)]
Output:
[['Bassett', 'Richard', '1745-04-02', 'M', 'sen', 'DE', 'Anti-Administration', 1745], ['Bland', 'Theodorick', '1742-03-21', 'M', 'rep', 'VA', '', 1742], ['Burke', 'Aedanus', '1743-06-16', 'M', 'rep', 'SC', '', 1742]]

Break a string into sequential substrings? [duplicate]

This question already has answers here:
How To Get All The Contiguous Substrings Of A String In Python?
(9 answers)
Closed 5 years ago.
I have a string "BANANA". I would like to generate a list of all possible sequential substrings:
[B, BA, BAN, BANA, BANAN, BANANA, A, AN, ANA, ...]
Is this something I can accomplish using a Python List Comprehension or would I just generate them in a brute force manner? Note: I am new to Python. TIA

Using list Comprehension:
s = "BANANA"
l = len(s)
ar = [s[j:] for i in range(l) for j in range(i,l)]
print(*ar)
Using nested loop:
s = "BANANA"
l = len(s)
ar = []
for i in range(l):
for j in range(i,l):
ar.append(s[j:])
print(*ar)
Both output:
BANANA ANANA NANA ANA NA A ANANA NANA ANA NA A NANA ANA NA A ANA NA A NA A A
N.B.: The itertools has already been explained in A.J.'s answer.

Try the following with itertools:
str = "BANANA"
all = [[''.join(j) for j in itertools.product(str, repeat=i)] for i in range(1, len(str)+1)]
>>> all[0]
['B', 'A', 'N', 'A', 'N', 'A']
>>> all[1]
['BB', 'BA', 'BN', 'BA', 'BN', 'BA', 'AB', 'AA', 'AN', 'AA', 'AN', 'AA', 'NB', 'NA', 'NN', 'NA', 'NN', 'NA', 'AB', 'AA', 'AN', 'AA', 'AN', 'AA', 'NB', 'NA', 'NN', 'NA', 'NN', 'NA', 'AB', 'AA', 'AN', 'AA', 'AN', 'AA']
>>>

If you want all the posible sublist, you can use two for in one list comprehension:
def sublists(lst):
return [lst[m:n+1] for m in range(0,len(lst)+1) for n in range(m,len(lst)+1)]
sublists("banana")
=> ['b', 'ba', 'ban', 'bana', 'banan', 'banana', 'banana', 'a', 'an', 'ana', 'anan', 'anana', 'anana', 'n', 'na', 'nan', 'nana', 'nana', 'a', 'an', 'ana', 'ana', 'n', 'na', 'na', 'a', 'a', '']
if you dont want repeated elements:
def sublistsWithoutRepeated(lst):
return list(set(sublists(lst)))
sublistsWithoutRepeated("banana")
=> ['a', '', 'b', 'ba', 'nana', 'na', 'nan', 'an', 'anana', 'anan', 'n', 'bana', 'ban', 'banan', 'banana', 'ana']

Sorting a list using both length and alphabetically [duplicate]

This question already has answers here:
Length-wise-sorted list but, same length in alphabetical-order in a step
(2 answers)
Closed 5 years ago.
I have a list of all combinations of word HACK like this:
lista = ['H', 'A', 'C', 'K', 'HA', 'HC', 'HK', 'AC', 'AK', 'CK']
I tried sorting the above using :
lista.sort(lambda x,y:cmp(len(x),len(y)))
gives me the same result.
How can I sort with both the length and alphabetically.
Expected Output:
['A', 'C', 'H', 'K', 'AC', 'AH', 'AK', 'CH', 'CK', 'HK']
Update:
from itertools import combinations
inp = "HACK 2".split(" ")
lista = []
for i in range(1,int(inp[1])+1):
for item in list(combinations(inp[0],i)):
lista.append("".join(item))
lista = sorted(lista, key=lambda x: (len(x), x))
print lista
#Output
['A', 'C', 'H', 'K', 'AC', 'AK', 'CK', 'HA', 'HC', 'HK']
#Expected Output
['A', 'C', 'H', 'K', 'AC', 'AH', 'AK', 'CH', 'CK', 'HK']
Also is there anything wrong with how I am iterating the combinations ?

list.sort, sorted accept an optional keyword argument key. Return value of the key function is used to compare elements instead of the elements themselves.
For your case, you can use a key function that returns a tuple of (length, string itself):
>>> lista = ['H', 'A', 'C', 'K', 'HA', 'HC', 'HK', 'AC', 'AK', 'CK']
>>> sorted(lista, key=lambda x: (len(x), x))
['A', 'C', 'H', 'K', 'AC', 'AK', 'CK', 'HA', 'HC', 'HK']

You want to sort not just the lista list, but also all the strings in it. So
>>> lista = ['H', 'A', 'C', 'K', 'HA', 'HC', 'HK', 'AC', 'AK', 'CK']
>>> for i, string in enumerate(lista):
... lista[i] = ''.join(sorted(list(string)))
...
>>> lista
['H', 'A', 'C', 'K', 'AH', 'CH', 'HK', 'AC', 'AK', 'CK']
>>> lista.sort(key=lambda s: (len(s), s))
>>> lista
['A', 'C', 'H', 'K', 'AC', 'AH', 'AK', 'CH', 'CK', 'HK']