Develop Reference

Turn a string into a list before appending

I have 3 lists I created and I want to append them from the sql query output. I thought of created them as string as lst and append the output but I get the error below.
list_A1 = []
list_A2 = []
list_A3 = []
for i, name in enumerate(dataset['name'][1]['price']):
.
.
.
id_list = pd.read_sql_query(q, con=db)[price].tolist()
lst = f"{lst_name}_{id}"
#lst created the 3 list above.
lst.append(id_list).astype(int)
Error : AttributeError: 'str' object has no attribute 'append'
Edit:
What I mean is on each loop lst = list_A1..
and each of those lists gets populated with the data from the sql query

You can try follows,
list_A1 = []
list_A2 = []
list_A3 = []
for id_ in range(1,4,1):
lst = f"list_A1"
vars()[lst].append(id_)
print(list_A1)
Output
[1, 2, 3]
Reference
How do I create variable variables?

You can't create variable using string f"{lst_name}_{id}". You may only use it as key in dictionary all_lists[f"{lst_name}_{id}"] = []. And you should use dictionary
all_lists = {
"list_A1": [],
"list_A2": [],
"list_A3": [],
}
for ... in ...:
id_list = ....astype(int).tolist()
key = f"{lst_name}_{id}"
lst = all_lists[key]
lst.append( id_list )
You could do it even dinamically
all_lists = {} # empty dictionary
for ... in ...:
id_list = ....astype(int).tolist()
key = f"{lst_name}_{id}"
# create list if not exists yet
if key not in all_lists:
all_lists[key] = []
lst = all_lists[key]
lst.append( id_list )
BTW: You can create variable with vars() or similar functions but it is not preferred. You have dictionary for this.

use python to combine data lists

If I have two separated lists:
list1 = [['2021-05-24', '31220'],....., ['2021-05-24', '6640'],['2021-05-10', '8830'].....]
list2 = [['2021-05-24', '77860'],.....,['2021-05-24', '438000'],['2021-05-10', '9990'].....]
How could I combine them to
[['2021-05-24', 'add all numbers in '2021-05-24' together'],['2021-05-10', 'add all numbers in '2021-05-10' together']]
, '.....' means there are many list-tuples
I am considering delete the duplicated date in each list and then add two lists up:
import networkx as nx
G = nx.Graph()
G.add_nodes_from(sum(list2, []))
q = [[(s[i],s[i+1]) for i in range(len(s)-1)] for s in list2]
for i in q:
G.add_edges_from(i)
print([list(i) for i in nx.connected_components(G)])
but my code not only deleted the same dates but also deleted the same numbers.
Thanks in advance.

I'd recommend using a defaultdict:
from collections import defaultdict
result = defaultdict(int)
for k,v in (list1 + list2):
result[k] += v
Then you can convert the dict back to a list. Of course if you have several lists you may want to use itertools.chain instead of list1 + list2

You can go with creating new dictionary and do calculation and then create list out of it. Here is the code for that
result={}
for i in list1+list2: #creating dict and doing calculation
if i[0] in result.keys():
result[i[0]] += int(i[1])
else:
result[i[0]] = int(i[1])
result_list = [[key, result[key]] for key in result] #converting dict to expected list
print(result_list)

Find the Set substraction or the difference between the list of dictionaries

I have two lists of dictionaries as below , both are in different length.
list A = [{'key1':'001', 'key2':'AAA', 'key3':'BBB'},
{'key1':'002', 'key2':'BBB', 'key3':'CCC'},
{'key1':'003', 'key2':'XYZ', 'key3':'ZYX'}]
list B = [{'key1':'001', 'key2':'AAA', 'key3':'BBB'},
{'key1':'002', 'key2':'ABX', 'key3':'GHT'}]
I want to do a List A - list B it should return me the list contains the newly added Dictionary ,and the dictionary in which any of the values are changed
Result of List A - List B would return as
List A - List B = [ {'key1':'003', 'key2':'XYZ', 'key3':'ZYX'},
{'key1':'002', 'key2':'BBB', 'key3':'CCC'}]
I different ways, Could any one please help me on this

You can try this:
list_A = [
{'key1':'001', 'key2':'AAA', 'key3':'BBB'},
{'key1':'002', 'key2':'BBB', 'key3':'CCC'},
{'key1':'003', 'key2':'XYZ', 'key3':'ZYX'}
]
list_B = [
{'key1':'001', 'key2':'AAA', 'key3':'BBB'},
{'key1':'002', 'key2':'ABX', 'key3':'GHT'}
]
def list_subtraction(from_list, to_list):
''' Subtracts to_list from from_list'''
list_c = []
for i in list_A:
if i not in list_B:
list_c.append(i)
return list_c
print(list_subtraction(list_A, list_B))
Hope this helps..

Group elements of a list based on repetition of values

I am really new to Python and I am having a issue figuring out the problem below.
I have a list like:
my_list = ['testOne:100', 'testTwo:88', 'testThree:76', 'testOne:78', 'testTwo:88', 'testOne:73', 'testTwo:66', 'testThree:90']
And I want to group the elements based on the occurrence of elements that start with 'testOne'.
Expected Result:
new_list=[['testOne:100', 'testTwo:88', 'testThree:76'], ['testOne:78', 'testTwo:88'], ['testOne:73', 'testTwo:66', 'testThree:90']]

Just start a new list at every testOne.
>>> new_list = []
>>> for item in my_list:
if item.startswith('testOne:'):
new_list.append([])
new_list[-1].append(item)
>>> new_list
[['testOne:100', 'testTwo:88', 'testThree:76'], ['testOne:78', 'testTwo:88'], ['testOne:73', 'testTwo:66', 'testThree:90']]

Not a cool one-liner, but this works also with more general labels:
result = [[]]
seen = set()
for entry in my_list:
test, val = entry.split(":")
if test in seen:
result.append([entry])
seen = {test}
else:
result[-1].append(entry)
seen.add(test)
Here, we are keeping track of the test labels we've already seen in a set and starting a new list whenever we encounter a label we've already seen in the same list.
Alternatively, assuming the lists always start with testOne, you could just start a new list whenever the label is testOne:
result = []
for entry in my_list:
test, val = entry.split(":")
if test == "testOne":
result.append([entry])
else:
result[-1].append(entry)

It'd be nice to have an easy one liner, but I think it'd end up looking a bit too complicated if I tried that. Here's what I came up with:
# Create a list of the starting indices:
ind = [i for i, e in enumerate(my_list) if e.split(':')[0] == 'testOne']
# Create a list of slices using pairs of indices:
new_list = [my_list[i:j] for (i, j) in zip(ind, ind[1:] + [None])]

Not very sophisticated but it works:
my_list = ['testOne:100', 'testTwo:88', 'testThree:76', 'testOne:78', 'testTwo:88', 'testOne:73', 'testTwo:66', 'testThree:90']
splitting_word = 'testOne'
new_list = list()
partial_list = list()
for item in my_list:
if item.startswith(splitting_word) and partial_list:
new_list.append(partial_list)
partial_list = list()
partial_list.append(item)
new_list.append(partial_list)

joining the list into a string with delimiter |
step1="|".join(my_list)
splitting the listing based on 'testOne'
step2=step1.split("testOne")
appending "testOne" to the list elements to get the result
new_list=[[i for i in str('testOne'+i).split("|") if len(i)>0] for i in step2[1:]]

Python remove elements matching pattern from array

I have a dictionary that contains strings as keys and lists as values.
I'd like to remove all list elements that contain the strings "food", "staging", "msatl" and "azeus". I have the below code already, but am having a hard time applying the logic I have in filterIP to the rest of the strings I have.
def filterIP(fullList):
regexIP = re.compile(r'\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}$')
return filter(lambda i: not regexIP.search(i), fullList)
groups = {key : [domain.replace('fake.com', 'env.fake.com')
for domain in filterIP(list(set(items)))]
for (key, items) in groups.iteritems() }
for key, value in groups.iteritems():
value.sort()
meta = { "_meta" : { "hostvars" : hostvars } }
groups.update(meta)
print(self.json_format_dict(groups, pretty=True))
Example of current output
"role_thumper": [
"thumper-msatl1-prod-1.env.fake.com",
"thumper-msatl1-prod-2.env.fake.com",
"thumper-rwva1-prod-1.env.fake.com",
"thumper-rwva1-prod-2.env.fake.com",
"thumper-rwva1-prod-3.env.fake.com",
"thumper-rwva1-prod-4.env.fake.com",
"thumper-rwva1-prod-5.env.fake.com",
"thumper-rwva1-prod-6.env.fake.com",
"thumper-staging-1.env.fake.com"
],
"role_thumper_mongo": [
"thumper-mongo-staging-1.env.fake.com",
"thumper-mongo-staging-2.env.fake.com",
"thumpermongo-rwva1-staging-1.env.fake.com",
"thumpermongo-rwva1-staging-2.env.fake.com"
],
"role_thumper_mongo_arb": [
"thumper-mongo-arb-staging-1.env.fake.com",
"thumpermongo-arb-rwva1-staging-1.env.fake.com"
],

A list comprehension is what you're after
x= ["a", "b", "aa", "aba"]
x_filtered = [i for i in x if "a" not in i]
print(x_filtered)
>>> ['b']
This is just shorthand for a for loop.
x_filtered = []
for i in x:
if "a" not in i:
x_filtered.append(i)

A simple way to accomplish your task would be to iterate over each lists in the dictionary. Create new lists based upon your criteria, and assign the new lists to the same keys but in a new dictionary. Here is how that would look like in code:
def filter_words(groups, words):
d = {}
for key, domains in groups.iteritems():
new_domains = []
for domain in domains:
if not any(word in domain for word in words):
new_domains.append(domain)
d[key] = new_domains
return d
And you would call it like so:
groups = filter_words(groups, {"food", "staging", "msatl" and "azeus"})
The "meat" of the code above is the second for loop:
for domain in domains:
if not any(word in domain for word in words):
new_domains.append(domain)
This code goes over each string in the current key's list, and filters out all invalid strings according to a list of invalid words.

If I understand you correctly, this might help.
Set up an exclude list:
exclude= ["food", "staging", "msatl", "azeus"]
Test list ( I couldn't really find instances in your examples)
test= ["food", "staging", "msatl", "azeus", "a", "bstaging"]
Run list comprehension (the name of iterators don't matter, you can pick more appropriate ones)
result= [i for i in test if not any([e for e in exclude if e in i])]
result
['a']
The answer above by #Julian gives a good explanation of what list comprehensions do. This uses two of them, the any part is True if there is any match in the exclude list.
Hope this helps.