I want to alter the values in my 2D list keyed within a dictionary. My code is altering all the values in my dictionary which is not intended.
cache_memory = {}
zero_list = ["0"] * 2 + ["00"] * 5
empty_lists = []
# append zero_lists to empty_list to make a 2D LIST
for count in range(2):
empty_lists.append(zero_list)
# store 2D lists with keys
for index in range(2):
cache_memory[index] = empty_lists
cache_memory[0][0][0] = "1"
print(cache_memory)
Output is:
{0: [['1', '0', '00', '00', '00', '00', '00'], ['1', '0', '00', '00', '00', '00', '00']], 1: [['1', '0', '00', '00', '00', '00', '00'], ['1', '0', '00', '00', '00', '00', '00']]}
I want it to be:
{0: [['1', '0', '00', '00', '00', '00', '00'], ['0', '0', '00', '00', '00', '00', '00']], 1: [['0', '0', '00', '00', '00', '00', '00'], ['0', '0', '00', '00', '00', '00', '00']]}
Is there a way to achieve this in python or should I try a different workaround?
In the following lines of your code
for count in range(2):
empty_lists.append(zero_list)
# store 2D lists with keys
for index in range(2):
cache_memory[index] = empty_lists
You are creating a shallow copy of the list which is nothing but a reference to the list. When you make changes in the copy, the same changes are reflected in the original list. That's why all the lists in your dictionary 'cache_memory' are references of the same list and modifying the element in one list modifies the list for all the references.
To get your favourable output you need to create and store the deep copies of the list which can be achieved using copy module available in python. So you need to modify your code as:
import copy #importing copy module
cache_memory = {}
zero_list = ["0"] * 2 + ["00"] * 5
empty_lists = []
# append zero_lists to empty_list to make a 2D LIST
for count in range(2):
empty_lists.append(copy.deepcopy(zero_list))
# store 2D lists with keys
for index in range(2):
cache_memory[index] = copy.deepcopy(empty_lists)
cache_memory[0][0][0] = "1"
print(cache_memory)
You can learn more about shallow copies and deep copies in this article Shallow vs deep copies in Python
When you just say cache_memory[index] = empty_lists, Python will take that as cache_memory[index] and empty_lists is one object. If you want to set cache_memory[index] to a copy of empty_lists, then you can either put .copy() after empty_lists in that line or [:]. Both of these get a copy of empty_lists instead of taking them both as one object.
If you want to explore this more, just use the following code snippet.
my_list = [1,2,3]
new_list = my_list
print('Prev my_list:',my_list)
print('Prev new_list:',new_list)
my_list.append(4)
print('New my_list:',my_list)
print('New new_list:',new_list) # As you can see new_list also gets updated
Related
I have a file say, file1.txt which looks something like below.
27,28,29,30,1,0.67
31,32,33,34,1,0.84
35,36,37,38,1,0.45
39,40,41,42,1,0.82
43,44,45,46,1,0.92
43,44,45,46,1,0.92
51,52,53,54,2,0.28
55,56,57,58,2,0.77
59,60,61,62,2,0.39
63,64,65,66,2,0.41
75,76,77,78,3,0.51
90,91,92,93,3,0.97
Where the last column is the fitness and the 2nd last column is the class.
Now I read this file like :
rule_file_name = 'file1.txt'
rule_fp = open(rule_file_name)
list1 = []
for line in rule_fp.readlines():
list1.append(line.replace("\n","").split(","))
Then a default dictionary was created to ensure the rows are separated according to the classes.
from collections import defaultdict
classes = defaultdict(list)
for _list in list1:
classes[_list[-2]].append(_list)
Then they are paired up within each class using the below logic.
from random import sample, seed
seed(1)
for key, _list in classes.items():
_list=sorted(_list,key=itemgetter(-1),reverse=True)
length = len(_list)
middle_index = length // 2
first_half = _list[:middle_index]
second_half = _list[middle_index:]
result=[]
result=list(zip(first_half,second_half))
Later using the 2 rows of the pair, a 3rd row is being created using the below logic:
ans=[[random.choice(choices) for choices in zip(*item)] for item in result]
So if there were initially 12 rows in the file1, that will now form 6 pairs and hence 6 new rows will be created. I simply want to append those newly created rows to the file1 using below logic:
list1.append(ans)
print(ans)
with open(f"output.txt", 'w') as out:
new_rules = [list(map(str, i)) for i in list1]
for item in new_rules:
out.write("{}\n".format(",".join(item)))
#out.write("{}\n".format(item))
But now my output.txt looks like:
27,28,29,30,1,0.67
31,32,33,34,1,0.84
35,36,37,38,1,0.45
39,40,41,42,1,0.82
43,44,45,46,1,0.92
43,44,45,46,1,0.92
51,52,53,54,2,0.28
55,56,57,58,2,0.77
59,60,61,62,2,0.39
63,64,65,66,2,0.41
75,76,77,78,3,0.51
90,91,92,93,3,0.97
['43', '44', '41', '46', '1', '0.82'],['27', '28', '45', '46', '1', '0.92'],['35', '36', '33', '38', '1', '0.84']
['55', '60', '57', '58', '2', '0.77'],['51', '64', '53', '66', '2', '0.28']
['75', '91', '77', '93', '3', '0.51']
But my desired outcome is:
27,28,29,30,1,0.67
31,32,33,34,1,0.84
35,36,37,38,1,0.45
39,40,41,42,1,0.82
43,44,45,46,1,0.92
43,44,45,46,1,0.92
51,52,53,54,2,0.28
55,56,57,58,2,0.77
59,60,61,62,2,0.39
63,64,65,66,2,0.41
75,76,77,78,3,0.51
90,91,92,93,3,0.97
43,44,41,46,1,0.82
27,28,45,46,1,0.92
35,36,33,38,1,0.84
55,60,57,58,2,0.77
51,64,53,66,2,0.28
75,91,77,93,3,0.51
I would use numpy, it is flexible and compact.
import numpy as np
fin = 'file1.txt'
col1, col2, col3, col4, jclass, fitness = np.loadtxt(fin, unpack=True, delimiter=',')
rows = np.column_stack((col1, col2, col3, col4, jclass, fitness))
print(rows[0])
print(rows[-1])
print(fitness)
Then apply your logic to the rows array
I have a list of stings that have some repeating elements that I want to combine into a shorter list.
The original list contents look something like this:
lst = [['0.1', '0', 'RC', '100'],
['0.2', '10', 'RC', '100'],
['0.3', '5', 'HC', '20'],
['0.4', '5', 'HC', '20'],
['0.5', '5', 'HC', '20'],
['0.6', '5', 'HC', '20'],
['0.7', '5', 'HC', '20'],
['0.8', '5', 'HC', '20'],
['0.9', '10', 'RC', '100'],
['1.0', '0', 'RC', '100']]
After running it through the function it would become:
lst = [['0.1', '0', 'RC', '100'],
['0.2', '10', 'RC', '100'],
['0.3', '5', 'HC', '20'],
['0.9', '10', 'RC', '100'],
['1.0', '0', 'RC', '100']]
The list will always have this general structure, so essentially I want to combine the list based on whether or not the last 3 columns are exactly the same.
I want it to be a callable function so it would look some thing like:
def combine_list(lst):
if sublist[1:3] == next_sublist[1:3]:
let.remove(next_sublist)
My initial research on this showed many methods to remove a sublist based on its index, but that is not necessarily known before hand. I also found the re module, however I have never used it and unsure on how to implement it. Thank you in advanced
If you want to remove sub lists that are the same for the last three elements and consecutive, you would need itertools.groupby keyed on the last three elements:
from itertools import groupby
[next(g) for _, g in groupby(lst, key=lambda x: x[1:])]
#[['0.1', '0', 'RC', '100'],
# ['0.2', '10', 'RC', '100'],
# ['0.3', '5', 'HC', '20'],
# ['0.9', '10', 'RC', '100'],
# ['1.0', '0', 'RC', '100']]
Maybe just use a set to keep track of duplicates?
def combine_list(lst):
out = []
seen = set()
for item in lst:
if not tuple(item[1:]) in seen:
out.append(item)
seen.add(tuple(item[1:]))
return out
Lists are a mutable data structure. And so there is no guarantee that the contents of a list does not change over time. That means it cannot be used in a hashing function (which the set uses). The tuple, on the other hand, is immutable, and hence hashable.
for index in range(len(lst) - 1, 0, -1):
if lst[index][1:] == lst[index - 1][1:]:
lst.pop(index)
By going through the list backwards, we remove the problems with indices changing when we remove elements. This results in an in-place reduction.
If you'd like to make a new list, this can be done via list comprehension following the same idea, but since we're not doing it in place, we don't have to work in reverse:
lst[0] + [lst[ind] for ind in range(1, len(lst)) if lst[ind][1:] != lst[ind-1][1:]]
Again, lst[0] is trivially non-duplicate and therefore automatically included.
def combine_list(ls):
cpy = ls[:]
for i, sub in enumerate(ls[:len(ls) - 1]):
if sub[1:] == ls[i + 1][1:]:
cpy.remove(ls[i + 1])
return cpy
This function should work. It creates a new copy of the list, to avoid modifying the original. Then it iterates over the original list (except the last value), as that stays the same.
It then checks if the last values of the list are equal to the last values of the next list. If they are, the next list is deleted.
The function then returns the new list.
Whats the best way to filter out some subsets from a generator. For example I have a string "1023" and want to produce all possible combinations of each of the digits. All combinations would be:
['1', '0', '2', '3']
['1', '0', '23']
['1', '02', '3']
['1', '023']
['10', '2', '3']
['10', '23']
['102', '3']
['1023']
I am not interested in a subset that contains a leading 0 on any of the items, so the valid ones are:
['1', '0', '2', '3']
['1', '0', '23']
['10', '2', '3']
['10', '23']
['102', '3']
['1023']
I have two questions.
1) If using a generator, whats the best way to filter out the ones with leading zeroes. Currently, I generate all combinations then loop through it afterwards and only continuing if the subset is valid. For simplicity I am only printing the subset in the sample code. Assuming the generator that was created is very long or if it constains a lot of invalid subsets, its almost a waste to loop through the entire generator. Is there a way to stop the generator when it sees an invalid item (one with leading zero) then filter it off 'allCombinations'
2) If the above doesn't exist, whats a better way to generate these combinations (disregarding combinations with leading zeroes).
Code using a generator:
import itertools
def isValid(subset): ## DIGITS WITH LEADING 0 IS NOT VALID
valid = True
for num in subset:
if num[0] == '0' and len(num) > 1:
valid = False
break
return valid
def get_combinations(source, comb):
res = ""
for x, action in zip(source, comb + (0,)):
res += x
if action == 0:
yield res
res = ""
digits = "1023"
allCombinations = [list(get_combinations(digits, c)) for c in itertools.product((0, 1), repeat=len(digits) - 1)]
for subset in allCombinations: ## LOOPS THROUGH THE ENTIRE GENERATOR
if isValid(subset):
print(subset)
Filtering for an easy and obvious condition like "no leading zeros", it can be more efficiently done at the combination building level.
def generate_pieces(input_string, predicate):
if input_string:
if predicate(input_string):
yield [input_string]
for item_size in range(1, len(input_string)+1):
item = input_string[:item_size]
if not predicate(item):
continue
rest = input_string[item_size:]
for rest_piece in generate_pieces(rest, predicate):
yield [item] + rest_piece
Generating every combination of cuts, so long it's not even funny:
>>> list(generate_pieces('10002', lambda x: True))
[['10002'], ['1', '0002'], ['1', '0', '002'], ['1', '0', '0', '02'], ['1', '0', '0', '0', '2'], ['1', '0', '00', '2'], ['1', '00', '02'], ['1', '00', '0', '2'], ['1', '000', '2'], ['10', '002'], ['10', '0', '02'], ['10', '0', '0', '2'], ['10', '00', '2'], ['100', '02'], ['100', '0', '2'], ['1000', '2']]
Only those where no fragment has leading zeros:
>>> list(generate_pieces('10002', lambda x: not x.startswith('0')))
[['10002'], ['1000', '2']]
Substrings that start with a zero were never considered for the recursive step.
One common solution is to try filtering just before using yield. I have given you an example of filtering just before yield:
import itertools
def my_gen(my_string):
# Create combinations
for length in range(len(my_string)):
for my_tuple in itertools.combinations(my_string, length+1):
# This is the string you would like to output
output_string = "".join(my_tuple)
# filter here:
if output_string[0] != '0':
yield output_string
my_string = '1023'
print(list(my_gen(my_string)))
EDIT: Added in a generator comprehension alternative
import itertools
my_string = '1023'
my_gen = ("".join(my_tuple)[0] for length in range(len(my_string))
for my_tuple in itertools.combinations(my_string, length+1)
if "".join(my_tuple)[0] != '0')
This question already has answers here:
Add SUM of values of two LISTS into new LIST
(22 answers)
Closed 6 years ago.
I'm pretty new to Python although I have learned most the basic's although I need to be able to read from a csv file (which so far works), then append the data from this csv into lists which is working, and the part I am unsure about is using two of these lists and / 120 and * 100
for example the list1 first score is 55 and list2 is 51, I want to merge these together into a list to equal 106 and then add something which can divide then times each one as there is 7 different numbers in each list.
import csv
list1 = []
list2 = []
with open("scores.csv") as f:
reader = csv.reader(f)
for row in reader:
list1.append(row[1])
list2.append(row[2])
print (list1)
print (list2)
OUTPUT
['55', '25', '40', '21', '52', '42', '19']
['51', '36', '50', '39', '53', '33', '40']
EXPECTED OUTPUT (WANTED OUTPUT)
['106', '36', '90', '60', '105', '75', '59']
which then needs to be divided by 120 and * 100 for each one.
Check out zip.
for a, b in zip(list1, list2):
# .... do stuff
so for you maybe:
output = [((int(a)+int(b))/120)*100 for a, b in zip(list1, list2)]
Make a new list that takes your desired calculations into account.
>>> list1 = ['55', '25', '40', '21', '52', '42', '19']
>>> list2 = ['51', '36', '50', '39', '53', '33', '40']
>>> result = [(int(x)+int(y))/1.2 for x,y in zip(list1, list2)]
>>> result
[88.33333333333334, 50.833333333333336, 75.0, 50.0, 87.5, 62.5, 49.16666666666667]
all_tags = ['24', '02', '26', '03', '33', '32', '31', '30', '29', '68', '11']
ref_tag = str('24')
union_tags = set(all_tags) | set(ref_tag)
left_tags = set(all_tags) - set(ref_tag)
print(union_tags)
print(left_tags)
The above is the simple code which I expect elements in union_tags should be the same as those in all_tags. However, the result is
set
(['24', '02', '26', '03', '33', '32', '31', '30', '29', '68', '2', '4', '11'])
The union_tags instead contains two extra elements '2' and '4', which I think it is the result splitting the str '24'.
Again, left_tags should exclude element '24'. However, the result still have the '24'.
Please let me know why. I use the python 2.7 as the interpreter.
Set function accept an iterable with hashable items and convert it to a set object, and since strings are iterables when you pass the string 24 to your set function it converts your string to following set:
{'2', '4'}
And at last the unioin of this set with all_tags would contain items 2 and 4.
If you want to put the 24 in a set as one item you can use {} in order to create your expected set:
>>> ref_tag = {'24'}
set(['24'])