Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I am looking for tips and tricks to better code the following in Python (e.g. remove superfluous loops and copying, use more splicing)
I have coded this to create all possible combinations of weights for a portfolio of N securities subject to the constraints:
Weights come from a list of possibilities (in this case 0,.1,.2,.3,.4,.5)
The sum of weights for a valid portfolio must = 1 (Fully Invested)
This is not a valid approach as the number of combinations quickly becomes unmanageable. It's just me trying to get to grips with this language.
Thank you all in advance for you help!
import copy as cp
def generateWeights (weights,possibleWeights,N):
"Generate all possible combinations of weights"
# cycle over number of desired coloumns
for i in range(0,N):
# copy weights to iterate over every element while i pop stuff out of
# the original list
weightsCopy = cp.deepcopy(weights)
for w in weightsCopy:
# make a copy to edit
wtemp = cp.deepcopy(w)
for p in possibleWeights:
# append every possibility
wtemp.append(p)
# I only want combinations with sum == 1 so I can start
# avoiding those that are > 1
if sum(wtemp) <= 1:
weights.append(cp.deepcopy(wtemp))
# get the original wtemp back so I can work on it with my next p
wtemp.pop()
# finished developing the first line of the table. Pop it out and
# move on.
weights.pop(0)
# once again copy weights to iterate over every element while I edit the
# original list
weightsCopy = cp.deepcopy(weights)
for w in weightsCopy:
# remove all possibilities whose sum < 1
# all those > 1 were never added
if sum(w) < 1:
weights.remove(w)
return weights
N=6 # Number of securities
possibleWeights = [0.0,0.1,0.2,0.3,0.4,0.5]
# weights is a coloumn because I want to access its elements and still get
# lists instead of floats.
weights = [[0.0],[0.1],[0.2],[0.3],[0.4],[0.5]]
weights = generateWeights(weights,possibleWeights,N)
You should use the itertools module that already has the algorithms to do most of what you want.
from itertools import combinations
def valid_combinations(weights):
'''generator of possible combinations of weights elements that add up to 1'''
list_length = len(weights) # we will need this
for lengths in range(list_length):
for possible in combinations(weights, lengths): # all possible orderings of weights
if sum(possible[:lengths]) == 1: # only generate valid ones
yield possible[:lengths]
>>> original = [0, .1, .2, .3, .4, .5]
>>> print list(valid_combinations(original))
[(0.1, 0.4, 0.5), (0.2, 0.3, 0.5), (0, 0.1, 0.4, 0.5), (0, 0.2, 0.3, 0.5), (0.1, 0.2, 0.3, 0.4), (0, 0.1, 0.2, 0.3, 0.4)]
If you are only interested in unique combinations of the weights (the order does not matter), you need to use combinations, if it does matter you should use permutations the following way:
from itertools import permutations
def valid_combinations(weights):
'''generator of possible combinations of weights elements that add up to 1'''
list_length = len(weights) # we will need this
for possible in permutations(weights): # all possible orderings of weights
for lengths in range(list_length): # get all prefix sublists
if sum(possible[:lengths]) == 1: # only generate valid ones
yield possible[:lengths]
>>> original = [0, .1, .2, .3, .4, .5]
>>> print list(valid_combinations(original))
>>> [(0, 0.1, 0.2, 0.3, 0.4), (0, 0.1, 0.2, 0.4, 0.3), (0, 0.1, 0.3, 0.2, 0.4), (0, 0.1, 0.3, 0.4, 0.2), (0, 0.1, 0.4, 0.2, 0.3), (0, 0.1, 0.4, 0.3, 0.2), (0, 0.1, 0.4, 0.5), (0, 0.1, 0.4, 0.5), (0, 0.1, 0.5, 0.4), (0, 0.1, 0.5, 0.4), (0, 0.2, 0.1, 0.3, 0.4), (0, 0.2 ...
You can use itertools.combinations(), however you will have to increase the size of combinations until you reach the length of the dataset.
>>> input_list = [0,.1,.2,.3,.4,.5]
>>> from itertools import combinations
>>> valid_combinations = []
>>> for comb_length in range(1,len(input_list)+1):
possible_combinations = combinations(input_list,comb_length)
for comb in possible_combinations:
if sum(comb) ==1:
valid_combinations.append(comb)
>>>valid_combinations
[(0.1, 0.4, 0.5), (0.2, 0.3, 0.5), (0, 0.1, 0.4, 0.5), (0, 0.2, 0.3, 0.5), (0.1, 0.2, 0.3, 0.4), (0, 0.1, 0.2, 0.3, 0.4)]
Read your requirements and updated to have combinations ==1, not <= 1.
Note -- if your dataset for input is very large, you will need a better algorithm as this is bruteforce.
Related
I need to perform something similar to the built-in torch.argmax() function on a one-dimensional tensor, but instead of picking the index of the first of the maximum values, I want to be able to pick a random index of one of the maximum values. For example:
my_tensor = torch.tensor([0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.1])
index_1 = random_max_val_index_fn(my_tensor)
index_2 = random_max_val_index_fn(my_tensor)
print(f"{index_1}, {index_2}")
> 5, 1
You can get the indexes of all the maximums first and then choose randomly from them:
def rand_argmax(tens):
max_inds, = torch.where(tens == tens.max())
return np.random.choice(max_inds)
sample runs:
>>> my_tensor = torch.tensor([0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.1])
>>> rand_argmax(my_tensor)
2
>>> rand_argmax(my_tensor)
5
>>> rand_argmax(my_tensor)
2
>>> rand_argmax(my_tensor)
1
I think this should work:
import numpy as np
import torch
your_tensor = torch.tensor([0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.1])
argmaxes = np.argwhere(your_tensor==torch.max(your_tensor)).flatten()
rand_argmax = np.random.choice(argmaxes)
print(rand_argmax)
make sure you adjust for np.random.choice to account for replacement
below I put the code I would like to get the result like: 0.1, 0.2, 0.3, 0.4 .... but I get this result [0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9] how can I remove those zeros after the decimal point?
squares = []
for i in range(10):
squares.append(i * (0.1))
print(squares)
You can use something like this:
>>> ['{:.2}'.format(i * 0.1) for i in range(10)]
Use the str method format to specify how many decimals to display.
squares = []
for i in range(10):
squares.append(i * (0.1))
print(*["{:.1f}".format(s) for s in squares], sep=', ')
0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9
Sup, Kozinski. Hope you're having a great time.
squares = []
for i in range(10):
squares.append(round(i * (0.1), 1)) #integers will be stored in a proper format
print(squares)
Check out this round function
I have a Numpy array, and I need to find the N maximum product subarrays of M elements. For example, I have the array p = [0.1, 0.2, 0.8, 0.5, 0.7, 0.9, 0.3, 0.5] and I want to find the 5 highest product subarrays of 3 elements. Is there a "fast" way to do that?
Here is another quick way to do it:
import numpy as np
p = [0.1, 0.2, 0.8, 0.5, 0.7, 0.9, 0.3, 0.5]
n = 5
m = 3
# Cumulative product (starting with 1)
pc = np.cumprod(np.r_[1, p])
# Cumulative product of each window
w = pc[m:] / pc[:-m]
# Indices of the first element of top N windows
idx = np.argpartition(w, n)[-n:]
print(idx)
# [1 2 5 4 3]
Approach #1
We can create sliding windows and then perform prod reduction and finally np.argpartition to get top N ones among them -
from skimage.util.shape import view_as_windows
def topN_windowed_prod(a, W, N):
w = view_as_windows(a,W)
return w[w.prod(1).argpartition(-N)[-N:]]
Sample run -
In [2]: p = np.array([0.1, 0.2, 0.8, 0.5, 0.7, 0.9, 0.3, 0.5])
In [3]: topN_windowed_prod(p, W=3, N=2)
Out[3]:
array([[0.8, 0.5, 0.7],
[0.5, 0.7, 0.9]])
Note that the order is not maintained with np.argpartition. So, if we need the top N in descending order of prod values, use range(N) with it. More info.
Approach #2
For smaller window lengths, we can simply slice and get our desired result, like so -
def topN_windowed_prod_with_slicing(a, W, N):
w = view_as_windows(a,W)
L = len(a)-W+1
acc = a[:L].copy()
for i in range(1,W):
acc *= a[i:i+L]
idx = acc.argpartition(-N)[-N:]
return w[idx]
I need to generate list of values from provided that satisfy this requirements:
Sum of all generated values should be equal of total, only providedValues should be used to get the sum, providedValues and total can be any double.
For example:
total = 1.0
providedValues = [0.5, 0.25]
Values in output list should be randomly distributed, for example output can be: [0.5, 0.25, 0.25], [0.25, 0.5, 0.25] or [0.25, 0.25, 0.5]
In case sum can't be equal total:
total = 1.0
providedValues = [0.3]
algorithm should throw error.
Language for implementation not so matter, I'll try to read any.
This algorithm will return all the possible combinations that sum to total.
import itertools
import numpy as np
def find_combination(total, providedValues):
i = 1
rv = []
while True:
combs = list(itertools.combinations_with_replacement(providedValues,i))
validCombs = [comb for comb in combs if np.isclose(sum(comb),total)]
if validCombs:
rv.extend(validCombs)
elif not [comb for comb in combs if sum(comb) <= total]:
return rv
i += 1
Output:
>>> find_combination(1.0, [0.5, 0.25])
[(0.5, 0.5), (0.5, 0.25, 0.25), (0.25, 0.25, 0.25, 0.25)]
>>> find_combination(1.0, [0.3])
[]
If you want to get all permutations of the results, you can use
>>> set(itertools.permutations((0.5, 0.25, 0.25)))
{(0.25, 0.25, 0.5), (0.25, 0.5, 0.25), (0.5, 0.25, 0.25)}
For example:
>>> set(y for x in find_combination(1.0, [0.5, 0.25]) for y in itertools.permutations(x))
{(0.25, 0.25, 0.25, 0.25),
(0.25, 0.25, 0.5),
(0.25, 0.5, 0.25),
(0.5, 0.25, 0.25),
(0.5, 0.5)}
Here is my solution based on there are two values provided, you may want to change it for you need
from itertools import permutations, combinations
def get_scala(x,y,t):
# get list of scala combinations
# find a,b that a*x+b*y = total
scala_list = []
amax = int(t // x) # possible max scala for x
bmax = int(t // y) # possible max scala for y
for i in range(1, amax+1):
for j in range(1, bmax+1):
if i*x + j*y == t: # find the scala combination that == total
scala_list.append((i, j))
if scala_list:
return scala_list
else:
print("Warning: cannot add up to the total")
def dist(x, y, scala):
a, b = scala
# get a base list with a number of x and b number of y [x,x,y,y,y]
bl = [x]*a + [y]*b
# get permutations and using set to get rid of duplicate items
return set(permutations(bl))
for l in get_scala(0.3, 0.2, 1):
for d in dist(0.3, 0.2, l):
print(d)
the output would look look:
(0.2, 0.3, 0.2, 0.3)
(0.2, 0.2, 0.3, 0.3)
(0.3, 0.2, 0.2, 0.3)
(0.3, 0.2, 0.3, 0.2)
(0.3, 0.3, 0.2, 0.2)
(0.2, 0.3, 0.3, 0.2)
I have a list of lists that I want to re-order:
qvalues = [[0.1, 0.3, 0.6],[0.7, 0.1, 0.2],[0.3, 0.4, 0.3],[0.1, 0.3, 0.6],[0.1, 0.3, 0.6],[0.1, 0.3, 0.6]]
I know how to reorder this list if I have a list with the order I want (example here). The tricky part is getting this order.
What I have is this:
locations = [(['Loc1','Loc1'], 3), (['Loc2'], 1), (['Loc3', 'Loc3', 'Loc3'], 2)]
This is a list of tuples, where the first element of each tuple is a list with the location name, repeated for each individual in that location, and the second element is the order these individuals are in on the qvalues list (qvalues[0] is 'Loc2', qvalues[1:4] are 'Loc3' and qvalues[4:6] are 'Loc1'.
What I want is to change the order of the lists in qvalues to the order they show up in locations: First 'Loc1', then 'Loc2' and finally 'Loc3'.
This is just a small example, my real dataset has hundreds of individuals and 17 locations.
Thanks in advance for any help you may provide.
You will need to build a list of offsets and length instead of length and positions as provided in your locations list. Then, you’ll be able to reorder based on the answer you linked to:
qvalues = [[0.1, 0.3, 0.6],[0.7, 0.1, 0.2],[0.3, 0.4, 0.3],[0.1, 0.3, 0.6],[0.1, 0.3, 0.6],[0.1, 0.3, 0.6]]
locations = [(['Loc1','Loc1'], 3), (['Loc2'], 1), (['Loc3', 'Loc3', 'Loc3'], 2)]
locations_dict = {pos:(index,len(loc)) for index,(loc,pos) in enumerate(locations)}
# if python2: locations_dict = dict([(pos,(index,len(loc))) for index,(loc,pos) in enumerate(locations)])
offsets = [None]*len(locations)
def compute_offset(pos):
# compute new offset from offset and length of previous position. End of recursion at position 1: we’re at the beginning of the list
offset = sum(compute_offset(pos-1)) if pos > 1 else 0
# get index at where to store current offset + length of current location
index, length = locations_dict[pos]
offsets[index] = (offset, length)
return offsets[index]
compute_offset(len(locations))
qvalues = [qvalues[offset:offset+length] for offset,length in offsets]
You’ll end up with qvalues being a list of lists of lists instead of a "simple" list of lists. If you want to flatten it to keep your initial layout use this list comprehension instead:
qvalues = [value for offset,length in offsets for value in qvalues[offset:offset+length]]
Output with first version
[[[0.1, 0.3, 0.6], [0.1, 0.3, 0.6]], [[0.1, 0.3, 0.6]], [[0.7, 0.1, 0.2], [0.3, 0.4, 0.3], [0.1, 0.3, 0.6]]]
Output with second version
[[0.1, 0.3, 0.6], [0.1, 0.3, 0.6], [0.1, 0.3, 0.6], [0.7, 0.1, 0.2], [0.3, 0.4, 0.3], [0.1, 0.3, 0.6]]