Related
I am trying to create a list of 6 numbers lists from 1 to 49 throw looping from 1 to 49 and creating all possible sets of 1 to 49 .
the issue is that code stops at number 15 and in Pycharm nothing is being printed (excel file is being written but stops at 38759 record)
import itertools
import pandas as pd
stuff = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
all=[]
for L in range(0, len(stuff)+1):
for subset in itertools.combinations(stuff, L):
alist=list(subset)
if len(subset)==6:
all.append(alist)
all_tuple=tuple(all)
df = pd.DataFrame(all_tuple,columns=['z1','z2','z3','z4','z5','z6'])
print(df)
df.to_excel('test.xlsx')
If I understand correctly, you are trying to find the possible combinations of 6 numbers sampled from the list [1, 2, 3, ..., 49] without replacement.
But your code calculates the combinations of all lengths and then only saves those of length 6.
To get a clue as to why your code does not terminate quickly, consider the number of combinations of 6 numbers:
>>> print(len(list(itertools.combinations(range(1, 50), 6))))
13983816
So, if there are 14 million possible combinations of 6 numbers, imagine how many combinations there are of 7, 8, 9, ...
Here is some code to calculate only the 14 million combinations of length 6:
combs = list(itertools.combinations(range(1, 50), 6))
Or, if you really want to build the dataframe:
# Warning, this takes about 25 seconds
combs = itertools.combinations(range(1, 50), 6)
df = pd.DataFrame(combs, columns=['z1','z2','z3','z4','z5','z6'])
Bear in mind that this will take up quite a bit of memory. I'm not sure if Excel can handle 14 million rows so I didn't risk it.
Also, don't use reserved keywords for variable names. all is a built in Python function.
Using my algorithm, I can take space-time data as an input list & generate accurate planetary & astrological data as a 2D lists as follows:
Input_list: [Year, Month, Day, Hour, Minute, Second, Latitude, Longitude, Altitude]
Input_sample: [1990, 4, 9, 21, 12, 30, 22.51361111111111, 88.3411111111111, 9.14]
Output:2 2D lists "GRAHA POSITIONS" & "HOUSE CUSPS"
--------------------GRAHA POSITIONS-----------------------------
[Planet, Z.sign, degree, minute, second, nakshatra, pada, angle_in_float]
['Sun', 'Aries', 19, 30, 30, 'Revati', 4, 19.508579713622044]
['Moon', 'Libra', 14, 36, 20, 'Chitra', 1, 14.60574883925213]
['Mercury', 'Taurus', 8, 14, 2, 'Bharani', 2, 8.23414995759422]
['Venus', 'Pisces', 3, 21, 55, 'Shatavishak', 2, 3.3653055864194243]
['Mars', 'Aquarius', 21, 33, 43, 'Dhanistha', 3, 21.562001016537806]
['Jupiter', 'Cancer', 3, 45, 55, 'Ardra', 3, 3.765369522224674]
['Saturn', 'Capricorn', 24, 49, 1, 'U.Ashadha', 3, 24.81699905482151]
['Rahu', 'Aquarius', 14, 21, 35, 'Dhanistha', 1, 14.359770567077646]
['Ketu', 'Leo', 14, 21, 35, 'Ashlesha', 3, 14.359770567077646]
----------------------HOUSE CUSPS-------------------------------
[Cusp, Z.sign, degree, minute, second, nakshatra, pada, angle_in_float]
[1, 'Scorpio', 29, 16, 11, 'Anuradha', 2, 29.269861999821757]
[2, 'Sagittarius', 28, 55, 44, 'Moola', 3, 28.929165427870714]
[3, 'Aquarius', 0, 37, 54, 'Sravana', 1, 0.6317628121900043]
[4, 'Pisces', 3, 49, 57, 'Shatavishak', 3, 3.8325043513459605]
[5, 'Aries', 5, 53, 38, 'U.Bhadrapada', 4, 5.893938324894634]
[6, 'Taurus', 4, 20, 36, 'Bharani', 1, 4.343341833871818]
[7, 'Taurus', 29, 16, 11, 'Krittika', 4, 29.269861999821728]
[8, 'Gemini', 28, 55, 44, 'Ardra', 1, 28.929165427870714]
[9, 'Leo', 0, 37, 54, 'Pushya', 3, 0.6317628121900043]
[10, 'Virgo', 3, 49, 57, 'P.Phalguni', 1, 3.832504351345932]
[11, 'Libra', 5, 53, 38, 'Hasta', 2, 5.893938324894634]
[12, 'Scorpio', 4, 20, 36, 'Svati', 3, 4.34334183387179]
Now I want to use this data to create visualizations as as South Indian horoscope chart. I've added an example chart to show what I want to achieve.Sample chart
In the image, the chart background with zodiac symbols in the bottom right is static, I can simply use an image for that. The planets & asc symbol i.e. HOUSE CUSPS [0][0] (red slanted line in Scorpio) are vectors, which I'd like to import into my program & assign to planet names i.e. GRAHA POSITIONS[i][0] from the above 2D list.
The number above a planet is its angle in the rashi i.e. GRAHA POSITIONS[i][2,3,4] & the text below is its Nakshatra & pada i.e. GRAHA POSITIONS[i][5,6].
The little black numbers in the bottom left of a rashi are the cusps in that rashi i.e. HOUSE CUSPS[i][0].
Is it reasonable to try to build this idea in Python? If yes, then how? If no, then please guide me.
I'm an amateur programmer by need, with only few months of experience in Python. I'm very eager to build this software for research in statistical astrology aimed to predict natural calamities decades ahead & at a fraction of the current cost.
Thank you for any help in advance.
Yes you can most certainly do this in Python and I would suggest looking into Pillow/PIL. Basically you need to look at this as more of image manipulation rather than a chart/graph. You start with a basic background image and add all details on top of it.
Just out of curiosity, how are you generating the planetary & astrological data?
I'm struggling with the implementation of an algorithm in python (2.7) to parallelize the computation of a physics problem. There's a parameter space over two variables (let's say a and b) over which I would like to run my written program f(a,b) which returns two other variables c and d.
Up to now, I worked with two for-loops over a and b to calculate two arrays for c and d which are then saved as txt documents. Since the parameter space is relatively large and each calculation of a point f(a,b) in it takes relatively long, it would be great to use all of my 8 CPU cores for the parameter space scan.
I've read about multithreading and multiprocessing and it seems that multiprocessing is what I'm searching for. Do you know of a good code example for this application or resources to learn about the basics of multiprocessing for my rather simple application?
Here is an example of how you might use multiprocessing with a simple function that takes two arguments and returns a tuple of two numbers, and a parameter space over which you want to do the calculation:
from itertools import product
from multiprocessing import Pool
import numpy as np
def f(a, b):
c = a + b
d = a * b
return (c, d)
a_vals = [1, 2, 3, 4, 5, 6]
b_vals = [10, 11, 12, 13, 14, 15, 16, 17]
na = len(a_vals)
nb = len(b_vals)
p = Pool(8) # <== maximum number of simultaneous worker processes
answers = np.array(p.starmap(f, product(a_vals, b_vals))).reshape(na, nb, 2)
c_vals = answers[:,:,0]
d_vals = answers[:,:,1]
This gives the following:
>>> c_vals
array([[11, 12, 13, 14, 15, 16, 17, 18],
[12, 13, 14, 15, 16, 17, 18, 19],
[13, 14, 15, 16, 17, 18, 19, 20],
[14, 15, 16, 17, 18, 19, 20, 21],
[15, 16, 17, 18, 19, 20, 21, 22],
[16, 17, 18, 19, 20, 21, 22, 23]])
>>> d_vals
array([[ 10, 11, 12, 13, 14, 15, 16, 17],
[ 20, 22, 24, 26, 28, 30, 32, 34],
[ 30, 33, 36, 39, 42, 45, 48, 51],
[ 40, 44, 48, 52, 56, 60, 64, 68],
[ 50, 55, 60, 65, 70, 75, 80, 85],
[ 60, 66, 72, 78, 84, 90, 96, 102]])
The p.starmap returns a list of 2-tuples, from which the c and d values are then extracted.
This assumes that you will do your file I/O in the main program after getting back all the results.
Addendum:
If p.starmap is unavailable (Python 2), then instead you can change your function to take a single input (a 2-element tuple):
def f(inputs):
a, b = inputs
# ... etc as before ...
and then use p.map in place of p.starmap in the above code.
If it is not convenient to change the function (e.g. it is also called from elsewhere), then you can of course write a wrapper function:
def f_wrap(inputs):
a, b = inputs
return f(a, b)
and call that instead.
range(5, 15) [1, 1, 5, 6, 10, 10, 10, 11, 17, 28]
range(6, 24) [4, 10, 10, 10, 15, 16, 18, 20, 24, 30]
range(7, 41) [9, 18, 19, 23, 23, 26, 28, 40, 42, 44]
range(11, 49) [9, 23, 24, 27, 29, 31, 43, 44, 45, 45]
range(38, 50) [1, 40, 41, 42, 44, 48, 49, 49, 49, 50]
I get the above outpout from a print command from a function. What I really want is a combined list of the range, for example in the top line 5,6,7...15,1,1,5,6 etc.
The output range comes from
range_draws=range(int(lower),int(upper))
which I naively thought would give a range. The other numbers come from a sliced list.
Could someone help me to get the desired result.
The range() function returns a special range object to save on memory (no need to keep all the numbers in memory when only the start, end and step size will do). Cast it to a list to 'expand' it:
list(yourrange) + otherlist
To quote the documentation:
The advantage of the range type over a regular list or tuple is that a range object will always take the same (small) amount of memory, no matter the size of the range it represents (as it only stores the start, stop and step values, calculating individual items and subranges as needed).
I've a huge number of entries, every one is a float number. These data x are accesible with an iterator. I need to classify all the entries using selection like 10<y<=20, 20<y<=50, .... where y are data from an other iterables. The number of entries is much more than the number of selections. At the end I want a dictionary like:
{ 0: [all events with 10<x<=20],
1: [all events with 20<x<=50], ... }
or something similar. For example I'm doing:
for x, y in itertools.izip(variable_values, binning_values):
thebin = binner_function(y)
self.data[tuple(thebin)].append(x)
in general y is multidimensional.
This is very slow, is there a faster solution, for example with numpy? I think the problem cames from the list.append method I'm using and not from the binner_function
A fast way to get the assignments in numpy is using np.digitize:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.digitize.html
You'd still have to split the resulting assignments up into groups. If x or y is multidimensional, you will have to flatten the arrays first. You could then get the unique bin assignments, and then iterate over those in conjunction with np.where to split the the assigments up into groups. This will probably be faster if the number of bins is much smaller than the number of elements that need to be binned.
As a somewhat trivial example that you will need to tweak/elaborate on for your particular problem (but is hopefully enough to get you started with with a numpy solution):
In [1]: import numpy as np
In [2]: x = np.random.normal(size=(50,))
In [3]: b = np.linspace(-20,20,50)
In [4]: assign = np.digitize(x,b)
In [5]: assign
Out[5]:
array([23, 25, 25, 25, 24, 26, 24, 26, 23, 24, 25, 23, 26, 25, 27, 25, 25,
25, 25, 26, 26, 25, 25, 26, 24, 23, 25, 26, 26, 24, 24, 26, 27, 24,
25, 24, 23, 23, 26, 25, 24, 25, 25, 27, 26, 25, 27, 26, 26, 24])
In [6]: uid = np.unique(assign)
In [7]: adict = {}
In [8]: for ii in uid:
...: adict[ii] = np.where(assign == ii)[0]
...:
In [9]: adict
Out[9]:
{23: array([ 0, 8, 11, 25, 36, 37]),
24: array([ 4, 6, 9, 24, 29, 30, 33, 35, 40, 49]),
25: array([ 1, 2, 3, 10, 13, 15, 16, 17, 18, 21, 22, 26, 34, 39, 41, 42, 45]),
26: array([ 5, 7, 12, 19, 20, 23, 27, 28, 31, 38, 44, 47, 48]),
27: array([14, 32, 43, 46])}
For dealing with flattening and then unflattening numpy arrays, see:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.unravel_index.html
http://docs.scipy.org/doc/numpy/reference/generated/numpy.ravel_multi_index.html
np.searchsorted is your friend. As I read somewhere here in another answer to the same topic, it's currently a good bit faster than digitize, and does the same job.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.searchsorted.html