Develop Reference

How to speed up "for loop" in Python3

Below code is running so slow. I tried using numpy.argwhere instead of "if statement" to speed up the code and I got a pretty efficient result but it's still very slow. I also tried numpy.frompyfunc and numpy.vectorize but I failed. What would you suggest to speed up the code below?
import numpy as np
import time
time1 = time.time()
n = 1000000
k = 10000
velos = np.linspace(-1000, 1000, n)
line_centers = np.linspace(-1000, 1000, k)
weights = np.random.random_sample(k)
rvs = np.arange(-60, 60, 2)
m = len(rvs)
w = np.arange(10)
M = np.zeros((n, m))
for l, lc in enumerate(line_centers):
vi = velos - lc
for j in range(m - 1):
w = np.argwhere((vi < rvs[j + 1]) & (vi > rvs[j])).T[0]
M[w, j] = weights[l] * (rvs[j + 1] - vi[w]) / (rvs[j + 1] - rvs[j])
M[w, j + 1] = weights[l] * (vi[w] - rvs[j]) / (rvs[j + 1] - rvs[j])
time2 = time.time()
print(time2 - time1)
EDIT:
The size of the array M was incorrect. I fixed it.

This seems like a situation where a c++ interface could come in handy. With Pybind11 you can create c++ functions which take numpy arrays as argument, manipulate them and return them back to python. That would speed up you loops. Take a look at it!

Of course it is slow, you have two nested loops! You need to rethink your algorithm using vector operations, as in, no iteration over indices, but implement in terms of index or boolean arrays, and index shifts.
You have not given any background information, so it is incredibly hard for anyone to suggest something meaningful (given the soup of indices in the example). A few quick suggestions based on quickly gleaning over your example.
An expression like this (rvs[j + 1] - rvs[j]) is easily replaced with numpy.ediff1d.
You seem to be iterating through n in blocks of m, maybe numpy.nditer will be of use.
I have a hunch that your inner loop has an error, are you sure you really mean to iterate over range(m - 1)? That would mean you are iterating from 0 to m-2 (inclusive), I doubt you meant that.
We can help with more concrete answers if you provide more background information.

How to find negative imaginary parts of values in an array then turning them to positive?

I have a function a=x*V where x assumes thousands of values as x = arange(1,1000,0.1) and V is a combination of other constants. These make a always complex (has nonzero real and imaginary parts). However, because a depends on other values, the imag(a) can be negative for some x's.
For what I am doing, however, I need imag(a) to be always positive, so I need to take the negative values and turn them into positive.
I have tried doing
if imag(a)<0:
imag(a) = -1*imag(a)
That didn't seem to work because it gives me the error: SyntaxError: Can't assign to function call. I thought it was because it's an array so I tried any() and all(), but that didn't work either.
I'm out of options now.

IIUC:
In [35]: a = np.array([1+1j, 2-2j, 3+3j, 4-4j])
In [36]: a.imag *= np.where(a.imag < 0, -1, 1)
In [37]: a
Out[37]: array([ 1.+1.j, 2.+2.j, 3.+3.j, 4.+4.j])

You can't redefine a function that way. It would be like saying
sqrt(x) = 2*sqrt(x)
What you can do is reassign the value of a (not imag(a)).
if imag(a) < 0
a = a - 2*imag(a)*j
For example, if a = 3 - 5j, then it would give you
3 - 5j - 2(-5)j = 3 + 5j

It appears to be faster than doing subtraction. For a full function:
import numpy as np
def imag_abs(x):
mask = x.imag < 0
x[mask] = np.conj(x[mask])
return x

Manual fft not giving me same results as fft

import numpy as np
import matplotlib.pyplot as pp
curve = np.genfromtxt('C:\Users\latel\Desktop\kool\Neuro\prax2\data\curve.csv',dtype = 'float', delimiter = ',')
curve_abs2 = np.empty_like(curve)
z = 1j
N = len(curve)
for i in range(0,N-1):
curve_abs2[i] =0
for k in range(0,N-1):
curve_abs2[i] += (curve[i]*np.exp((-1)*z*(np.pi)*i*((k-1)/N)))
for i in range(0,N):
curve_abs2[i] = abs(curve_abs2[i])/(2*len(curve_abs2))
#curve_abs = (np.abs(np.fft.fft(curve)))
#pp.plot(curve_abs)
pp.plot(curve_abs2)
pp.show()
The code behind # gives me 3 values. But this is just ... different
Wrong ^^ this code: http://www.upload.ee/image/3922681/Ex5problem.png
Correct using numpy.fft.fft(): http://www.upload.ee/image/3922682/Ex5numpyformulas.png

There are several problems:
You are assigning complex values to the elements of curve_abs2, so it should be declared to be complex, e.g. curve_abs2 = np.empty_like(curve, dtype=np.complex128). (And I would recommend using the name, say, curve_fft instead of curve_abs2.)
In python, range(low, high) gives the sequence [low, low + 1, ..., high - 2, high - 1], so instead of range(0, N - 1), you must use range(0, N) (which can be simplified to range(N), if you want).
You are missing a factor of 2 in your formula. You could fix this by using z = 2j.
In the expression that is being summed in the inner loop, you are indexing curve as curve[i], but this should be curve[k].
Also in that expression, you don't need to subtract 1 from k, because the k loop ranges from 0 to N - 1.
Because k and N are integers and you are using Python 2.7, the division in the expression (k-1)/N will be integer division, and you'll get 0 for all k. To fix this and the previous problem, you can change that term to k / float(N).
If you fix those issues, when the first double loop finishes, the array curve_abs2 (now a complex array) should match the result of np.fft.fft(curve). It won't be exactly the same, but the differences should be very small.
You could eliminate that double loop altogether using numpy vectorized calculations, but that is a topic for another question.

Multiplying two polynomials mod n,x^r-1 using long integers: what is the correct window size?

Using a window multiplication algorithm to multiply two polynomials[coefficients in Z/nZ and the whole polynomial mod x^r-1 for some r] using long-integer multiplications, what size should I give to the window?
Where by "window" I mean the bit-length that the coefficients should use in the initial long-integers such that the result of the long-integer multiplication contains the correct coefficients of the result[the sums of the coefficients "don't overlap"].
At the beginning I thought that ceil(log(n**2,2)) + 1 would be enough, because each coefficient is at most n-1 so a product of these coefficients is at most (n-1)**2.
Then I realized that when doing a long-integer multiplication there can be some sums of these coefficients, thus the window should be ceil(log(number-of-additions * n**2,2)) + 1.
I thought that there could be at most the sum of the degrees of the two polynomials additions, but using ceil(log((deg_a + deg_b + 1) * n**2,2)) +1 works for some time, but eventually the coefficients overlap and I get incorrect results.
So how big should this "window" be?
By the way, here's the current (python) code:
def __mul__(self, other):
new = ModPolynomial(self._mod_r, self._mod_n)
#new = x mod self._mod_n,x^(self._mod_r -1)
try:
new_deg = self._degree + other.degree
new_coefs = []
# i've also tried with (new_deg + 1) ** 2 * ...
window = int(math.ceil(math.log((new_deg + 1) * self._mod_n**2,2))) + 1
A = 0
for i,k in enumerate(self._coefs):
A += k << (window * i)
B = 0
for i,k in enumerate(other.coefficients):
B += k << (window * i)
res = A * B
mask = 2**window - 1
while res:
new_coefs.append((res & mask) % self._mod_n)
res >>= window
new._coefs = new_coefs
new._degree = self._finddegree(new_coefs)
except AttributeError:
new._coefs = [(c * other) % self._mod_n for c in self._coefs]
new._degree = self._finddegree(new._coefs)
new._mod()
return new
edit 1:
I'm starting to think that the window size may not be the problem.
I've tried to increase it up to ceil(log2((new_deg+1)**3 * self._mod_n ** 5))+1 and this yields the same results as using ceil(log2((new_deg+1) * self._mod_n**2))+1, and since the difference between these two size is really big[about 55-60 bits difference in my tests, which is a lot if you think...],this means that probably the smallest of these size if already okay, but there is some other problem somewhere.
edit 2:
An example of wrong result is:
#ModPolynomial(r,n) = x mod n, x^r-1
>>> pol = polys.ModPolynomial(20,100) # uses integer-multiplication
>>> pol += 2
>>> pol2 = polynomials.ModPolynomial(20,100) #uses the naive algorithm
>>> pol2 += 2
>>> (pol2**52).coefficients #should be the correct result[first is coef of x^0]
(76, 76, 44, 0, 0, 16, 16, 4, 0, 0, 24, 24, 81, 0, 0, 80, 80, 20)
>>> (pol**52).coefficients #the wrong result
(12L, 52L, 8L, 20L, 60L, 12L, 92L, 28L, 60L, 80L, 68L, 48L, 22L, 0L, 0L, 20L, 20L, 80L)
I'll try to find some smaller example, so that I can verify it by hand.
edit 3:
Okay, I found out that there is some problem with the degree. I found an example in which the degree becomes negative, which obviously shouldn't be. So i'll try to dig more checking when and why the degree changes in this unexpected way.
edit 4:
I've found the bug. When creating the integer I was iterating over all the _coefs sequence, but my implementation does not guarantee that all coefficients that correspond to a degree > of the polynomial degree are 0. This fixes the issue.
edit 5:
Just some performance results I've obtained testing this implementation.
1) Using long-integer multiplication is faster than using numpy.convolve iff the coefficients are bigger than ints. Otherwise numpy.convolve is faster.
2) About 80% of the time is spent converting lists of coefficients to integers and integers to lists of coefficients. There is not much you can do about this, since these operations are O(n).
Now i'm wondering if there is a way to implement the "mod x^r-1" operation in an efficient way using only long-integers... this could probably give a big speed-up, since at that point you don't have to do the conversions anymore.

The correct calculation is
window = int(math.ceil(math.log((max(self._degree, other.degree) + 1) *
(self._mod_n - 1)**2, 2)))
However this will definitely be less than the window you calculated, so there must be some other reason you're getting incorrect results. Are you sure the degree is being calculated correctly? Can you give an example of a polynomial that is calculated incorrectly?
Unless there's a particularly good reason to use long integer multiplication, I'd recommend using NumPy:
new.coeffs = np.convolve(self.coeffs, other.coeffs) % self.mod
This will usually be at least as efficient as long integer multiplication (which is a form of convolution anyway), and you've got a lot less Python code to worry about. In fact NumPy has a polynomial library; although it's designed for floating-point coefficients, you can look at it to get an idea how to implement your code efficiently.

Numpy conditional arithmetic operations on two arrays

Still trying to earn my numpy stripes: I want to perform an arithmetic operation on two numpy arrays, which is simple enough:
return 0.5 * np.sum(((array1 - array2) ** 2) / (array1 + array2))
Problem is, I need to be able to specify the condition that, if both arrays are element-wise 0 at the same element i, don't perform the operation at all--would be great just to return 0 on this one--so as not to divide by 0.
However, I have no idea how to specify this condition without resorting to the dreaded nested for-loop. Thank you in advance for your assistance.
Edit: Would also be ideal not to have to resort to a pseudocount of +1.

Just replace np.sum() by np.nansum():
return 0.5 * np.nansum(((array1 - array2) ** 2) / (array1 + array2))
np.nansum() treats nans as zero.

return numpy.select([array1 == array2, array1 != array2], [0.5 * np.sum(((array1 - array2) ** 2) / (array1 + array2)), 0])
should do the trick... numpy.where might also be used.

You could also try post-applying numpy.nan_to_num:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.nan_to_num.html
although I found that when I have a divide by zero in your code, it gives a warning, but fills that element with zero (when doing integer math) and NaN when using floats.
If you want to skip the sum when you have a divide by zero, you could also just do the calculation and then test for the NaN before returning:
xx = np.sum(((array1 - array2) ** 2) / (array1 + array2))
if np.isnan(xx):
return 0
else:
return xx
Edit: To silence warnings you could try messing around with numpy.seterr:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.seterr.html