Develop Reference

what's the difference between lambda x : np.sqrt & lambda x : np.sqrt(x)?

Hi I encountered a problem:
df.apply(lambda x : np.sqrt)
I wonder why it hasn't any x for sqrt?

This does not give the result you would expect. Instead of computing the square of each row value, it returns the function np.sqrt which is a constant python object.
To apply the square root to all elments of df, you should use:
df.apply(lambda x : np.sqrt(x))
or
df.apply(np.sqrt)
Actually, to obtain the same result, you can directly call np.sqrt on the dataframe since it is a vectorized function.
np.sqrt(df)
This would be the fastest way to compute the square root of all elements. You can read more here on numpy's vectorization functionality that maps a function over a sequence very efficiently.

Using a matrix multiplication operator in a sympy expression

I am trying to implement a function using a Sympy expression with multiple parameters. For example, I used the following code:
import sympy
a = sympy.symbols("a")
ad = sympy.symbols("ad")
x = sympy.symbols("x")
c = sympy.symbols("c")
f = (ad*a)*c + x
func = sympy.lambdify((a,ad,x,c),f)
And what I would like to evaluate is the following:
func(M_A,M_B,0,1)
When I use two matrices M_A and M_B, the function performs just an element-wise multiplication, but I need it to be a matrix multiplication for the objects a and ad. I do know that it is possible to do so when I define the variables using MatrixSymbol instead of symbols, but this is not possible in my case as I have implemented a scenario which uses diagonal matrices where element-wise or matrix multiplication would not make a difference. Further, it is also possible to do something like this with normal symbols
x_vars = [symbols("x"+i) for i in range(1,4)]
trans_mat = np.random.random([3,3])
y_vars = trans_mat.dot(x_vars)
which just does not seem to work when I am using MatrixSymbol.
So, I was thinking if I could just compute the expression and perform all the manipulations using the regular symbols and at the end replace all the multiplication operators with numpy.matmul. Please let me know if this is possible somehow, or any other suggestion which can help is also welcome.
Thanks!

Doing help(func) we can see the code produced by lambdify:
Help on function _lambdifygenerated:
_lambdifygenerated(a, ad, x, c)
Created with lambdify. Signature:
func(a, ad, x, c)
Expression:
a*ad*c + x
Source code:
def _lambdifygenerated(a, ad, x, c):
return a*ad*c + x
That's a straightforward translation to python/numpy. Sounds like you want (a#ad)*c+x.

Numerically stable way to compute conditional covariance matrix using linalg.solve

I know that the recommendation is not to use linalg.inv and use linalg.solve when inverting matrices. This makes sense when I have situation like Ax = b and I want to get x, but is there a way to compute something like: A - B * D^{-1} * C without using linalg.inv? Or what is the most numerically stable way to deal with the inverse in the expression?
Thanks!

Please don't inv—it's not as bad as most people think, but there's easier ways: you mentioned how np.linalg.solve(A, b) equals A^{-1} . b, but there's no requirement on what b is. You can use solve to solve your question, A - np.dot(B, np.linalg.solve(D, C)).
(Note, if you're doing blockwise matrix inversion, C is likely B.transpose(), right?)

Is there a way to define a function through symbolic derivation?

Is there a way to define a function through symbolic derivation? For example,
def f(x):return x**x
def df(x) : return diff(f(x),x)
This code doesn't work, since df(1) would not be possible (diff(f(1),1) doesn't make sense.) But is there a way to take advantage of the result of "diff"? When printing df(x), there is a pretty function form with variable x, so neat.

Don't need Maple or Mathematica. Python has sympy.
import numpy as np
from sympy import symbols, diff, lambdify
x = symbols('x')
def f(x):
return x**x
def df(x):
return diff(f(x))
print(df(x))
This returns the symbolic derivative, x**x*(log(x) + 1). Now, it depends on what you want to do with that. It seems like you wanted to calculate numeric values without necessarily knowing that the derivative is x**x*(log(x) + 1). sympy's recommended way for calculating many numeric values is to use its lambdify.
df_numeric = lambdify(x, df(x), modules=['numpy'])
df_numeric(5)
This gives 8154.4934763565643. You can check it against a regular, manual function:
def df_manual(n):
return n**n*(np.log(n) + 1)
df_manual(5)

Python - Difficulty Calculating Partition Function using Large NumPy arrays

I had a pretty compact way of computing the partition function of an Ising-like model using itertools, lambda functions, and large NumPy arrays. Given a network consisting of N nodes and Q "states"/node, I have two arrays, h-fields and J-couplings, of sizes (N,Q) and (N,N,Q,Q) respectively. J is upper-triangular, however. Using these arrays, I have been computing the partition function Z using the following method:
# Set up lambda functions and iteration tuples of the form (A_1, A_2, ..., A_n)
iters = itertools.product(range(Q),repeat=N)
hf = lambda s: h[range(N),s]
jf = lambda s: np.array([J[fi,fj,s[fi],s[fj]] \
for fi,fj in itertools.combinations(range(N),2)]).flatten()
# Initialize and populate partition function array
pf = np.zeros(tuple([Q for i in range(N)]))
for it in iters:
hterms = np.exp(hf(it)).prod()
jterms = np.exp(-jf(it)).prod()
pf[it] = jterms * hterms
# Calculates partition function
Z = pf.sum()
This method works quickly for small N and Q, say (N,Q) = (5,2). However, for larger systems (N,Q) = (18,3), this method cannot even create the pf array due to memory issues because it has Q^N nontrivial elements. Any ideas on how to either overcome this memory issue or how to alter the code to work on subarrays?
Edit: Made a small mistake in the definition of jf. It has been corrected.

You can avoid the large array just by initializing Z to 0, and incrementing it by jterms * iterms in each iteration. This still won't get you out of calculating and summing Q^N numbers, however. To do that, you probably need to figure out a way to simplify the partition function algebraically.

Not sure what you are trying to compute but I tested your code with ChrisB suggestion and jf will not work for Q=3.

Perhaps you shouldn't use a dense numpy array to encode your function? You could try sparse arrays or just straight Python with Numba compilation. This blogpost shows using Numba on the simple Ising model with good performance.