Develop Reference

Symbolic matrix multiplication on GPU using sympy?

I have a matrix M with approximately 300 rows and columns. Each entry contains a symbolic expression, and there are around 40 symbols in total. Doing something like M*M can take a long time (hours). Is there a way to do this symbolic matrix multiplication on a gpu using sympy, or more generally in python?

You're in luck! I just levelled up on this a few minutes ago. The optimal solution for parallelizing SymPy expressions on [G|T]PU hardware is (as of mid-2021) Aesara, a well-maintained fork of the now-defunct Theano package foolishly discontinued in the mistaken belief that Tensorflow was better. Spoiler alert: it wasn't.
Since the OP sadly failed to supply a minimal length example (MLE) for us to mutilate, we'll just copypasta a mostly useless example demonstrating the core API concepts:
# SymPy -> Aesara translater, bundled with SymPy.
from sympy.printing.aesaracode import aesara_function
# Arbitrary SymPy expression.
from sympy import sin
from sympy.abc import x
expr = sin(x)/x
# GPU-parallelized Aesara function compiled from that expression! Yup.
f = aesara_function([x], [expr])
Generally speaking, the optimal solution is whatever SymPy's "Numeric Computation" documentation says it is. Currently, that means Aesara; since data science in Python changes faster than a King Cobra biting the village hand that feeds it, everything has probably already changed by the time you read this. Since the aforementioned documentation is surprisingly intelligible, go there first instead. That's your optimum landing page for all things optimizing SymPy.

Solving reaction-diffusion system with Theano

I am new to Theano, and I try to implement a numerical integrator of a reaction-diffusion system - FitzHugh–Nagumo model of this version:
For now my expressions are:
import theano as th
import theano.tensor as T
u = T.dmatrix('u')
v = T.dmatrix('v')
e = T.dscalar('e')
a0 = T.dscalar('a0')
a1 = T.dscalar('a1')
dudt = u - u**3 -v
dvdt = e*(u - a1*v - a0)
So I haven't implemented the finite-differences laplacian operator yet. My question is whether there is a smart way of doing it in Theano?

Is there any reason for using Theano? There are other ways in Python to solve a system of coupled non-linear ODEs.
The reaction-diffusion system definition from Google seems to suggest that u(x,y,t), v(x,y,t).
I am not a user of Theano, but it looks like casting the problem in the form of an equation like b = Ax is the way to go.
Some resources that I came across on Google for using Theano and generally solving PDEs are below.
Expressing the Laplacian using Theano
Solving a reaction-diffusion problem using numpy
Github project using Theano to solve the shallow water PDE

An interesting example of a similar, but simpler problem, solved using convolutional networks on Google's tensorflow can be found here:
https://www.tensorflow.org/versions/r0.7/tutorials/pdes/index.html
In particular they use the following definition of the diffusion kernel:
laplace_k = make_kernel([[0.5, 1.0, 0.5],
[1.0, -6., 1.0],
[0.5, 1.0, 0.5]])

I see two coupled, first order, non-linear, ordinary differential equations here.
Update: Now your equations are clear - Laplacians are there; two coupled nonlinear PDEs. Much better.
You need a finite difference or finite element approach for your spatial discretization. Your choice, of course, but I'd prefer a finite element approach over finite differences.
You also need some kind of numerical integration in time. An implicit error correcting scheme would be best.
I looked quickly at the Theano docs. I didn't see anything to help you with your spatial discretization problem. Once you accomplish that you'll have matrix equations that you can solve, but I don't believe Theano will help you formulate the problem.
I'll admit that I'm not a Theano maven.

binary linear programming solver in Python

I have a Python script in which I need to solve a linear programming problem. The catch is that the solution must be binary. In other words, I need an equivalent of MATLAB's bintprog function. NumPy and SciPy do not seem to have such a procedure. Does anyone have suggestions on how I could do one of these three things:
Find a Python library which includes such a function.
Constrain the problem such that it can be solved by a more general linear programming solver.
Interface Python with MATLAB so as to make direct use of bintprog.

Just to be rigorous, if the problem is a binary programming problem, then it is not a linear program.
You can try CVXOPT. It has a integer programming function (see this). To make your problem a binary program, you need to add the constrain 0 <= x <= 1.
Edit: You can actually declare your variable as binary, so you don't need to add the constrain 0 <= x <= 1.
cvxopt.glpk.ilp = ilp(...)
Solves a mixed integer linear program using GLPK.
(status, x) = ilp(c, G, h, A, b, I, B)
PURPOSE
Solves the mixed integer linear programming problem
minimize c'*x
subject to G*x <= h
A*x = b
x[I] are all integer
x[B] are all binary

This is a half-answer, but you can use Python to interface with GLPK (through python-glpk). GLPK supports integer linear programs. (binary programs are just a subset of integer programs).
http://en.wikipedia.org/wiki/GNU_Linear_Programming_Kit
Or you could simply write your problem in Python and generate an MPS file (which most standard LP/MILP (CPLEX, Gurobi, GLPK) solvers will accept). This may be a good route to take, because as far as I am aware, there aren't any high quality MILP solvers that are native to Python (and there may never be). This will also allow you to try out different solvers.
http://code.google.com/p/pulp-or/
As for interfacing Python with MATLAB, I would just roll my own solution. You could generate a .m file and then run it from the command line
% matlab -nojava myopt.m
Notes:
If you're an academic user, you can get a free license to Gurobi, a high performance LP/MILP solver. It has a Python interface.
http://www.gurobi.com/
OpenOpt is a Python optimization suite that interfaces with different solvers.
http://en.wikipedia.org/wiki/OpenOpt

I develop a package called gekko (pip install gekko) that solves large-scale problems with linear, quadratic, nonlinear, and mixed integer programming (LP, QP, NLP, MILP, MINLP) and is released under the MIT License. A binary variable is declared as an integer variable type with lower bound 0 and upper bound 1 as b=m.Var(integer=True,lb=0,ub=1). Here is a more complete problem with the use of m.Array() to define multiple binary variables:
from gekko import GEKKO
m = GEKKO()
x,y = m.Array(m.Var,2,integer=True,lb=0,ub=1)
m.Maximize(y)
m.Equations([-x+y<=1,
3*x+2*y<=12,
2*x+3*y<=12])
m.options.SOLVER = 1
m.solve()
print('Objective: ', -m.options.OBJFCNVAL)
print('x: ', x.value[0])
print('y: ', y.value[0])

Libraries for manipulating multivariate polynomials

I need to write some code that deals with generating and manipulating multivariable polynomials. I'll outline my task with a simplified example.
Lets say I am given three expressions: 2x^2, 3y + 1, and 1z. I then need to multiply these together which would give me 6x^2yz + 2x^2z. Then I would like to find the partial derivatives of this expression with respect to x, y, and z. This would give me 12xyz + 4xz, 6x^2z, and 6x^2y + 2x^2.
My problem deals with doing simple manipulations like this on expressions containing thousands of variables and I need an easy way to do this systematically. I would really like to use python since I already have a lot of project related functionality completed using numpy/scipy/matplotlib, but if there is a robust toolbox out there in another language I am open to using that as well. I am doing university research so I am open to using Matlab as well.
I haven't been able to find any good python libraries that could do this for me easily and ideally would like something similar to the scipy polynomial routines that could work on multidimensional polynomials. Does anyone know of a good library that seems suitable for this problem and that would be easy to integrate into already existing python code?
Thanks!
Follow up: I spent a couple of days working with sympy which turned out to be very easy to use. However, it was much to slow for the size of the problem I am working on so I will now go explore matlab. To give an extremely rough estimate of the speed using a small sample size, it took approximately 5 seconds to calculate each of the partial derivatives of an order 2 polynomial containing 250 variables.
Follow up #2: I probably should have done this back when I was still working on this problem, but I might as well let everyone know that the matlab symbolic library was extremely comparable in speed to sympy. In other words, it was brutally slow for large computations. Both libraries were amazingly easy to work with, so for small computations I do highly recommend either.
To solve my problem I computed the gradients by hand, simplified them, and then used the patterns I found to hard code some values in my code. It was more work, but made my code exponentially faster and finally usable.

Sympy is perfect for this: http://code.google.com/p/sympy/
Documentation: http://docs.sympy.org/
Examples of differentiation from the tutorial: http://docs.sympy.org/tutorial.html#differentiation
import sympy
x, y, z = sympy.symbols('xyz')
p1 = 2*x*x
p2 = 3*y + 1
p3 = z
p4 = p1*p2*p3
print p4
print p4.diff(x)
print p4.diff(y)
print p4.diff(z)
Output:
2*z*x**2*(1 + 3*y)
4*x*z*(1 + 3*y)
6*z*x**2
2*x**2*(1 + 3*y)

If you are using MATLAB, then the symbolic TB works well, IF you have it. If not, then use my sympoly toolbox. Just download it from the file exchange.
sympoly x y z
A = 2*x^2; B = 3*y + 1;C = 1*z;
gradient(A*B*C)
ans =
Sympoly array has size = [1 3]
Sympoly array element [1 1]
4*x*z + 12*x*y*z
Sympoly array element [1 2]
6*x^2*z
Sympoly array element [1 3]
2*x^2 + 6*x^2*y
Note that this points out that you made a mistake in differentiating the result with respect to z in your question.

Matlab and other tools you mention typically do numerical computing. You should consider using Mathematica or an alternative Computer Algebra System (CAS) for symbolic computation. See the wiki link: http://en.wikipedia.org/wiki/Comparison_of_computer_algebra_systems for a comparison of various CASs.

computer algebra soft to minimize the number of operations in a set of polynomials

I have systems of polynomials, fairly simple polynomial expressions but rather long
to optimize my hand. Expressions are grouped in sets, and in a given set there are common terms in several variables.
I would like to know if there is a computer algebra system, such as Mathematica, Matlab, or sympy, which can optimize multiple polynomials with common terms to minimize number of operations. It would be also great if such system can minimize the number of intermediate terms to reduce number of registers.
If such system is not existing, I am going to do my own, using Python symbolic algebra Sympy. If you are working on such package or are interested in developing or using one, please let me know.
here is a made-up example
x0 = ((t - q*A)*x + B)*y
y0 = ((t - q*A)*y + B)*z
z0 = ((t - q*A)*z + B)*x
so you can obviously factor the (t - qA) term. Now if you make number of terms very large with various combinations of common terms it becomes difficult to do by hand. The equations I have involve up to 40 terms and the size of set is around 20. Hope that helps
Thank you

Is sympy what you're looking for? I do believe it has support for polynomials although I don't know if it supports all the features you may desire (still, tweaking it to add what you think it might be missing has to be easier than writing your own from scratch;-).

Have you considered Maxima?
It is an impressive symbolical computation package that is free, open source, and has a strong and active community that provides valuable assistance when dealing with non-obvious formulations. It is readily availability for all three major operating systems, and has a precompiled Windows binary.
You have a variety of algebraic manipulation commands available for expressions and for systems of equations (such as yours): expand, factor, simplify, ratsimp, linsolve, etc.
This page (Maxima for Symbolic Computation)should get you started — downloading, installing, a few examples, and then pointing out additional resources to guide you on your way, including a quick command reference / cheat sheet, and some guidlines for writing your own scripts.

Well Mathematica can certainly do all sorts of transformations on sets of polynomial equations such as yours, and some of those transformations could be to reduce the number of terms. Whether that is the right answer for you is open to question, as you don't seem to have a copy available. I expect that the same is true for Maple and for most of the other CAS out there.
But your mention of
reduce number of registers
suggests that you are actually trying to do some data-flow analysis for compilation. You might want to look at the literature on that topic too. Some of that literature does indeed refer to computer-algebra-like transformations on expressions.

I'm late to the party, but anyway there is a function optimize in Maxima (https://maxima.sourceforge.io) which identifies common subexpressions and emits a blob of code which can be evaluated. For the example shown in the problem statement, I get:
(%i11) optimize ([x0 = ((t-A*q)*x+B)*y,
y0 = ((t-A*q)*y+B)*z,
z0 = x*((t-A*q)*z+B)]);
(%o11) block([%1],
%1 : t - A q,
[x0 = (%1 x + B) y,
y0 = (%1 y + B) z,
z0 = x (%1 z + B)])
As you can see, t - A*q was pulled out and assigned to a made-up variable %1 (percent sign being an allowed character for symbols in Maxima) which is then reused to compute other results.
? optimize at the input prompt shows some documentation about it.