I'm about to reinstall numpy and scipy on my Ubuntu Lucid. As these things carry quite a few dependencies, I'm wondering if there is a comprehensive test suite to check if the new install really works.
Of course, I can just take a bunch of my scripts and run them one by one to see if they keep working, but that won't guard against a situation where at some point in the future I'll try to use something I didn't use before and it'll break (or, worse, silently produce nonsence).
Yes. Both packages have a test method for this.
import numpy
numpy.test('full')
import scipy
scipy.test('full')
You will need to have pytest and hypothesis installed to run numpy.test.
Note that binary packages for the mathematical libraries Scipy and
Numpy depend on, shipped by Linux distributions, have in some cases
showed to be subtly broken. Running Numpy and Scipy test suites with
numpy.test() and scipy.test() is recommended, as a first step to
confirm that your installation functions properly. If it doesn't, you
may want to try another set of binaries if available, or buy some
above-mentioned commercial packages.
from http://www.scipy.org/Download
Related
ChatGPT's point of view is : "It's generally recommended to upgrade to the latest version of Numpy step by step, rather than directly from version 1.19 to 1.24. This is because newer versions of Numpy may have breaking changes, so by upgrading incrementally, you can minimize the risk of encountering compatibility issues and ensure a smoother transition."
From my personal point of view, the gap between the versions of NumPy is not so big. I would create a new virtual environment and install the desired version of NumPy. Then by running the code, either you will get Runtime errors for unsupported and no longer existing functions or everything will run just smoothly.
In case you have errors, you can try searching them online to find the required fix.
In case you have no errors at all, I would still try to come up with a Test script that testes some of the basic functionalities that are used and could break through out the code, you could try copying some values and hardcode set them to see the behaviour.
The above, would apply to any kind of package. If you still feel you need to go step by step, feel free.
I have written a python package that does stuff (the specifics are irrelevant). When using the package, the user can specify whether or not to make use of available GPUs. In my code, I use the CuPy package to utilize the available GPUs.
I would now like to distribute the package - ideally package it with setuptools, put it up on PyPI and have users install it using pip.
The issue:
I want it to be possible to install it on systems regardless of whether they have CUDA available or not. The problem is, if the system does not have CUDA, the installation (and consequently the import) of CuPy will fail. Therefore, I gather that I need to have two separate versions of the package, one that supports CUDA (and imports CuPy etc) and one that does not. Moreover, I need some mechanism that will choose the appropriate version during installation.
What I was hoping to find is some sort of tool that will check (during the installation process) whether CUDA is available (or, more practically - whether CuPy can be installed), and accordingly choose the proper version of my package to download and install.
Any ideas whether something of the sorts exists, and if not, then on the right way to implement it?
NOTE: I assume I could list CuPy in the dependencies in setuptools regardless of whether the system has CUDA or not, and assuming it does not, have my package catch any exceptions that arise due to trying to install/import/use it. But I feel this is rather barbaric, and was hoping to find something more elegant.
NOTE 2: Of course, I could let the user choose the right version for his system, but I would rather not, if possible.
I want to use the "rbf" function from scipy.interpolate without installing neither scipy nor cython. First I tried to import only the source code of the rbf-function. But Rbf itselfs want to import other functions among others .pxd-files wherein cython is used, which is also not installed on the maschine the python script is running on. Is there a way to use the 2dinterpolation rbf without compiling scipy?
Many thanks in advance
Assuming you cannot get a scipy wheel for your platform, nor you can use e.g. Anaconda, you can of course just take https://github.com/scipy/scipy/blob/v0.18.1/scipy/interpolate/rbf.py#L57-L240
and try to use the code standalone. There will be two issues to solve:
scipy.special.xlogy. This is just x*log(y) with special care taken so
that it's equal zero for x=0. Can easily replicate in pure python or numpy if needs be.
scipy.linalg.solve. If you have numpy, you can replace it with numpy.linalg.solve. If you don't have numpy, uhm, well, you've a problem.
I have a Python package that I'm distributing and I need to include in it a function that does some heavy computation that I can't find programmed in Numpy as Scipy (namely, I need to include a function to compute a variogram with two variables, also called a cross-variogram).
Since these have to be calculated for arrays of over 20000 elements, I need to optimize the code. I have successfully optimized the code (very easily) using Numba and I'm also trying to optimize it using Cython. From what I've read, there is little difference on the final run-time with both, just the steps change.
The problem is: optimizing this code on my computer is relatively easy, but I don't know how to include the code and its optimized (compiled) version in my github package for other users.
I'm thinking I'm going to have to put only the python/cython source code and tweak the setup.py around so it re-compiles in for every user that install the package. If that is the case, I'm not sure if I should use Numba or Cython since Numba seems so much easier to use (at least from by experience) but is such a hassle to install (I don't want to force my users to install anaconda!).
To sum up, two questions:
1 Should this particular piece of code indeed be re-compiled in every user's computer?
2 If so, it is more portable to use Numba or Cython? If not, should I just provide the .so I compiled in my computer?
There is a project called JyNI that allows you to run NumPy in Jython. However I haven't come across anywhere on how to get NumPy into Jython. I've tried 'pip install numpy' (which will work for normal python 3.4.3) but gives an error about a missing py3k module. Does anybody have a bit more information about this?
JyNI does state NumPy-support as its main goal, but cannot do it yet, as long as it is still in alpha-state.
However until it is mature enough you can use NumPy via
JEP (https://github.com/mrj0/jep) or
JPY (https://github.com/bcdev/jpy).
Alternatively you can use a Java numerical library for your computation, e.g. one of these:
https://github.com/mikiobraun/jblas
https://github.com/fommil/matrix-toolkits-java
Both are Java-libs that do numerical processing natively backed by blas or lapack (i.e. the same backends NumPy uses), so the performance should equal that of NumPy more or less. However they don't feature such a nice multiarray implementation as NumPy does afaik.
If you need NumPy indirectly to fulfill dependencies of some other framework, these solutions won't do it out of the box. If the dependencies are only marginal you can maybe rewrite/substitute the corresponding calls based on one of the named projects. Otherwise you'll have to wait for JyNI...
If you can make some framework running on Jython this way, please consider to make your work publicly available, ideally as a fork of the framework.