Can MlPy / SciPy be used on GAE?
I believe I have imported NumPy correctly, as it does not raise any errors so far (GAE 1.6 ships with support for NumPy). However, From what I've read, I still need to import SciPy and MlPy, and I haven't been able to do it so far. Is there any documentation out there that could explain accurately how to setup MlPy on GAE (if this is even possible)?
The main reason as to why I need MlPy is that I need to do a k-means analysis (finding a cluster center point). Isn't there a "lite" library to do this that would avoid all the hassle of setting up NumPy and MlPy?
Thanks.
EDIT:
I'm trying to import scipy. What I did is:
Downloaded scipy-0.11.0b1.tar.gz
Extracted the 'scipy' folder into my GAE App folder
From a python file, call 'import scipy'
The error that I get is:
ImportError: Error importing scipy: you cannot import scipy while
being in scipy source directory; please exit the scipy source
tree first, and relaunch your python intepreter.
Libraries written in pure Python that don't require C modules should be supported.
Libraries written in Python that utilize C modules MAY be supported. The following is a link to supported and not-supported C modules. This may help in determining whether or not the library you want to use will be supported.
Google App Engine Python Library Support
Related
I want to run my code on AWS lambda function. To do so, i need to import some python packages (i.e. pandas, numpy, sklearn, scipy)
I have two problems:
First of all, the size of (unzip) packaged python zip files is greater than 250MB.
Secondly, I got some error using scipy as well as sklearn as:
Unable to import module 'lambda_function': cannot import name
'_ccallback_c'
of
Unable to import module 'lambda_function': No module named
'sklearn.check_build._check_build'
___________________________________________________________________________ Contents of /var/task/sklearn/__check_build:
__pycache _check_build.cpython-35m-x86_64-linux-gnu.sosetup.py
init.py
___________________________________________________________________________ It seems that scikit-learn has not been built correctly.
I tried to reinstall many times...
But still problems in sklearn and scipy.
Any idea?
sample code in AWS LambdaFunction:
import json
import numpy
import pandas
import sklearn
import scipy
def lambda_handler(event, context):
# TODO implement
print(event)
return
You appear to have two issues.
The first (and easiest to solve) is that you need to install the relevant modules on a Linux distro comparable to Amazon Linux.
You can either do this using EC2 or in a Docker container with Amazon Linux on it.
The second issue (which is a bit trickier if not impossible to solve given the size of the modules you want to use) is that you need to get your deployment size down to under 250MB unzipped and under 50MB zipped.
Using relevant CFLAG when installing may get you some of the way there. See here for an idea of what might work.
If you are still over limit (which I suspect you will be) your only choice left will be to delete some of the files in the modules which you believe will not be used in your particular program. This is risky, often error prone and usually takes many attempts to get right. Using code coverage tools may help you here, as they can indicate which files are actually being used.
In Julia I am using the module PyCall
using PyCall: #pyimport
When I then try to use the scikitlearn library's module called ensamble, there is no problem, it works and I can use the module:
#pyimport sklearn.ensemble as skle
However when I try to do the same with the PIL library or PIL library's module Images, it doesn't work.
#pyimport PIL.Image as PILI
I get the following error: ERROR: PyError (:PyImport_ImportModule)
ImportError('No module named PIL.Image',)
[inlined code] from /home/lara/.julia/v0.4/PyCall/src/exception.jl:81
in pyimport at /home/lara/.julia/v0.4/PyCall/src/PyCall.jl:387
Can someone please talk me through the steps to get this working becasue I don't see how this is different from the scikit learn library and ensamble module.
Based on discussion in the comments, the issue appears to be that PyCall is using its own Python installation which does not have the requisite library installed. There are two options:
follow these instructions to change the Python installation referenced by PyCall to your local one in /home/lara/anaconda2.
use Conda.jl to add PIL to the Conda.jl Python install:
Conda.add("Pillow") (you could probably use PIL, but Pillow is a compatible fork that is actively maintained from what I can tell)
The second option is probably slightly preferred, but if you have many packages installed already it may be simpler to try the first option (you can always switch back to the Conda.jl version if something doesn't work).
I have ARCGIS 10.2 . When we install this software, python 2.7.3 (on win32) is also installed which we can develop the software by python and by arcpy module. I want to add h5py to it's library. My windows is 64 bit and I downloaded both hy5py 32 and 64 bit and copied them to the Lib folder. When I open IDLE an try to import it, I encounter with the following error:
"ValueError: numpy.dtype has the wrong size, try recompiling".
I also copied six. and I think the problem is with numpy. I replaced the numpy from anaconda(numpy 1.9) with numpy in ARCGIS python Library. Then I could import h5py but I could not open arcpy.
I really appreciate for your help
It sounds like the NumPy that came with your system was built one way, with one version, then you "installed" (by copying) h5py which was built another way, either with a different version or different options for NumPy.
To fix this, you should try building h5py yourself, which you can do by downloading the Source from here: https://pypi.python.org/pypi/h5py/2.5.0
Or perhaps easier, consider using Pandas. It can also read and write hdf5 files (though perhaps with fewer arcane features), and it's quite likely that someone has already provided a package for Pandas which is compatible with your existing installation.
I'm on Windows using/embedding Python 3.4
I'm trying to embed Python in an existing C++ application. I've pretty much done the job, but I can't get numpy to work.
I have numpy installed via the whl file from http://www.lfd.uci.edu/~gohlke/pythonlibs/ Everything works just fine in python.exe, but when I do something like...
PyRun_SimpleString("import numpy\n");
...in the C++ application, I get a traceback which ends with:
\Python\lib\site-packages\numpy\core\__init__.py, line 6, in <module>
from . import multiarray
cannot import 'multiarray'
Well, this made some sense to me. The module multiarray is in ...\Python34\Lib\site-packages\numpy\core\multiarray.pyd I think these .pyd files are basically DLLs, which means an import library is needed.
For example, ...\Python34\lib contains import libraries (I think) for other modules that are contained in pyd files. You have to link against the import lib. in order to import the module in the C++\Python (eg via PyRun_SimpleString("import _elementtree\n"); )
I suppose I need an import library for numpy to link my C++ application against. Anybody know off hand if building numpy from source pops these out? The build process looks tricky, but maybe it's the only options here.
Also, obviously when Python is built, it can't be linked against import libraries for modules that aren't installed yet. In other words, my python.exe wasn't linked against this magic import library I'm looking for, but it can still import numpy, how? Can I recreate this in my application?
I'm using PyBrain in a project over Windows 7 and I've had not problem with this library until I had to write the trained network to a XML file.
I tried this "from pybrain.tools.xml.networkwriter import NetworkWriter" but I got an importation error.
Can anyone tell me if there's a requirement to get this job done?
I tried installing the library called "lxml", because I have it installed on my linux pc, but it doesn't seem to work along side with pybrain.
PyBrain declares in it's setup.py only one required package scipy.
Scipy is a bit complex stuff and it is best installed from binaries (at least on Windows). So if you manage installing scipy, you shall make PyBrain running.