Pypy has a fork of numpy. How to use it in the pypy sandbox?
The sandbox prompt 'No module named numpy'.
However, when I tried to copy the numpy library in the lib_pypy directory, I received this error message. Is there a way to import the numpy in pypy sandbox?
import numpyTraceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/bin/lib_pypy/numpy/__init__.py", line 160, in <module>
raise ImportError(msg)
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
You can't import numpy in a sandboxed PyPy at all, sorry.
This is the case for almost all extension modules. Sandboxing in PyPy is a good proof of concept, really safe imho, but it is only a proof of concept. It seriously requires someone working on it. Before this occurs, it can only be used it for "toy" examples: programs that import mostly nothing (or only pure Python modules, recursively).
Related
I'm attempting to figure out the mechanics of running Python scripts in Power BI for Reasons and I've hit a snag. I was running through the steps in this somewhat basic tutorial and I came to the section in which I am supposed to paste the script into they Python Script screen, which is Step 3 of the 'Run the Script and Import Data' section.
When I followed the steps, which are essentially to paste the example script into the window and hit Okay, I got this 'helpful' error:
`
Details: "ADO.NET: Python script error.
<pi>Traceback (most recent call last):
File "C:\Users\my.username\PythonScriptWrapper_aca634c7-30c3-4bf4-881b-d1e47bb0a919\PythonScriptWrapper.PY", line 2, in <module>
import os, pandas, matplotlib
File "C:\Users\my.username\Anaconda3\lib\site-packages\matplotlib\__init__.py", line 109, in <module>
from . import _api, _version, cbook, docstring, rcsetup
File "C:\Users\my.username\Anaconda3\lib\site-packages\matplotlib\rcsetup.py", line 27, in <module>
from matplotlib.colors import Colormap, is_color_like
File "C:\Users\my.username\Anaconda3\lib\site-packages\matplotlib\colors.py", line 51, in <module>
from PIL import Image
File "C:\Users\my.username\Anaconda3\lib\site-packages\PIL\Image.py", line 89, in <module>
from . import _imaging as core
ImportError: DLL load failed while importing _imaging: The specified module could not be found.
</pi>"
`
I have verified that the matplotlib and pandas packages were installed via pip list, os doesn't show up, which surprised me but I know it's part of the standard library so I'm not stressing about it unless someone thinks I should. Is anyone an expert in this? Is there a better way? Am I doomed to scream into the Power BI void for all eternity?
The somewhat basic tutorial is based on using standard python from python.org. However, you are using the Anaconda distribution, which basically requires the environment to be activated before any modules - especially pandas' C-library - can be accessed.
You can achieve that in the cmd shell by running
conda activate
C:\Users\my.username\AppData\Local\Microsoft\WindowsApps\PBIDesktopStore.exe
which assumes that you are using the Power BI Desktop version from the Microsoft store.
I am trying to run a script which involves numpy through the terminal on WinSCP, but whenever I do, I get the following error:
import gensim
File "/data/work/worker/gensim/init.py", line 5, in
from gensim import parsing, corpora, matutils, interfaces, models, similarities, summarization, utils # noqa:F401
File "/data/work/worker/gensim/parsing/init.py", line 4, in
from .preprocessing import (remove_stopwords, strip_punctuation, strip_punctuation2, # noqa:F401
File "/data/work/worker/gensim/parsing/preprocessing.py", line 42, in
from gensim import utils
File "/data/work/worker/gensim/utils.py", line 38, in
import numpy as np
File "/data/work/worker/numpy/init.py", line 142, in
from . import core
File "/data/work/worker/numpy/core/init.py", line 50, in
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
The Python version is: Python2.7 from "/usr/bin/python"
The NumPy version is: "1.18.5"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named _multiarray_umath>
I suspect that the issue is that the version of Python being quoted here is outdated; however, after having done some searching, I cannot find any literature on how to update Python on WinSCP. I have Python 3.8 installed on my machine, and I have tried moving the installer and the .exe file into the WinSCP directory to no avail. Is there any way to update python directly in the terminal? Alternately, is this issue actually nothing to do with a stale version of Python at all?
I am running Spark programs on a large cluster (for which, I do not have administrative privileges). numpy is not installed on the worker nodes. Hence, I bundled numpy with my program, but I get the following error:
Traceback (most recent call last):
File "/home/user/spark-script.py", line 12, in <module>
import numpy
File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 170, in <module>
File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in <module>
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in <module>
File "/usr/local/lib/python2.7/dist-packages/numpy/core/__init__.py", line 6, in <module>
ImportError: cannot import name multiarray
The script is actually quite simple:
from pyspark import SparkConf, SparkContext
sc = SparkContext()
sc.addPyFile('numpy.zip')
import numpy
a = sc.parallelize(numpy.array([12, 23, 34, 45, 56, 67, 78, 89, 90]))
print a.collect()
I understand that the error occurs because numpy dynamically loads multiarray.so dependency and even if my numpy.zip file includes multiarray.so file, somehow the dynamic loading doesn't work with Apache Spark. Why so? And how do you othewise create a standalone numpy module with static linking?
Thanks.
There are at least two problems with your approach and both can be reduced to a simple fact that NumPy is a heavyweight dependency.
First of all Debian packages come with multiple dependencies including libgfortran, libblas, liblapack and libquadmath. So you cannot simply copy NumPy installation and expect that things will work (to be honest you shouldn't do anything like this if it wasn't the case). Theoretically you could try to build it using static linking and this way ship it with all the dependencies but it hits the second issue.
NumPy is pretty large by itself. While 20MB doesn't look particularly impressive and with all the dependencies it shouldn't be more 40MB it has to be shipped to the workers each time you start your job. The more workers you have the worse it gets. If you decide you need SciPy or SciKit it can get much worse.
Arguably this makes NumPy a really bad candidate for being shipped with pyFile method.
If you hadn't have direct access to the workers but all the dependencies, including header files and a static library were present, you could simply try to install NumPy in the user space from the task itself (it assumes that pip is installed as well) with something like this:
try:
import numpy as np
expect ImportError:
import pip
pip.main(["install", "--user", "numpy"])
import numpy as np
You'll find other variants of this method in How to install and import Python modules at runtime?
Since you have access to the workers a much better solution is to create a separate Python environment. Probably the simplest approach is to use Anaconda which can be used to package non-Python dependencies as well and doesn't depend on the system-wide libraries. You can easily automate this task using tools like Ansible or Fabric, it doesn't require administrative privileges and all you really need is bash and some way to fetch basic installers (wget, curl, rsync, scp).
See also: shipping python modules in pyspark to other nodes?
I am trying to use ryp with my arcpy scripts but I have the following error:
import rpy2.robjects as robjects Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module> import rpy2.robjects as robjects
File "C:\Python26\ArcGIS10.0\lib\site-packages\rpy2\robjects\__init__.py", line 12, in <module> import rpy2.rinterface as rinterface
File "C:\Python26\ArcGIS10.0\lib\site-packages\rpy2\rinterface\__init__.py", line 39, in <module> import win32api
ImportError: No module named win32api
This error comes even after the installation of the pywin32 for my version of python.
I've noticed that this seems to be a common error that is usually solved with the installation of pywin32.
I also have a problem with the matplotlib installation, every time i try to use it (import matplotlib.pyplot as plt), python crashes...
Versions:
Python 2.6.6
matplotlib installation: matplotlib-1.1.0.win32-py2.6.exe
You will need to run these scripts with PROPER Python. It seems to me that the ArcPy distribution does not include the win32api module (It also does not exist from example in Python on Mac or Linux).
I would install PythonXY which includes R bindings, and see if your scripts run there. If they run there, then I (guess) I am correct, and ArcPy does not include these modules.
A nice BONUS of PythonXY is it's an excellent Python IDE (Spyder), but the real bonus is what the commenter above me said:
different compiler versions can cause hell of a lot of Problems.
So, in PythonXY you get a whole bundle compiled with the same compiler.
Let us know if these made your RPy script run.
EDIT: after reading this http://projects.scipy.org/numpy/ticket/1322 it seems that the NumPy version I am using doesn't work with Mac OS 10.5.x. Does anyone have access to a version of NumPy that works with Mac OS 10.5? I can't get it to compile either.
Original post ...
I am trying to use NumPy, but I'm having difficulty installing it. I'm using the installer that is available on sourceforge.
NumPy v1.5.0 (Py2.7)
Python 2.7
Mac OS 10.5.8 (PPC)
After running the installer (all I did was double-click the .pkg and follow the instructions), I try to import it ...
>>> import numpy
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import numpy
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nump/__init__.py", line 153, in <module>
import random
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/random/__init__.py", line 87, in <module>
from mtrand import *
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/random/mtrand.so, 2): Symbol not found: _fopen$UNIX2003
Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/random/mtrand.so
Expected in: /usr/lib/libSystem.B.dylib
Does anyone have experience with this or have advice on how to fix it?
Thanks!
You can find discussion about current problems with Mac OS on the numpy mailing list, and I would recommend to discuss installation problems there, since that's were the developers are that try to fix the problems.
for example
http://groups.google.com/group/numpy/browse_thread/thread/a0ceb45b58feca2b#
and this is OS 10.5 specific
http://groups.google.com/group/numpy/browse_thread/thread/de75279785d56a25/2bfbb96e6d6c0a2e
You could join the effort, since they don't seem to have many OS 10.5 users available for testing.