How to import bitarray using pypy? - python

I am running pypy and after adding the path to the bitarray library to sys.path I still can't import the module:
Python 2.7.3 (87aa9de10f9ca71da9ab4a3d53e0ba176b67d086, Feb 10 2014, 05:26:49)
[PyPy 2.2.1 with GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``every VM should come with neural
network trained to recognize microbenchmarks and randomly fluctuate them
+/-9000%''
>>>> import sys
>>>> sys.path.append('/usr/local/lib/python2.7/dist-packages/')
>>>> sys.path.append('/usr/local/lib/python2.7/dist-packages/bitarray/')
>>>> import bitarray
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/bitarray/__init__.py", line 11, in <module>
from bitarray._bitarray import _bitarray, bitdiff, bits2bytes, _sysinfo
ImportError: No module named bitarray._bitarray
In regular python (CPython installed from aptitude) importing bitarray works fine and I haven't made any modifications to the module after installing it with pip. What's wrong here?
Just in case you need it here is the content of the bitarray dir:
$ls /usr/local/lib/python2.7/dist-packages/bitarray/
_bitarray.so __init__.py __init__.pyc test_bitarray.py test_bitarray.pyc
UPDATE
After creating a virtualenv for pypy and installing bitarray with pip as Sunny suggested I still get an ImportError:
$ ls my-pypy-env/site-packages/bitarray
_bitarray.pypy-22.so __init__.py __init__.pyc test_bitarray.py test_bitarray.pyc
$ cd my-pypy-env/
~/my-pypy-env$ ./bin/activate
~/my-pypy-env$ pypy
Python 2.7.3 (87aa9de10f9ca71da9ab4a3d53e0ba176b67d086, Feb 10 2014, 05:26:49)
[PyPy 2.2.1 with GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``__xxx__ and __rxxx__ vs operation
slots: particle quantum superposition kind of fun''
>>>> import sys
>>>> sys.path.append('/home/sofia/my-pypy-env/site-packages/bitarray')
>>>> import bitarray
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named bitarray
UPDATE 2
Sunny's solution is correct. The new problem was appending /site-packages/bitarray
instead of just /site-packages/. I would have though that virtualenv would add this to the path automatically but it seems that it doesn't.


The _bitarray module looks like a CPython extension, which is written directly in C.
CPython extension modules does not work directly in PyPy without any changes. You either need to install PyPy compatible version of the module, or do the required changes manually.
To install pypy compatible version, use the following commands:
# If pypy is installed globally
/path/to/pypy/pypy-2.1/bin/pip install bitarray
# If using virtualenv
source /path/to/virtualenv/env/bin/activate
pip install bitarray
Here are a couple of links from PyPy FAQs about this issue:
http://doc.pypy.org/en/latest/faq.html#module-xyz-does-not-work-with-pypy-importerror
http://doc.pypy.org/en/latest/faq.html#do-cpython-extension-modules-work-with-pypy

Related

Linking errors when using setuptools

I am extending C++ library with functionality that requires GRPC. GRPC dependencies are added through VCPKG (example from CMakeLists.txt):
find_package(gRPC CONFIG REQUIRED)
target_link_libraries(
mylib PRIVATE
gRPC::grpc++)
Now, that same library has python bindings (where I enter into, for me, an uncharted territory).
The library is built through setuptools. The setup itself initially went ok but when I try to load the library I get:
❯ python3
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python3.8/dist-packages/mylib-0.0.0-py3.8-linux-x86_64.egg/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZTIN6google8protobuf2io20ZeroCopyOutputStreamE
Missing _ZTIN6google8protobuf2io20ZeroCopyOutputStreamE clearly comes from GRPC dependency. I tried playing with setup.py and including grpc lib folder:
toolchain_args += ['-I/home/atomic/vcpkg/installed/x64-linux/lib']
I also tried extending required libraries list:
libraries += ['libgrpc++', 'libprotobuf']
Grpc is installed under /usr/local/lib/ and also as vcpkg package.
But without any luck. Including libs failed with following error:
/usr/bin/ld: cannot find -llibgrpc++
/usr/bin/ld: cannot find -llibprotobuf

Python on Gentoo is missing sqlite

I use a Gentoo-based Docker image for CI with multiple versions of Python. Recently, I've started experiencing errors because one tool (coveralls) requires sqlite, which is missing. sqlite is part of the Python standard library.
This can be checked from the command line
>>> removing all .pyc files
>>> executing command
me#5b35f99c08af /source $ python
Python 3.6.9 (default, Dec 27 2019, 12:15:49)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'sqlite3'
I couldn't find any notes in the Gentoo packaging database about this, but I'm not really familiar with it as an OS. I assume there must have been a problem building some relevant library.
Python has been installed like this:
RUN emerge -q -u dev-lang/python:3.6
But the error occurs for all the versions I'm currently testing with: >= 3.5. Any ideas as to what I'm doing wrong?

How to use mechanize in pypy?

I used mechanize in Python, but peppy said
$ pypy
Python 2.7.9 (9c4588d731b7fe0b08669bd732c2b676cb0a8233, Mar 31 2015, 07:55:22)
[PyPy 2.5.1 with GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import mechanize
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named mechanize
>>>> import sys
>>>> sys.path
['', '/usr/local/Cellar/pypy/2.5.1/libexec/site-packages/setuptools-14.3.1-py2.7.egg', '/usr/local/Cellar/pypy/2.5.1/libexec/site-packages/pip-6.0.8-py2.7.egg', '/usr/local/Cellar/pypy/2.5.1/libexec/lib_pypy/__extensions__', '/usr/local/Cellar/pypy/2.5.1/libexec/lib_pypy', '/usr/local/Cellar/pypy/2.5.1/libexec/lib-python/2.7', '/usr/local/Cellar/pypy/2.5.1/libexec/lib-python/2.7/lib-tk', '/usr/local/Cellar/pypy/2.5.1/libexec/lib-python/2.7/plat-darwin', '/usr/local/Cellar/pypy/2.5.1/libexec/lib-python/2.7/plat-mac', '/usr/local/Cellar/pypy/2.5.1/libexec/lib-python/2.7/plat-mac/lib-scriptpackages', '/usr/local/Cellar/pypy/2.5.1/libexec/site-packages']
Please tell me how to use mechanize in pypy.
I think I haven't create path but I don't know way.

PyPy does not automatically pick CPython paths. You either need to install mechanize separately (I suggest virtualenv) or you need to put path to it into sys.path or PYTHONPATH environment variable.

OSX Port Python Library Path? Cant find ctypes

I am trying to use an application that has a dependency of ctypes, but am getting this error:
$ python peach.py -t ~/Desktop/fuzz/wav/template.xml
] Peach 2.3.6 Runtime
] Copyright (c) Michael Eddington
Traceback (most recent call last):
File "peach.py", line 335, in <module>
from Peach.Engine import *
File "/opt/Peach-2.3.6/Peach/__init__.py", line 40, in <module>
import Publishers, Transformers
File "/opt/Peach-2.3.6/Peach/Publishers/__init__.py", line 37, in <module>
import file, sql, stdout, tcp, udp, com, process, http, icmp, raw, remote, dll, smtp
File "/opt/Peach-2.3.6/Peach/Publishers/file.py", line 37, in <module>
from Peach.Engine.engine import Engine
File "/opt/Peach-2.3.6/Peach/Engine/engine.py", line 835, in <module>
from Peach.Engine.state import StateEngine
File "/opt/Peach-2.3.6/Peach/Engine/state.py", line 38, in <module>
import sys, re, types, time, struct, ctypes
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/ctypes/__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: No module named _ctypes
I have installed py-ctypes from ports, but it seems to only be a Python 2.4 version:
$ port contents py-ctypes
Port py-ctypes contains:
/opt/local/lib/python2.4/site-packages/_ctypes.so
/opt/local/lib/python2.4/site-packages/_ctypes_test.so
/opt/local/lib/python2.4/site-packages/ctypes/__init__.py
/opt/local/lib/python2.4/site-packages/ctypes/__init__.pyc
/opt/local/lib/python2.4/site-packages/ctypes/_endian.py
/opt/local/lib/python2.4/site-packages/ctypes/_endian.pyc
/opt/local/lib/python2.4/site-packages/ctypes/macholib/__init__.py
/opt/local/lib/python2.4/site-packages/ctypes/macholib/__init__.pyc
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dyld.py
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dyld.pyc
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dylib.py
/opt/local/lib/python2.4/site-packages/ctypes/macholib/dylib.pyc
I then tried to run the application via python2.4, but it seems the application uses syntax that is only available in 2.5:
$ python2.4 peach.py -t ~/Desktop/fuzz/wav/template.xml
File "peach.py", line 498
finally:
^
SyntaxError: invalid syntax
My python install is also from OSX ports, and I noticed in the Peach application, it defines python as:
#!/usr/bin/python
Will that mess with anything if my default python executable points to my port installation (and I am running 'python peach.py')?
$ which python
/opt/local/bin/python
Is there any work around for this?
ctypes for python2.5?
Ability to add 2.4 libraries to 2.5 path?

A simple solution would be to use the native Python build that is included with Mac OS. This definitely works with the latest release of Mac OS X 10.6.4, which has Python 2.6.
Here is an example showing that '_ctypes' is being imported successfully:
mariah:~ joet3ch$ /usr/bin/python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from _ctypes import Union, Structure, Array
>>>
If you have issues after this, try looking at the sys.path attribute to see which modules and versions are in your path.
Here is an example of viewing the contents of sys.path on a fresh Mac OS 10.6.4 build:
mariah:~ joet3ch$ /usr/bin/python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload', '/Library/Python/2.6/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']
>>>

ctypes is a standard Python library since version 2.5, so py-ctypes is not needed. The line at which you get an ImportError still exists in my 2.6.5 installation.
I do not own OSX, so my question is: does /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5 belong to the standard installation of Python 2.5, or may this belong to a possibly broken installation of some framework?
In a running python shell, you could inspect the value of sys.path. Maybe there is some broken library that precedes the standard library.
The shebang line #!/usr/bin/python is interpreted by the OS if it is the first line of an executable script that is invoked directly like an ordinary program. In all other cases, this is just a comment. In particular, the line is ignored if you invoke the script as in python myscript.py or if it is imported by other Python code.

Python 2.5.2 and Solaris 8 (gcc 3.4.2) build issues

I'm trying to build python 2.5.2 on Solaris 8 using gcc 3.4.2. I can't see any immediate errors in the ./configure step but, once built and i enter the python shell doing an import time errors with :
Python 2.5.2 (r252:60911, Nov 21 2008, 18:45:42)
[GCC 3.4.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named time
What am i doing wrong? From what i can see with a cursory google is that there might be an error with libstdc++.so, but i can't find any hard details.
Any suggestions would be most welcome.
Many thanks,
Al.

The time module is not built by default in Python, if you build from a source distribution you need to explicitly enable all the modules you want to compile.
Open up Modules/Setup.dist in the python source tree and comment out the line which says:
#time timemodule.c
To enable the build of time module. Also remember that you need to recompile Python for this to take an effect.

Categories

Resources