importing an external c module to python - python

I'm trying to run a python code (test.py) which uses an external module written in C (swapped). Here is the directory structure,
test.py
setup.py
swapped.so
ext_src/
c_swapped.c
c_swapped.h
swapped.c
In order to use the swapped module, I first run setup.py, which is configured as following:
from distutils.core import setup
from distutils.extension import Extension
from distutils.command import build_ext
import numpy as np
import sys
sys.argv[1:] = ['build_ext', '--inplace']
ext_modules = [Extension(
name="swapped",
sources=["ext_src/swapped.c", "ext_src/c_swapped.c"],
language="c", include_dirs=[np.get_include()])
]
setup(
name = 'RLScore',
ext_modules = ext_modules,
)
And in test.py, swapped is imported as:
import swapped
So when I run these on python 2.7.6, test.py works just fine, but on python 3.4.5, I get the following error:
import swapped
File "swapped.pyx", line 2, in init rlscore.utilities.swapped (ext_src/swapped.c:3533)
System error parent module '' not loaded cannot perform relative import
I wish to run this code on python 3.4.5, so is there a way that I can make swapped module working?
Thanks a lot!
Edit: I installed virtual environment as python=2.7, so it works now. But still, if there is a way to run the package without using virtualenv, I'd like to learn.

Related

Cython ImportError: dynamic module does not define module export function (PyInit_world)

I successfully used Cython to convert .py to .pyd, but have been running into difficulties trying to import cythonized modules afterwards.
setup.py
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize("world.py")
)
world.py
print('Hello World')
When I try to import using import world, it gives me the following error:
ImportError: dynamic module does not define module export function (PyInit_world)
I am using Windows 8.1 64bit and Python 3.7.7 32bit.
I have been trying different strategies for hours. Any help will be appreciated.

how to import modules from local repository with virtualenv and pip

I have a question that I assume has a simple answer, but for some reason I am struggling to find it on my own. I have created and activated a virtual environment with virtualenv, and I am trying to install all the necessary packages in order to create a requirements.txt file.
I have, for example, a Python file that begins like this:
import xml.etree.ElementTree as ET
from lib.project import Projector
from lib import writer
import os
import datetime
from datetime import timedelta
from datetime import datetime
import pprint
When I try to run this file from the virtual machine, I receive the following error:
Traceback (most recent call last):
File "readMap.py", line 2, in <module>
from lib.project import Projector
ModuleNotFoundError: No module named 'lib.project'
My problem is that I'm not sure why the virtual environment can't find project.py. My directory structure is:
regiaoSul
lib
__init__.py
arrival_conversion.py
coord_conversion.py
message_conversion.py
project.py
route_conversion.py
stop_conversion.py
wkt_parser.py
writer.py
readMap.py
json_generator.py
The import on line 2 implies lib is a module rather than "a simple repository".
I will try running the script with the flag -m. Something like this -
python -m script_name
make sure to drop the .py extension when you run with -m flag.
Another advice: you don't need to install python files to the virtual environment, they are not some external libraries. They only need to be present (with the same order of packaging) when you run your script.
Thanks to everyone who responded. I believe the issue was some sort of dependency problem. In readMap.py I had imported writer from lib, and in writer.py I had imported Projector from project. I moved the function that required Projector from writer.py to readMap.py and it worked.
I still don't fully understand why this was a problem. Until recently I had been running my scripts in PyCharm and they all worked with the structure I had. It was only when I tried to run them from the command line in my virtual machine that they didn't work.
If anybody would like to explain the distinction to me and what the exact problem was with my imports, feel free to.
I sometimes face the same issue. A solution is to add the path to sys.path by:
import sys
sys.path.insert(0, "/path/to/your/package_or_module")

How do you resolve 'hidden imports not found!' warnings in pyinstaller for scipy?

I'm working on using pyinstaller to create an .exe for a python program that uses pandas and sklearn. The pyinstaller process completes and produces the dist folder with the executable as expected. However, when I run the .exe I get module import errors related to sklearn and scipy.
I created a test script (test.py) to test imports, which only imports pandas and sklearn and then prints a success message:
import time
import pandas as pd
import sklearn
def main():
print('hello world!')
time.sleep(5)
if __name__ == '__main__':
main()
I'm aware of pyinstaller hooks and I was able to resolve the pandas errors by adding a hook to the pyinstaller hooks directory. I added similar hooks for sklearn and scipy it looks like they're running, but in the pyinstaller output I'm getting warnings that 'Hidden import "sklearn.utils.sparsetools._graph_validation" not found!' and similar one for '._graph_tools'.
Here's the hook for scipy (hook-scipy.py):
print('loading custome hook for scipy')
from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('scipy')
Here's a snapshot of the warnings generated from running pyinstaller
Here's a snapshot of the error when running test.exe
I'm working in a virtual environment where pyinstaller, pandas, sklearn, scipy and all dependencies are installed (at least I can get the regular test.py script running in this venv). Using PyInstaller 3.3.1, Python 3.6.4 on Windows 10.10.0.
Any help is appreciated!
You need to go into the hook-scipy.py (or create one) and have it look like this:
from PyInstaller.utils.hooks import collect_submodules
from PyInstaller.utils.hooks import collect_data_files
hiddenimports = collect_submodules('scipy')
datas = collect_data_files('scipy')
then go into the hook-sklearn.metrics.cluster.py file and modify it to look like this:
from PyInstaller.utils.hooks import collect_data_files
hiddenimports = ['sklearn.utils.sparsetools._graph_validation',
'sklearn.utils.sparsetools._graph_tools',
'sklearn.utils.lgamma',
'sklearn.utils.weight_vector']
datas = collect_data_files('sklearn')
I do not know if this part is necessary but I also created a hook-sklearn.py file that looks like this:
from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('sklearn')
In the cmd I used pyinstaller test.py -F to create one file.
Then it should work:

pyttsx compilation error in windows using py2xe

I have written a python application with using pyttsx library . its working without any issue from tt.py file. I compiled this tt.py file using py2exe module. after compilation i tried to run .exe file. then its shows error like given image
my
setup.py file like this
from distutils.core import setup
import py2exe, sys, os
#includes =['drivers','drivers.sapi5'] #this tried. but making error
sys.argv.append('py2exe')
setup(
options = {'py2exe': {}},
console=['tt.py'],
)
i compiled by this command
python setup.py py2exe install
i am importing following modules in tt.py
import pyttsx
import pyttsx.drivers.sapi5
import win32com
from time import sleep
How can i fix this?
Looking at your imports you need to add import time
If your problem consist use cx freeze it is like py to exe and use a code like this.
It might also be a good idea to upgrade to python 3.3. This may sort some problems.
This might help it is a article on your error. https://mail.python.org/pipermail/python-win32/2006-January/004184.html
import pyttsx
import pyttsx.drivers.sapi5
import win32com
from time import sleep
import sys
from cx_Freeze import setup, Executable
setup(
name = "tt.py",
version = "0.1",
description = "your discription",
executables = [Executable("The file name", base = "Win32GUI")])
here's the link to cx freeze http://cx-freeze.sourceforge.net/
Here's the tutorial on how to use it http://cx-freeze.readthedocs.org/en/latest/overview.html
I tried cx_Freeze for a couple of minutes, but when it didn't work out immediately I tried some more with py2exe and got this working:
from distutils.core import setup
import py2exe
py2exe_options = { 'includes': ['pyttsx.drivers.sapi5', 'win32com.gen_py.C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4'],
'typelibs': [('{C866CA3A-32F7-11D2-9602-00C04F8EE628}', 0, 5, 4)] }
setup(console=['tt.py'], options = {'py2exe': py2exe_options})
Note though that this requires you to run the same version (v5.4 in my case) on both machines. If you want to circumvent that you probably need to try something more advanced.

Having an issue creating an exe with py2exe and script importing xlrd

My goal is to create a python script that loops over cells of an excel document. This is my python script called reader.py, and it works just fine.
import xlrd
import os
exceldoc = raw_input("Enter the path to the doc [C:\\folder\\file.xlsx]: ")
wb = xlrd.open_workbook(exceldoc,'rb')
.... some code....
The problem I'm encountering is attempting to use py2exe to create an executable so this script can be used elsewhere.
Here is my setup.py file:
from distutils.core import setup
import py2exe
import sys
from glob import glob
setup(name='Excel Document Checker',console=['reader.py'])
I run the following command: python setup.py py2exe
It appears to run fine; it creates the dist folder that has my reader.exe file, but near the end of the command I get the following:
The following modules appear to be missing
['cElementTree', 'elementtree.ElementTree']
I did some searching online, and tried the recommendations here Re: Error: Element Tree not found, this changing my setup.py file:
from distutils.core import setup
import py2exe
import sys
from glob import glob
options={
"py2exe":{"unbuffered": True,"optimize": 2,
'includes':['xml.etree.ElementPath', 'xml.etree.ElementTree', 'xml.etree.cElementTree'],
"packages": ["elementtree", "xml"]}}
setup(name='Excel Document Checker',options = options,console=['reader.py'])
I'm now getting an error:
ImportError: No module named elementtree
I'm sort of at an impasse here. Any help or guidance is greatly appreciate.
Just some information - I'm running Python 2.6 on a 32 bit system.
You explicitly told setup.py to depend on a package named elementtree here:
"packages": ["elementtree", "xml"]}}
There is no such package in the stdlib. There's xml.etree, but obviously that's the same name.
The example you found is apparently designed for someone who has installed the third-party package elementtree, which is necessary if you need features added after Python 2.6's version of xml.etree, or if you need to work with Python 1.5-2.4, but not if you just want to use Python 2.6's version. (And anyway, if you do need the third-party package… then you have to install it or it won't work, obviously.)
So, just don't do that, and that error will go away.
Also, if your code—or the code you import (e.g., xlrd) is using xml.etree.cElementTree, then, as the py2exe FAQ says, you must also import xml.etree.ElementTree before using it to get it working. (And you also may need to specify it manually as a dependency.)
You presumably don't want to change all the third-party modules you're using… but I believe that making sure to import xml.etree.ElementTree before importing any of those third-party modules works fine.

Categories

Resources