Develop Reference

Cant import local module

So I am trying to use this repository on my computer but I cant seem to import a local module (Agent.pyx) and use functions from that file. The error I get is:
ModuleNotFoundError: No module named 'RLAgent.Agent
Screenshot of error
Link to repository

You have to install the package manually

Since Agent.pyx is a .pyx file, you will need to first build it. The README file in the linked repo also mentions this. So you need to move to the RLAgent directory and execute the setup instruction as:
cd RLAgent
python3 setup.py build_ext --inplace
This will build the required .c and .so file which will then enable it's import in other files such as Train.py which is where the import is throwing an error.

Py2App error: ModuleNotFoundError: No module named 'cmath' when using Pandas

I am trying to build a standalone app that utilises Pandas. This is my setup.py file:
from setuptools import setup
APP = ['MyApp.py']
DATA_FILES = ['full path to/chromedriver']
PKGS = ['pandas','matplotlib','selenium','xlrd']
OPTIONS = {'packages': PKGS, 'iconfile': 'MyApp_icon.icns'}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app','pandas','matplotlib','selenium','xlrd'],
)
The making of the *.app file goes smoothly, but when I try to run it, it gives me the following error:
...
import pandas._libs.testing as _testing
File "pandas/_libs/testing.pyx", line 1, in init pandas._libs.testing
ModuleNotFoundError: No module named 'cmath'
I tried to include ‘cmath’ in my list of PKGS and in setup_requires in the setup.py file, but when I tried to build the app using py2app it gave me the error:
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cmath')
I am stuck. I couldn't find anything useful online. cmath should be automatically included from what I have been reading. Any ideas on where is the problem and how can I fix it?

I think I have found a solution: downgrade to Python version 3.6.6.
See: python 3.6 module 'cmath' is not found
To uninstall Python I followed this process: https://www.macupdate.com/app/mac/5880/python/uninstall
Then I installed Python 3.6.6: https://www.python.org/downloads/release/python-366/
With Python 3.6.6, Py2App seem to work no problem and includes Pandas smoothly.
It seems that for some reasons cmath is not included in the latest versions of Python? I might be wrong. Please let me know what you think and if you have any questions.
P.S.: I am using MacOS (Mojave 10.14.6) and PyCharm.

I had a similar issue with py2app and cmath. I solved this by adding import cmath into the main script. (MyApp.py in your case) I think doing so may have the modulegraph to add the cmath library files.

openpyxl not working as EXE

Error message from running my exe:
ModuleNotFoundError: No module named 'openpyxl'
testHi.py
#simple test to see if openpyxl module works
import openpyxl
print ("hi")
input()
hook-openpyxl.py
# taken from pyinstaller dev team, store in same dir as testHi.py
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files('openpyxl')
cmd line input:
pyinstaller.exe --onefile --icon=py.ico --additional-hooks-dir=. hiTest.py
I run the the hiTest and get the error above.
I have looked everywhere for this solution. Can anyone tell me what I am doing wrong.

I fixed my issue by installing it through Pip, rather than install the package through Pycharm, and Pyinstaller was able to find the package.
I got the idea from looking through the text in the command prompt and saw it was loading modules that I had installed via Pip and not through Pycharm.

I was able to get this working using auto-py-to-exe (which uses pyinstaller) by including the following folders/files from my python library into the same folder that I run pyinstaller from:
jdcal.py
openpyxl (folder)
et_xmlfile (folder)
pyinstaller command:
pyinstaller -y -F "[directory]/myscript.py"
Notes on Library Location:
Windows library location for me was: C:\users[username]\AppData\Local\Programs\Python\Python37-32\Lib
The packages were in the "site_packages" folder

use
--hiddenimport openpyxl
long with the previous solutions, it worked for me, I was able to enforce the import of openpyxl.

You was quite close. :-)
I fixed the problem by modifying the "hook-openpyxl.py" file. The command collect_data_files('openpyxl') actually returns an empty list. But there is another command collect_submodules which seems to do what we want. So my "hook-openpyxl.py" file looks like this.
from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('openpyxl')
I placed the "hook-openpyxl.py" file in the same directory like my spec file. In the spec file I set to location of the new hook file
a = Analysis(
...
...
hookspath=['.'],
...
...
...
I guess, you will have the same result with your command line parameter
pyinstaller.exe --onefile --icon=py.ico --additional-hooks-dir=. hiTest.py
My environment
Python=3.8
openpyxl=3.0.10
PyInstaller=4.8

Cython unable to find shared object file

I am trying to link to my own C library from Cython, following the directions I've found on the web, including this answer:
Using Cython To Link Python To A Shared Library
I am running IPython through Spyder.
My setup.py looks like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
import numpy as np
setup(
ext_modules = cythonize(
[Extension("*",["*.pyx"],
libraries =["MyLib"],
extra_compile_args = ["-fopenmp","-O3"],
extra_link_args=["-L/path/to/lib"])
]),
include_dirs = [np.get_include()],
)
The file libMyLib.so is in /path/to/lib and it compiles fine.
I have a Python script in my IPython profile startup folder that does this
try:
os.environ["LD_LIBRARY_PATH"] += ":/path/to/lib"
except KeyError:
os.environ["LD_LIBRARY_PATH"] = "/path/to/lib"
I can confirm that this is running, because if I type os.environ["LD_LIBRARY_PATH"] into the IPython interpreter, it returns /path/to/lib
But when I try to load the Cython module (i.e. import mycythonmodule) I get:
ImportError: libMyLib.so: cannot open shared object file: No such file or directory
I've also tried putting libMyLib.so in other places to see if cython would find it:
In the directory where Python is running
On the Python path
In the same folder as the cython module
But it still doesn't find the shared library. The only way I can get it to find the library is by dropping it in /usr/lib, but I don't want it there, I want to be able to set the library path.
Am I missing something?

I'm self-answering, in case anyone else runs into the same problem. Looks like the answers are here:
Set LD_LIBRARY_PATH before importing in python
Changing LD_LIBRARY_PATH at runtime for ctypes
According to these answers (and my experience), the linker reads LD_LIBRARY_PATH when python is launched, so changing it from within python doesn't have any useful effect, at least not the effect I was hoping for. The only solution is to either wrap python in a shell script that sets LD_LIBRARY_PATH, or else drop the shared object somewhere on the linker search path.
Kind of a pain, but it is what it is.

I have fixed it by change setup.py.
I have a C++ dynamic shared library called "libtmsmdreader.so". and a header file named "TmsMdReader.hpp"
I wrapper C++ shared library to cython library called "tmsmdreader-pythonxxxxxx.so"
from setuptools import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
name="tmsmdreader",
ext_modules=cythonize([
Extension(
name="tmsmdreader",
language="c++",
sources=["TmsMdReaderApi.pyx"],
libraries=["tmsmdreader"],
library_dirs=["."],
include_dirs=["."],
extra_compile_args=["-std=c++14"],
compiler_directives={'language_level': 3},
runtime_library_dirs=["."])
]))
library_dirs=["."] and runtime_library_dirs=["."] can fixed LD_LIBRARY_PATH if libtmsmdreader.so in python scripy directory

Compile Python C Extensions with cx_Freeze

I am trying to create an exe on Windows from a Python 3 package with a C extension module. In distutils, you can create an extension like this:
from distutils.core import setup, Extension
module1 = Extension('demo',
sources = ['demo.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
Than, the extension will be compiled with the appropriate compiler and placed alongside your other modules with the command:
python setup.py build_ext --inplace
cx_Freeze is a module that can package your code into an exe file along with a Python interpreter and the relevant packages. Then, an end user can use your program without having a Python installation. Unfortunately, cx_Freeze doesn't have an Extension class, and I cannot find a way to handle compilation with cx_Freeze.
One solution I am unsure about is to first build the extensions in place with distutils/setuptools, and then use cx_Freeze to create the executable. I don't want to reinvent the wheel though, so I wonder if someone else with more experience in this area has a solution.

I found a working solution. I can import Extension from distutils, and pass it into the setup from cx_Freeze:
from cx_Freeze import setup
from distutils.core import Extension
...
setup=(...
ext_modules=Extension(...))
This makes sense, since cx_Freeze is built on top of distutils. Originally, I was trying to use setuptools.setup, but that doesn't work.