Here's my build.py:
import os
import shutil
import subprocess
from pybuilder.core import use_plugin, init, task, Author
use_plugin('python.core')
use_plugin('python.unittest')
use_plugin('python.coverage')
use_plugin('python.distutils')
use_plugin('python.install_dependencies')
use_plugin('pypi:elasticsearch')
authors = [Author('eagle', 'email')]
version = '1.0'
description = 'Pipeline thing test'
requires_python = '>=3.0'
default_task = ['install_dependencies', 'publish', 'setup', 'analyze']
#init
def initialize(project):
project.build_depends_on('elasticsearch')
project.set_property('dir_source_unittest_python','tests')
project.set_property('dir_source_main_python','src')
project.set_property('unittest_module_glob','test_*')
project.set_property('coverage_branch_partial_threshold_warn ',80)
project.set_property('coverage_threshold_warn',80)
project.set_property('coverage_break_build',False)
project.set_property('distutils_use_setuptools',True)
project.set_property('distutils_classifiers',['Development Status :: 3 - Alpha', 'Intended Audience :: Developers', 'Topic :: Software Development :: Pipeline', 'License :: OSI Approved :: MIT License', 'Programming Language :: Python :: 3.6'])
#task
def setup():
subprocess.check_output(['python3.6','target/dist/pythonutilities-1.0/setup.py', 'install'])
It's specifically looking for the test directory under projectDir/tests
as specified in project.set_property('dir_source_unittest_python','tests')
And it does look there and find the correct test file:
File "/Users/adc1blz/Desktop/Work/pythonutilities/tests/test_elastic_controller.py", line 4, in <module>
But I still receive the following error:
ImportError: Failed to import test module: test_elastic_controller
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
module = __import__(module_name)
File "/Users/adc1blz/Desktop/Work/pythonutilities/tests/test_elastic_controller.py", line 4, in <module>
from src.elastic_controller import ElasticController
ModuleNotFoundError: No module named 'src'
I've tried using the default project structure by deleting
project.set_property('dir_source_unittest_python','tests')
project.set_property('dir_source_main_python','src')
and then moving around my test and src directory, but still I get the same error. The specific commands that I'm running that cause this are:
> python3.6 -m virtualenv env
> source env/bin/activate
> pip3.6 install pybuilder
> pyb -X
Why can't PyBuilder find the src module while in a virtual environment? Please, this is has been so frustrating.
Chris
Related
ISSUE: I would like to install local package in a specific conda environment. To do that, I read the current documentation (python-packaging).
package structure:
$ pwd
~/…/test
.
|- requirements.txt
|- my_package
| |-- __init__.py
| |-- base.py
|- setup.py
setup.py
# -*- coding: utf-8 -*-
import os
from setuptools import setup
with open('requirements.txt') as f:
requirements = f.read().splitlines()
setup(
name='my_package',
version='2.0.0',
author='B.Gees',
author_email='B.Gees#gmail.com',
license='MIT',
packages=['my_package'],
description='my package description',
long_description='my package long description',
keywords='chemistry machine learning cheminformatics',
classifiers=[
'Environment :: Console',
'Intended Audience :: Developers',
'Intended Audience :: Healthcare Industry',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.5.5',
'Topic :: Scientific/Engineering',
'Topic :: Scientific/Engineering :: Bio-Informatics',
'Topic :: Scientific/Engineering :: Chemistry',
'Topic :: Scientific/Engineering :: Pharmacokinetic',
'Topic :: Software Development :: Libraries :: Python Modules',
],
install_requires=requirements,
zip_safe=False
)
requirements.txt
pandas==0.19.2
dill==0.2.7.1
cython==0.23.4
__init__.py
# -*- coding: UTF-8 -*-
"""
my_package
~~~~~~~~~~
my package full description
:copyright: (c) 2018 by B.Gees.
:license: MIT, see LICENSE file for more details.
"""
from __future__ import print_function
from __future__ import unicode_literals
from __future__ import division
import logging
__title__ = 'my_package'
__version__ = '2.0.0'
__author__ = 'B.Gees'
__email__ = 'B.Gees#gmail.com'
__license__ = 'MIT'
__copyright__ = 'Copyright 2018 B.Gees'
log = logging.getLogger(__name__)
log.addHandler(logging.NullHandler())
base.py
# -*- coding: UTF-8 -*-
def titi(x):
return x**2
I install my package in a specific conda environment with the folowing code lines:
conda activate my_env
pip install . # In my package repository
Nevertheless, when I try to import my_package in jupyter notebook, I obtain the following error:
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-9-daa52839320b> in <module>()
----> 1 import my_package
ImportError: No module named 'my_package'
This installation works fine when I used python pip outer conda environment.
QUESTION: I don't know how to install my package correctly in a specific conda environment. I need your lights to enlighten me.
CONFIGURATION: MacOSX with conda3 and python3.5 ; Need to be compatible with Linux 7
You are using MacOSX, so you should firstly use source activate yourenvname, then you can use what you did to install your package. for more information How to activate an Anaconda environment
so start with : conda create --name my_env python=3.5
then source activate my_env
OS: Windows 7
Python: 3.6
I'm trying to create and installing a python wheel package. The building works fine but when i import the module into project after installing it, i get a "ModuleNotFound" error. My project has the following structure:
my_lib/
__init__.py
phlayer/
__init___.py
uart.py
utils/
__init___.py
ctimer.py
My setup.py for creating the wheel package:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="my_lib",
version="0.0.1",
author="",
author_email="",
description="",
packages=setuptools.find_packages(),
classifiers=(
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
),
)
In uart.py i do:
from utils import ctimer
After installing i import the package into another project:
#Test.py
from my_lib.phlayer.uart import Uart
def main(args=None):
pass
if __name__ == "__main__":
main()
And i get the error:
File "C:/.../.../.../Test.py", line 9, in <module>
from my_lib.phlayer.uart import Uart
File "C:\...\...\...\...\...\...\test\env\lib\site-packages\my_lib\phlayer\uart.py", line 3, in <module>
from utils import ctimer
ModuleNotFoundError: No module named 'utils'
So it seems that python cannot find the correct module in the other package. Do i need to specify the correct paths in the setup.py
before creating the wheel package?
You have to specify full module names:
from my_lib.utils import ctimer
I'm trying to create a one-file executable from a python file using pyinstaller. The executable fails when trying to import cups and gives the following error:
Failed to execute script ImportPyCups
Traceback (most recent call last):
File "ImportPyCups.py", line 1, in <module>
import cups
File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
exec(bytecode, module.__dict__)
File "build/bdist.macosx-10.8-intel/egg/cups.py", line 7, in <module>
File "build/bdist.macosx-10.8-intel/egg/cups.py", line 6, in __bootstrap__
ImportError: dlopen(/var/folders/jp/8v0hvshd585dw9v_7bnlxsqh0000gn/T/_MEILQoRVs/cups.so, 2): image not found
Here's the python file (yes, just a single import statement):
import cups
Here are the commands I run to turn the python file into a single file executable:
$ /usr/local/bin/pyi-makespec --onefile --console ImportPyCups.py
$ pyinstaller ImportPyCups.spec
I then run:
$ dist/ImportPyCups
This is where I get the error message that I pasted above. If I just run the python file from the terminal like below, no errors occur:
$ python ImportPyCups.py
The environment this runs in is (Mac OS X 10.8.5):
PyInstaller: 3.2
Python: 2.7.2
Platform: Darwin-12.5.0-x86_64-i386-64bit
The installed pycups is:
Metadata-Version: 1.0
Name: pycups
Version: 1.9.68
Summary: Python bindings for libcups
Home-page: http://cyberelk.net/tim/software/pycups/
Author: Tim Waugh
Author-email: twaugh#redhat.com
License: GPLv2+
Location: /Library/Python/2.7/site-packages/pycups-1.9.68-py2.7-macosx-10.8-intel.egg
Requires:
Classifiers:
Intended Audience :: Developers
Topic :: Software Development :: Libraries :: Python Modules
License :: OSI Approved :: GNU General Public License (GPL)
Development Status :: 5 - Production/Stable
Operating System :: Unix
Programming Language :: C
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 3
From the error message it seems that something is missing, but I haven't been able to figure out what.
I've finally found the answer here: https://github.com/pyinstaller/pyinstaller/issues/1728
If you use easy_install to install PyCups, you'll get an .egg file. Pyinstaller doesn't look for C extensions inside zipped .egg files and never will.
The solution is simple: Remove the .egg file and install PyCups using pip.
This is not specific to PyCups, but to easy_install.
I am using python 2.7.5. I created a simple python project pythontest. The folder structure of my project is as follows:
pythontest
|
-setup.py
-README.md
-src
|
-- pack1
|
test1.py
test2.py
|
--pack2
|
test3.py
My files are like this:
test1.py
print "hello"
test2.py
import test1
import src.pack2.test3
print "hai"
test3.py
print "how are you?"
I did run the like this :
sudo python setup.py install
python src/pack1/test2.py
But i am getting the following error:
hello
Traceback (most recent call last):
File "src/pack1/test2.py", line 7, in <module>
import src.pack2.test3
ImportError: No module named pack2.test3
But i can run the code from eclipse. What is wrong with my code?
setup.py file is as follows:
import os
from setuptools import setup
def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()
setup(
name = "pythontest",
version = "0.0.4",
author = "Andrew Carter",
author_email = "andrewjcarter#gmail.com",
description = ("An demonstration of how to create, document, and publish "
"to the cheese shop a5 pypi.org."),
license = "BSD",
keywords = "example documentation tutorial",
url = "http://packages.python.org/an_example_pypi_project",
packages=[],
long_description=read('README.md'),
classifiers=[
"Development Status :: 3 - Alpha",
"Topic :: Utilities",
"License :: OSI Approved :: BSD License",
],
)
You need to place __init__.py-files into packages (in your case, src, pack1 and pack2) for this to work. Also, you should get rid of src as package-name (it's generic and makes no sense), but instead use something pertaining to your actual codebase, e.g. supercoolapp or amazinglib or whatever you are writing on.
Then you need to add that top-level package to your packages-list in the setup.py.
I've just registered my new package with PIP
python setup.py register
python setup.py sdist upload
And I wanted to install it on other machine globally (i.e. no virtualenv) using 'PIP':
sudo pip install standardiser
This all went fine, since I have one file registered as a script:
setup(
...
scripts=['standardiser/bin/standardiser.py'],
)
'standariser.py' is now available as CLI command system wide. But if I execute it, I'm getting:
mnowotka#candela:~/Documents/ci/curation_interface/trunk/src$ standardiser.py
Traceback (most recent call last):
File "/usr/local/bin/standardiser.py", line 32, in <module>
from standardiser import standardise, SDF
File "/usr/local/bin/standardiser.py", line 32, in <module>
from standardiser import standardise, SDF
ImportError: cannot import name standardise
I get the same when I explicitly call python:
python /usr/local/bin/standardiser.py
But if I copy this to some local folder:
sudo cp python /usr/local/bin/standardiser.py bla.py
And run it from there:
mnowotka#candela:~$ python bla.py
usage: bla.py [-h] [-V] [-r] infile
bla.py: error: too few arguments
I don't have any ImportErors. What I'm doing wrong? Can you help me?
My setyp.py:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
__author__ = 'mnowotka'
import sys
try:
from setuptools import setup
except ImportError:
from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup
setup(
name='standardiser',
version='0.1.4',
author='Francis Atkinson',
author_email='francis#ebi.ac.uk',
description='Provides a simple way of standardising molecules as a prelude to e.g. molecular modelling exercises.',
url='https://www.ebi.ac.uk/chembldb/index.php/ws',
license='Apache License, Version 2.0',
scripts=['standardiser/bin/standardiser.py'],
packages=['standardiser'],
long_description=open('ReadMe.txt').read(),
package_data={
'standardiser': ['bin/*', 'data/*', 'docs/*', 'knime/*', 'test/*',],
},
classifiers=['Development Status :: 2 - Pre-Alpha',
'Environment :: Console',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Topic :: Scientific/Engineering :: Chemistry'],
zip_safe=False,
)
You are asking for a script to be installed named standardiser.py; that script (at least, the one you've uploaded to the cheese shop) contains the line:
from standardiser import standardise, SDF
But thats an ambigious import; the script you're executing, /usr/local/bin/standardise.py appears in sys.path, because the main script is located there. it's importing itself!
you should be using the console_scripts feature of setuptools anyway.
alter your script file from
#! /guess/path/to python
from standardise import import *
do_things()
do_more_things()
to
from __future__ import absolute_import
from standardise import import *
def main():
do_things()
do_more_things()
if __name__ == '__main__':
main()
Which is to say;
get rid of the shebang; you never need it in python!
use the absolute_import feature to get the module named foo.bar.foo to be able to import foo instead of only foo.bar.foo (you can still import foo.bar.foo as either from foo.bar import foo or import .foo). __future__ imports must appear first in the source file, before any other non-comment lines (including other imports)
most importantly; wrap import time side effects in a function, and only invoke those side effects if this is the "main script"
then change your setup.py around from
setup(
scripts=['standardiser/bin/standardiser.py'],
...)
to
setup(
entry_points={
'console_scripts': [
'standardiser=standardiser.bin.standardiser:main']},
...)
which is to say:
use an entry point; setuptools knows how to get your installed package on sys.path correctly in this case, and it knows how to connect to the proper python interpreter, the one that was used to run setup.py. This matters when there are multiple versions of python, or when running in a virtualenv. this is why you never need a shebang.
remove the '.py' from the installed executable name. this shouldn't ever be there for scripts that are to be on your executable path.