Develop Reference

Configuring Python C extensions for pip

I wrote a Python C extension, and it works great. Installing via python setup.py install works. However, pip cannot find my header files - so a pip installation doesn't work.
> pip install
Collecting jcalg1==1.0.1
Downloading https://files.pythonhosted.org/packages/a1/83/08b5fc2fbd36c8ac0668ae64c05cc88f3f6bd8fe68f058b19b11a463afa1/jcalg1-1.0.1.tar.gz
Installing collected packages: jcalg1
Running setup.py install for jcalg1 ... error
ERROR: Command errored out with exit status 1:
(...)
src\main.cpp(7): fatal error C1083: Cannot open include file: 'jcalg1.h': No such file or directory
(...)
This is my setup.py, the header file is located in the src folder.
from setuptools import setup,Extension
import setuptools
from setuptools import find_packages
import pathlib
# The actual C extension
jc_module = Extension('jcalg1', include_dirs=["src"], sources = ['src\main.cpp'], libraries =["src\jcalg1_static"])
# The directory containing this file
HERE = pathlib.Path(__file__).parent
# The text of the README file
README = (HERE / "README.md").read_text()
# This call to setup() does all the work
setup(
name="jcalg1",
version="1.0.1",
description="Interface to the JCALG1 compression library",
long_description=README,
long_description_content_type="text/markdown",
url="https://github.com/CallMeAlexO/jcalg1",
author="Alex Osheter",
author_email="alex.osheter#gmail.com",
license="MIT",
classifiers=[
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
],
ext_modules = [ jc_module ],
packages=find_packages()
)

You can create a MANIFEST.in file in your project root directory:
include src\*.h
include src\*.lib
Then rebuild your package. It will add the header files and library files into your package.

ModuleNotFoundError after making a setup.py and running "pip install -e ."

I am trying to import the classes SqlQuery, SqlPost, and SqlPage to the file Query.py from the file "database.py" with the path "C:\Users\myName\dir1\2020\Indeed-Scraper\database\database.py". My first attempt at this was to use relative imports: from ...database.database import SqlQuery which I presumed would work since the "database" folder is up two levels from Query.py. Not so. That returned ValueError: attempted relative import beyond top-level package.
I posted about this on StackOverflow and a helpful user suggested this fix: "make a setup.py for your project. Then run pip install -e".
Well I tried that. I now have a setup.py folder in the root directory of my project that looks like this:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="indeedscraper",
version="0.1.1",
author="myName",
author_email="myName#provider.com",
description="A tool used to scrape & display data from Indeed",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/myUsername/myProject",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
And I ran pip install -e . in the root directory of my project. Now PyCharm autocompletes "from in" to "from indeedscraper", so something changed. But from indeedscraper.database.database import SqlQuery still gives me:
Traceback (most recent call last):
File "C:/Users/myName/dir1/2020/Indeed-Scraper/scraper/classes/Query.py", line 17, in <module>
from indeedscraper.database.database import SqlQuery
ModuleNotFoundError: No module named 'indeedscraper'
Any suggestions on how to proceed? I'm kind of stuck here learning to make a package and import from that package, because the alternative is to copy ~50 lines of code from my "database.py" file, which is ugly.
Edit: I do have __init__.py files in my top level dir, "database.py", and the directory housing the file I am trying to import into. Like this:
project_name/
__init__.py
database/
__init__.py
database.py
scraper/
__init__.py # I do have this file here, sorry, didn't realize it was important
classes/
__init__.py
Query.py
Edit2: I realize the problem may be stemming from the fact that I am using Anaconda. I import in Query.py right, but the first 2 lines of my output in PyCharm read: "C:\Users\myname\Anaconda3\envs\august_2019_django_project\python.exe C:/Users/myname/dir1/2020/Indeed-Scraper/scraper/classes/Query.py"
Could this be it?

cant run python package after uploading to pip

I have a python 3 package I'm trying to upload to pip called vlcradio, I upload it successfully, then download it with pip install vlcradio, but am unable to run it using python -m vlcradio, as I get an error saying:
C:\Program Files\Python36\python.exe: No module named vlcradio
This is what the structure of my python package folder looks like:
/VLC-Radio/
/.git
/vlcradio
__init__.py
__main__.py
LICENSE
README.md
setup.py
And these are my files:
#__ init__.py
name = "vlcradio"
.
#__ main __.py
import sys
import os
#libraries to help encode / decode utf-8 chars to their corresponding ascii
#from unidecode import unidecode
from urllib.parse import unquote
import html
import html.parser
from shutil import copyfile
from sys import exit
import os.path
import requests
import time
from PIL import Image
###############################################
print("\n\nprogram start")
#... lots more code
setup.py
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="vlcradio",
version="0.0.1",
author="martinbarker99",
author_email="martinbarker99#gmail.com",
description="export VLC metadata",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/MartinBarker/VLC-Radio",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
inside the folder where setup.py is, I run these commands:
python setup.py sdist bdist_wheel
Which creates the /dist folder successfully.
python -m twine upload --repository-url https://upload.pypi.org/legacy/ dist/*
I sign in and the package gets uploaded correctly, I can see it in my pip account online. Then in a different cmd window I run:
pip install vlcradio
Which installs correctly, but when I try to run it with:
python -m vlcradio
I get an error saying:
C:\Program Files\Python36\python.exe: No module named vlc-radio
is there some step for specifying my package name that I'm missing? Thanks

had different versions of python running, uninstalled them and was able to get it working

Custom package installed but not found in another project

I created a Python folder/project and published the code on Github. The folder has the following structure:
/modulename/__init__.py
/modulename/setup.py
/modulename/somefunctions.py
/modulename/README.md
The name of my package is module_helloworld and setup.py looks as follows:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="module_helloworld",
version="0.0.1",
author="Hello World",
author_email="hello#world.com",
description="Hello world module",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://www.website.com",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
In another project, I installed it in Pycharm using the command
pip install git+https://github.com/Username/module-helloworld.git
That worked fine and in my Project Settings I see the package installed (noticed that it was installed with the name module-helloworld however) .
Now when I open the python console (or a new Python file) and I type
import module_helloworld
Then I get the error:
ModuleNotFoundError: No module named 'module_helloworld'
What did I do wrong?

In my case I had to restructure the folder structure as follows:
root/module_helloworld/__init__.py
root/module_helloworld/somefunctions.py
root/setup.py
And then in the other project I could call it in the normal way.
Inside the function __init__.py to import the functions I had to change it to the following:
# import somefunctions changed to:
from module_helloworld import somefunctions

PIP-installed global script throws exception, local copy works flawlessly

I've just registered my new package with PIP
python setup.py register
python setup.py sdist upload
And I wanted to install it on other machine globally (i.e. no virtualenv) using 'PIP':
sudo pip install standardiser
This all went fine, since I have one file registered as a script:
setup(
...
scripts=['standardiser/bin/standardiser.py'],
)
'standariser.py' is now available as CLI command system wide. But if I execute it, I'm getting:
mnowotka#candela:~/Documents/ci/curation_interface/trunk/src$ standardiser.py
Traceback (most recent call last):
File "/usr/local/bin/standardiser.py", line 32, in <module>
from standardiser import standardise, SDF
File "/usr/local/bin/standardiser.py", line 32, in <module>
from standardiser import standardise, SDF
ImportError: cannot import name standardise
I get the same when I explicitly call python:
python /usr/local/bin/standardiser.py
But if I copy this to some local folder:
sudo cp python /usr/local/bin/standardiser.py bla.py
And run it from there:
mnowotka#candela:~$ python bla.py
usage: bla.py [-h] [-V] [-r] infile
bla.py: error: too few arguments
I don't have any ImportErors. What I'm doing wrong? Can you help me?
My setyp.py:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
__author__ = 'mnowotka'
import sys
try:
from setuptools import setup
except ImportError:
from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup
setup(
name='standardiser',
version='0.1.4',
author='Francis Atkinson',
author_email='francis#ebi.ac.uk',
description='Provides a simple way of standardising molecules as a prelude to e.g. molecular modelling exercises.',
url='https://www.ebi.ac.uk/chembldb/index.php/ws',
license='Apache License, Version 2.0',
scripts=['standardiser/bin/standardiser.py'],
packages=['standardiser'],
long_description=open('ReadMe.txt').read(),
package_data={
'standardiser': ['bin/*', 'data/*', 'docs/*', 'knime/*', 'test/*',],
},
classifiers=['Development Status :: 2 - Pre-Alpha',
'Environment :: Console',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Topic :: Scientific/Engineering :: Chemistry'],
zip_safe=False,
)

You are asking for a script to be installed named standardiser.py; that script (at least, the one you've uploaded to the cheese shop) contains the line:
from standardiser import standardise, SDF
But thats an ambigious import; the script you're executing, /usr/local/bin/standardise.py appears in sys.path, because the main script is located there. it's importing itself!
you should be using the console_scripts feature of setuptools anyway.
alter your script file from
#! /guess/path/to python
from standardise import import *
do_things()
do_more_things()
to
from __future__ import absolute_import
from standardise import import *
def main():
do_things()
do_more_things()
if __name__ == '__main__':
main()
Which is to say;
get rid of the shebang; you never need it in python!
use the absolute_import feature to get the module named foo.bar.foo to be able to import foo instead of only foo.bar.foo (you can still import foo.bar.foo as either from foo.bar import foo or import .foo). __future__ imports must appear first in the source file, before any other non-comment lines (including other imports)
most importantly; wrap import time side effects in a function, and only invoke those side effects if this is the "main script"
then change your setup.py around from
setup(
scripts=['standardiser/bin/standardiser.py'],
...)
to
setup(
entry_points={
'console_scripts': [
'standardiser=standardiser.bin.standardiser:main']},
...)
which is to say:
use an entry point; setuptools knows how to get your installed package on sys.path correctly in this case, and it knows how to connect to the proper python interpreter, the one that was used to run setup.py. This matters when there are multiple versions of python, or when running in a virtualenv. this is why you never need a shebang.
remove the '.py' from the installed executable name. this shouldn't ever be there for scripts that are to be on your executable path.