I am trying to import the classes SqlQuery, SqlPost, and SqlPage to the file Query.py from the file "database.py" with the path "C:\Users\myName\dir1\2020\Indeed-Scraper\database\database.py". My first attempt at this was to use relative imports: from ...database.database import SqlQuery which I presumed would work since the "database" folder is up two levels from Query.py. Not so. That returned ValueError: attempted relative import beyond top-level package.
I posted about this on StackOverflow and a helpful user suggested this fix: "make a setup.py for your project. Then run pip install -e".
Well I tried that. I now have a setup.py folder in the root directory of my project that looks like this:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="indeedscraper",
version="0.1.1",
author="myName",
author_email="myName#provider.com",
description="A tool used to scrape & display data from Indeed",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/myUsername/myProject",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
And I ran pip install -e . in the root directory of my project. Now PyCharm autocompletes "from in" to "from indeedscraper", so something changed. But from indeedscraper.database.database import SqlQuery still gives me:
Traceback (most recent call last):
File "C:/Users/myName/dir1/2020/Indeed-Scraper/scraper/classes/Query.py", line 17, in <module>
from indeedscraper.database.database import SqlQuery
ModuleNotFoundError: No module named 'indeedscraper'
Any suggestions on how to proceed? I'm kind of stuck here learning to make a package and import from that package, because the alternative is to copy ~50 lines of code from my "database.py" file, which is ugly.
Edit: I do have __init__.py files in my top level dir, "database.py", and the directory housing the file I am trying to import into. Like this:
project_name/
__init__.py
database/
__init__.py
database.py
scraper/
__init__.py # I do have this file here, sorry, didn't realize it was important
classes/
__init__.py
Query.py
Edit2: I realize the problem may be stemming from the fact that I am using Anaconda. I import in Query.py right, but the first 2 lines of my output in PyCharm read: "C:\Users\myname\Anaconda3\envs\august_2019_django_project\python.exe C:/Users/myname/dir1/2020/Indeed-Scraper/scraper/classes/Query.py"
Could this be it?
Related
When I install one of my own Python applications from PyPi, it fails to run citing
File "/home/me/.local/lib/python3.9/site-packages/refrapt/refrapy.py", line 20, in
from classes import (
ModuleNotFoundError: No module named 'classes'.
I have the following directory layout in my local area:
/refrapt
setup.py
/refrapt
classes.py
helpers.py
refrapt.conf
refrapt.py
settings.py
__init__.py
To build the project, I'm using setuptools, running the following command:
python setup.py sdist bdist_wheel
This builds and works happily enough, and I'm able to upload the resulting /dist.
I then install the project using pip3 install refrapt. When I run it using refrapt, I get the error ModuleNotFoundError above.
When I run the development code locally, it runs fine, but installed via pip, it is failing. I assume it's a problem with my setup.py, but this is my first time and I haven't really a clue what is correct. I tried adding the init.py (which is empty) as suggested by some python docs, but to no avail. The contents of setup.py are as follows:
import pathlib
from setuptools import setup, find_packages
HERE = pathlib.Path(__file__).parent
README = (HERE / "README.md").read_text()
setup(
name='Refrapt',
version='0.1.5',
description='A tool to mirror Debian repositories for use as a local mirror.',
python_requires='>=3.9',
long_description=README,
long_description_content_type="text/markdown",
packages=find_packages(),
install_requires=[
'Click >= 7.1.2',
'Colorama >= 0.4.4',
'tqdm >= 4.60.0',
'wget >= 3.2',
'filelock >= 3.0.12'
],
classifiers=[
"Development Status :: 4 - Beta",
"Operating System :: Microsoft :: Windows :: Windows 10",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: Implementation",
"Topic :: System :: Archiving :: Mirroring"
],
keywords=['Mirror', 'Debian', 'Repository'],
entry_points='''
[console_scripts]
refrapt=refrapt:refrapt
''',
)
If anyone could help, I'd greatly appreciate. I'm out of my depth on this one, and haven't been able to find an answer so far.
from classes import …
In Python 2 this was relative import — the statements imports classes from the current directory of the importing module.
But in Python 3 it was changed to absolute import. The import fails because there is no a global module or a package named classes. You need to convert the import to absolute or relative. Either
from refrapt.classes import …
or
from .classes import …
Potentially I've found out the answer to my question, but it's not the answer I wanted.
I spun up a virtual environment, and installed an application that I've used before via pip. When I went to run the app, I got the ModuleNotFoundError: No module named 'app'.
I tried to run it manually via the .py file by using python .\.venv\Lib\site-packages\app\cli.py, which resulting in the same error.
Seems to be something about the environment setup in Windows VS Code that just doesn't operate the same as on a Linux machine.
I guess I'll just have to remove the "refrapt." prefix from import statements when developing locally, and then add it back when pushing to GitHub.
I have a python 3 package I'm trying to upload to pip called vlcradio, I upload it successfully, then download it with pip install vlcradio, but am unable to run it using python -m vlcradio, as I get an error saying:
C:\Program Files\Python36\python.exe: No module named vlcradio
This is what the structure of my python package folder looks like:
/VLC-Radio/
/.git
/vlcradio
__init__.py
__main__.py
LICENSE
README.md
setup.py
And these are my files:
#__ init__.py
name = "vlcradio"
.
#__ main __.py
import sys
import os
#libraries to help encode / decode utf-8 chars to their corresponding ascii
#from unidecode import unidecode
from urllib.parse import unquote
import html
import html.parser
from shutil import copyfile
from sys import exit
import os.path
import requests
import time
from PIL import Image
###############################################
print("\n\nprogram start")
#... lots more code
setup.py
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="vlcradio",
version="0.0.1",
author="martinbarker99",
author_email="martinbarker99#gmail.com",
description="export VLC metadata",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/MartinBarker/VLC-Radio",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
inside the folder where setup.py is, I run these commands:
python setup.py sdist bdist_wheel
Which creates the /dist folder successfully.
python -m twine upload --repository-url https://upload.pypi.org/legacy/ dist/*
I sign in and the package gets uploaded correctly, I can see it in my pip account online. Then in a different cmd window I run:
pip install vlcradio
Which installs correctly, but when I try to run it with:
python -m vlcradio
I get an error saying:
C:\Program Files\Python36\python.exe: No module named vlc-radio
is there some step for specifying my package name that I'm missing? Thanks
had different versions of python running, uninstalled them and was able to get it working
I have followed this tutorial to upload a package to PyPI. My package is now uploaded to test instance of the index: https://test.pypi.org/project/ilya-ezplots/#files. However when I install it with pip, I still can't import it.
>>> import ilya_ezplots
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ilya_ezplots'
I have inspected the site-packages of my python interpreter, and found only ilya_ezplots-0.0.2.dist-info directory. For other packages there would be two directories, e.g. h5py-2.9.0.dist-info and h5py.
My setup.py:
import setuptools
setuptools.setup(
name="ilya_ezplots",
version="0.0.2",
packages=['plots'],
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
Upload script:
#!/usr/bin/env bash
rm -r dist
python setup.py sdist bdist_wheel
python -m twine upload -r testpypi dist/* -u ikamensh
File structure:
./
./plots/__init__.py
./setup.py
./upload.sh
Let's me point you to my previous answer: https://stackoverflow.com/a/54599368/7976758
ilya_ezplots is the name of your distribution but it's not importable name. The importable name after installation is plots. So:
import plots
As #phd answered, you should import: import plots.
or you should change you directory structure to:
./ilya_ezplots/__init__.py
And then import ilya_ezplots can do the trick.
The name of the import should match the parent directory.
Refer this post to maintain the correct structure and for good practices.
The good practice is maintain some namespace or same directory name as your package name, so that the user won't get confused.
if your project name is xyz then you should maintain your directory name as xyz. So, the use can do import xyz to import your package and won't get confused with the other package. For eg. if you use yyy as your directory name (and your package name is xyz) then users might get confused and install package yyy instead.
One real life example:
package beautifulsoup needs import bs4 to import it and hence they maintain bs4 package (dummy package) so that no one can create the package with the name bs4 as that might become a security risk.
I am using python 2.7.5. I created a simple python project pythontest. The folder structure of my project is as follows:
pythontest
|
-setup.py
-README.md
-src
|
-- pack1
|
test1.py
test2.py
|
--pack2
|
test3.py
My files are like this:
test1.py
print "hello"
test2.py
import test1
import src.pack2.test3
print "hai"
test3.py
print "how are you?"
I did run the like this :
sudo python setup.py install
python src/pack1/test2.py
But i am getting the following error:
hello
Traceback (most recent call last):
File "src/pack1/test2.py", line 7, in <module>
import src.pack2.test3
ImportError: No module named pack2.test3
But i can run the code from eclipse. What is wrong with my code?
setup.py file is as follows:
import os
from setuptools import setup
def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()
setup(
name = "pythontest",
version = "0.0.4",
author = "Andrew Carter",
author_email = "andrewjcarter#gmail.com",
description = ("An demonstration of how to create, document, and publish "
"to the cheese shop a5 pypi.org."),
license = "BSD",
keywords = "example documentation tutorial",
url = "http://packages.python.org/an_example_pypi_project",
packages=[],
long_description=read('README.md'),
classifiers=[
"Development Status :: 3 - Alpha",
"Topic :: Utilities",
"License :: OSI Approved :: BSD License",
],
)
You need to place __init__.py-files into packages (in your case, src, pack1 and pack2) for this to work. Also, you should get rid of src as package-name (it's generic and makes no sense), but instead use something pertaining to your actual codebase, e.g. supercoolapp or amazinglib or whatever you are writing on.
Then you need to add that top-level package to your packages-list in the setup.py.
I've just registered my new package with PIP
python setup.py register
python setup.py sdist upload
And I wanted to install it on other machine globally (i.e. no virtualenv) using 'PIP':
sudo pip install standardiser
This all went fine, since I have one file registered as a script:
setup(
...
scripts=['standardiser/bin/standardiser.py'],
)
'standariser.py' is now available as CLI command system wide. But if I execute it, I'm getting:
mnowotka#candela:~/Documents/ci/curation_interface/trunk/src$ standardiser.py
Traceback (most recent call last):
File "/usr/local/bin/standardiser.py", line 32, in <module>
from standardiser import standardise, SDF
File "/usr/local/bin/standardiser.py", line 32, in <module>
from standardiser import standardise, SDF
ImportError: cannot import name standardise
I get the same when I explicitly call python:
python /usr/local/bin/standardiser.py
But if I copy this to some local folder:
sudo cp python /usr/local/bin/standardiser.py bla.py
And run it from there:
mnowotka#candela:~$ python bla.py
usage: bla.py [-h] [-V] [-r] infile
bla.py: error: too few arguments
I don't have any ImportErors. What I'm doing wrong? Can you help me?
My setyp.py:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
__author__ = 'mnowotka'
import sys
try:
from setuptools import setup
except ImportError:
from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup
setup(
name='standardiser',
version='0.1.4',
author='Francis Atkinson',
author_email='francis#ebi.ac.uk',
description='Provides a simple way of standardising molecules as a prelude to e.g. molecular modelling exercises.',
url='https://www.ebi.ac.uk/chembldb/index.php/ws',
license='Apache License, Version 2.0',
scripts=['standardiser/bin/standardiser.py'],
packages=['standardiser'],
long_description=open('ReadMe.txt').read(),
package_data={
'standardiser': ['bin/*', 'data/*', 'docs/*', 'knime/*', 'test/*',],
},
classifiers=['Development Status :: 2 - Pre-Alpha',
'Environment :: Console',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Topic :: Scientific/Engineering :: Chemistry'],
zip_safe=False,
)
You are asking for a script to be installed named standardiser.py; that script (at least, the one you've uploaded to the cheese shop) contains the line:
from standardiser import standardise, SDF
But thats an ambigious import; the script you're executing, /usr/local/bin/standardise.py appears in sys.path, because the main script is located there. it's importing itself!
you should be using the console_scripts feature of setuptools anyway.
alter your script file from
#! /guess/path/to python
from standardise import import *
do_things()
do_more_things()
to
from __future__ import absolute_import
from standardise import import *
def main():
do_things()
do_more_things()
if __name__ == '__main__':
main()
Which is to say;
get rid of the shebang; you never need it in python!
use the absolute_import feature to get the module named foo.bar.foo to be able to import foo instead of only foo.bar.foo (you can still import foo.bar.foo as either from foo.bar import foo or import .foo). __future__ imports must appear first in the source file, before any other non-comment lines (including other imports)
most importantly; wrap import time side effects in a function, and only invoke those side effects if this is the "main script"
then change your setup.py around from
setup(
scripts=['standardiser/bin/standardiser.py'],
...)
to
setup(
entry_points={
'console_scripts': [
'standardiser=standardiser.bin.standardiser:main']},
...)
which is to say:
use an entry point; setuptools knows how to get your installed package on sys.path correctly in this case, and it knows how to connect to the proper python interpreter, the one that was used to run setup.py. This matters when there are multiple versions of python, or when running in a virtualenv. this is why you never need a shebang.
remove the '.py' from the installed executable name. this shouldn't ever be there for scripts that are to be on your executable path.