Develop Reference

Pylint disagrees with VSCode and python in imports

I am not finding the way to properly code so that both pylint and the execution of the code (within VSCode or from the command line) would work.
There are some similar questions but none seems to apply to my project structure with a src directory under which there will be multiple packages. Here's the simplified project structure:
.
├── README.md
├── src
│   ├── rssita
│   │   ├── __init__.py
│   │   ├── feeds.py
│   │   ├── rssita.py
│   │   └── termcolors.py
│   └── zanotherpackage
│   ├── __init__.py
│   └── anothermodule.py
└── tests
├── __init__.py
└── test_feeds.py
From what I understand rssita is one of my my packages (because of the init.py file) with some modules under it amongst which rssita.py file contains the following imports:
from feeds import RSS_FEEDS
from termcolors import PC
The rssita.py code as shown above runs well from both within VSCode and from command line python ( python src/rssita/rssita.py ) from the project root, but at the same time pylint (both from within VSCode and from the command line (pylint src or pylint src/rssita)) flags the two imports as not found.
If I modify the code as follows:
from rssita.feeds import RSS_FEEDS
from rssita.termcolors import PC
pylint will then be happy but the code will not run anymore since it would not find the imports.
What's the cleanest fix for this?

As far as I'm concerned pylinty is right, your setup / PYTHONPATH is screwed up: in Python 3, all imports are absolute by default, so
from feeds import RSS_FEEDS
from termcolors import PC
should look for top-level packages called feeds and termcolors which I don't think exist.
python src/rssita/rssita.py
That really ain't the correct invocation, it's going to setup a really weird PYTHONPATH in order to run a random script.
The correct imports should be package-relative:
from .feeds import RSS_FEEDS
from .termcolors import PC
Furthermore if you intend to run a package, that should be either a runnable package using __main__:
python -m rssita
or you should run the sub-package as a module:
python -m rssita.rssita
Because you're using an src-package, you'll either need to create a pyproject.toml so you can use an editable install, or you'll have to PYTHONPATH=src before you run the command. This ensures the packages are visible at the top-level of the PYTHONPATH, and thus correctly importable. Though I'm not a specialist in the interaction of src layouts & runnable packages, so there may be better solutions.

Python executable script ModuleNotFound

I am trying to better understand importing modules. I read about how to do this from here https://stackoverflow.com/a/14132912/14179793 and I can get it to work using solution 1. There is an additional variable that I need to figure out though.
This is a dummy project I am testing with:
.
├── a_package
│   ├── __init__.py
│   └── lib_a.py
├── b_package
│   ├── __init__.py
│   └── test_script.py
├── main.py
└── src
└── src_lib
└── src_lib.py
With this setup I can do:
python -m b_package.test_script
this is lib a function
This is src_lib_function.
test_script.py:
from a_package.lib_a import lib_a_function
from src.src_lib.src_lib import src_lib_function
if __name__ == '__main__':
lib_a_function()
src_lib_function()
pass
The goal is to make b_package/test_script.py executable without using python test_script ie ./test_script
However, adding the shebang at the top #!/usr/bin/env python causes an import error:
$ ./b_package/test_script.py
Traceback (most recent call last):
File "./b_package/test_script.py", line 2, in <module>
from a_package.lib_a import lib_a_function
ModuleNotFoundError: No module named 'a_package'
I assume it is because python is not loading it as a module based off the above mentioned question but I am not sure how to resolve this.

I ended up using setuptools as suggested by kosciej16 to achieve the desired results.
New project structure:
.
├── a_package
│   ├── __init__.py
│   └── lib_a.py
├── b_package
│   ├── __init__.py
│   └── test_script.py
├── main.py
├── pyproject.toml
├── setup.cfg
└── src
├── __init__.py
└── src_lib
├── __init__.py
└── src_lib.py
setup.cfg:
[metadata]
name = cli_test
version = 0.0.1
[options]
packages = find:
[options.entry_points]
console_scripts =
test_script = b_package.test_script:main
This allows the user to clone the repo and run pip install . from the top level then they can execute the script by just typing test_script

How can I fix "bad interpreter" error after renaming my django project in PyCharm?

I recently renamed my Django project in Pycharm and now I have many errors associated with Python. From my MAC terminal when I try and run:
$ django-admin startproject mobileproject
or
$ pip install django
I get:
-bash: /usr/local/bin/pip: /usr/local/opt/python/bin/python3.7: bad interpreter: No such file or directory
In Pycharm when I open most of my Django projects I now get:
invalid python interpreter selected for the project
or when i try to install pip:
$ sudo easy_install pip
i get:
sudo: unable to execute /usr/local/bin/easy_install: No such file or directory
Below was the original structure of my project:
---project
---project
---app
---models.py
...etc
---project
---settings.py
...etc
---manage.py
---venv
To rename my Django project in Pycharm I right clicked on the root folder "project" and selected refactor/rename/rename project. For the rest of the folders I right clicked and selected refactor/rename and I was only given rename directory so I chose that. Below shows how I renamed my project folders.
---newproject
---newp
---newapp
---models.py
...etc
---newp
---settings.py
...etc
---manage.py
---venv
After doing this I have all the errors listed above. I think choosing "rename project" instead of "rename directory" messed things up. But why do I have problems outside of Pycharm when trying to start a new project with django-admin startproject or tell pip to install django? I feel like the Python executable is in the wrong directory but I have no idea. Please Help!

The Problem
Unfortunately PyCharm does not update the venv path containing the Python interpreter when renaming the project/ directory, hence it cannot find the Python interpreter. There is an issue for this on YouTrack too.
Solution
You have to update the path to the virtual environment manually. To do this open the Interpreter settings with Ctrl+Alt+S | Project <project name> | Python Interpreter (see JetBrains Docs), click on Settings | Show All.... Then select the venv for your application marked [invalid] and click Edit. In created modal window you may now update the Interpreter Path by replacing the previous directory name with the new one and PyCharm will work like a charm (sorry for the dad joke) again.
Walkthrough for a simple application
Given a project pythonProject with a venv pythonProject/venv containing the Python interpreter as follows:
pythonProject/
├── src
│   └── app.py
└── venv
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate.ps1
│   ├── activate_this.py
│   ├── activate.xsh
│   ├── pip
│   ├── pip3
│   ├── pip-3.9
│   ├── pip3.9
│   ├── python -> /usr/bin/python3.9
│   ├── python3 -> python
│   ├── python3.9 -> python
│   ├── wheel
│   ├── wheel3
│   ├── wheel-3.9
│   └── wheel3.9
├── lib
│   └── python3.9
└── pyvenv.cfg
Now when we check the Python interpreter settings with Ctrl+Alt+S | Project pythonProject | Python Interpreter (see JetBrains Docs) we can see that the interpreter path is set to path/to/project/pythonProject/venv/bin/python.
When you now rename the directory (and project) to renamedProject using PyCharm`s Refactor | Rename option the file structure will look like this:
renamedProject/
├── src
│   └── app.py
└── venv
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate.ps1
│   ├── activate_this.py
│   ├── activate.xsh
│   ├── pip
│   ├── pip3
│   ├── pip-3.9
│   ├── pip3.9
│   ├── python -> /usr/bin/python3.9
│   ├── python3 -> python
│   ├── python3.9 -> python
│   ├── wheel
│   ├── wheel3
│   ├── wheel-3.9
│   └── wheel3.9
├── lib
│   └── python3.9
└── pyvenv.cfg
Checking the Python interpreter settings again will show you that the interpreter is invalid now, that is because the path still is path/to/project/pythonProject/venv/bin/python but as we renamed the directory the pythonProject directory does no longer exist hence this path is in fact invalid (See file tree above).
To resolve this click on Settings | Show All... (the cogwheel symbol next to the dropdown menu) within the Python interpreter settings. In the created dialog window select your venv i. e. [invalid] Python 3.9 (pythonProject) in this example and click Edit (the pen symbol on top of the listing).
There is another window popping up where you can now change the Intepreter Path. Replace pythonProject with renamedProject in the path and accept your changes and you will see that PyCharm immediately recognizes the venv as valid again.

Python no module named X - absolute imports

Before I even start I want to say - I know, that there are like bazillion questions similar to this, but I couldn't find the answer to my problem. I have a dir structure like this:
.
├── project
│   ├── A
│   │   ├── __init__.py
│   │   └── somelib.py
│   ├── B
│   ├── C
│   │   └── C
│   │   ├── foo.py
│   │   └── __init__.py
│   └── __init__.py
└── run.sh
run.sh:
python3 project/C/C/foo.py
foo.py:
from project.A.somelib import somefunc
VS Code actually gets the intellisense in foo.py - it tells me what funcitons/variables I can import from somelib. But when I run run.sh, I get this error message:
from project.A.somelib import somefunc
ModuleNotFoundError: No module named 'project'
Is there a way to solve this while preserving this directory structure?
adding project/__init__.py changed nothing
the sys.path in foo.py looks like this:
['/home/dabljues/projects/project/project/C/C', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3.7/site-packages']
restrictions:
I can't modify neither sys.path in the files nor PYTHONPATH before running the scripts
I can't pip-install anything
I don't have sudo-access
I can't create a virtualenv, since the scripts are supposed to be downloadable and quickly executable

IDEs like VSCode or Pycharm make their own assumptions about a project, and will usually correctly link modules even if the interpreter that will ultimately run the code can't.
The reason why project.A.somelib can't be found is visible in your sys.path output, which gives you the places where python will search for modules. Since '/home/dabljues/projects/project/project' is not included, there is no way for python to resolve it during runtime.
A quick hack
You can just add the path manually to sys.path, either in the source file by running import sys; sys.insert(0, '/home/dabljues/projects/project/project/') in foo.py before any other imports happen, or by running export PYTHONPATH="${PYTHONPATH}:/home/dabljues/projects/project/project/" in your shell before run.sh.
Installing the project
Since it looks like you're developing a library, you might as well use the mechanisms python offers to make libraries shareable and thereby fixing any import issues. Add a minimal setup.py to the project root (i.e. /home/dabljues/projects/project/project/setup.py):
from setuptools import setup, find_packages
setup(
name='project',
version='0.1.0',
packages=find_packages('project'),
)
And install your project in editable mode:
$ python3 -m pip install -e .
This will put a link in your python3 executable's site-packages that points to the project root, which makes it accessible whenever you run anything with python3.
Tests
I included print(__name__) at the top of all python files to get some output.
running run.sh without installing the package:
$ sh run.sh
Traceback (most recent call last):
File "project/C/C/foo.py", line 1, in <module>
from project.A.somelib import somefunc
ModuleNotFoundError: No module named 'project'
after installing it
$ sh run.sh
__main__
project.A.somelib
As you can see, project.C.C.foo is executed as a script, yet it finds all imports that start with project because project is installed.

Run python in package mode helps.
1) Add __init__.py for every path:
.
├── project
│ ├── A
│ │ ├── __init__.py
│ │ └── somelib.py
│ ├── B
│ ├── C
│ │ ├── __init__.py
│ │ └── C
│ │ ├── foo.py
│ │ └── __init__.py
│ └── __init__.py
└── run.sh
2) Import module with relative path in foo.py:
from ...A.somelib import somefunc
3) Run python in package mode:
python -m project.C.C.foo
It works for me.

I'm unable to reproduce this given your code (assuming run.sh just starts a script).
Are you sure it's not a case of e.g. circular imports?
$ mkdir -p project/A project/C/C
$ cat > project/C/C/foo.py
print('moof!')
$ cat > project/A/somelib.py
print('beef!')
$ cat > script.py
import project.A.somelib
import project.C.C.foo
$ tree
.
├── project
│   ├── A
│   │   └── somelib.py
│   └── C
│   └── C
│   └── foo.py
└── script.py
$ python3 script.py
beef!
moof!

Change your run.sh script to run foo as a module. python3 -m proj.C.C.foo

Python - No module named foo

I'm new to python and I am trying to write a simple program for an led cube. First I need to execute one of the other sample programs provided, but I'm getting the error "No module named cube_interface" when I try and run a python program.
Now before you mark this question as duplicate, and refer me to the 7,776 similar questions stack overflow, let me tell disclose that I've already tried implementing the solutions provided on 14 other SO questions, and followed 3 blogs covering how to fix this error, with no luck.
OS: Mac OsX 10.8.1
Python: 2.7.5
Repo: https://github.com/chadharrington/all_spark_cube
The example program is src/tetris.py
The module is src/all_spark_cube_client
├── Makefile
└── src
├── LICENSE.txt
├── all_spark_cube_client
│   ├── __init__.py
│   └── __init__.pyc
├── all_spark_cube_client.egg-info
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   ├── dependency_links.txt
│   ├── not-zip-safe
│   ├── requires.txt
│   └── top_level.txt
├── build
│   ├── bdist.macosx-10.9-intel
│   └── lib
│   └── all_spark_cube_client
│   └── __init__.py
├── client_demo.py
├── colors.py
├── dist
│   └── all_spark_cube_client-0.1-py2.7.egg
├── load_test.py
├── setup.py
├── supervisord.conf
├── supervisord_init_script.debian
├── supervisord_init_script.redhat
└── tetris.py
When I try and run the tetris.py program, I get an error "No module named.."
python tetris.py
Traceback (most recent call last):
File "tetris.py", line 5, in <module>
from all_spark_cube_client import CubeClient
File "/Users/sowen/Code/all_spark_cube/software/clients/python_client/src/all_spark_cube_client/__init__.py", line 8, in <module>
from cube_interface import CubeInterface
ImportError: No module named cube_interface
Many of the answers suggest adding pwd . /Library/Python ect.. to the PYTHONPATH. I've tried implementing all of them independently, and even combined all suggestions into one frakenstein path as shown, with no luck.
cd ~/Code/all_spark_cube/software/clients/python_client
export PYTHONPATH=$PATH:$PYTHONPATH:`pwd`:`pwd`/src:.:/Library/Python/2.7/site-packages/
I've tried building the python module
$sudo python setup.py build
running build
running build_py
file all_spark_cube_client.py (for module all_spark_cube_client) not found
file all_spark_cube_client.py (for module all_spark_cube_client) not found
I've tried installing the python module (It doesn't give any errors, but I still am unable to run tetris.py)
cd ~/Code/all_spark_cube/software/clients/python_client/src/
sudo python setup.py install
How can I execute the tetris.py program?
Additional Resources
http://docs.python.org/2/using/mac.html
http://www.confusedcoders.com/random/python-module-importerror-no-module-named-pocketsphinx

You forgot to build cube_interface:
See: https://github.com/chadharrington/all_spark_cube/tree/master/software/thrift
You need to build this with the gen_py tool.
i.e: (I assume:):
cd /path/to/all_spark_cube/software/thrift/
make
cp cube_interface.py /path/to/python/site-packages