Develop Reference

How can I fix "bad interpreter" error after renaming my django project in PyCharm?

I recently renamed my Django project in Pycharm and now I have many errors associated with Python. From my MAC terminal when I try and run:
$ django-admin startproject mobileproject
or
$ pip install django
I get:
-bash: /usr/local/bin/pip: /usr/local/opt/python/bin/python3.7: bad interpreter: No such file or directory
In Pycharm when I open most of my Django projects I now get:
invalid python interpreter selected for the project
or when i try to install pip:
$ sudo easy_install pip
i get:
sudo: unable to execute /usr/local/bin/easy_install: No such file or directory
Below was the original structure of my project:
---project
---project
---app
---models.py
...etc
---project
---settings.py
...etc
---manage.py
---venv
To rename my Django project in Pycharm I right clicked on the root folder "project" and selected refactor/rename/rename project. For the rest of the folders I right clicked and selected refactor/rename and I was only given rename directory so I chose that. Below shows how I renamed my project folders.
---newproject
---newp
---newapp
---models.py
...etc
---newp
---settings.py
...etc
---manage.py
---venv
After doing this I have all the errors listed above. I think choosing "rename project" instead of "rename directory" messed things up. But why do I have problems outside of Pycharm when trying to start a new project with django-admin startproject or tell pip to install django? I feel like the Python executable is in the wrong directory but I have no idea. Please Help!

The Problem
Unfortunately PyCharm does not update the venv path containing the Python interpreter when renaming the project/ directory, hence it cannot find the Python interpreter. There is an issue for this on YouTrack too.
Solution
You have to update the path to the virtual environment manually. To do this open the Interpreter settings with Ctrl+Alt+S | Project <project name> | Python Interpreter (see JetBrains Docs), click on Settings | Show All.... Then select the venv for your application marked [invalid] and click Edit. In created modal window you may now update the Interpreter Path by replacing the previous directory name with the new one and PyCharm will work like a charm (sorry for the dad joke) again.
Walkthrough for a simple application
Given a project pythonProject with a venv pythonProject/venv containing the Python interpreter as follows:
pythonProject/
├── src
│   └── app.py
└── venv
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate.ps1
│   ├── activate_this.py
│   ├── activate.xsh
│   ├── pip
│   ├── pip3
│   ├── pip-3.9
│   ├── pip3.9
│   ├── python -> /usr/bin/python3.9
│   ├── python3 -> python
│   ├── python3.9 -> python
│   ├── wheel
│   ├── wheel3
│   ├── wheel-3.9
│   └── wheel3.9
├── lib
│   └── python3.9
└── pyvenv.cfg
Now when we check the Python interpreter settings with Ctrl+Alt+S | Project pythonProject | Python Interpreter (see JetBrains Docs) we can see that the interpreter path is set to path/to/project/pythonProject/venv/bin/python.
When you now rename the directory (and project) to renamedProject using PyCharm`s Refactor | Rename option the file structure will look like this:
renamedProject/
├── src
│   └── app.py
└── venv
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate.ps1
│   ├── activate_this.py
│   ├── activate.xsh
│   ├── pip
│   ├── pip3
│   ├── pip-3.9
│   ├── pip3.9
│   ├── python -> /usr/bin/python3.9
│   ├── python3 -> python
│   ├── python3.9 -> python
│   ├── wheel
│   ├── wheel3
│   ├── wheel-3.9
│   └── wheel3.9
├── lib
│   └── python3.9
└── pyvenv.cfg
Checking the Python interpreter settings again will show you that the interpreter is invalid now, that is because the path still is path/to/project/pythonProject/venv/bin/python but as we renamed the directory the pythonProject directory does no longer exist hence this path is in fact invalid (See file tree above).
To resolve this click on Settings | Show All... (the cogwheel symbol next to the dropdown menu) within the Python interpreter settings. In the created dialog window select your venv i. e. [invalid] Python 3.9 (pythonProject) in this example and click Edit (the pen symbol on top of the listing).
There is another window popping up where you can now change the Intepreter Path. Replace pythonProject with renamedProject in the path and accept your changes and you will see that PyCharm immediately recognizes the venv as valid again.

Python how to import local module?

I have the following directory structure in Ubuntu. I'm trying to import the module config from my local package my_app into my script my_app_script.py
$ tree
.
├── my_app/
│   ├── config.py
│   ├── __init__.py
│   └── test/
├── my_app-info # created by pip install -e .
│   ├── dependency_links.txt
│   ├── PKG-INFO
│   ├── requires.txt
│   ├── SOURCES.txt
│   └── top_level.txt
├── bin/
│   └── my_app_script.py
├── LICENSE
├── README.md
└── setup.py
# setup.py
setup(
name='my_app',
version='0.1.2',
description='',
url='',
packages=['my_app'],
scripts=['bin/my_app_script.py'],
install_requires=[],
python_requires='>=3.6',
)
# my_app_script.py
from my_app import config
When I run my_app_script.py it results in "ImportError: cannot import name 'config'
What am I doing wrong?
Edit:
I am trying to follow this guide on packaging a program.

You need an __init__.py file in the parent directory as well as in bin directory.

You can use either of below approaches.
The first approach seems best to me as the script will always set the path relative to it, and will also work if you clone your repos.
Add an __init__.py in parent directory(Next to setup.py).
And add below line in my_app_script.py
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir, "my_app")))
What this will do is add .../my_app to PYTHONPATH at runtime, when my_app_script.py is executed.
Add an env.sh in parent directory. Below should be the contents of env.sh.
export PYTHONPATH=$(pwd):$PYTHONPATH
Now cd int the directory where env.sh is kept and then source it
source env.sh
Now run your my_app_script.py
python bin/my_app_script.py
Set PYTHONPATH from commandline.
PYTHONPATH=$(pwd) python bin/my_app_script.py

Building pipenv with requirements file on Docker

I am having a big trouble building my dockerised version of flask app.
At first I am not able to install any dependencies from requirements.txt file which is provided inside the container itself. Here is a folder structure:
.
├── app
│   ├── ASRModule.py
│   ├── auth.py
│   ├── config
│   ├── files
│   ├── index.py
│   ├── __init__.py
│   ├── Interval.py
│   ├── MySQLDBHandler.py
│   ├── __pycache__
│   ├── SIPCall.py
│   ├── SOAPClient.py
│   ├── static
│   ├── stats.py
│   ├── templates
│   ├── TrunkOccupation.py
│   └── TrunkTraffic.py
├── Dockerfile
├── instance
└── requirements.txt
And my Dockerfile on which I want to build a containter:
FROM python:3.5.2-alpine
COPY . /flask
WORKDIR /flask
RUN pip install --upgrade pip
RUN pip install pipenv
CMD ["pipenv", "shell", "testshell"]
CMD ["pipenv","install", "-r ./requirements.txt"]
To my understanding, after completed build I should have same folder structure except one above directory called flask which will hold all aforementioned files and dirs. I also should have a virtualenv called testshell to which all dependencies from requirements.txt should be installed. And up until now everything works silky-smooth-fine.
To my disappointment, though, after I try to run this container I see a correctly build virtual-env and such error:
Requirements file doesn't appear to exist. Please ensure the file exists in your project directory or you provided the correct path.
I tried various paths for the requirements file but nothing helped.
I would appreciate any help which will point me, where I am making mistake.

The error is rather simple, but made hard to notice by the bad error message - it does not say which file it is trying to load. The file it tries to load is '/flask/ ./requirements.txt', i.e. requirements.txt in a subdirectory named space. - c.f. with the error message from pip:
% "pip" "install" "-r ./requirements.txt"
Could not open requirements file:
[Errno 2] No such file or directory: ' ./requirements.txt'
The fix is to either remove the space, or correctly split the arguments:
CMD ["pipenv", "install", "-r./requirements.txt"]
or
CMD ["pipenv", "install", "-r", "./requirements.txt"]
both ought to work.
I suggest that you will complain about the bad error message to the pipenv issue tracker.

pytest-django could not find a Django project

Trying to configure pytest with django, the project already has a lot of test not written with pytest (written with unittest) but I am trying to get them run with pytest so I can write pytest tests and get it work with old tests.
I know pytest-django checks for the manage.py file in the root dir of a django project but this project the manage.py file is not in the root dir so I guess that's why the error below is thrown when I run pytest however running pytest and supplying a particular file works. How do I specify where manage.py is? As I can't find this in the documentation
pytest-django could not find a Django project (no manage.py file could be found).
You must explicitly add your Django project to the Python path to have it picked up.

you can define a python path to python commands that you want to run:
PYTHONPATH=/your/path/to/your/django/project/ pytest
or export your pythonpath before you run the pytest command:
export PYTHONPATH=/your/path/to/your/django/project/
pytest

As a standard practice, you should add a setup.cfg file to your root, with the following block -
[tool:pytest]
DJANGO_SETTINGS_MODULE=<package_name>.settings.py
You can later use the same file for linters by adding specific blocks for them.

In my case, I created a configure file named pytest.ini in the tests folder with the following content:
[pytest]
pythonpath = ../bitpin
DJANGO_SETTINGS_MODULE = bitpin.settings
python_files = tests.py test_*.py *_tests.py
Repository tree:
bitpin-repo
├── bitpin
│   ├── bitpin
│   │   ├── asgi.py
│   │   ├── __init__.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   ├── __init__.py
│   └── manage.py
├── tests
│ ├── test_models.py
│ ├── pytest.ini
│ └── conftest.py
└── setup.py
[NOTE]
If the tests/ is next to the project, replace bitpin with .:
[pytest]
pythonpath = .

How to run my python program with -m (full path)?

I have a structure similar to this:
/home/user/nginx_http/
├── nginx_http
│   ├── common
│   │   ├── autodiscovery.py
│   │   ├── __init__.py
│   ├── __init__.py
│   ├── __main__.py
│   ├── nginx_http.py
├── README.rst
└── setup.py
When I'm in the /home/user/nginx_http folder and I run this:
python -m nginx_http
it works fine. But soon as I try to run it with full path it gives me an error;
python -m /home/user/nginx_http/nginx_http
/bin/python: Import by filename is not supported.; '/home/user/nginx_http/nginx_http' is a package and cannot be directly executed
I want to run my python program as a cronjob which would require the full path. The two options I wanted clarification on:
Should I consider moving my python program into python's module folder?
/usr/lib/python2.7/site-packages
Or should I set PYTHONPATH env to the location of /home/user/nginx_http. Will that break anything as far as other modules are concerned?
Thanks.