Develop Reference

How to build a Python project for a specific version of Python?

I have an app that I would like to deploy to AWS Lambda and for this reason it has to have Python 3.9.
I have the following in the pyproject.toml:
name = "app"
readme = "README.md"
requires-python = "<=3.9"
version = "0.5.4"
If I try to pip install all the dependencies I get the following error:
ERROR: Package 'app' requires a different Python: 3.11.1 not in '<=3.9'
Is there a way to specify the Python version for this module?
I see there is a lot of confusion about this. I simply want to specify 3.9 "globally" for my build. So when I build the layer for the lambda with the following command it runs:
pip install . -t pyhon/
Right now it has only Python 3.11 packaged. For example:
❯ ls -larth python/ | grep sip
siphash24.cpython-311-darwin.so
When I try to use the layer created this way it fails to load the required library.

There are multiple ways of solving this.
Option 1 (using pip's built in facilities to restrict Python version)
pip install . \
--python-version "3.9" \
--platform "manylinux2010" \
--only-binary=:all: -t python/
Another way of solving this is with Docker:
FROM python:3.9.16-bullseye
RUN useradd -m -u 5000 app || :
RUN mkdir -p /opt/app
RUN chown app /opt/app
USER app
WORKDIR /opt/app
RUN python -m venv venv
ENV PATH="/opt/app/venv/bin:$PATH"
RUN pip install pip --upgrade
RUN mkdir app
RUN touch app/__init__.py
COPY pyproject.toml README.md ./
RUN pip install . -t python/
This way there is no change to create a layer for AWS Lambda that is newer than Python 3.9.

Python on Gitlab has ModuleNotFound, But Not When I Run Locally

The following snapshot shows the file structure:
When I run on Gitlab CI, here is what I am seeing:
Why is this error occurring when Gitlab runs it but not when I run locally?
Here is my .gitlab-ci.yml file.
Note that this had been working before.
I recently made win_perf_counters a Git submodule instead of being an actual subdirectory. (Again, it works locally.)
test:
before_script:
- python -V
- pip install virtualenv
- virtualenv venv
- .\venv\Scripts\activate.ps1
- refreshenv
script:
- python -V
- echo "******* installing pip ***********"
- python -m pip install --upgrade pip
- echo "******* installing locust ********"
- python -m pip install locust
- locust -V
- python -m pip install multipledispatch
- python -m pip install pycryptodome
- python -m pip install pandas
- python -m pip install wmi
- python -m pip install pywin32
- python -m pip install influxdb_client
- set LOAD_TEST_CONF=load_test.conf
- echo "**** about to run locust ******"
- locust -f ./src/main.py --host $TARGET_HOST -u $USERS -t $TEST_DURATION -r $RAMPUP -s 1800 --headless --csv=./LoadTestsData_VPOS --csv-full-history --html=./LoadTestsReport_VPOS.html --stream-file ./data/stream_jsons/streams_vpos.json --database=csv
- Start-Sleep -s $SLEEP_TIME
variables:
LOAD_TEST_CONF: load_test.conf
PYTHON_VERSION: 3.8.0
TARGET_HOST: http://10.10.10.184:9000
tags:
- win2019
artifacts:
paths:
- ./LoadTests*
- public
only:
- schedules
after_script:
- ls src -r
- mkdir .public
- cp -r ./LoadTests* .public
- cp metrics.csv .public -ErrorAction SilentlyContinue
- mv .public public
When I tried with changing the Gitlab CI file to use requirements.txt:

Probably the python libraries you are using in your local environment are not the same you are using in gitlab. Run a pip list or pip freeze in your local machine and see which versions do you have there. Then pip install those in your gitlab script. A good practice is to have a requirements.txt or a setup.py file with specific versions rather than pulling the latest versions every time.

Probably the module you are developing doesn't have the __init__.py file and thus it cannot be found when imported from the external.

zsh: abort python error when I try to run the app in venv

I setup Python 3.6 using pyenv so I could manage multiple Python versions (e.g. 3.7 and 3.8) in the future. I didn't use Homebrew to install Python since it changes the system version. It's my first time to use zsh shell since it's the default shell in Catalina OS. Currently, I use 3.6.8 version for my existing project.
So here's my current setup:
% pyenv versions
result:
system
* 3.6.8 (set by /Users/macbook/.python-version)
3.7.3
% python -V results to Python 3.6.8
% which python results to /Users/macbook/.pyenv/shims/python
% echo $PATH results to /Users/macbook/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
The content of my ~/.zshrc is PATH=$(pyenv root)/shims:$PATH
I created a virtual env using % python -m venv venv, installed all the necessary packages, and when I activate it and get the python path,
(venv) % python -V
Python 3.6.8
(venv) % which python
/Users/macbook/python-project/venv/bin/python
(venv) % echo $PATH
/Users/macbook/python-project/venv/bin:/Users/macbook/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
Finally, when I try to run the app, I always get zsh: abort error:
(venv) % python app.py
zsh: abort python app.py
(venv) % export FLASK_APP=app.py
(venv) % flask run
zsh: abort flask run
I don't know what else is still missing or are there anything wrong with my python path?
Thanks!

After searching through the web, I think this is a common issue with the latest MacOS or Homebrew. This thread fixed the issue.
brew update && brew upgrade && brew install openssl
copy the two files from /usr/local/Cellar/openssl#1.1/1.1.1g to /usr/local/lib/
cd /usr/local/Cellar/openssl#1.1/1.1.1g/
sudo cp libssl.1.1.1.dylib libcrypto.1.1.1.dylib /usr/local/lib/
add symlink to missing openssl libs
cd /usr/local/lib
sudo ln -s libssl.1.1.1.dylib libssl.dylib
sudo ln -s libcrypto.1.1.1.dylib libcrypto.dylib

For me, the below worked:
Python 3.6.9
MacOs Catalina 10.15.7
cd /usr/local/Cellar/openssl#1.1/1.1.1h/
cp lib/libssl.1.1.dylib lib/libcrypto.1.1.dylib /usr/local/lib
cd /usr/local/lib
sudo ln -s libssl.1.1.dylib libssl.dylib
sudo ln -s libcrypto.1.1.dylib libcrypto.dylib
Thanks to the answer by Zhanrah

Travis-ci installs packages from requirements.txt file automatically

I have Django application based on docker-compose file.
Somehow travis autoinstalls packages from requirements.txt in project repo and its failing my build cause I don't have gcc package.
I want to run all actions (tests, linters) in docker container not directly in project repo.
Here is my travis-ci.yml file:
---
dist: xenial
services:
- docker
language: python
python:
- "3.7"
script:
- docker compose up --build
- docker exec web flake8
- docker exec web mypy my_project
- docker exec web safety check -r requirements.txt
- docker exec web python -m pytest --cov my_project -vvv -s
And begin of travis log:
$ git checkout -qf bab09dee57a707a5cd0a353d6f50bb66fd90a095
0.01s$ source ~/virtualenv/python3.7/bin/activate
$ python --version
Python 3.7.1
$ pip --version
pip 19.0.3 from /home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/pip (python 3.7)
$ pip install -r requirements.txt
...
py_GeoIP.c:23:19: fatal error: GeoIP.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
...
Do you have any idea why travis behaves like this?

According to https://docs.travis-ci.com/user/languages/python/#dependency-management travisci automatically install requirements.txt dependencies.
To ommit this behaviour I had to add following line to travis.yml to overwrite it:
install: pip --version

you can also use: install: skip

A strange virtualenv issue-- even with -p it installs multiple pythons

I'm trying to create a virtual environment with 3.4. I have followed instructions found through searching stackoverflow, as well as posting this issue on reddit, and continue to have the same problem: even when I specify python3.4, it installs both 3.4 AND 2.7. This then leads to all of the problems I was trying to avoid by using a virtual environment in the first place.
$ which python3
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3
$ mkdir test
$ cd test
$ virtualenv -p /Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4 venv
(various output saying it's creating venv/bin/python with 3.4)
$ source venv/bin/activate
(venv)$ python --version
Python 2.7.9
when I look in the venv/bin folder it has binaries for both 2.7.9 and 3.4.
after activating the environment:
$ echo $PATH
/Users/theinevitable/Documents/python/test/venv/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Postgres.app/Contents/Versions/9.4/bin
$ which python
/Users/theinevitable/Documents/python/test/venv/bin/python
$ which python3
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3
one person on reddit suggested trying to use venv instead:
$ python3.4 -m venv my_venv
/Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4: No module named venv
$ pyvenv tester
File "/Library/Frameworks/Python.framework/Versions/3.4/bin/pyvenv", line 10
print('Error: %s' % e, file=sys.stderr)
^
SyntaxError: invalid syntax