Develop Reference

Why does 'pipenv lock -r' errors with 'No such option: -r'?

With this minimal Dockerfile
FROM public.ecr.aws/lambda/python:3.8
RUN pipenv lock -r > requirements.txt
The pipenv command fails with the error message:
Usage: pipenv lock [OPTIONS]
Try 'pipenv lock -h' for help.
Error: No such option: -r
And, so, the docker build fails with
The command '/bin/sh -c pipenv lock -r > requirements.txt' returned a non-zero code: 2
My pipenv is in version 2023.2.4.
pipenv lock -h doesn't show the option -r exists and I can't find anything online for this. Is the option deprecated? Is it changed?

You are correct, this command does not work anymore
pipenv lock -r > requirements.txt
The recommended replacement is
pipenv requirements > requirements.txt
As pointed in the issue 5253 from there issue tracker.

Install python packages from github with Docker

I am trying to install a package that is not on PyPi. i.e from github. Adding the repo as git+url to the requirements file gives
ERROR: Error [Errno 2] No such file or directory: 'git' while executing command git clone -q https://github.com/Rapptz/discord-ext-menus /tmp/pip-req-build-147rct22
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH?
Installing the packages is done with
RUN python3 -m pip install -r requirements.txt
as specified in the docs
I also tried the solutions from this, but the answers mess up my other packages.
The dockerfile is almost directly from the docs
FROM python:3.8-slim-buster
WORKDIR /app
COPY requirements.txt requirements.txt
RUN python3 -m pip install -r requirements.txt
COPY . .
CMD [ "python3", "main.py"]
requirements.txt
asyncpg==0.21.0
git+https://github.com/Rapptz/discord-ext-menus
discord.py==1.7.0
pre-commit==2.10.1
pyclean==2.0.0
pylint==2.6.0
python-dotenv==0.15.0

As the error tells us, we have to simply install git, so that pip can clone the repo and run the setup file.
We can install git with
RUN apt-get update && apt-get install -y git
We also have to build from a python image, the above answer works with python:3.8-slim-buster

Not able to install any python package in docker container

I am trying to create an docker image with ubutu 16.04 as base. I want to install few python packages like pandas, flask etc. I have kept all packages in "requirements.txt". But when I am trying to build image, I am getting
Could not find a version that satisfies the requirement requests (from -r requirements.txt (line 1)) (from versions: )
No matching distribution found for requests (from -r requirements.txt (line 1))
Basically, I have not mentioned any version in "requirements.txt". I guess it should take the latest available and compatible version of that package. But for every package same issue I am getting.
My DockerFile is as follows.
FROM ubuntu:16.04
RUN apt-get update -y && \
apt-get install -y python3-pip python3-dev build-essential cmake pkg-config libx11-dev libatlas-base-dev
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /testing/requirements.txt
WORKDIR /testing
RUN pip3 install -r requirements.txt
and requirements.txt is.
pandas
requests
PyMySQL
Flask
Flask-Cors
Pillow
face-recognition
Flask-SocketIO
Where I am doing wrong ? Can anybody help ?

I too ran into the same situation. I observed that, python packages is looking for the network within docker. It is thinking that, it is running in a standalone without network so its not able to locate the package. In these type of situations either
No matching distribution found
or sometimes
Retrying ...
error may occur.
I used a --network option in the docker build command like below to overcome this error where the command insists python to use the host network to download the required packages.
docker build --network=host -t tracker:latest .

Try using this:
RUN python3.6 -m pip install --upgrade pip \
&& python3.6 -m pip install -r requirements.txt
by using it in this way, you are specifying the version of python in which you want to search for those packages.
Change it to python3.7 if you wish to use 3.7 version.

I suggest using the official python image instead. As a result, your Dockerfile will now become:
FROM python:3
WORKDIR /testing
COPY ./requirements.txt /testing/requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
... etc ...
Now re: Angular/Node. You have two options from here: 1) Install Angular/Node on the Python image; or 2) Use Docker's multi-stage build feature so you build the Angular and Python-specific images before merging them together. Option 2 is recommended but it would take some work. It would probably look like this:
FROM node:8 as node
# Angular-specific build
FROM python:3 as python
# Python-specific build
# Then copy your data from the Angular image to the Python one:
COPY --from=node /usr/src/app/dist/angular-docker /usr/src/app

ERROR: Directory is not installable. Neither 'setup.py' nor 'pyproject.toml'

I've got the following error:
ERROR: Directory is not installable. Neither 'setup.py' nor 'pyproject.toml'
Background is that I'm following a guide online to expose an ML model via API Gateway on AWS that can be found here:
Hosting your ML model on AWS Lambdas + API Gateway
I'm trying to pull some python packages into a local folder using the following command:
pip install -r requirements.txt --no-deps --target python/lib/python3.6/site-packages/
I have also tried this:
pip install -r requirements.txt --no-deps -t python/lib/python3.6/site-packages/
and all I get is the above error.
Google is pretty bare when it comes to help with this issue, any ideas?
thanks,

Does this work?
You can create a new folder e.g. lib, and run this command:
pip3 install <your_python_module_name> -t lib/

Would suggest making the path explicit to requirements.txt, e.g. ./requirements.txt if you're running the command in the same directory
Also may need to add a basic setup.py to the folder where you're trying to install. The pip docs mention that this will happen if there's no setup.py file:
When looking at the items to be installed, pip checks what type of
item each is, in the following order:
Project or archive URL.
Local directory (which must contain a
setup.py, or pip will report an error).
Local file (a sdist or wheel
format archive, following the naming conventions for those formats).
A
requirement, as specified in PEP 440.
https://pip.pypa.io/en/stable/cli/pip_install/#argument-handling

Please try this:
ADD requirements.txt ./
pip install -r requirements.txt --no-deps -t python/lib/python3.6/site-packages/
syntax: ADD source destination
'ADD requirements.txt ./' adds requirements.txt (assumed to be at the cwd) to the docker image's './' folder.
Thus creating a layer from which the daemon has the context to the location of requirements.txt in the docker image.
More about it in dockerfile-best-practices

you can change your directory as follow
import os
os.chdir(path)
instead of:
cd path
also try to use:
!pip freeze > requirements.txt
instead of:
pip install -r requirements.txt
then execute your code:
!pip install .
or
!pip install -e .
in conclusion try this:
import os
os.chdir(path)
!pip freeze > requirements.txt
!pip install .

Split requirements files in pip

To create Python virtual environments I use virtualenv and pip. The workflow is very simple:
$ virtualenv project
$ cd project
$ . bin/activate
$ pip install -r /path/to/requirements/req1.txt
$ pip install -r /path/to/requirements/req2.txt
The number of different requirement files can grow enough to make handy to have a way to include them at once, so I'd rather prefer to say:
$ pip install -r /path/to/requirements/req1_req2.txt
with req1_req2.txt containing something like:
include /path/to/requirements/req1.txt
include /path/to/requirements/req2.txt
or otherwise:
$ pip install -r /path/to/requirements/*.txt
None of that works and however much simple it could be, I can't figure out how to do what I want.
Any suggestion?

The -r flag isn't restricted to command-line use only, it can also be used inside requirements files. So running pip install -r req-1-and-2.txt when req-1-and-2.txt contains this:
-r req-1.txt
-r req-2.txt
will install everything specified in req-1.txt and req-2.txt.

Just on a note, you can also split the requirements based on your groupings and embed them in a single file ( or again can prepare multiple requirements file based on your environment), that you can execute.
For example, the test requirements here:
requirements-test.txt
pylint==2.4.4
pytest==5.3.2
The dev requirements here:
requirements-dev.txt
boto3>=1.12.11
Master requirements file containing your other requirements:
requirements.txt
-r requirements-dev.txt
-r requirements-test.txt
Now, you can just install the requirements file embedding your other requirements
pip3 install -r requirements.txt