This question already has answers here:
How to install packages offline?
(12 answers)
Closed 3 years ago.
I'm an experienced programmer, but very new to python. My company requires us to do development on a private network for some of our projects. There is a pypi index on the private network which can be used to install packages using pip. Recently, while needing to install a package, the pypi index when down and was down for several hours. Although it did come back up eventually, the situation begs the question, how do I install packages (maybe manually without pip) in the absense of an index? I've tried to google this, but came up empty. I'm sure there's a way, but I'm probably not searching for the right phrase. Thanks for any help with.
You can manually install Python packages if you have read access to the package repositories. Every Python package has a setup.py file in the root directory and you can do something like
python setup.py sdist
This creates a subdirectory called dist which contains a compressed archived file, tar.gz or .zip depending in your OS. You can pass this archived file to pip and install the package
pip3 install some-python-package.tar.gz
I would download the wheel and install that. For this to you do need to install the wheel package:
pip install wheel
You can then tell pip to install the project (and it'll download the wheel if available), or the wheel file directly:
pip install project_name # download and install
pip install wheel_file.whl # directly install the wheel
The wheel module is also runnable from the command line and you can use it to install already-downloaded wheels:
python -m wheel install wheel_file.whl
There are a few ways you can get around this issue. The two that I know of are:
Use a proxy to get to the standard PyPI. If your company permits it, then you can tunnel your traffic through their proxy and install packages from PyPA's standard locations.
Use a locally hosted index. All you need is a directory structured like https://pypi.org/simple/, and you can then pip install -i ~/my/personal/index/path and packages will be installed from there.
Related
I created a repository, on Artifactory, which includes a zip containing 2 folders.
https://artifactory.healthcareit.net:443/artifactory/pi-generic-local/paymentintegrity-airflow-lib-plugins/paymentintegrity-airflow-libs-plugins-202211031330.zip
Is there a way to download that zip and extract the directories during a pip install? Basically, the developer just runs a pip install and gets the directories where needed.
I'm not looking for something in the [scripts] section since installing these directories would be the only thing currently needed in the Pipfile (its a weird project). Is this possible?
You can abuse the install command in the setup.py to call any arbitrary code, such as a one that unzips the assets and install it at the right place. I have seen this being done to package c++ binaries using python pypi in a place I worked for. See Post-install script with Python setuptools for hints on how to override the install command.
As per my observations, you have created generic repo and trying to fetch the packages using pip install. I would recommend creating a PyPI repository in the Artifactory and then try fetching the packages. This will. help in creating the metadata which will maintain all the package versions in the repository.
If you have the packages in your local then push them in the PyPI local repo and when you resolve them from Artifactory it will automatically download for the pip install based on your requirement.
If your requirement is to zip up multiple packages and push the archive file to the Artifactory and want the Artifactory to unzip and give the dependeciense during the pip install - then this is not possible from the Artifactory side we need to use a Post-install script with Python setup tools as mentioned .
This question already has answers here:
How to install packages offline?
(12 answers)
Closed 3 years ago.
I want to install a python3 package (in this specific case is torch and torchvision) on a cluster that is not connected to the external world. I have tried to do pip3 download <package> and then I move the file to the cluster and run pip3 install <downloaded-file> -t /custom/folder (because I am installing in one of the folders in my personal account, and not on the entire cluster).
When I run the install command, pip3 is trying to download numpy package -- for some reason (it is actually already installed and working on ipython3, I do not know why it is trying to download that).
Of course, the installation fails because there is no connection to the external world. Can I force python to download everything with any flags?
Moreover, I would like to be sure I am downloading the proper thing: the location from which I am downloading the file is different from the location where I want to install, the first it is a regular CPU and the second runs GPU and has CUDA installed.
Formal answer here. Pip is attempting to install a different version of numpy specified in your package's requirements. You can easily compare the version installed with pip freeze and the version it is attempting to download (before it fails).
To make the download and installation fool-proof you probably should create a requirements.txt file and then run pip download -r requirements.txt in the environment that is connected to the internet and then transfer to the non-connected environment and carry on with the installation.
Regarding the GPU/CPU difference between the packages, it is hard to tell without actually knowing which package this is, but if it's similar to tensorflow where you have completely different package names when using pip the lack of GPU should not affect the download process. See pip download documentation for more information.
Is it possible to collect many Python extensions an install them in one step? I have a Python build environment for an open source project that often needs to be recreated on multiple machine. It's a pain to double click through a bunch of python extension exes every time we need to do this.
Ideally I'd like to package a complete build environment, Python, extensions, system environment variables, and all, into a one step install process. But a single step extension install would also be helpful. Is this possible?
Yes, you can do that... do you have pip(python indexed package) installed in your system?
if not, then install it... and put all the extensions into a single text file... say requirements.txt...
This is done by running
pip freeze > requirements.txt
then by using pip you can install it... by using this command...
pip install -r requirements.txt...
it will install all the extensions mentioned in the file...
you can find the pip package here pip
might help you...
You can with Distribute define dependencies of a package, and easy_install or pip will install all dependencies when you ask to install the package.
This question already has answers here:
How do I install from a local cache with pip?
(11 answers)
Closed 9 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
How do you prevent PIP from re-downloading previously downloaded packages? I'm testing the install of matplotlib, an 11MB package that depends on several distro-specific packages. Everytime I run pip install matplotlib, it re-downloads matplotlib. How do I stop this?
NOTE: Only wheels downloaded over HTTPS are cached. If you are using a custom repo over plain old HTTP, the cache is disabled.
For new Pip versions:
Newer Pip versions by default now cache downloads. See this documentation:
https://pip.pypa.io/en/stable/topics/caching/
For old Pip versions:
Create a configuration file named ~/.pip/pip.conf, and add the following contents:
[global]
download_cache = ~/.cache/pip
In one command:
printf '[global]\ndownload_cache = ~/.cache/pip\n' >> ~/.pip/pip.conf
You can use a specific environment variable PIP_DOWNLOAD_CACHE and make it point to a directory where your packages will be stored. If they are to be installed again, they will be taken from this directory.
There seems to be also an additional option for PIP pip --download-cache which ought to do something similar, but I have never tried it myself. For your example, to avoid re-downloading matplotlib every time, you would do the following:
pip install --download-cache /path/to/pip/cache matplotlib
Does that answer your question?
You could
# download and extract package to build path
pip install --no-install matplotlib
# the build path could be found by
pip install --help|grep Unpack\ packages\ into -A 2
# then rm pip-delete-this-directory.txt inside the build path
# this prevent pip from removing package from the build directory after install
# you could check the content of the file
rm build/pip-delete-this-directory.txt
# from now on you could install matplotlib quickly
# this uses single build directory
# and can speed up compiling by caching intermediate objects.
pip install --no-download matplotlib
Also, you could manually download the package
pip install -d dir_for_packages matplotlib
Then install it by un-tar and python setup install later.
The pip install --download-cache works in a similar way w/ extra checking: it firstly search for the latest or specified version of the target package from web, if the search has result and there is cached package in the directory specified by download-cache, the cached package will be used instead of downloading. For example,
pip install --download-cache . pymongo
will download pymongo package to current directory:
http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpymongo%2Fpymongo-2.1.1.tar.gz
http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpymongo%2Fpymongo-2.1.1.tar.gz.content-type
I'm wondering if there's a way to "install" single-file python modules using pip (i.e. just have pip download the specified version of the file and copy it to site-packages).
I have a Django project that uses several 3rd-party modules which aren't proper distributions (django-thumbs and a couple others) and I want to pip freeze everything so the project can be easily installed elsewhere. I've tried just doing
pip install git+https://github.com/path/to/file.git
(and tried with the -e tag too) but pip complains that there's no setup.py file.
Edit: I should have mentioned - the reason I want to do this is so I can include the required module in a requirements.txt file, to make setting up the project on a new machine or new virtualenv easier.
pip requires a valid setup.py to install a python package. By definition every python package has a setup.py... What you are trying to install isn't a package but rather a single file module... what's wrong with doing something like:
git clone git+https://github.com/path/to/file.git /path/to/python/install/lib
I don't quite understand the logic behind wanting to install something that isn't a package with a package manager...