This question already has answers here:
How to install packages offline?
(12 answers)
Closed 3 years ago.
I want to install a python3 package (in this specific case is torch and torchvision) on a cluster that is not connected to the external world. I have tried to do pip3 download <package> and then I move the file to the cluster and run pip3 install <downloaded-file> -t /custom/folder (because I am installing in one of the folders in my personal account, and not on the entire cluster).
When I run the install command, pip3 is trying to download numpy package -- for some reason (it is actually already installed and working on ipython3, I do not know why it is trying to download that).
Of course, the installation fails because there is no connection to the external world. Can I force python to download everything with any flags?
Moreover, I would like to be sure I am downloading the proper thing: the location from which I am downloading the file is different from the location where I want to install, the first it is a regular CPU and the second runs GPU and has CUDA installed.
Formal answer here. Pip is attempting to install a different version of numpy specified in your package's requirements. You can easily compare the version installed with pip freeze and the version it is attempting to download (before it fails).
To make the download and installation fool-proof you probably should create a requirements.txt file and then run pip download -r requirements.txt in the environment that is connected to the internet and then transfer to the non-connected environment and carry on with the installation.
Regarding the GPU/CPU difference between the packages, it is hard to tell without actually knowing which package this is, but if it's similar to tensorflow where you have completely different package names when using pip the lack of GPU should not affect the download process. See pip download documentation for more information.
Related
This question already has answers here:
How to install packages offline?
(12 answers)
Closed 5 months ago.
I am not able to use PIP on this device, so I need to install a module manually. The only problem is that I have to install like 30 dependencies of this module. I would have to download, unzip, include and install them all one by one.
Is there any faster way, like downloading a module with all its dependencies included?
If all requirements are pip packages, a quick solution might involve creating a lean python environment on another machine, installing the package using pip, and then copying over all of the resultant wheel files to the restricted machine either via SSH or another method.
This question already has answers here:
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings
(22 answers)
Closed 1 year ago.
F:\Hacking\GitHub\sherlock>python3 -m pip install -r requirements.txt
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
You provided limited information
First off check if you have installed python: I suggest using Anaconda. It's quite straightforward.
After that create a conda (Anaconda) environment (imagine it as a space that you can have different versions of packages- Python packages tend to be very dependent on the version of related packages, conda usually check the compatibility of them when you are installing them)
Using pip and conda together can be tricky. I suggest first install the main packages then go for more specialized one, maybe using pip
Try to have an issue on the github page of the project. If the package is popular or fresh, they usually respond fast.
It's a great hassle when installing some packages in a VE and conda or pip downloads them again even when I already have it in my base environment. Since I have limited internet bandwidth and I'm assuming I'll work with many different VE's, it will take a lot of time to download basic packages such as OpenCV/Tensorflow.
By default, pip caches anything it downloads, and will used the cached version whenever possible. This cache is shared between your base environment and all virtual environments. So unless you pass the --no-cache-dir option, pip downloading a package means it has not previously downloaded a compatible version of that package. If you already have that package installed in your base environment or another virtual environment and it downloads it anyway, this probably means one or more of the following is true:
You installed your existing version with a method other than pip.
There is a newer version available, and you didn't specify, for example, pip install pandas=1.1.5 (if that's the version you already have elsewhere). Pip will install the newest compatible version for your environment, unless you tell it otherwise.
The VE you're installing to is a different Python version (e.g. created with Pyenv), and needs a different build.
I'm less familiar with the specifics of conda, and I can't seem to find anything in its online docs that focuses on the default caching behavior. However, a how-to for modifying the cache location seems to assume that the default behavior is similar to how pip works. Perhaps someone else with more Anaconda experience can chime in as well.
So except for the caveats above, as long as you're installing a package with the same method you did last time, you shouldn't have to download anything.
If you want to simplify the process of installing all the same packages (that were installed via pip) in a new VE that you already have in another environment, pip can automate that too. Run pip freeze > requirements.txt in the first environment, and copy the resulting file to your newly created VE. There, run pip install -r requirements.txt and pip will install all the packages that were installed (via pip) in the first environment. (Note that pip freeze records version numbers as well, so this won't install newer versions that may be available -- whether this is a good or bad thing depends on your needs.)
This question already has answers here:
How to install packages offline?
(12 answers)
Closed 3 years ago.
I'm an experienced programmer, but very new to python. My company requires us to do development on a private network for some of our projects. There is a pypi index on the private network which can be used to install packages using pip. Recently, while needing to install a package, the pypi index when down and was down for several hours. Although it did come back up eventually, the situation begs the question, how do I install packages (maybe manually without pip) in the absense of an index? I've tried to google this, but came up empty. I'm sure there's a way, but I'm probably not searching for the right phrase. Thanks for any help with.
You can manually install Python packages if you have read access to the package repositories. Every Python package has a setup.py file in the root directory and you can do something like
python setup.py sdist
This creates a subdirectory called dist which contains a compressed archived file, tar.gz or .zip depending in your OS. You can pass this archived file to pip and install the package
pip3 install some-python-package.tar.gz
I would download the wheel and install that. For this to you do need to install the wheel package:
pip install wheel
You can then tell pip to install the project (and it'll download the wheel if available), or the wheel file directly:
pip install project_name # download and install
pip install wheel_file.whl # directly install the wheel
The wheel module is also runnable from the command line and you can use it to install already-downloaded wheels:
python -m wheel install wheel_file.whl
There are a few ways you can get around this issue. The two that I know of are:
Use a proxy to get to the standard PyPI. If your company permits it, then you can tunnel your traffic through their proxy and install packages from PyPA's standard locations.
Use a locally hosted index. All you need is a directory structured like https://pypi.org/simple/, and you can then pip install -i ~/my/personal/index/path and packages will be installed from there.
This question already has answers here:
How do I install from a local cache with pip?
(11 answers)
Closed 9 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
How do you prevent PIP from re-downloading previously downloaded packages? I'm testing the install of matplotlib, an 11MB package that depends on several distro-specific packages. Everytime I run pip install matplotlib, it re-downloads matplotlib. How do I stop this?
NOTE: Only wheels downloaded over HTTPS are cached. If you are using a custom repo over plain old HTTP, the cache is disabled.
For new Pip versions:
Newer Pip versions by default now cache downloads. See this documentation:
https://pip.pypa.io/en/stable/topics/caching/
For old Pip versions:
Create a configuration file named ~/.pip/pip.conf, and add the following contents:
[global]
download_cache = ~/.cache/pip
In one command:
printf '[global]\ndownload_cache = ~/.cache/pip\n' >> ~/.pip/pip.conf
You can use a specific environment variable PIP_DOWNLOAD_CACHE and make it point to a directory where your packages will be stored. If they are to be installed again, they will be taken from this directory.
There seems to be also an additional option for PIP pip --download-cache which ought to do something similar, but I have never tried it myself. For your example, to avoid re-downloading matplotlib every time, you would do the following:
pip install --download-cache /path/to/pip/cache matplotlib
Does that answer your question?
You could
# download and extract package to build path
pip install --no-install matplotlib
# the build path could be found by
pip install --help|grep Unpack\ packages\ into -A 2
# then rm pip-delete-this-directory.txt inside the build path
# this prevent pip from removing package from the build directory after install
# you could check the content of the file
rm build/pip-delete-this-directory.txt
# from now on you could install matplotlib quickly
# this uses single build directory
# and can speed up compiling by caching intermediate objects.
pip install --no-download matplotlib
Also, you could manually download the package
pip install -d dir_for_packages matplotlib
Then install it by un-tar and python setup install later.
The pip install --download-cache works in a similar way w/ extra checking: it firstly search for the latest or specified version of the target package from web, if the search has result and there is cached package in the directory specified by download-cache, the cached package will be used instead of downloading. For example,
pip install --download-cache . pymongo
will download pymongo package to current directory:
http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpymongo%2Fpymongo-2.1.1.tar.gz
http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpymongo%2Fpymongo-2.1.1.tar.gz.content-type