Tried the following command for installing fasttext
conda install -c "conda-forge/label/cf202003" fasttext
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
- fasttext
Current channels:
- https://conda.anaconda.org/conda-forge/label/cf202003/osx-arm64
- https://conda.anaconda.org/conda-forge/label/cf202003/noarch
- https://repo.anaconda.com/pkgs/main/osx-arm64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/osx-arm64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/osx-arm64
- https://repo.anaconda.com/pkgs/r/noarch
- https://conda.anaconda.org/conda-forge/osx-arm64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Tried this approach
$ git clone https://github.com/facebookresearch/fastText.git
$ cd fastText
$ pip install .
and got this error
Building wheels for collected packages: fasttext
Building wheel for fasttext (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [67 lines of output]
.....
raise RuntimeError(
RuntimeError: libc++ is needed! Failed to compile with -stdlib=libc++ -mmacosx-version-min=10.7 and -stdlib=libc++.
[end of output]
I am trying to install geopandas in python with the command:
conda install -c "conda-forge/label/cf202003" geopandas
(I use intel compiler for miniconda and linux centos)
But, the feedback is :
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages. failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your system:
feature:/linux-64::__glibc==2.17=0
feature:|#/linux-64::__glibc==2.17=0
Your installed version is: 2.17
I tried to install geopandas.
I try to build a docker container with a python script on a raspberry pi.
I' using the arm32v7/python latest image (3.12), but also tried it with 3.9 and 3.10 images.
In my Dockerfile I first update pip and then try to install my requirements. But when it tries to install PyWavelets or scipy it wont move further. I just get "installing build dependencies: still running".
Downloading PyWavelets-1.4.1.tar.gz (4.6 MB)
=> => # 4.6/4.6 MB 11.9 MB/s eta 0:00:00
=> => # Installing build dependencies: started
=> => # Installing build dependencies: still running...
After 30 mins it fails with
� Building wheel for ninja (pyproject.toml) did not run successfully.
#0 2857.1 exit code: 1
#0 2857.1 > [11 lines of output]
#0 2857.1 Traceback (most recent call last):
#0 2857.1 File "/tmp/pip-build-env-roddjnvw/overlay/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 612, in setup
#0 2857.1 cmkr = cmaker.CMaker(cmake_executable)
#0 2857.1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#0 2857.1 File "/tmp/pip-build-env-roddjnvw/overlay/lib/python3.11/site-packages/skbuild/cmaker.py", line 149, in __init__
#0 2857.1 self.cmake_version = get_cmake_version(self.cmake_executable)
#0 2857.1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#0 2857.1 File "/tmp/pip-build-env-roddjnvw/overlay/lib/python3.11/site-packages/skbuild/cmaker.py", line 104, in get_cmake_version
#0 2857.1 raise SKBuildError(
#0 2857.1
#0 2857.1 Problem with the CMake installation, aborting build. CMake executable is cmake
#0 2857.1 [end of output]
I already tried to install them via their apt packages (python3-pywt, python3-scipy) and skip the ones in my requirements.txt, but then I get an error in the installer.
I also tried to use https://www.piwheels.org/simple as index url in my install command but then it wont find any scipy installation candidate.
My Dockerfile:
FROM arm32v7/python
RUN apt-get update && apt-get -y install vim python3-pywt python3-scipy
WORKDIR /app
COPY main.py /app/main.py
COPY requirements.txt /app/requirements.txt
RUN pip install --upgrade pip
#RUN pip install --index-url=https://www.piwheels.org/simple --no-cache-dir -r requirements.txt
RUN pip install -r requirements.txt
My requirements.txt
certifi==2022.12.7
charset-normalizer==2.1.1
idna==3.4
mysql-connector-python==8.0.25
numpy==1.24.0
Pillow==9.3.0
protobuf==3.20.1
python-dotenv==0.21.0
PyWavelets==1.4.1
requests==2.28.1
scipy==1.9.3
update-checker==0.18.0
urllib3==1.26.13
websocket-client==1.4.2
I am trying to install xarray-spatial using anaconda. I am getting an UnsatisfiableError, but there are no deatils of any incompatible specifications.
Running the following command:
(geo_env) C:\Users\Alex>conda install -c conda-forge xarray-spatial
Here are the details of the error:
(geo_env) C:\Users\Alex>conda install -c conda-forge xarray-spatial
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Note that strict channel priority may have removed packages required for satisfiability.
I'm hoping to get my pip install instructions inside my docker builds as fast as possible.
I've read many posts explaining how adding your requirements.txt before the rest of the app helps you take advantage of Docker's own image cache if your requirements.txt hasn't changed. But this is no help at all when dependencies do change, even slightly.
The next step would be if we could use a consistent pip cache directory. By default, pip will cache downloaded packages in ~/.cache/pip (on Linux), and so if you're ever installing the same version of a module that has been installed before anywhere on the system, it shouldn't need to go and download it again, but instead simply use the cached version. If we could leverage a shared cache directory for docker builds, this could help speed up dependency installs a lot.
However, there doesn't appear to be any simple way to mount a volume while running docker build. The build environment seems to be basically impenetrable. I found one article suggesting a genius but complex method of running an rsync server on the host and then, with a hack inside the build to get the host IP, rsyncing the pip cache in from the host. But I'm not relishing the idea of running an rsync server in Jenkins (which isn't the most secure platform at the best of times).
Does anyone know if there's any other way to achieve a shared cache volume more simply?
I suggest you to use buildkit, also see this.
Dockerfile:
# syntax = docker/dockerfile:experimental
FROM python:3.6-alpine
RUN --mount=type=cache,target=/root/.cache/pip pip install pyyaml
NOTE: # syntax = docker/dockerfile:experimental is a must，you have to add it at the beginning of Dockerfile to enable this feature.
1.
The first execute build:
export DOCKER_BUILDKIT=1
docker build --progress=plain -t abc:1 . --no-cache
The first log:
#9 [stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install...
#9 digest: sha256:55b70da1cbbe4d424f8c50c0678a01e855510bbda9d26f1ac5b983808f3bf4a5
#9 name: "[stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install pyyaml"
#9 started: 2019-09-20 03:11:35.296107357 +0000 UTC
#9 1.955 Collecting pyyaml
#9 3.050 Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB)
#9 5.006 Building wheels for collected packages: pyyaml
#9 5.007 Building wheel for pyyaml (setup.py): started
#9 5.249 Building wheel for pyyaml (setup.py): finished with status 'done'
#9 5.250 Created wheel for pyyaml: filename=PyYAML-5.1.2-cp36-cp36m-linux_x86_64.whl size=44104 sha256=867daf35eab43c2d047ad737ea1e9eaeb4168b87501cd4d62c533f671208acaa
#9 5.250 Stored in directory: /root/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030
#9 5.267 Successfully built pyyaml
#9 5.274 Installing collected packages: pyyaml
#9 5.309 Successfully installed pyyaml-5.1.2
#9completed: 2019-09-20 03:11:42.221146294 +0000 UTC
#9 duration: 6.925038937s
From above, you can see the first time, the build will download pyyaml from internet.
2.
The second execute build:
docker build --progress=plain -t abc:1 . --no-cache
The second log:
#9 [stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install...
#9 digest: sha256:55b70da1cbbe4d424f8c50c0678a01e855510bbda9d26f1ac5b983808f3bf4a5
#9 name: "[stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install pyyaml"
#9 started: 2019-09-20 03:16:58.588157354 +0000 UTC
#9 1.786 Collecting pyyaml
#9 2.234 Installing collected packages: pyyaml
#9 2.270 Successfully installed pyyaml-5.1.2
#9completed: 2019-09-20 03:17:01.933398002 +0000 UTC
#9 duration: 3.345240648s
From above, you can see the build no longer download package from internet, just use the cache. NOTE, this is not the traditional docker build cache as I have use --no-cache, it's /root/.cache/pip which I mount into build.
3.
The third execute build which delete buildkit cache:
docker builder prune
docker build --progress=plain -t abc:1 . --no-cache
The third log:
#9 [stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install...
#9 digest: sha256:55b70da1cbbe4d424f8c50c0678a01e855510bbda9d26f1ac5b983808f3bf4a5
#9 name: "[stage-0 2/2] RUN --mount=type=cache,target=/root/.cache/pip pip install pyyaml"
#9 started: 2019-09-20 03:19:07.434792944 +0000 UTC
#9 1.894 Collecting pyyaml
#9 2.740 Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB)
#9 3.319 Building wheels for collected packages: pyyaml
#9 3.319 Building wheel for pyyaml (setup.py): started
#9 3.560 Building wheel for pyyaml (setup.py): finished with status 'done'
#9 3.560 Created wheel for pyyaml: filename=PyYAML-5.1.2-cp36-cp36m-linux_x86_64.whl size=44104 sha256=cea5bc4689e231df7915c2fc3abca225d4ee2e869a7540682aacb6d42eb17053
#9 3.560 Stored in directory: /root/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030
#9 3.580 Successfully built pyyaml
#9 3.585 Installing collected packages: pyyaml
#9 3.622 Successfully installed pyyaml-5.1.2
#9completed: 2019-09-20 03:19:12.530742712 +0000 UTC
#9 duration: 5.095949768s
From above, you can see if delete buildkit cache, the package download again.
In a word, it will give you a shared cache between several times build, and this cache will only be mounted when image build. But, the image self will not have these cache, so avoid a lots of intermediate layer in image.
EDIT for folks who are using docker compose and are lazy to read the comments...:
You can also do this with docker-compose if you set
COMPOSE_DOCKER_CLI_BUILD=1. For example: COMPOSE_DOCKER_CLI_BUILD=1
DOCKER_BUILDKIT=1 docker-compose build –
UPDATE according to folk's question 2020/09/02:
I don't know from which version (my version now is 19.03.11), if not specify mode for cache directory, the cache won't be reused by next time build.
Don't know the detail reason, but you could add mode=0755, to Dockerfile to make it work again:
Dockerfile:
# syntax = docker/dockerfile:experimental
FROM python:3.6-alpine
RUN --mount=type=cache,mode=0755,target=/root/.cache/pip pip install pyyaml