Develop Reference

How can I install the FeniCS dolfin module?

So I'm trying to install FEniCS from the instructions here. I did the
pip3 install fenics-ffc --upgrade
inside my virtualenv and it worked but when I try to import dolfin I get a ModuleNotFound error. I'm not sure how to get dolfin installed. I did
pip install pybind11
to get pybind11 installed then copied the code for dolfin installation into my cmd
FENICS_VERSION=$(python3 -c"import ffc; print(ffc.__version__)")
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/dolfin
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/mshr
mkdir dolfin/build && cd dolfin/build && cmake .. && make install && cd ../..
mkdir mshr/build && cd mshr/build && cmake .. && make install && cd ../..
cd dolfin/python && pip3 install . && cd ../..
cd mshr/python && pip3 install . && cd ../..
but it just spat out dozens of errors like:
FENICS_VERSION=$(python3 -c"import ffc; print(ffc.version)") 'FENICS_VERSION' is not recognized as an internal or external command, operable program or batch file.
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/dolfin Cloning into 'dolfin'...
fatal: Remote branch $FENICS_VERSION not found in upstream origin
git clone --branch=$FENICS_VERSION https://bitbucket.org/fenics-project/mshr Cloning into 'mshr'...
fatal: Remote branch $FENICS_VERSION not found in upstream origin
There were lots more errors after too. Am I not supposed to paste the dolfin code into cmd? I don't know much about this stuff so unsure of how to get the dolfin module. I've previously only used pip to get my packages but this does not work for dolfin as it doesn't appear to be on PyPI.

Do you have cmake? It says in the docs you need it. Also its says to to this to install pybind11 not pip install pybind11
For building optional Python interface of DOLFIN and mshr, pybind11 is needed since version 2018.1.0. To install it:
wget -nc --quiet https://github.com/pybind/pybind11/archive/v${PYBIND11_VERSION}.tar.gz
tar -xf v${PYBIND11_VERSION}.tar.gz && cd pybind11-${PYBIND11_VERSION}
mkdir build && cd build && cmake -DPYBIND11_TEST=off .. && make install
Also what is your os?

So here is how you can install fenics 2019.1 using conda (miniconda):
Install Conda:
First go to https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html
and follow the installation instructions.
Create a conda environment for fenics:
Open a terminal and type:
conda create -n fenics
To activate the created environment "fenics", type:
conda activate fenics
If you want the fenics environment to be activated automatically everytime you open a new terminal, then open you .bashrc file (should be under /home/username/.bashrc) and add the line "source activate fenics" below the ">>> conda initialize >>>" block.
Install fenics:
Type all these commands:
conda install -c conda-forge h5py=*=*mpich*
conda install -c conda-forge fenics
pip install meshio
pip install matplotlib
pip install --upgrade gmsh
conda install -c conda-forge paraview
pip install scipy
The second command will take a while. I added a few nice to have programs like gmsh and paraview which will help you to create meshes and view your solutions.

Install python modules on Elastic Beanstalk with conda

Problem
I am trying to install python packages that have C dependencies on AWS Elastic Beanstalk (namely : fbprophet and xgboost)
Elastic Beanstalk python installs packages from requirements.txt by default with pip or pipenv on Amazon Linux 2
However, fbprophet and xgboost have dependencies in C that need to be compiled before installing them with pip. conda comes with these libraries precompiled so they are a lot easier to install with conda.
What I have tried
Here is my attempt at installing them with conda using a .config file in .ebextensions folder :
commands:
00_download_conda:
command: 'wget http://repo.continuum.io/archive/Anaconda3-2020.02-Linux-x86_64.sh'
test: test ! -d /anaconda
01_install_conda:
command: 'bash Anaconda3-2020.02-Linux-x86_64.sh -b -f -p /anaconda'
test: test ! -d /anaconda
02_reload_bash:
command: 'source ~/.bashrc'
03_create_home:
command: 'mkdir -p /home/wsgi'
04_conda_env:
command: 'conda env create -f environment.yml'
05_activate_env:
command: 'conda activate demo_forecast'
However this does not work and throws this error :
[2020-04-21T18:18:22.285Z] INFO [3699] - [Application update app-8acc-200421_201612#4/AppDeployStage0/EbExtensionPreBuild/Infra-EmbeddedPreBuild/prebuild_0_test_empty_dash/Command 03_conda_env] : Activity execution failed, because: /bin/sh: conda: command not found
(ElasticBeanstalk::ExternalInvocationError)
[2020-04-21T18:18:22.285Z] INFO [3699] - [Application update app-8acc-200421_201612#4/AppDeployStage0/EbExtensionPreBuild/Infra-EmbeddedPreBuild/prebuild_0_test_empty_dash/Command 03_conda_env] : Activity failed.
So it seems that sourcing the .bashrc does not create the conda alias
I am aware of this question and its answer , however it is a little bit old and does not provide enough details for my case, because it does not go through with installing packages using conda.
Another way would be to try and install and compile the C dependencies before pip installing the requirement, but I had no success there for now.
Thank you for the help !

Add Miniconda binaries to path in Docker container

I'm using the following commands in my Dockerfile to install Miniconda. After I install it, I want to use the binaries in ~/miniconda3/bin like python and conda. I tried exporting the PATH with this new path prepended to it, but the subsequent pip command fails (pip is located in ~/miniconda3/bin.
Curiously, if I run the container in interactive terminal mode, the path is set correctly and I'm able to call the binaries as expected. It seems as though the issue is only when building the container itself.
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y python3.7
RUN apt-get install -y curl
RUN curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh --output miniconda.sh
RUN bash miniconda.sh -b
RUN export PATH="~/miniconda3/bin:$PATH"
RUN pip install pydub # errors out when building
Here's the result of echo $PATH
~/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Here's the error I get
/bin/sh: 1: pip: not found

export won't work. Try ENV
Replace
RUN export PATH="~/miniconda3/bin:$PATH"
with
ENV PATH="~/miniconda3/bin:$PATH"

Even though Miniconda is located in ~, it default installs to the root directory unless otherwise specified.
Here's the right command.
RUN export PATH="/root/miniconda3/bin:$PATH"

It looks like your export PATH ... command is putting the literal symbol ~ into the path. Try this:
ENV PATH="$HOME/miniconda3/bin:$PATH"

How to copy modules of Python between different machines

I have two machine that one does not have internet access. I want to install modules with anaconda and copy them to offline computer from the other computer that has internet access.
I tried looking for dependencies and install tar. files manually one by one and sent them to the offline machine but it is very time-consuming.
What is the easiest way? Does miniconda helpful ??
P.S: I forgot to mention that I am using anaconda in both machines. So I guess I need to create an env., install packages then export it for offline computer. Are there any other way to install number of packages to offline comp. from a copy <dir> in the online computer ??
Edit: I tried conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file://C:\Users\myName\Desktop\OfflineInstall\pkgs2 but offline machine still tried to connect internet. I also used --no-deps
Edit2: For those who stuck on the same problem, I solved using conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file:///C:\Users\myName\Desktop\OfflineInstall\pkgs2 --override-channels The tricky way is the file:/// prefix. You need to put ///. Also remember to put --override-channels flag to prevent connection to default channels.

It sounds like Conda-pack is what you are looking for.
Installing:
$ conda install conda-pack
On the source machine:
# Pack environment my_env into my_env.tar.gz
$ conda pack -n my_env
On the target machine:
# Unpack environment into directory `my_env`
$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env
# Use python without activating or fixing the prefixes. Most python
# libraries will work fine, but things that require prefix cleanups
# will fail.
$ ./my_env/bin/python
# Activate the environment. This adds `my_env/bin` to your path
$ source my_env/bin/activate
# Run python from in the environment
(my_env) $ python
# Cleanup prefixes from in the active environment.
# Note that this command can also be run without activating the environment
# as long as some version of python is already installed on the machine.
(my_env) $ conda-unpack
The caveat being that conda-pack will take the whole environment.

Had this problem the other day, very simple implementation.
First make a .txt file which contains all your python libraries. Now you can just pass this .txt file to whatever machine you want the solution to be installed under and issue the following command :
pip install -r packages.txt
Where "packages" is the name of your .txt file. Hope this helps!
Edit using Conda :
while read requirement; do conda install --yes $requirement; done < requirements.txt

Docker and Conda: Differences when building the same container on Mac and on Ubuntu

I'm using to Docker to build a Python container with the intention of having a reproducible environment on several machines, which are a bunch of development Macbooks and several AWS EC2 servers.
The container is based on continuumio/miniconda3, i.e. Dockerfile starts with
FROM continuumio/miniconda3
A few days ago on Ubuntu the conda install and conda upgrade commands in the Docker file complained about a new Conda version (4.11) being available:
==> WARNING: A newer version of conda exists. <==
current version: 4.4.10
latest version: 4.4.11
Please update conda by running
$ conda update -n base conda
If I ignore this, package installations quit with an error:
Downloading and Extracting Packages
The command '/bin/sh -c conda install -y pandas=0.22.0 matplotlib
scikit-learn=0.19.1 pathos lazy openpyxl pytables dill pydro psycopg2
sqlalchemy pyarrow arrow-cpp parquet-cpp scipy tensorflow keras
xgboost' returned a non-zero code: 1
When I add this conda update... to the Docker file, things work again.
What's really annoying, however, is that the update that makes things run in Ubuntu does not work on Mac Docker. I get the following error:
CondaEnvironmentNotFoundError: Could not find environment: base .
You can list all discoverable environments with `conda info --envs`.
Note that I get this error when I docker build the same Docker file that works on the Ubuntu machine, which kind of ruins the whole point about using Docker in the first place. On the Mac, the old version of the file (without conda update -n base conda) still runs fine and installs all packages.
Docker / Conda experts, any ideas?
Edit: Here's the full Dockerfile (the one that works in Ubuntu):
# Use an official Python runtime as a parent image
FROM continuumio/miniconda3
WORKDIR /app/dev/predictive.analytics
RUN apt-get update; \
apt-get install -y gcc tmux htop
RUN conda update -y -n base conda
RUN conda config --add channels babbel; \
conda config --add channels conda-forge;
RUN conda install -y pandas=0.22.0 matplotlib scikit-learn=0.19.1 pathos lazy openpyxl pytables dill pydro psycopg2 sqlalchemy pyarrow arrow-cpp parquet-cpp scipy tensorflow keras xgboost
RUN pip install recordclass sultan
RUN conda upgrade -y python
ENV DATA_DIR /host/data
ENV PYTHONPATH /host/predictive.analytics/python
ENV PATH="/host/predictive.analytics:${PATH}"

Perhaps you're using an outdated miniconda on one of the build machine, try doing docker build --pull --no-cache.
Docker doesn't necessarily pull the latest image from the repository, so unless you do a --pull, it is possible that some of your machines may be starting the build with outdated base image.