Develop Reference

deploying azure function using custom image

I am developing an azure app function that will be using a custom image with docker.
This is my DockerFile:
FROM mcr.microsoft.com/azure-functions/python:3.0-python3.7
ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
# Adding "apt-get install make" here
RUN apt-get update && apt-get install make && apt-get install -y gcc g++ && rm -rf /var/lib/apt/lists/*
RUN pip install numpy
RUN pip install pandas
RUN pip install scipy
RUN wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz && \
tar -xvzf ta-lib-0.4.0-src.tar.gz && \
cd ta-lib/ && \
./configure --prefix=/usr && \
make && \
make install
RUN rm -R ta-lib ta-lib-0.4.0-src.tar.gz
COPY requirements.txt /
RUN pip install -r /requirements.txt
COPY . /home/site/wwwroot
Everything works fine when I am using my local docker desktop app.
When I am trying to deploy to azure using the VS-Code build it interface the docker file fails to build here is the output I get:
14:57:08 : building 'talib._ta_lib' extension
14:57:08 : creating build/temp.linux-x86_64-3.7
14:57:08 : creating build/temp.linux-x86_64-3.7/talib
14:57:08 : gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/tmp/pip-install-rgqyohyf/ta-lib/.eggs/numpy-1.21.2-py3.7-linux-x86_64.egg/numpy/core/include -I/opt/python/3.7.9/include/python3.7m -c talib/_ta_lib.c -o build/temp.linux-x86_64-3.7/talib/_ta_lib.o
14:57:08 : talib/_ta_lib.c:613:28: fatal error: ta-lib/ta_defs.h: No such file or directory
14:57:08 : #include "ta-lib/ta_defs.h"
14:57:08 : ^
14:57:08 : compilation terminated.
14:57:09 : error: command 'gcc' failed with exit status 1
14:57:09 : ----------------------------------------
14:57:09 : ERROR: Failed building wheel for TA-Lib
14:57:09 : DEPRECATION: Could not build wheels for TA-Lib which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
14:57:16 : [11:57:16+0000] Running setup.py install for TA-Lib: started
14:57:16 : [11:57:16+0000] Running setup.py install for TA-Lib: finished with status 'error'
14:57:16 : ERROR: Command errored out with exit status 1:
I am not sure what to make of this error and why it works fine in docker.
Thanks
Amit

Can't install nmslib in docker

I can't install nmslib using pip in docker.
This is my Dockerfile:
FROM continuumio/anaconda3:4.4.0
MAINTAINER UNP, https://unp.education
EXPOSE 8000
RUN apt-get update && apt-get install -y apache2 \
apache2-dev \
emacs \
&& apt-get clean \
&& apt-get autoremove \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /var/www/devise-api/
COPY ./devise-api.wsgi /var/www/devise-api/devise-api.wsgi
COPY ./devise-api /var/www/devise-api/
RUN pip install -r requirements.txt
RUN /opt/conda/bin/mod_wsgi-express install-module
RUN mod_wsgi-express setup-server devise-api.wsgi --port=8000 \
--user www-data --group www-data \
--server-root=/etc/mod_wsgi-express-80
CMD /etc/mod_wsgi-express-80/apachectl start -D FOREGROUND
For now requirements.txt contains only the word nmslib
This is the output I get when running sudo docker build -t devise-api .:
Sending build context to Docker daemon 306.2MB
Step 1/11 : FROM continuumio/anaconda3:4.4.0
---> 795ad88c47ff
Step 2/11 : MAINTAINER UNP, https://unp.education
---> Using cache
---> cd5b1f7e6188
Step 3/11 : EXPOSE 8000
---> Using cache
---> 21ad868f0823
Step 4/11 : RUN apt-get update && apt-get install -y apache2 apache2-dev emacs && apt-get clean && apt-get autoremove && rm -rf /var/lib/apt/lists/*
---> Using cache
---> ebfc7c30b394
Step 5/11 : WORKDIR /var/www/devise-api/
---> Using cache
---> 8228e4b4d4fd
Step 6/11 : COPY ./devise-api.wsgi /var/www/devise-api/devise-api.wsgi
---> Using cache
---> c8cf5cfcf7dc
Step 7/11 : COPY ./devise-api /var/www/devise-api/
---> Using cache
---> 856e67f0b1de
Step 8/11 : RUN pip install -r requirements.txt
---> Running in 7260901af476
Collecting nmslib (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/e1/95/1f7c90d682b79398c5ee3f9296be8d2640fa41de24226bcf5473c801ada6/nmslib-1.7.3.6.tar.gz (255kB)
Collecting pybind11>=2.0 (from nmslib->-r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/f2/7c/e71995e59e108799800cb0fce6c4b4927914d7eada0723dd20bae3b51786/pybind11-2.2.4-py2.py3-none-any.whl (145kB)
Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from nmslib->-r requirements.txt (line 1))
Building wheels for collected packages: nmslib
Running setup.py bdist_wheel for nmslib: started
Running setup.py bdist_wheel for nmslib: finished with status 'error'
Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-4qfop8hr/nmslib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpl9tnakclpip-wheel- --python-tag cp36:
running bdist_wheel
running build
running build_ext
creating tmp
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/include/python3.6m -c /tmp/tmp2pnwck3x.cpp -o tmp/tmp2pnwck3x.o -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/include/python3.6m -c /tmp/tmp5ktoxd0l.cpp -o tmp/tmp5ktoxd0l.o -fvisibility=hidden
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
building 'nmslib' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/nmslib
creating build/temp.linux-x86_64-3.6/nmslib/similarity_search
creating build/temp.linux-x86_64-3.6/nmslib/similarity_search/src
creating build/temp.linux-x86_64-3.6/nmslib/similarity_search/src/space
creating build/temp.linux-x86_64-3.6/nmslib/similarity_search/src/method
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./nmslib/similarity_search/include -I/opt/conda/include/python3.6m -I/root/.local/include/python3.6m -I/opt/conda/lib/python3.6/site-packages/numpy/core/include -I/opt/conda/include/python3.6m -c nmslib.cc -o build/temp.linux-x86_64-3.6/nmslib.o -O3 -march=native -fopenmp -DVERSION_INFO="1.7.3.6" -std=c++14 -fvisibility=hidden
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
nmslib.cc:16:31: fatal error: pybind11/pybind11.h: No such file or directory
#include <pybind11/pybind11.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for nmslib
Running setup.py clean for nmslib
Failed to build nmslib
Installing collected packages: pybind11, nmslib
Running setup.py install for nmslib: started
Then it stays like this forever.
Does anyone know what I could try to fix this?
Thank you a lot in advance!
Best regards
F

I just faced the same issue and fixed it by adding RUN step before installing nmslib:
RUN pip install pip==9.0.3 pybind11
See: https://github.com/nmslib/nmslib/issues/307#issuecomment-384113900. Hope this helps in your case as well :-)

How to ignore compiler flags in pip?

I'm trying to install mujoco-py, when I try pip install mujoco_py in my maxOS 10.12 with python 3.5 in anaconda, I got
/usr/local/bin/gcc-7 -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Users/Shangtong/anaconda2/envs/Python3.5/include -I/Users/Shangtong/anaconda2/envs/Python3.5/include -Wno-error=unused-command-line-argument-hard-error-in-future -Wno-error=unused-command-line-argument-hard-error-in-future -Wno-error=unused-command-line-argument-hard-error-in-future -DONMAC -Imujoco_py -I/private/var/folders/6j/gtq1cx25611ck3bft9rnfxkh0000gn/T/pip-build-3r460esi/mujoco-py/mujoco_py -I/Users/Shangtong/.mujoco/mjpro150/include -I/Users/Shangtong/anaconda2/envs/Python3.5/lib/python3.5/site-packages/numpy/core/include -I/Users/Shangtong/anaconda2/envs/Python3.5/include/python3.5m -c /private/var/folders/6j/gtq1cx25611ck3bft9rnfxkh0000gn/T/pip-build-3r460esi/mujoco-py/mujoco_py/cymj.c -o /private/var/folders/6j/gtq1cx25611ck3bft9rnfxkh0000gn/T/pip-build-3r460esi/mujoco-py/mujoco_py/generated/_pyxbld_MacExtensionBuilder/temp.macosx-10.6-x86_64-3.5/private/var/folders/6j/gtq1cx25611ck3bft9rnfxkh0000gn/T/pip-build-3r460esi/mujoco-py/mujoco_py/cymj.o -fopenmp -w
cc1: error: -Werror=unused-command-line-argument-hard-error-in-future: no option -Wunused-command-line-argument-hard-error-in-future
cc1: error: -Werror=unused-command-line-argument-hard-error-in-future: no option -Wunused-command-line-argument-hard-error-in-future
cc1: error: -Werror=unused-command-line-argument-hard-error-in-future: no option -Wunused-command-line-argument-hard-error-in-future
error: command '/usr/local/bin/gcc-7' failed with exit status 1
I have to install gcc because it doesn't accept clang. It seems gcc doesn't ignore this unknown flag. I tried
ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install mujoco_py
But it didn't help. Is there any way that gcc can ignore this unknown flag?

I finally find a solution.
First install clang-omp via brew install clang-omp. Then link clang-omp to gcc-7 via sudo ln -s /usr/local/bin/clang-omp /usr/local/bin/gcc-7. Then you should be able to run mujoco151. Note you may also need to brew uninstall glfw to solve a conflict.

installing cx_freeze on ubuntu

i have kubuntu 15.10 with python2.7 and python 3.4m installed
if i install cxfreeze with sudo apt-get cx_freeze it is installed with no issues but the problem is when i compile my python modules the are compiled for python2.7 not python 3
So i am trying to build cx_freeze from source and i am facing this error when i run sudo python3 setup.py build
here's the error
adding base module named weakref
running build
running build_py
running build_ext
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict
prototypes -g -fstack-protector-strong -Wformat -Werror=format
security -D_FORTIFY_SOURCE=2 -fPIC -Ibuild/temp.linux-x86_64-3.4
I/usr/include/python3.4m -c source/bases/Console.c -o
build/temp.linux-x86_64-3.4/source/bases/Console.o
x86_64-linux-gnu-gcc -pthread build/temp.linux-x86_64
3.4/source/bases/Console.o -L/usr/lib/python3.4/config-3.4m-x86_64
linux-gnu -lpython3.4 -o build/lib.linux-x86_64
3.4/cx_Freeze/bases/Console -Xlinker -export-dynamic -Wl,-O1 -Wl,
Bsymbolic-functions -lpthread -ldl -lutil -lm -lrt -lexpat -L/usr/lib
-lz -lexpat -s
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1'

Install zlib1g-dev :
sudo apt install zlib1g-dev
then install cx_freeze for python:
pip install cx_freeze
I've tried that on Ubuntu 18.04 and Python 3.6 and worked.

I had exactly the same problem. As suggested in this answer you just need to install the dependencies first:
pip3 install zlib1g-dev
Then sudo python3 setup.py build

I've got a CPython C++ module with C++11 code, but I can't seem to build on travis-ci

This project work fine on my local Ubuntu 12.04 and Mac OSX 10.10 (with fink python) machines. I can't seem to figure out how to configure the .travis.yml to get the .cpp files to build with g++-4.8 (4.9 or 5.x) would be fine too.
Project: https://github.com/schwehr/libais
My most recent failed attempt:
language: python
python:
- "2.7"
- "3.4"
before_install:
- sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
- sudo apt-get update -qq
- if [ "$CXX" = "g++" ]; then export CXX="g++-4.8" CC="gcc-4.8"; fi
install:
- sudo apt-get install -qq gcc-4.8 g++-4.8
- python setup.py install
script:
- python setup.py test
Gives:
gcc -pthread -fno-strict-aliasing -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/python/2.7.9/include/python2.7 -c src/libais/ais_py.cpp -o build/temp.linux-x86_64-2.7/src/libais/ais_py.o -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for Ada/C/ObjC but not for C++ [enabled by default]
cc1plus: error: unrecognized command line option ‘-std=c++11’
The key portion of my setup.py:
EXTRA_COMPILE_ARGS = []
if sys.platform in ('darwin', 'linux', 'linux2'):
EXTRA_COMPILE_ARGS = ['-std=c++11']
AIS_MODULE = Extension(
'_ais',
extra_compile_args=EXTRA_COMPILE_ARGS,

Thanks Dominic. I tried printing things and that was helpful. That got me thinking that I could just get explicit and force python to use the correct compiler. That makes it easier to see what is happening.
install:
- sudo apt-get install -qq gcc-4.8 g++-4.8
- CC=g++-4.8 python setup.py install
Which works.