Develop Reference

Azure DevOps AzureFunctionApp#1 not installing python dependencies

Using the Azure Devops task with current setup :
task: AzureFunctionApp#1
displayName: Deploy Lab
inputs:
azureSubscription: 'serviceConnection'
appType: 'functionAppLinux'
appName: 'myAwesomeApp'
package: '.'
runtimeStack: 'PYTHON|3.8'
Also, in the root directory (ie . in the ado task), I have a requirements.txt file that contains numpy
The deployment runs successfully but when I test the app, I have the following error:
Failure Exception: ModuleNotFoundError: No module named 'numpy'
Which simply means that the requirements.txt file is not considered or pip install -r requirements.txt didn't run successfully in the remote compute, what am I doing wrong ?
EDIT: Added a step to install the packages locally did work, but as numpy is a C built library it's dependent on the host machine and that doesn't work in the remote function app :lol:
The step that I added is this one:
- bash: pip install -r requirements.txt --target="./.python_packages/lib/site-packages"
displayName: 'Install dependencies'
And the error:
Result: Failure Exception: ImportError: IMPORTANT: PLEASE READ THIS
FOR ADVICE ON HOW TO SOLVE THIS ISSUE! Importing the numpy
C-extensions failed. This error can happen for many reasons, often due
to issues with your setup or how NumPy was installed. We have compiled
some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html Please
note and check the following: * The Python version is: Python3.9 from
"/usr/local/bin/python" * The NumPy version is: "1.22.3" and make sure
that they are the versions you expect. Please carefully study the
documentation linked above for further help. Original error was: No
module named 'numpy.core._multiarray_umath' . Troubleshooting Guide:
https://aka.ms/functions-modulenotfound Stack: File
"/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py",
line 305, in _handle__function_load_request func =
loader.load_function( File
"/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/utils/wrappers.py",
line 42, in call raise extend_exception_message(e, message) File
"/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/utils/wrappers.py",
line 40, in call return func(*args, **kwargs) File
"/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/loader.py",
line 85, in load_function mod = importlib.import_module(fullmodname)
File "/usr/local/lib/python3.9/importlib/init.py", line 127, in
import_module return _bootstrap._gcd_import(name[level:], package,
level) File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in
_find_and_load_unlocked File "", line 680, in _load_unlocked File "",
line 850, in exec_module File "", line
228, in _call_with_frames_removed File
"/home/site/wwwroot/bilans/main.py", line 24, in from .
import bilan File "/home/site/wwwroot/bilans/bilan.py", line 16, in
import numpy as np File
"/home/site/wwwroot/.python_packages/lib/site-packages/numpy/init.py",
line 144, in from . import core File
"/home/site/wwwroot/.python_packages/lib/site-packages/numpy/core/init.py",
line 49, in raise ImportError(msg)

Maybe you will have to use python 3.9 and the latest ubuntu agent in the pipeline
https://github.com/Azure/azure-functions-python-worker/issues/904

Azure / Django / Celery / Ubuntu | tkinter & libtk8.6.so import issue

UPDATE / SOLUTION
Per Sytech's answer....
Did not realize that the build was in Ubuntu which has all the packages but when Azure deploys it to a Linux container, the needed packages were missing.
Like in other questions/answers just add these installs to a startup script that Azure will use
ex.
#!/bin/bash
apt-get update
apt-get install tk --yes
python manage.py wait_for_db
python manage.py migrate
gunicorn --bind=0.0.0.0 --timeout 600 app.wsgi --access-logfile '-' --error-logfile '-' &
celery -A app worker -l info --uid=1
Original Post:
When Azure builds & deploys a Python3.9 Django/Django-Rest WebApp it has been failing in it's start up.
Error in question ( full logs below )
2022-03-08T21:13:30.385999188Z File "/tmp/8da0147da65ec79/core/models.py", line 1, in <module>
2022-03-08T21:13:30.386659422Z from tkinter import CASCADE
2022-03-08T21:13:30.387587669Z File "/opt/python/3.9.7/lib/python3.9/tkinter/__init__.py", line 37, in <module>
2022-03-08T21:13:30.387993189Z import _tkinter # If this fails your Python may not be configured for Tk
2022-03-08T21:13:30.388227101Z ImportError: libtk8.6.so: cannot open shared object file: No such file or directory
I have come across other answers to this needing to make sure that tkinter is installed with sudo apt-get python3-tk which I have added to the deployment yml file
Though it still seems to have issue. Reverting back to previous code for deployment is successful and the only feature that has been added to the application is Celery. Not sure if that has anything to do with it or not.
Am I adding the installation of the tk/tkinter in the wrong sequence?
When I revert the to previous code and have a successful build/deploy I ssh into the container and run the python shell and try to manually import the tkinter module.
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/opt/python/3.9.7/lib/python3.9/tkinter/__init__.py", line 37, in <module>
import _tkinter # If this fails your Python may not be configured for Tk
ImportError: libtk8.6.so: cannot open shared object file: No such file or directory
it errors out like expected.
when I run apt-get update && apt-get install python3-tk --yes manually in the container, then go back to the shell on the container there is not error importing tkinter.
Which leads me to believe something is not installing in the right place? virtualenv? Or is being overwritten in the build process?
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python version
uses: actions/setup-python#v1
with:
python-version: "3.9"
- name: Create and start virtual environment
run: |
python -m venv venv
source venv/bin/activate
- name: Install TK dependency
run: |
sudo apt-get update
sudo apt-get install python3-tk
- name: Install dependencies
run: pip install -r requirements.txt
- name: Upload artifact for deployment jobs
uses: actions/upload-artifact#v2
with:
name: python-app
path: |
.
!venv/
App Log spit out below...
2022-03-08T21:13:27.830330743Z Updated PYTHONPATH to ':/opt/startup/code_profiler:/tmp/8da0147da65ec79/antenv/lib/python3.9/site-packages'
2022-03-08T21:13:30.370903021Z Traceback (most recent call last):
2022-03-08T21:13:30.371872470Z File "/tmp/8da0147da65ec79/manage.py", line 22, in <module>
2022-03-08T21:13:30.372648510Z main()
2022-03-08T21:13:30.373176037Z File "/tmp/8da0147da65ec79/manage.py", line 18, in main
2022-03-08T21:13:30.373892773Z execute_from_command_line(sys.argv)
2022-03-08T21:13:30.374862922Z File "/tmp/8da0147da65ec79/antenv/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_comma
nd_line
2022-03-08T21:13:30.374880323Z utility.execute()
2022-03-08T21:13:30.378586012Z File "/tmp/8da0147da65ec79/antenv/lib/python3.9/site-packages/django/core/management/__init__.py", line 420, in execute
2022-03-08T21:13:30.378603012Z django.setup()
2022-03-08T21:13:30.378607713Z File "/tmp/8da0147da65ec79/antenv/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
2022-03-08T21:13:30.378612113Z apps.populate(settings.INSTALLED_APPS)
2022-03-08T21:13:30.378679216Z File "/tmp/8da0147da65ec79/antenv/lib/python3.9/site-packages/django/apps/registry.py", line 116, in populate
2022-03-08T21:13:30.378689817Z app_config.import_models()
2022-03-08T21:13:30.378694417Z File "/tmp/8da0147da65ec79/antenv/lib/python3.9/site-packages/django/apps/config.py", line 304, in import_models
2022-03-08T21:13:30.379003533Z self.models_module = import_module(models_module_name)
2022-03-08T21:13:30.381756173Z File "/opt/python/3.9.7/lib/python3.9/importlib/__init__.py", line 127, in import_module
2022-03-08T21:13:30.383257849Z return _bootstrap._gcd_import(name[level:], package, level)
2022-03-08T21:13:30.383423757Z File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
2022-03-08T21:13:30.383857479Z File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
2022-03-08T21:13:30.384148694Z File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
2022-03-08T21:13:30.384836329Z File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
2022-03-08T21:13:30.384850030Z File "<frozen importlib._bootstrap_external>", line 850, in exec_module
2022-03-08T21:13:30.385281052Z File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
2022-03-08T21:13:30.385999188Z File "/tmp/8da0147da65ec79/core/models.py", line 1, in <module>
2022-03-08T21:13:30.386659422Z from tkinter import CASCADE
2022-03-08T21:13:30.387587669Z File "/opt/python/3.9.7/lib/python3.9/tkinter/__init__.py", line 37, in <module>
2022-03-08T21:13:30.387993189Z import _tkinter # If this fails your Python may not be configured for Tk
2022-03-08T21:13:30.388227101Z ImportError: libtk8.6.so: cannot open shared object file: No such file or directory
2022-03-08T21:13:36.193Z ERROR - Container <container_name>_0_fd6a978c for site <container_name> has exited, failing site start

Tkinter is already included in the ubuntu-latest image. No particular setup is needed.
jobs:
verify-tkinter:
name: verify-tkinter
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python version
uses: actions/setup-python#v1
with:
python-version: "3.9"
- name: show tk version
run: |
python -c "import tkinter;print(tkinter.TkVersion)"
If this error is occurring after deployment, you need to install tkinter in your deployment environment, which is separate from GitHub Actions runs.
On your server is running Ubuntu 20 and, make sure the tk package is installed, which provides the libtk8.6.so file needed.
apt install -y tk

I came across this error because a simple mistake.
The IDE add from turtle import up to my .py and I didn't notice

Module not found - how to install a package in a Django + poetry project?

This is such a basic question, I'm sorry. I installed django-parsley with poetry (poetry add django-parsley). It's clearly installed in my pyproject.toml file.
In my django project files, in forms.py, I have a line of code that imports a module from parsley: from parsley.decorators import parsleyfy
However, when I try to run python manage.py runserver, I get the following error:
from parsley.decorators import parsleyfy
ModuleNotFoundError: No module named 'parsley'
I also tried adding 'parsley' to my INSTALLED_APPS in settings.py. That gives me this error (which is maybe due to not adding it globally with pip install?):
...some more errors...
File "C:\Program Files\Python310\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'parsley'
What do I need to do to be able to import it in a python file in my project?

I figured it out. It's actually a VSCode issue - normally, VSCode automatically identifies the right virtual environment for the project (in this case, Poetry's default auto-created project-specific venv).
However, in this project, it didn't switch over. To fix, I ran the Python: Select Interpreter command and switched the venv over to the right project. It then recognized the site-packages folder and was able to import normally.

The problem involved WSL, Gunicorn, Docker and Flask

I am working on Windows 10 Pro, Git Bash, Docker Desktop.
Now I have a project which runs a Flask application in Docker through Gunicorn.
The entrypoint in Dockerfile:
ENTRYPOINT ["gunicorn", "-b",":8080","main.py"]
When run below command:
docker run -p 127.0.0.1:80:8080 jwt-api-test
It shows the error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
mod = importlib.import_module(module)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 962, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'main.py'; 'main' is not a package
If I am right, it is related to gunicorn, which isn't available in Windows.
After googling, it seems wsl is an option. In fact, there is wsl (already being turned on, running in Docker Desktop), info as below:
wsl.exe --list --all --verbose
NAME STATE VERSION
* docker-desktop-data Running 2
docker-desktop Running 2
When I clicked the wsl.exe, and tried to open bash, it didn't work: no error, just nothing happened. I did use shift +restart according to some instructions, but it didn't work either.
May I ask for your help on how to make this Flask application works? Thanks.
Edited: The structure of main.py:
JWT_SECRET = os.environ.get('JWT_SECRET', 'abc123abc1234')
LOG_LEVEL = os.environ.get('LOG_LEVEL', 'INFO')
LOG = _logger()
LOG.debug("Starting with log level: %s" % LOG_LEVEL )
APP = Flask(__name__)
if __name__ == '__main__':
APP.run(host='127.0.0.1', port=8080, debug=True)
The Dockerfile:
FROM python:stretch
COPY . /app
WORKDIR /app
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
ENTRYPOINT ["gunicorn", "-b", ":8080", "main.py"]

Assuming that the rest of your setup is correct (relative paths, ports, Dockerfile, etc.), the problem could be passing main.py to gunicorn.
Usually you need to pass your Flask variable, i.e in your case replace "main.py" in your ENTRYPOINT with "main.APP" (see docs)
Apart from that: If you get the container running it might be the case that you cannot reach your API. In this case change your gunicorn binding to "0.0.0.0:8080" in your ENTRYPOINT.

Problem with azure-cognitiveservices-speech on heroku

I would like deploy my app to heroku server.
When I send request i response this log:
2020-05-26T13:30:56.122694+00:00 app[web.1]: _speech_py_impl = swig_import_helper()
2020-05-26T13:30:56.122695+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/azure/cognitiveservices/speech/speech_py_impl.py", line 16, in swig_import_helper
2020-05-26T13:30:56.122695+00:00 app[web.1]: return importlib.import_module('_speech_py_impl')
2020-05-26T13:30:56.122696+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/importlib/init.py", line 126, in import_module
2020-05-26T13:30:56.122696+00:00 app[web.1]: return _bootstrap._gcd_import(name[level:], package, level)
2020-05-26T13:30:56.122702+00:00 app[web.1]: ModuleNotFoundError: No module named '_speech_py_impl'
I have problem with install ModuleNotFoundError: No module named '_speech_py_impl'
My file requirement.txt
Flask
requests
gunicorn
azure-cognitiveservices-speech
Also I have a problem run this command on heroku:
sudo apt-get install build-essential libssl1.0.0 libasound2
How to solve this problem?

Hello I got this problem too.
Here's how I fix it.
Add this buildpack to your heroku app.
https://github.com/hopkinschris/heroku-buildpack-apt
Create a file call Aptfile in your project.
Put the name of package you want to install in Aptfile just like the requirement.txt
build-essential
libssl1.0.0
libasound2