Develop Reference

Python, Django Celery job interface upgrade to ubuntu 20.04 celery jobs not running

Very New to Celery/Django
Over the weekend, we upgraded the ubuntu server to 20.04, and while the web interface is up and running and I can see the defined jobs when I attempt to run any job the job does not run.
I am not seeing any errors in the logs (assuming I am looking at the correct apache logs).
Everything is down and I am not sure where to debug celery. If I could find out where the jobs are located then I would try to run using the celery CLI.
Updated from ubuntu 16.04.
It is using mod_wsgi not using a virtual env that I can tell. I believe it is using the system python version 2.7.
I've tried restarting.
Any help/pointers are much appreciated.
The 2.7 python did stick around, probably as you suggested due to the upgrade.
It looks to be the default in the path:
/home/barrow/bin:/home/barrow/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
barrow#server:~$ which python
/usr/bin/python
barrow#server:~$ python
Python 2.7.18 (default, Mar 8 2021, 13:02:45)
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
There are, of course, other versions of python sitting around.
-rwxr-xr-x 1 root root 5490352 Jun 2 04:49 /usr/bin/python3.8
lrwxrwxrwx 1 root root 33 Jun 2 04:49 /usr/bin/python3.8-config -> x86_64-linux-gnu-python3.8-config
-rwxr-xr-x 1 root root 3674216 Mar 8 06:02 /usr/bin/python2.7
lrwxrwxrwx 1 root root 33 Mar 8 06:02 /usr/bin/python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root 33 Feb 20 16:02 /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config
lrwxrwxrwx 1 root root 34 Feb 20 16:02 /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config
-rwxr-xr-x 2 root root 4723808 Feb 20 16:02 /usr/bin/python3.6
-rwxr-xr-x 2 root root 4723808 Feb 20 16:02 /usr/bin/python3.6m
lrwxrwxrwx 1 root root 7 Apr 15 2020 /usr/bin/python -> python2
lrwxrwxrwx 1 root root 14 Apr 15 2020 /usr/bin/python-config -> python2-config
lrwxrwxrwx 1 root root 9 Mar 13 2020 /usr/bin/python2 -> python2.7
lrwxrwxrwx 1 root root 16 Mar 13 2020 /usr/bin/python2-config -> python2.7-config
lrwxrwxrwx 1 root root 9 Mar 13 2020 /usr/bin/python3 -> python3.8
lrwxrwxrwx 1 root root 16 Mar 13 2020 /usr/bin/python3-config -> python3.8-config
I am able to run the celery command in the CLI, so I removed that post here. I am "searching" for where the job might be. I can't wait until I understand this.
My thoughts are that I might be able to run the job using celery on the command line. Or worst case that would help with the debugging and eliminate is as a culprit.

python not found in the container but it is installed

I am using this command:
./configure --enable-2d-adaptivity
in order to create an executable for an open source code I would like to install in a folder outside the container.
But I get this error when it stops:
checking for Fortran 77 name-mangling scheme... lower case, underscore, no extra underscore
configure: Checking for Python automagic.
configure: No Python automagic.
checking for python... no
configure: error: Cannot find python in your system path`
This code require Python 2.7 and it is installed in /usr/bin.
When I use ls -l :
Singularity> ls -l /usr/bin/python*
-rwxr-xr-x 1 root root 3628976 Sep 30 09:38 /usr/bin/python2.7
lrwxrwxrwx 1 root root 33 Sep 30 09:38 /usr/bin/python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root 9 Oct 25 2018 /usr/bin/python3 -> python3.6
lrwxrwxrwx 1 root root 16 Oct 25 2018 /usr/bin/python3-config -> python3.6-config
-rwxr-xr-x 1 root root 384 Apr 11 2018 /usr/bin/python3-futurize
-rwxr-xr-x 1 root root 388 Apr 11 2018 /usr/bin/python3-pasteurize
-rwxr-xr-x 1 root root 364 Aug 22 2016 /usr/bin/python3-qr
-rwxr-xr-x 2 root root 4526456 Oct 8 08:12 /usr/bin/python3.6
lrwxrwxrwx 1 root root 33 Oct 8 08:12 /usr/bin/python3.6-config -> x86_64-linux-gnu-python3.6-config
-rwxr-xr-x 2 root root 4526456 Oct 8 08:12 /usr/bin/python3.6m
lrwxrwxrwx 1 root root 34 Oct 8 08:12 /usr/bin/python3.6m-config -> x86_64-linux-gnu-python3.6m-config
lrwxrwxrwx 1 root root 10 Oct 25 2018 /usr/bin/python3m -> python3.6m
lrwxrwxrwx 1 root root 17 Oct 25 2018 /usr/bin/python3m-config -> python3.6m-config
So probably a solution could be to set python to point at Python 2.7?
I tried many commands but they didn't worked.
In addition when I run which python it is empty.
How can I fix it?
Should the folder be in the container instead of my home directory?
The folder in which I run the command ./configure --enable-2d-adaptivity is located in the home directory of my account on a super computer. I can see it from the container.
Thank you!

How can I tidyup and fix my Python installation on Kali

so I suspect I have a Python installation problem. I was doing some coding in python. Everything was running just fine then it just stopped loading scapy for no particular reason at all. Unable to import Scapy. I tried lots of things and in the end gave up on it as it was taking too long to figure out and wasn't really that important anyway.
So now I am doing something else that has a Python dependency. I am using Maltego with some custom transforms that I have downloaded. They are Python dependent and getting these transforms working will save me a ton of time.
The error in Maltego when it runs is:
Traceback (most recent call last):
File "/opt/maltego-nmap/units/nmap_xml_parser.py", line 4, in <module>
from lxml import etree
ImportError: No module named lxml
The python root path in maltego is shown as /usr/bin/python
So I checked my python path:
KALI606:/usr/bin# type -a python
python is /usr/bin/python
python is /bin/python
KALI606:/usr/bin# cd bin/python
bash: cd: bin/python: No such file or directory
KALI606:/usr/bin# cd usr/bin/python
bash: cd: usr/bin/python: No such file or directory
KALI606:/usr/bin# ls -l python*
lrwxrwxrwx 1 root root 7 Nov 18 06:36 python -> python2
lrwxrwxrwx 1 root root 9 Nov 18 06:36 python2 -> python2.7
-rwxr-xr-x 1 root root 3647800 Jan 19 19:54 python2.7
lrwxrwxrwx 1 root root 33 Jan 19 19:54 python2.7-config -> x86_64-linux-gnu-python2.7-config
lrwxrwxrwx 1 root root 16 Nov 18 06:36 python2-config -> python2.7-config
-rwxr-xr-x 1 root root 385 Feb 2 03:10 python2-futurize
-rwxr-xr-x 1 root root 389 Feb 2 03:10 python2-pasteurize
lrwxrwxrwx 1 root root 9 Nov 14 14:13 python3 -> python3.7
-rwxr-xr-x 2 root root 5114928 Mar 10 13:18 python3.7
lrwxrwxrwx 1 root root 33 Mar 10 13:18 python3.7-config -> x86_64-linux-gnu-python3.7-config
-rwxr-xr-x 2 root root 5114928 Mar 10 13:18 python3.7m
lrwxrwxrwx 1 root root 34 Mar 10 13:18 python3.7m-config -> x86_64-linux-gnu-python3.7m-config
-rwxr-xr-x 1 root root 5453504 Feb 25 13:04 python3.8
lrwxrwxrwx 1 root root 16 Nov 14 14:13 python3-config -> python3.7-config
-rwxr-xr-x 1 root root 384 Feb 2 03:10 python3-futurize
-rwxr-xr-x 1 root root 398 Sep 7 2018 python3-jsonschema
lrwxrwxrwx 1 root root 10 Nov 14 14:13 python3m -> python3.7m
lrwxrwxrwx 1 root root 17 Nov 14 14:13 python3m-config -> python3.7m-config
-rwxr-xr-x 1 root root 388 Feb 2 03:10 python3-pasteurize
-rwxr-xr-x 1 root root 364 Dec 15 18:09 python3-qr
-rwxr-xr-x 1 root root 196 Oct 6 2019 python3-tor-prompt
-rwxr-xr-x 1 root root 5902 Oct 1 2018 python3-wsdump
lrwxrwxrwx 1 root root 14 Nov 18 06:36 python-config -> python2-config
lrwxrwxrwx 1 root root 7 Feb 4 14:30 python-faraday -> faraday
This is a mess right?
Any suggestions on how I can sort out Python pls? I feel like I want to just remove all python references and start clean but I have limited knowledge on this. The Maltego error is very similar to the scapy error I had.
I am running the latest version of Kali.

Try to install python2-lxml with something like
sudo apt-get install python2-lxml

Import error: oracle client library with python wsgi

When I deployment don't get any problem to run, but when I trying run with apache wsgi got this error: DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory"
Tried to use ldconfig and setup on environment LD_LIBRARY_PATH but didn't worked. The server is a linux x64 with cx_oracle 7.2.0 and django 1.11.22 with python 2.7.15+ and installing libaio.
root#webservice:/opt/instantclient_11_2# ls -la
lrwxrwxrwx 1 root root 17 Jul 10 15:41 libclntsh.so -> libclntsh.so.11.1
-rwxrwxr-x 1 root root 53865194 Aug 24 2013 libclntsh.so.11.1
-r-xr-xr-x 1 root root 7996693 Aug 24 2013 libnnz11.so
lrwxrwxrwx 1 root root 15 Jul 10 15:41 libocci.so -> libocci.so.11.1
-rwxrwxr-x 1 root root 1973074 Aug 24 2013 libocci.so.11.1
-rwxrwxr-x 1 root root 118738042 Aug 24 2013 libociei.so
-r-xr-xr-x 1 root root 164942 Aug 24 2013 libocijdbc11.so
-r--r--r-- 1 root root 2091135 Aug 24 2013 ojdbc5.jar
-r--r--r-- 1 root root 2739616 Aug 24 2013 ojdbc6.jar
-rwxrwxr-x 1 root root 192365 Aug 24 2013 uidrvci
-rw-rw-r-- 1 root root 66779 Aug 24 2013 xstreams.jarre
root#webservice:/opt/instantclient_11_2# cat /etc/ld.so.conf.d/oracle-instantclient.conf
/opt/instantclient_11_2

export LD_RUN_PATH:
export LD_RUN_PATH=$LD_LIBRARY_PATH

Using virtualenv and error message: - setuptools pip wheel failed with error code 1

There has been quite a few installations of Python and Anacondas on my Mac over the years, and to some extent these have been neglected and I suspect now resurface and cause me issues.
I'd like to setup a virtualenv environment and did a pip install virtualenv. I'd like to create a new environement, Django18, for Django1.8, but after executing the command virtualenv Django18 I receive an error message:
OSError: Command ~/Desktop/Django18/bin/python - setuptools pip wheel failed with error code 1
bash_profile
# added by Anaconda2 4.2.0 installer
export PATH="~/anaconda2/bin:$PATH"
command: which python
~/anaconda2/bin/python
In /usr/bin/:
-rwxr-xr-x 2 root wheel 58416 Jul 15 2015 python
-rwxr-xr-x 5 root wheel 925 Sep 10 2014 python-config
lrwxr-xr-x 1 root wheel 75 Nov 9 2014 python2.6 -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
lrwxr-xr-x 1 root wheel 82 Nov 9 2014 python2.6-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6-config
lrwxr-xr-x 1 root wheel 75 Nov 9 2014 python2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
lrwxr-xr-x 1 root wheel 82 Nov 9 2014 python2.7-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config
-rwxr-xr-x 2 root wheel 58416 Jul 15 2015 pythonw
lrwxr-xr-x 1 root wheel 76 Nov 9 2014 pythonw2.6 -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/pythonw2.6
lrwxr-xr-x 1 root wheel 76 Nov 9 2014 pythonw2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7
.profile
export PATH="/opt/local/lib/gmt4/bin:$PATH"
export PATH="~/teqc:$PATH"
export PATH="/usr/local/runpkr000:$PATH"
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
# MacPorts Installer addition on 2015-04-02_at_16:40:23: adding an appropriate PATH variable for use with MacPorts.
# export PATH="/opt/local/bin:/opt/local/sbin:$PATH"
# Finished adapting your PATH environment variable for use with MacPorts.
# added by Anaconda 2.3.0 installer
# export PATH="~/anaconda/bin:$PATH"
I suspect there needs to be some clearing up with this, but how can I resolve this error message and get virtualenv to work as expected? I have tried several methods already out there, but it's difficult for me to recognise what exactly in my case is causing this issue, before applying a potential remedy.
/opt/local/bin (is that macports installation)?:
lrwxr-xr-x 1 root admin 24 Oct 9 2016 python -> /opt/local/bin/python2.7
lrwxr-xr-x 1 root admin 31 Oct 9 2016 python-config -> /opt/local/bin/python2.7-config
lrwxr-xr-x 1 root admin 73 Dec 14 2014 python2.7 -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
lrwxr-xr-x 1 root admin 80 Dec 14 2014 python2.7-config -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config
lrwxr-xr-x 1 root admin 25 Oct 9 2016 pythonw -> /opt/local/bin/pythonw2.7
lrwxr-xr-x 1 root admin 74 Dec 14 2014 pythonw2.7 -> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7
/usr/Local/bin
lrwxr-xr-x 1 root staff 69 Dec 12 2012 python3 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3
lrwxr-xr-x 1 root staff 72 Dec 12 2012 python3-32 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3-32
lrwxr-xr-x 1 root staff 76 Dec 12 2012 python3-config -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3-config
lrwxr-xr-x 1 root staff 71 Dec 12 2012 python3.3 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3.3
lrwxr-xr-x 1 root staff 74 Dec 12 2012 python3.3-32 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3.3-32
lrwxr-xr-x 1 root staff 78 Dec 12 2012 python3.3-config -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3.3-config
lrwxr-xr-x 1 root staff 72 Dec 12 2012 python3.3m -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3.3m
lrwxr-xr-x 1 root staff 79 Dec 12 2012 python3.3m-config -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/python3.3m-config
lrwxr-xr-x 1 root staff 70 Dec 12 2012 pythonw3 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/pythonw3
lrwxr-xr-x 1 root staff 73 Dec 12 2012 pythonw3-32 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/pythonw3-32
lrwxr-xr-x 1 root staff 72 Dec 12 2012 pythonw3.3 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/pythonw3.3
lrwxr-xr-x 1 root staff 75 Dec 12 2012 pythonw3.3-32 -> ../../../Library/Frameworks/Python.framework/Versions/3.3/bin/pythonw3.3-32
import sys
sys.path
~/anaconda2/lib/python27.zip
~/anaconda2/lib/python2.7
~/anaconda2/lib/python2.7/plat-darwin
~/anaconda2/lib/python2.7/plat-mac
~/anaconda2/lib/python2.7/plat-mac/lib-scriptpackages
~/anaconda2/lib/python2.7/lib-tk
~/anaconda2/lib/python2.7/lib-old
~/anaconda2/lib/python2.7/lib-dynload
~/anaconda2/lib/python2.7/site-packages
~/anaconda2/lib/python2.7/site-packages/Sphinx-1.4.6-py2.7.egg
~/anaconda2/lib/python2.7/site-packages/aeosa
~/anaconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg
/opt/local/Library/Frameworks:
Python.framework
/System/Library/Frameworks
Python.framework
EDIT
which python produces :
~/anaconda2/bin/python
which -a python produces :
~/anaconda2/bin/python
/usr/bin/python
They're not the same. But then after deleting python installations in:
/opt/local/bin
/usr/Local/bin
/usr/bin
and using rm -rf python* and sudo rm -rf python* in
/opt/local/Library/Frameworks
the which outputs then become the same.
I've realised that the which python command actually refers to what is written in the .bash_profile file and not .profile (priority to .bash_profile).
In any case I commented out the $PATH in my .bash_profile, so forcing the environment variables only to be used in the .profile file. Then source ~/.bash_profile.
In my .profile I added:
export PATH="/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages:$PATH
Then source ~/.profile
However after which python I still get:
~/anaconda2/bin/python
This is getting a bit frustrating as even changing the relevant files and deleting certain python installations, the result is still the same :-/

For me the best workaround was to check first in which -a python, in my case there are two occurrences:
~/anaconda2/bin/python
/usr/bin/python
For some unknown reason, the virtualenv command doesn't seem to like the anaconda installation of python, so use instead:
virtualenv -p /usr/bin/python2.7 my_environment

I had the same error but then i just realized that i was trying to create a virtualenv folder (C:/myname/virtual_folder) at the same directory level as was my Python installation folder (C:/myname/python27) .
So I just created a folder and then inside of it (C:/myname/newfolder/..) i run ("virtualenv virtual_folder") with no issues .

If you have several disparate installations for python and anaconda, th only thing I can recommend is to just clean up. But also interested what other will tell.