On my Windows 10 machine, I am developing a database manager. Because the backend uses LDAP and the required development libraries are only available for Linux, I want to use Docker to set up an environment with the appropriate libs.
I managed to write a Dockerfile and compose file, that launch the (currently very basic) Django app in a Docker container with all the libs necessary.
I would like to play around with the django-ldapdb package and for that I want to apply the migrations.
When I open PyCharm's terminal and try to execute python manage.py migrate, I get an error telling me that the module ldapdb is not found. I suspect this is because the command does not use the remote Docker interpreter I set up with PyCharm.
The other thing I tried is using PyCharm's dedicated manage.py console. This does not initialize properly. It says the working directory is invalid and needs to be an absolute path, although the path it shows it the absolute path to the project.
I have to admit that I have no idea how this remote interpreter works and I don't see any Docker container running, so I might have not understood something properly here. I even tried running the app using PyCharm's Django run config, which started a container, but still I get the same errors.
I googled a lot, but I couldn't find more infos about remote interpreters nor something solving my issue.
The only way I managed to do this, is by executing the command inside the container.
To get inside a container named contr, use the docker command
docker exec -ti contr /bin/bash
Related
I recently followed the following tutorial to try to debug python code in a Docker container using VSCode:
https://www.youtube.com/watch?v=qCCj7qy72Bg&t=374s
My Dockerfile looks like this:
FROM ubuntu as base
#Do standard image stuff here
#Python Debugger
From base as debugger
RUN pip3 install debugpy
ENTRYPOINT ["python3","-m","debugpy","--listen","0.0.0.0:5678","--wait-for-client"]
I have alternately tried copying the tutorial exactly and using the following ENTRYPOINT instead:
ENTRYPOINT ["python3","-m","debugpy","--listen","0.0.0.0:5678","--wait-for-client","-m"]
I have also configured a VSCode remote attach debug instance to launch.json:
{"name":"Python: Remote Attach","type":"python","request":"attach","connect":{"host":"5678","port":5678},"pathMappings":[{"localRoot":"${workspaceFolder}","remoteRoot":"."}]},
I want the debugger to either debug the current file alone in isolation, or run a file I use to run the entire project, called init.py with the debugger in the docker container.
Currently, when I build and run the docker container with
docker run -p 5678:5678 CONTAINERNAME python3 /home/init.py
It hangs and times out on the Visual Studio side.
In the video, he uses this to run the python unittest module, which is why I tried taking out the -m from the end of the command in my modified version. However, it looks like debugpy doesn't know what to do. I have tried running the docker instance before the remote debugger, or the remote debugger after the docker instance, but the error remains and the debug does not work. How can I remote debug into a docker instance using VSCode?
EDIT:
Thank you FlorianLudwig for pointing out that my original code used commas for the IP rather than the periods required.
I have edited the question to reflect this change. It removed issues where python complained about a malformed address, but it seems I am still having some sort of connection issue to the debugger.
EDIT2:
I think I figured out what caused the connection issue. It appears the visual studio default is to use the same host as the port number in question. I changed my host to 0.0.0.0 and I was able to debug by running the container then connecting to it via Visual Studio Debugging.
In your Dockerfile:
"0,0,0,0:5678" should be "0.0.0.0:5678"
To make it a valid ip address. 0.0.0.0 basically means "any" ip address.
I am running succesfully a django app that is hosted inside a docker container. I change something on my code on purpose in order for my code to break. What I need is somehow to see the log of the running code as if I was running this locally on my computer. For example I forgot to import a library and when I run this locally I get a message on the terminal like "ModuleNotFoundError: No module named 'somemodule'". But when i run the same code from inside the container I get no log, just the container fails to start.
My question is: How can I get a log for my script from inside the container, so I can debug my code?
So, what I wanted to do was to somehow debug/run my own python code that was inside a container in order to see the log of my code.
I managed to do it using VSC and Remote SSH and Remote - Containers extensions.
Remote SSH
Remote - Containers
If the containers are hosted locally on your PC, you dont need the Remote - SSH extension
I am trying to do auto-deployment of a Python Flask application using Jenkins and then run it by using shell command on a Raspberry Pi server.
Here are some background info,
Before using Jenkins, my deployment and execution process was manual described below:
FTP to the directory where my Python scripts and Python venv are located
Replace Flask application scripts using FTP
Activate virtual environment to of Python(3.5) through the terminal on Raspberry Pi ("./venv/bin/activate")
Run myFlaskApp.py by executing "python myFlaskApp.py" in terminal
Now I have integrated Jenkins with the deployment/execution process described below:
Code change pushed to github
Jenkins automatically pulls from github
Jenkins deploy files to specified directories by executing shell commands
Jenkins then activates virtual environment and run myFlaskApp.py by bashing a .sh script in the shell terminal.
Now the problem that I am having is on step 4, because a Flask app has to always be alive, my Jenkins will never "finish building successfully", it will always be in a loading state as the Flask app is running on the shell terminal Jenkins is using.
Now my question:
What is the correct approach that I should be taking in order to activate myFlaskApp.py with Jenkins after deploying the files while not causing it to be "locked down" by the build process?
I have read up about Docker, SubShell and the Linux utility "Screen". Will any of these tools be useful to assist me in my situation right now and which approach should I be taking?
The simple and robust solution (in my opinion) is to use Supervisor which is available in Debian as supervisor package. It allows you do make a daemon from script like your app, it can spawn multiple processes, watch if app doesn't crash and if it does it can start it again.
Note about virtualenv - you don't need to activate venv to use it. You just need to point appropriate Python executable (your_venv/bin/python) instead of default one. For example:
$ ./venv/bin/python myFlaskApp.py
You need to create these files for deployment over jenkins.
Code can be found: https://github.com/ishwar6/django_ci_cd
This will work for both flask as well as django.
initial-setup.sh - This file is the first file to look at when setting up this project. It installs the required packages to make this project work such as Nginx, Jenkins, Python etc. Refer to the youtube video to see how and when it is used.
Jenkinsfile - This file contains the definition of the stages in the pipeline. The stages in this project's pipeline are Setup Python Virtual Environment, Setup gunicorn service and Setup Nginx. The stages in this pipeline just does two things. First it makes a file executable and then runs the file. The file carries out the commands that is described by the stage description.
envsetup.sh - This file sets up the python virtual environment, installs the python packages and then creates log files that will be used by Nginx.
gunicorn.sh - This file runs some Django management commands like migration commands and static files collection commands. It also sets up the gunicorn service that will be running the gunicorn server in the background.
nginx.sh - This file sets up Nginx with a configuration file that points Nginx to the gunicorn service that is running our application. This allows Nginx serve our application. I have followed a digital ocean article to setup this file. You can go through the video once to replicate sites-available and sites-enabled scanerio.
app.conf - This is an Nginx server configuration file. This file is used to setup Nginx as proxy server to gunicorn. For this configuration to work, change the value of server_name to the IP address or domain name of your server.
The docker plugin has a debug port for connecting to a container
I have a python application, but according to the docs the debug port is only supported for java.
How can I set breakpoints and debug my python container in intellij? Is there some way I could have the python container connect to the intellij python debugger?
Edit: I'm running Windows 10, docker for Windows, and the container is linux. Perhaps I need to manually setup some kind of remote debugging for the intellij Python debugger? Also, might as well ask, must I have the professional version for remote debugging or is there a workaround using community?
You can do that using Python Remote Debugging. Open the configurations window and click on + -> Python Remote Debug
Then you either set a port or keep it blank for Pycharm to find a available port.
Then click on Debug icon to launch the debug server which will show below kind of message
Starting debug server at port 57588
Use the following code to connect to the debugger:
import pydevd
pydevd.settrace('localhost', port=57588, stdoutToServer=True, stderrToServer=True)
Waiting for process connection...
Now you need to setup pydev debugging inside docker. You will need the pycharm-debug-py3k.egg for this. For me I copied to my current Dockerfile folder like below
cp "/Users/tarun.lalwani/Library/Application Support/IntelliJIdea2017.2/python/pycharm-debug-py3k.egg" .
The location for your will change based on the IntelliJ version installed. After that, we need to edit our Dockerfile
FROM python:3.6
WORKDIR /app
ENV PYTHONPATH=/app:/app/debug
COPY pycharm-debug-py3k.egg /app/debug
COPY debug_test.py /app/
CMD python debug_test.py
The debug_test.py when built will have below lines at the top
import pydevd
pydevd.settrace('docker.for.mac.localhost', port=55507, stdoutToServer=True, stderrToServer=True)
Note: I have used docker.for.mac.localhost as I use docker for mac, but if use Docker for windows then use docker.for.win.localhost. For toolbox or linux you will add the IP of your machine
Since it is docker, we probably want to keep port fixed instead of dynamic like I did. Now we build the docker file and run it.
This will open a popup in pycharm, click autodetect to detect the source mappings
And then you will have your code breakpointed at the main line of your file
We have a project on nginx/Django, using VirtualBox.
When we try to run command VBoxManage list runningvms in nginx, we have the next error:
Failed to initialize COM because the global settings directory '/.config/VirtualBox' is not accessible!
If we run this command in console, it works fine.
What can we do to make it working good in nginx?
Other details:
nginx is runned by user "www-data", console - by the other user (Administrator).
We have fixed the issue.
There was wrong environment variable "Home" (os.environ['HOME']). We changed it, and so the problem was gone.
Using Python API for VB instead of ssh can really help you with that problem, as RegularlyScheduledProgramming suggested - we added Python API too.
Thanks!