Develop Reference

Serve multiple web requests in Django

I am using Django with Nginx and want to serve multiple requests in parallel.
We have Docker configuration and one pod has 10 cores. I am trying to create multiple workers in uWSGI like (uwsgi --socket /tmp/main.sock --module main.wsgi --enable-threads --master --processes=10 --threads=1 --chmod-socket=666)
Request first lands to view and from there it calls service file which does heavy work.
Actually, I am using openCV library in service file which has loop over all pixels to remove colored ones(pretty time consuming..)
I also tried using multiple cores and 1 worker as
(uwsgi --socket /tmp/main.sock --module main.wsgi --enable-threads --master --processes=1 --threads=10 --chmod-socket=666).
But still performance did not improve. I think it is due to GIL which is getting acquired while doing heavy I/O operations, not sure how I can find a work around it. Or use all cores in some other efficient way? TIA!

django in docker not detecting SIGINT

This is a funny stackover flow question, because I have an answer, but the answer is a few years old. I can't find much content which is new, yet it seems like it would be quite high profile.
I am using docker-compose to start a few containers. Two of them use standard postgres and redis images.
The others are django 2.2.9 (and celery) This is a development environment, and I start them with docker compose, like this:
command: ./manage.py runserver 0.0.0.0:80
docker-compose stop sends a SIGINT. The redis and postgres containers exit quickly.
the django containers don't. docker-compose stop loses patience and kills them.
(and pycharm has infinite patience currently, and doesn't send a kill until I force it).
This post from 2015 referring to Django 1.9 (http://blog.lotech.org/fix-djangos-runserver-when-run-under-docker-or-pycharm.html) says that
"The quick fix is to specifically listen for SIGINT and SIGTERM in
your manage.py, and sys.kill() when you get them. So modify your
manage.py to add a signal handler:"
and it says how. The fix to change manage.py to catch SIGINT works and it's a handful of lines, although it doesn't work for celery which has its own startup.
So I can carry forward my own version of of manage.py and fix celery, but really is this still how to fix this?
I see the the dockerfile could have
STOPSIGNAL SIGINT
but it doesn't make and difference, I suppose because the entry point is managed by docker-compose.

Use the list variant of command:
command: ["./manage.py", "runserver", "0.0.0.0:80"]
See https://hynek.me/articles/docker-signals/ for details why.

How can I run luigid and luigi task within docker? [duplicate]

I have built a base image from Dockerfile named centos+ssh. In centos+ssh's Dockerfile, I use CMD to run ssh service.
Then I want to build a image run other service named rabbitmq,the Dockerfile:
FROM centos+ssh
EXPOSE 22
EXPOSE 4149
CMD /opt/mq/sbin/rabbitmq-server start
To start rabbitmq container，run：
docker run -d -p 222:22 -p 4149:4149 rabbitmq
but ssh service doesn't work, it sense rabbitmq's Dockerfile CMD override centos's CMD.
How does CMD work inside docker image?
If I want to run multiple service, how to? Using supervisor?

You are right, the second Dockerfile will overwrite the CMD command of the first one. Docker will always run a single command, not more. So at the end of your Dockerfile, you can specify one command to run. Not more.
But you can execute both commands in one line:
FROM centos+ssh
EXPOSE 22
EXPOSE 4149
CMD service sshd start && /opt/mq/sbin/rabbitmq-server start
What you could also do to make your Dockerfile a little bit cleaner, you could put your CMD commands to an extra file:
FROM centos+ssh
EXPOSE 22
EXPOSE 4149
CMD sh /home/centos/all_your_commands.sh
And a file like this:
service sshd start &
/opt/mq/sbin/rabbitmq-server start

Even though CMD is written down in the Dockerfile, it really is runtime information. Just like EXPOSE, but contrary to e.g. RUN and ADD. By this, I mean that you can override it later, in an extending Dockerfile, or simple in your run command, which is what you are experiencing. At all times, there can be only one CMD.
If you want to run multiple services, I indeed would use supervisor. You can make a supervisor configuration file for each service, ADD these in a directory, and run the supervisor with supervisord -c /etc/supervisor to point to a supervisor configuration file which loads all your services and looks like
[supervisord]
nodaemon=true
[include]
files = /etc/supervisor/conf.d/*.conf
If you would like more details, I wrote a blog on this subject here: http://blog.trifork.com/2014/03/11/using-supervisor-with-docker-to-manage-processes-supporting-image-inheritance/

While I respect the answer from qkrijger explaining how you can work around this issue I think there is a lot more we can learn about what's going on here ...
To actually answer your question of "why" ... I think it would for helpful for you to understand how the docker stop command works and that all processes should be shutdown cleanly to prevent problems when you try to restart them (file corruption etc).
Problem: What if docker did start SSH from it's command and started RabbitMQ from your Docker file? "The docker stop command attempts to stop a running container first by sending a SIGTERM signal to the root process (PID 1) in the container." Which process is docker tracking as PID 1 that will get the SIGTERM? Will it be SSH or Rabbit?? "According to the Unix process model, the init process -- PID 1 -- inherits all orphaned child processes and must reap them. Most Docker containers do not have an init process that does this correctly, and as a result their containers become filled with zombie processes over time."
Answer: Docker simply takes that last CMD as the one that will get launched as the root process with PID 1 and get the SIGTERM from docker stop.
Suggested solution: You should use (or create) a base image specifically made for running more than one service, such as phusion/baseimage
It should be important to note that tini exists exactly for this reason, and as of Docker 1.13 and up, tini is officially part of Docker, which tells us that running more than one process in Docker IS VALID .. so even if someone claims to be more skilled regarding Docker, and insists that you absurd for thinking of doing this, know that you are not. There are perfectly valid situations for doing so.
Good to know:
https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
http://www.techbar.me/stopping-docker-containers-gracefully/
https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/
https://github.com/phusion/baseimage-docker#docker_single_process

The official docker answer to Run multiple services in a container.
It explains how you can do it with an init system (systemd, sysvinit, upstart) , a script (CMD ./my_wrapper_script.sh) or a supervisor like supervisord.
The && workaround can work only for services that starts in background (daemons) or that will execute quickly without interaction and release the prompt. Doing this with an interactive service (that keeps the prompt) and only the first service will start.

To address why CMD is designed to run only one service per container, let's just realize what would happen if the secondary servers run in the same container are not trivial / auxiliary but "major" (e.g. storage bundled with the frontend app). For starters, it would break down several important containerization features such as horizontal (auto-)scaling and rescheduling between nodes, both of which assume there is only one application (source of CPU load) per container. Then there is the issue of vulnerabilities - more servers exposed in a container means more frequent patching of CVEs...
So let's admit that it is a 'nudge' from Docker (and Kubernetes/Openshift) designers towards good practices and we should not reinvent workarounds (SSH is not necessary - we have docker exec / kubectl exec / oc rsh designed to replace it).
More info
https://devops.stackexchange.com/questions/447/why-it-is-recommended-to-run-only-one-process-in-a-container

restarting redis taking long time

I'm using redis as a broker for celery in my django project. As part of my deployment process I restart the services at the end, so redis, celery, gunicorn (django) etc starting with redis. However I run into an issue where redis will not shutdown.
$ sudo systemctl restart redis
$
And there it hangs, at time of writing for 15 minutes. journalctl shows no entries (logs have rotated overnight I assume), systemctl status shows the redis unit as deactivating (sig-term) but no indication what it's doing, other than:
May 24 10:31:22 staging systemd[1]: Stopping Advanced key-value store...
May 24 10:31:22 staging run-parts[305]: run-parts: executing /etc/redis/redis-server.pre-down.d/00_example
I understand that sig-term allows redis to exit gracefully, so wondered if the celery beat tasks or the django server was accessing it, but having stopped those services it is still hung up. Are there any places I'm not aware of where I can check the status/what it's doing?
edit: Aha, ok so it turns out redis doesn't actually log to systemd by default, instead it logs to /var/log/redis which does actually yield some information:
31602:M 24 May 10:59:56.097 * 1 changes in 900 seconds. Saving...
31602:M 24 May 10:59:56.101 * Background saving started by pid 1151
1151:C 24 May 10:59:56.103 # Failed opening .rdb for saving: Read-only file system
31602:M 24 May 10:59:56.204 # Background saving error
I'm using the defaults for the dir and dbfilename directives
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /var/lib/redis
user#server:/var/lib/redis$ ls -l
total 252
-rw-r--r-- 1 redis redis 249649 May 23 02:44 dump.rdb
It's owned by redis.. why would it be set to read-only?
ok... so
user#server:/var/lib/redis$ redis-cli config get dir
1) "dir"
2) "/var/spool/cron"
user#server:/var/lib/redis$ redis-cli config get dbfilename
1) "dbfilename"
2) "root"
This is starting to seem very strange. What on earth could have set it to that?

It's starting to look more and more like this was the result of an attack: https://github.com/antirez/redis/issues/3594
The interesting thing is that I was running redis behind a firewall. Currently not sure how the config was changed. Luckily it's a staging server with no sensitive information on it so I can take it down until I work out exactly what the issue was.
Update: So after investigating it further, all I can ascertain is that the attack must have taken place at some point after redis had been installed on the server, but before the firewall was activated. The version of redis I was using binds to 0.0.0.0 by default. I used a deployment script to set the server up so the vulnerability time must have been ~10 seconds. Lesson learned, activate firewall before installing packages.
I will say I dislike redis binding to 0.0.0.0 by default, especially when it has the ability to entirely compromise your server if left exposed. But based on what I've read that argument has been done to death and things have improved in later versions.

Getting Error: You've reached your account limit of 3 concurrent processes with Heroku

I tried to run the following command :
(venv)prajjwal#ubuntu:~/herokuDjango$ heroku run python manage.py shell
Running `python manage.py shell` attached to terminal... failed
! You've reached your account limit of 3 concurrent processes.
! Please verify your account at http://heroku.com/verify to have this limit increased
I looked it up on Heroku website:
https://devcenter.heroku.com/articles/limits
But It says under the following subtopic
Concurrent one-off dynos
that:
"Heroku accounts that aren’t verified cannot have more than 3 one-off dynos running concurrently."
But where exactly are my 3 one-off dynos running?

Try typing
heroku ps
Then type
heroku ps:stop run.xxxx
where xxxx is the # of the dyno that you want to kill.

Each time you call the heroku run command, you're actually creating a one-off dyno. Those dynos can last up to an hour at a time before they're shut down. So if you've called it 3 times, there's a chance that all three are still "alive" in the sense that resources are still allocated to your now, non-running process. If you want to test out your process, try running locally with foreman.
Take a look at their documentation for one-off dynos. That should explain the purpose and use case behind them.