I have built a base image from Dockerfile named centos+ssh. In centos+ssh's Dockerfile, I use CMD to run ssh service.
Then I want to build a image run other service named rabbitmq,the Dockerfile:
FROM centos+ssh
EXPOSE 22
EXPOSE 4149
CMD /opt/mq/sbin/rabbitmq-server start
To start rabbitmq container，run：
docker run -d -p 222:22 -p 4149:4149 rabbitmq
but ssh service doesn't work, it sense rabbitmq's Dockerfile CMD override centos's CMD.
How does CMD work inside docker image?
If I want to run multiple service, how to? Using supervisor?
You are right, the second Dockerfile will overwrite the CMD command of the first one. Docker will always run a single command, not more. So at the end of your Dockerfile, you can specify one command to run. Not more.
But you can execute both commands in one line:
FROM centos+ssh
EXPOSE 22
EXPOSE 4149
CMD service sshd start && /opt/mq/sbin/rabbitmq-server start
What you could also do to make your Dockerfile a little bit cleaner, you could put your CMD commands to an extra file:
FROM centos+ssh
EXPOSE 22
EXPOSE 4149
CMD sh /home/centos/all_your_commands.sh
And a file like this:
service sshd start &
/opt/mq/sbin/rabbitmq-server start
Even though CMD is written down in the Dockerfile, it really is runtime information. Just like EXPOSE, but contrary to e.g. RUN and ADD. By this, I mean that you can override it later, in an extending Dockerfile, or simple in your run command, which is what you are experiencing. At all times, there can be only one CMD.
If you want to run multiple services, I indeed would use supervisor. You can make a supervisor configuration file for each service, ADD these in a directory, and run the supervisor with supervisord -c /etc/supervisor to point to a supervisor configuration file which loads all your services and looks like
[supervisord]
nodaemon=true
[include]
files = /etc/supervisor/conf.d/*.conf
If you would like more details, I wrote a blog on this subject here: http://blog.trifork.com/2014/03/11/using-supervisor-with-docker-to-manage-processes-supporting-image-inheritance/
While I respect the answer from qkrijger explaining how you can work around this issue I think there is a lot more we can learn about what's going on here ...
To actually answer your question of "why" ... I think it would for helpful for you to understand how the docker stop command works and that all processes should be shutdown cleanly to prevent problems when you try to restart them (file corruption etc).
Problem: What if docker did start SSH from it's command and started RabbitMQ from your Docker file? "The docker stop command attempts to stop a running container first by sending a SIGTERM signal to the root process (PID 1) in the container." Which process is docker tracking as PID 1 that will get the SIGTERM? Will it be SSH or Rabbit?? "According to the Unix process model, the init process -- PID 1 -- inherits all orphaned child processes and must reap them. Most Docker containers do not have an init process that does this correctly, and as a result their containers become filled with zombie processes over time."
Answer: Docker simply takes that last CMD as the one that will get launched as the root process with PID 1 and get the SIGTERM from docker stop.
Suggested solution: You should use (or create) a base image specifically made for running more than one service, such as phusion/baseimage
It should be important to note that tini exists exactly for this reason, and as of Docker 1.13 and up, tini is officially part of Docker, which tells us that running more than one process in Docker IS VALID .. so even if someone claims to be more skilled regarding Docker, and insists that you absurd for thinking of doing this, know that you are not. There are perfectly valid situations for doing so.
Good to know:
https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
http://www.techbar.me/stopping-docker-containers-gracefully/
https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/
https://github.com/phusion/baseimage-docker#docker_single_process
The official docker answer to Run multiple services in a container.
It explains how you can do it with an init system (systemd, sysvinit, upstart) , a script (CMD ./my_wrapper_script.sh) or a supervisor like supervisord.
The && workaround can work only for services that starts in background (daemons) or that will execute quickly without interaction and release the prompt. Doing this with an interactive service (that keeps the prompt) and only the first service will start.
To address why CMD is designed to run only one service per container, let's just realize what would happen if the secondary servers run in the same container are not trivial / auxiliary but "major" (e.g. storage bundled with the frontend app). For starters, it would break down several important containerization features such as horizontal (auto-)scaling and rescheduling between nodes, both of which assume there is only one application (source of CPU load) per container. Then there is the issue of vulnerabilities - more servers exposed in a container means more frequent patching of CVEs...
So let's admit that it is a 'nudge' from Docker (and Kubernetes/Openshift) designers towards good practices and we should not reinvent workarounds (SSH is not necessary - we have docker exec / kubectl exec / oc rsh designed to replace it).
More info
https://devops.stackexchange.com/questions/447/why-it-is-recommended-to-run-only-one-process-in-a-container
I have set up a cronjob for root user in ubuntu environment as follows by typing crontab -e
34 11 * * * sh /srv/www/live/CronJobs/daily.sh
0 08 * * 2 sh /srv/www/live/CronJobs/weekly.sh
0 08 1 * * sh /srv/www/live/CronJobs/monthly.sh
But the cronjob does not run. I have tried checking if the cronjob is running using pgrep cron and that gives process id 3033. The shell script calls a python file and is used to send an email. Running the python file is ok. There's no error in it but the cron doesn't run. The daily.sh file has the following code in it.
python /srv/www/live/CronJobs/daily.py
python /srv/www/live/CronJobs/notification_email.py
python /srv/www/live/CronJobs/log_kpi.py
WTF?! My cronjob doesn't run?!
Here's a checklist guide to debug not running cronjobs:
Is the Cron daemon running?
Run ps ax | grep cron and look for cron.
Debian: service cron start or service cron restart
Is cron working?
* * * * * /bin/echo "cron works" >> /tmp/file
Syntax correct? See below.
You obviously need to have write access to the file you are redirecting the output to. A unique file name in /tmp which does not currently exist should always be writable.
Probably also add 2>&1 to include standard error as well as standard output, or separately output standard error to another file with 2>>/tmp/errors
Is the command working standalone?
Check if the script has an error, by doing a dry run on the CLI
When testing your command, test as the user whose crontab you are editing, which might not be your login or root
Can cron run your job?
Check /var/log/cron.log or /var/log/messages for errors.
Ubuntu: grep CRON /var/log/syslog
Redhat: /var/log/cron
Check permissions
Set executable flag on the command: chmod +x /var/www/app/cron/do-stuff.php
If you redirect the output of your command to a file, verify you have permission to write to that file/directory
Check paths
Check she-bangs / hashbangs line
Do not rely on environment variables like PATH, as their value will likely not be the same under cron as under an interactive session. See How to get CRON to call in the correct PATHs
Don't suppress output while debugging
Commonly used is this suppression: 30 1 * * * command > /dev/null 2>&1
Re-enable the standard output or standard error message output by removing >/dev/null 2>&1 altogether; or perhaps redirect to a file in a location where you have write access: >>cron.out 2>&1 will append standard output and standard error to cron.out in the invoking user's home directory.
If you don't redirect output from a cron job, the daemon will try to send you any output or error messages by email. Check your inbox (maybe simply more $MAIL if you don't have a mail client). If mail is not available, maybe check for a file named dead.letter in your home directory, or system log entries saying that the output was discarded. Especially in the latter case, probably edit the job to add redirection to a file, then wait for the job to run, and examine the log file for error messages or other useful feedback.
If you are trying to figure out why something failed, the error messages will be visible in this file. Read it and understand it.
Still not working? Yikes!
Raise the cron debug level
Debian
in /etc/default/cron
set EXTRA_OPTS="-L 2"
service cron restart
tail -f /var/log/syslog to see the scripts executed
Ubuntu
in /etc/rsyslog.d/50-default.conf
add or comment out line cron.* /var/log/cron.log
reload logger sudo /etc/init.d/rsyslog restart
re-run cron
open /var/log/cron.log and look for detailed error output
Reminder: deactivate log level, when you are done with debugging
Run cron and check log files again
Cronjob Syntax
# Minute Hour Day of Month Month Day of Week User Command
# (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)
0 2 * * * root /usr/bin/find
This syntax is only correct for the root user. Regular user crontab syntax doesn't have the User field (regular users aren't allowed to run code as any other user);
# Minute Hour Day of Month Month Day of Week Command
# (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)
0 2 * * * /usr/bin/find
Crontab Commands
crontab -l
Lists all the user's cron tasks.
crontab -e, for a specific user: crontab -e -u agentsmith
Starts edit session of your crontab file.
When you exit the editor, the modified crontab is installed automatically.
crontab -r
Removes your crontab entry from the cron spooler, but not from crontab file.
Another reason crontab will fail: Special handling of the % character.
From the manual file:
The entire command portion of the line, up to a newline or a
"%" character, will be executed by /bin/sh or by the shell specified
in the SHELL variable of the cronfile. A "%" character in the
command, unless escaped with a backslash (\), will be changed into
newline characters, and all data after the first % will be sent to
the command as standard input.
In my particular case, I was using date --date="7 days ago" "+%Y-%m-%d" to produce parameters to my script, and it was failing silently. I finally found out what was going on when I checked syslog and saw my command was truncated at the % symbol. You need to escape it like this:
date --date="7 days ago" "+\%Y-\%m-\%d"
See here for more details:
http://www.ducea.com/2008/11/12/using-the-character-in-crontab-entries/
Finally I found the solution. Following is the solution:-
Never use relative path in python scripts to be executed via crontab.
I did something like this instead:-
import os
import sys
import time, datetime
CLASS_PATH = '/srv/www/live/mainapp/classes'
SETTINGS_PATH = '/srv/www/live/foodtrade'
sys.path.insert(0, CLASS_PATH)
sys.path.insert(1,SETTINGS_PATH)
import other_py_files
Never supress the crontab code instead use mailserver and check the mail for the user. That gives clearer insights of what is going.
I want to add 2 points that I learned:
Cron config files put in /etc/cron.d/ should not contain a dot (.). Otherwise, it won't be read by cron.
If the user running your command is not in /etc/shadow. It won't be allowed to schedule cron.
Refs:
http://manpages.ubuntu.com/manpages/xenial/en/man8/cron.8.html
https://help.ubuntu.com/community/CronHowto
To add another point, a file in /etc/cron.d must contain an empty new line at the end. This is likely related to the response by Luciano which specifies that:
The entire command portion of the line, up to a newline or a "%"
character, will be executed
I found useful debugging information on an Ubuntu 16.04 server by running:
systemctl status cron.service
In my case I was kindly informed I had left a comment '#' off of a remark line:
Aug 18 19:12:01 is-feb19 cron[14307]: Error: bad minute; while reading /etc/crontab
Aug 18 19:12:01 is-feb19 cron[14307]: (*system*) ERROR (Syntax error, this crontab file will be ignored)
It might also be a timezone problem.
Cron uses the local time.
Run the command timedatectl to see the machine time and make sure that your crontab is in this same timezone.
https://askubuntu.com/a/536489/1043751
I had a similar problem to the link below.
similar to my problem
my original post
My Issue
My issue was that cron / crontab wouldn't execute my bash script. that bash script executed a python script.
original bash file
#!/bin/bash
python /home/frosty/code/test_scripts/test.py
python file (test.py)
from datetime import datetime
def main():
dt_now = datetime.now()
string_now = dt_now.strftime('%Y-%m-%d %H:%M:%S.%f')
with open('./text_file.txt', 'a') as f:
f.write(f'wrote at {string_now}\n')
return None
if __name__ == '__main__':
main()
the error I was getting
File "/home/frosty/code/test_scripts/test.py", line 7
string_to_write = f'wrote at {string_now}\n'
^
SyntaxError: invalid syntax
this error didn't make sense because the code executed without error from the bash file and the python file.
** Note -> ensure in the crontab -e file you don't suppress the output. I sent the output to a file by adding >>/path/to/cron/output/file.log 2>&1 after the command. below is my crontab -e entry
*/5 * * * * /home/frosty/code/test_scripts/echo_message_sh >>/home/frosty/code/test_scripts/cron_out.log 2>&1
the issue
cron was using the wrong python interpreter, probably python 2 from the syntax error.
how I solved the problem
I changed my bash file to the following
#!/bin/bash
conda_shell=/home/frosty/anaconda3/etc/profile.d/conda.sh
conda_env=base
source ${conda_shell}
conda activate ${conda_env}
python /home/frosty/code/test_scripts/test.py
And I changed my python file to the following
from datetime import datetime
def main():
dt_now = datetime.now()
string_now = dt_now.strftime('%Y-%m-%d %H:%M:%S.%f')
string_file = '/home/frosty/code/test_scripts/text_file.txt'
string_to_write = 'wrote at {}\n'.format(string_now)
with open(string_file, 'a') as f:
f.write(string_to_write)
return None
if __name__ == '__main__':
main()
No MTA installed, discarding output
I had a similar problem with a PHP file executed as a CRON job.
When I manually execute the file it works, but not with CRON tab.
I got the output message: "No MTA installed, discarding output"
Postfix is the default Mail Transfer Agent (MTA) in Ubuntu and can be installed it using
sudo apt-get install postfix
But this same message can be also output when you add a log file as below and it does not have proper write permission to /path/to/logfile.log
/path/to/php -f /path/to/script.php >> /path/to/logfile.log
The permission issue can occur if you create the cron-log file manually using a command like touch while you are logged in as a different user and you add CRONs in the tab of another user(group) like www-data using: sudo crontab -u www-data -e. Then CRON daemon tries to write to the log file and fail, then tries to send the output as an email using Ubuntu's MTA and when it's not found, outputs "No MTA installed, discarding output".
To prevent this:
Create the file with proper permission.
Avoid creating the relevant CRON log file manually, add the log in CRON tab and let the log file get created automatically when the cron is run.
I've found another reason for user's crontab not running: the hostname is not present on the hosts file:
user#ubuntu:~$ cat /etc/hostname
ubuntu
Now the hosts file:
user#ubuntu:~$ cat /etc/hosts
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
This is on a Ubuntu 14.04.3 LTS, the way to fix it is adding the hostname to the hosts file so it resembles something like this:
user#ubuntu:~$ cat /etc/hosts
127.0.0.1 ubuntu localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
For me, the solution was that the file cron was trying to run was in an encrypted directory, more specifcically a user diretory on /home/. Although the crontab was configured as root, because the script being run exisited in an encrypted user directory in /home/ cron could only read this directory when the user was actually logged in. To see if the directory is encrypted check if this directory exists:
/home/.ecryptfs/<yourusername>
if so then you have an encrypted home directory.
The fix for me was to move the script in to a non=encrypted directory and everythig worked fine.
As this is becoming a canonical for troubleshooting cron issues, allow me to add one specific but rather complex issue: If you are attempting to run a GUI program from cron, you are probably Doing It Wrong.
A common symptom is receiving error messages about DISPLAY being unset, or the cron job's process being unable to access the display.
In brief, this means that the program you are trying to run is attempting to render something on an X11 (or Wayland etc) display, and failing, because cron is not attached to a graphical environment, or in fact any kind of input/output facility at all, beyond being able to read and write files, and send email if the system is configured to allow that.
For the purposes of "I'm unable to run my graphical cron job", let's just point out in broad strokes three common scenarios for this problem.
Probably identify the case you are trying to implement, and search for related questions about that particular scenario to learn more, and find actual solutions with actual code.
If you are trying to develop an interactive program which communicates with a user, you want to rethink your approach. A common, but nontrivial, arrangement is to split the program in two: A back-end service which can run from cron, but which does not have any user-visible interactive facilities, and a front-end client which the user runs from their GUI when they want to communicate with the back-end service.
Probably your user client should simply be added to the user(s)' GUI startup script if it needs to be, or they want to, run automatically when they log in.
I suppose the back-end service could be started from cron, but if it requires a GUI to be useful, maybe start it from the X11 server's startup scripts instead; and if not, probably run it from a regular startup script (systemd these days, or /etc/rc.local or a similar system startup directory more traditionally).1
If you are trying to run a GUI program without interacting with a real user 2, you may be able to set up a "headless" X11 server 3 and run a cron job which starts up that server, runs your job, and quits.
Probably your job should simply run a suitable X11 server from cron (separate from any interactive X11 server which manages the actual physical display(s) and attached graphics card(s) and keyboard(s) available to the system), and pass it a configuration which runs the client(s) you want to run once it's up and running. (See also the next point for some practical considerations.)
You are running a computer for the sole purpose of displaying a specific application in a GUI, and you want to start that application when the computer is booted.
Probably your startup scripts should simply run the GUI (X11 or whatever) and hook into its startup script to also run the client program once the GUI is up and running. In other words, you don't need cron here; just configure the startup scripts to run the desktop GUI, and configure the desktop GUI to run your application as part of the (presumably automatic, guest?) login sequence.4
There are ways to run X11 programs on the system's primary display (DISPLAY=:0.0) but doing that from a cron job is often problematic, as that display is usually reserved for actual interactive use by the first user who logs in and starts a graphical desktop. On a single-user system, you might be able to live with the side effects if that user is also you, but this tends to have inconvenient consequences and scale very poorly.
An additional complication is deciding which user to run the cron job as. A shared system resource like a back-end service can and probably should be run by root (though ideally have a dedicated system account which it switches into once it has acquired access to any privileged resources it needs) but anything involving a GUI should definitely not be run as root at any point.
A related, but distinct problem is to interact in any meaningful way with the user. If you can identify the user's active session (to the extent that this is even well-defined in the first place), how do you grab their attention without interfering with whatever else they are in the middle of? But more fundamentally, how do you even find them? If they are not logged in at all, what do you do then? If they are, how do you determine that they are active and available? If they are logged in more than once, which terminal are they using, and is it safe to interrupt that session? Similarly, if they are logged in to the GUI, they might miss a window you spring up on the local console, if they are actually logged in remotely via VNC or a remote X11 server.
As a further aside: On dedicated servers (web hosting services, supercomputing clusters, etc) you might even be breaking the terms of service of the hosting company or institution if you install an interactive graphical desktop you can connect to from the outside world, or even at all.
1
The #reboot hook in cron is a convenience for regular users who don't have any other facility for running something when the system comes up, but it's just inconvenient and obscure to hide something there if you are root anyway and have complete control over the system. Use the system facilities to launch system services.
2
A common use case is running a web browser which needs to run a full GUI client, but which is being controlled programmatically and which doesn't really need to display anything anywhere, for example to scrape sites which use Javascript and thus require a full graphical browser to render the information you want to extract.
Another is poorly designed scientific or office software which was not written for batch use, and thus requires a GUI even when you just want to run a batch job and then immediately quit without any actual need to display anything anywhere.
(In the latter case, probably review the documentation to check if there isn't a --batch or --noninteractive or --headless or --script or --eval option or similar to run the tool without the GUI, or perhaps a separate utility for noninteractive use.)
3
Xvfb is the de facto standard solution; it runs a "virtual framebuffer" where the computer can spit out pixels as if to a display, but which isn't actually connected to any display hardware.
4
There are several options here.
The absolutely simplest is to set up the system to automatically log in a specific user at startup without a password prompt, and configure that user's desktop environment (Gnome or KDE or XFCE or what have you) to run your script from its "Startup Items" or "Login Actions" or "Autostart" or whatever the facility might be called. If you need more control over the environment, maybe run bare X11 without a desktop environment or window manager at all, and just run your script instead. Or in some cases, maybe replace the X11 login manager ("greeter") with something custom built.
The X11 stack is quite modular, and there are several hooks in various layers where you could run a script either as part of a standard startup process, or one which completely replaces a standard layer. These things tend to differ somewhat between distros and implementations, and over time, so this answer is necessarily vague and incomplete around these matters. Again, probably try to find an existing question about how to do things for your specific platform (Ubuntu, Raspbian, Gnome, KDE, what?) and scenario. For simple scenarios, perhaps see Ubuntu - run bash script on startup with visible terminal
I experienced same problem where crons are not running.
We fixed by changing permissions and owner by
Crons made root owner as we had mentioned in crontab AND
Cronjobs 644 permission given
There is already a lot of answers, but none of them helped me so I'll add mine here in case it's useful for somebody else.
In my situation, my cronjobs were working find until there was a power shortage that cut the power to my Raspberry Pi. Cron got corrupted. I think it was running a long python script exactly when the shortage happened. Nothing in the main answer above worked for me. The solution was however quite simple. I just had to force reinstallation of cron with:
sudo apt-get --reinstall install cron
It work right away after this.
Copying my answer for a duplicated question here.
cron may not know where to find the Python interpreter because it doesn't share your user account's environment variables.
There are 3 solutions to this:
If Python is at /usr/bin/python, you can change the cron job to use an absolute path: /usr/bin/python /srv/www/live/CronJobs/daily.py
Alternatively you can also add a PATH value to the crontab with PATH=/usr/bin.
Another solution would be to specify an interpreter in the script file, make it executable, and call the script itself in your crontab:
a. Put shebang at the top of your python file: #!/usr/bin/python.
b. Set it to executable: $ chmod +x /srv/www/live/CronJobs/daily.py
c. Put it in crontab: /srv/www/live/CronJobs/daily.py
Adjust the path to the Python interpreter if it's different on your system.
Reference
CRON uses a different TIMEZONE
A very common issue is: cron time settings may is different than your. In particular, the timezone could be not be the same:
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
You can run:
* * * * * echo $(date) >> /tmp/test.txt
This should generate a file like:
# cat test.txt
Sun 03 Apr 2022 09:02:01 AM UTC
Sun 03 Apr 2022 09:03:01 AM UTC
Sun 03 Apr 2022 09:04:01 AM UTC
Sun 03 Apr 2022 09:05:01 AM UTC
Sun 03 Apr 2022 09:06:01 AM UTC
If you are using a TZ other than UTC, you can try:
timedatectl set-timezone America/Sao_Paulo
replace America/Sao_Paulo according to you settings.
I'm not sure if it is actually necessary, but you can run:
sudo systemctl restart cron.service
After that, cron works as I expected:
# cat test.txt
Sun 03 Apr 2022 09:02:01 AM UTC
Sun 03 Apr 2022 09:03:01 AM UTC
Sun 03 Apr 2022 09:04:01 AM UTC
Sun 03 Apr 2022 09:05:01 AM UTC
Sun 03 Apr 2022 09:06:01 AM UTC
Sun 03 Apr 2022 09:07:01 AM UTC
Sun 03 Apr 2022 09:08:01 AM UTC
Sun 03 Apr 2022 09:09:01 AM UTC
Sun 03 Apr 2022 09:10:01 AM UTC
Sun 03 Apr 2022 06:11:01 AM -03
Sun 03 Apr 2022 06:12:01 AM -03
Sun 03 Apr 2022 06:13:01 AM -03
Sun 03 Apr 2022 06:14:01 AM -03
Try
service cron start
or
systemctl start cron
In my case I was trying to run cron locally.
I checked status:
service cron status
It showed me:
* cron is not running
Then I simply started the service:
service cron start
Sometimes the command that cron needs to run is in a directory where cron has no access, typically on systems where users' home directories' permissions are 700 and the command is in that directory.
Although answer has been accepted for this question, I will like to add what worked for me.
it's a good idea to quote the URL, if it contains a query it may not work without everything being quoted.
DONT FORGET TO PUT YOUR URL WHICH CONTAINS "?, =, #, %" IN A QUOTE.
Example.
https://paystack.com/indexphp?docs/api/#transaction-charge-authorization&date=today
should be in a quote like so
"https://paystack.com/indexphp?docs/api/#transaction-charge-authorization&date=today"
I have a remote server through Blue Host that's intended to run a server based on Twisted for Python. The only access I have to it is over SSH, so to keep Python running after I log out I tried using nohup python server.py & and screen -dm python server.py, getting the same results for each. Everything works fine until I log out of SSH - even though Python is running in the background as expected, once I've logged out, my client can no longer communicate with the server. The strange part is that if I log back in over SSH and check the running processes with ps aux, I see Python running and my client can successfully communicate with the server again. Even if I don't type anything at all once I log back in, everything works as expected. But, of course, as soon as I log back out, it's as if the server is gone.
I've contacted support for the hosting service in case this is some oddity on their end, but hopefully this is something that can be resolved on my end instead.
Edit: Looks like Blue Host doesn't want me doing server-y stuff without buying the VPS upgrade so it looks like that's the big problem.
Edit 2: Okay, so in case anybody ends up having a similar problem, here's what the main issue turned out to be. I was mistaken in my original description; I was able to connect to the server but I was getting kicked off immediately for what turned out to be a MySQL error. I guess trying to connect to a localhost database with no active connection somehow causes problems, so instead I changed the MySQL connection command to connect to my site's IP address instead, even though it was the same IP as the server. That seemed to do the trick in terms of my main issue.
Don't use this method to keep the server process running. Instead try using supervisor (apt-get install supervisor). It allows you to daemonize your process, and ability to stop/restart etc.
Here's a sample config entry (/etc/supervisor/supervisord.conf):
[program:my_server]
command=python /path/to/server/server.py
directory=/path/to/server/
autostart=true
autorestart=true
stdout_logfile=/var/log/server.log
stderr_logfile=/var/log/server_error.log
user=your_linux_user_name
After you edit your config, do
sudo service supervisor stop
sudo service supervisor start #need to do this - doing a `restart` doesn't reload the config file!
your server should now be running properly. You can manage its lifecycle via sudo supervisorctl
I have an instance on google compute engine, connecting to it by terminal: gcutil ssh, on it I have several DJango servieces. I run the server using: python manage.py runserver 0.0.0.0:8000. the services are being called from an iPhone application IOS 6.1
the problem I'm facing is that every few minutes (between 10- 15) I'm getting disconnected and have to reconnect and run the server again.
Why is my server being disconnected and how can I keep the it running?
Try using supervisor.d. It sounds like for what your trying to do, supervisor can keep your process up and running. http://supervisord.org/
Here's an example conf:
[program:app]
process_name = app-%(process_num)s
command =python /home/ubuntu/production/current/app/src/app.py --port=%(process_num)s
# Increase numprocs to run multiple processes on different ports.
# Note that the chat demo won't actually work in that configuration
# because it assumes all listeners are in one process.
numprocs = 4
numprocs_start = 8000
This is for running multiple processes of the same program. Just change around the args and it should work for you.
SSH normally times out after a period of inactivity, and that may be what is happening here. If so, this article might be useful to help configure SSH to send a regular message so connections are less likely to be dropped.
However, the core issue is that you'd like software you started at the terminal to keep running even when you're logged out. Consider using screen or tmux to host your shell sessions. This will allow your shell software to run even when you are not connected, and for you to pick up right where you left off when you reconnect. Here is a nice getting started post about tmux.
Once you're ready for production, take a look at the Django deployment docs.
I have written a module in Python and want it to run continuously once started and need to stop it when I need to update other modules. I will likely be using monit to restart it, if module has crashed or is otherwise not running.
I was going through different techniques like Daemon, Upstart and many others.
Which is the best way to go so that I use that approach through out my all new modules to keep running them forever?
From your mention of Upstart I will assume that this question is for a service being run on an Ubuntu server.
On an Ubuntu server an upstart job is really the simplest and most convenient option for creating an always on service that starts up at the right time and can be stopped or reloaded with familiar commands.
To create an upstart service you need to add a single file to /etc/init. Called <service-name>.conf. An example script looks like this:
description "My chat server"
author "your#email-address.com"
start on runlevel [2345]
stop on runlevel [!2345]
env AN_ENVIRONMENTAL_VARIABLE=i-want-to-set
respawn
exec /srv/applications/chat.py
This means that everytime the machine is started it will start the chat.py program. If it dies for whatever reason it will restart it. You don't have to worry about double forking or otherwise daemonizing your code. That's handled for you by upstart.
If you want to stop or start your process you can do so with
service chat start
service chat stop
The name chat is automatically found from the name of the .conf file inside /etc/init
I'm only covering the basics of upstart here. There are lots of other features to make it even more useful. All available by running man upstart.
This method is much more convenient, than writing your own daemonization code. A 4-8 line config file for a built in Ubuntu component is much less error prone than making your code safely double fork and then having another process monitor it to make sure it doesn't go away.
Monit is a bit of a red herring. If you want downtime alerts you will need to run a monitoring program on a separate server anyway. Rely on upstart to keep the process always running on a server. Then have a different service that makes sure the server is actually running. Downtime happens for many different reasons. A process running on the same server will tell you precisely nothing if the server itself goes down. You need a separate machine (or a third party provider like pingdom) to alert you about that condition.
You could check out supervisor. What it is capable of is starting a process at system startup, and then keeping it alive until shutdown.
The simplest configuration file would be:
[program:my_script]
command = /home/foo/bar/venv/bin/python /home/foo/bar/scripts/my_script.py
environment = MY_ENV_VAR=FOO, MY_OTHER_ENV_VAR=BAR
autostart = True
autorestart = True
Then you could link it to /etc/supervisord/conf.d, run sudo supervisorctl to enter management console of supervisor, type in reread so that supervisor notices new config entry and update to display new programs on the status list.
To start/restart/stop a program you could execute sudo supervisorctl start/restart/stop my_script.
I used old-style initscript with start-stop-daemon utility.Look at skel in /etc/init.d