I am using BaseHTTP since its very simple. Please advice if I'm using it wrong.
if self.path.endswith("nginxstart"):
xxxxxx
status=os.system("/home/karthik/nginx/nginx-1.0.4/sbin/nginx"); #tried ls -l and works
self.wfile.write("nginx restarted")
nginx gets started but doesn't write "nginx restarted" to browser until I kill the python server.
When I do netstat -anp | grep 'nginx's pid' there are two listening ports :
python server was listening (port 8000: which I have killed)
on which nginx was supposed to be running.
Works great if i run simple shell commands os.system("ls -l") and so on.
Works great if I run the same as a normal python script, not as a web server.
Tried starting some other services also not working.
I tried with try catch, catch part is not getting executed.
The browser is in connecting/receiving state forever.
Any help on this please ?
Your code is stuck because os.system block until the command is finished. So if the nginx command is not going to background, the process will not end, and your code will stuck.
An alternative is to use subprocess module:
from subprocess import Popen
# start a new command
cmd = Popen("/home/.../nginx", shell=True)
# you can regulary check if the command is still running with:
cmd.poll()
if cmd.returncode is None:
print "process is still running !"
else:
print "process exited with code %d" % cmd.returncode
# or even kill it
cmd.kill()
Related
Got a script for activating a python venv and running a server in the background, but right now I am trying to keep the pid when I start the process and then kill the process with pid after I am done. However, it is not all the time is gets killed.
My question is, can I run the process with a name, then killing it by using pkill name after? and how will that look
#!/bin/sh
ROOT_DIR=$(pwd)
activate(){
source $ROOT_DIR/.venv/bin/activate
python3 src/server.py -l & pid=$! # <== This is the process
python3 src/client.py localhost 8080
}
activate
sleep 10
kill "$pid"
printf "\n\nServer is done, terminating processes..."
You can run programs with a specific command name by using the bash buildin exec. Note that exec replaces the shell with the command so you have to run it in a subshell environment like:
( exec -a my_new_name my_old_command ) &
However, it probably won't help you much because this sets the command line name, which is apparently different from the command name. So executing the above snippet will show your process as "my_new_name" for example in top or htop, but pkill and killall are filtering by the command name and will thus not find a process called "my_new_name".
While it is interesting, how one can start a command with a different name than the executable, it is most likely not the cause of your problem. PIDs never change, so I assume that the problem lays somewhere different.
My best guess is that the server binds a socket to listen on a specific port. If the program is not shutdown gracefully but killed the port number remains occupied and is only freed by the kernel after some time during some kind of kernel garbage collect. If the program is restarted after a short period of time it finds the port already been occupied and prints a misleading message, that says it is already running. If that is indeed the cause of your problem I would strongly consider implementing a way to graceful shutdown the server. (may be already closing the socket in a destructor or something similar could help)
I think you should have to use systemd for this case:
https://github.com/torfsen/python-systemd-tutorial
I have setup a Django project on a virtual environment on my PC. When using the command
python manage.py runserver 0.0.0.0:8000
Bit Bash stops doing anything and I have to end the program to start over. I have waited several minutes and when I end the session, a dialogue says:
Processes are running in session:
WPID PID COMMAND
14904 1534 c:\Users\mine\AppData\Loca
Close anyway?
I have looked at every related question to this and tried every solution but I cannot get this to work, on or off the virtual environment.
Not sure if this applies, but I also noticed that in my task manager, python3.9.exe appears twice when trying to start the server. The status says running, and the PIDs are different numbers.
This is because something already running on port 8000. You can run the Django server by hosting on another port like 8080 but, you can also kill the already running task on the port. Follow the link to know how exactly you can kill any running process.
It is highly likely that another application is running using the port 8000. Try running the server using another port, say 8088 and share if the same issue persists.
To run on specific port kill all pids which is showing you on Git bash
(ps -ef)... kill them using (kill -9 pid_no)... then run your runserver command.
Ex:
ps -ef
kill -9 123
I ended up paying someone to fix this. Here is the conversation I had with the person who found the issue. Hope this helps someone out there.
Also, I noticed sometimes it still does freeze up on the watching for file changes with StateReloader until I go the browser and type in http://localhost:8000/. Once I do that the rest of the text loads and my browser shows The install worked successfully! Congratulations!
it works, what did you do?
- We have existing processes on port 8000.I think, you had the server running and kept trying over and over. So, I killed process using 8000 port.
- Next, I run Django project again.
- So, now it's working well.
ok. Can you tell me what you did at the start to make it work? Or was it always working?
- I started git bash session again when I started. Maybe git bash session had problems.
But this was going on for a day, I restarted the computer several times and restarted git bash. Same issue.
You must have done something.
- Sorry, I didn't do anything except restart git bash
how did you restart it?
- right click top of window, select NEW
How do I turn server off?
- press Ctrl + C
- And you didn't quit the server exactly before. Maybe you closed git bash directly and server was still live
- So, we had many 8000 ports.
Ok. How can I see what is on that port now?
- You can use this cli
- netstat -ano | findstr :<PORT>
- We don't have running 8000 port now.
I have never found solutions to this problem even with killing ports, changing port etc.
It never display the following part:
Starting development server at http://127.0.0.1:8000/
However, when I use the Git Bash terminal include into vscode, it displays the url.
Terminal > New Terminal > then:
Result :
python manage.py runserver 127.0.0.1:8080
I'm running a flask server and an apscheduler background scheduler. I want the program to periodically restart itself.
I restart the application using the following shell script:
#!/bin/bash
kill $(ps aux | grep 'python' | awk '{print $2}')
sleep 5
cd "$( dirname "${BASH_SOURCE[0]}" )"
python server.py
When I start the server without this script, then execute this script from another terminal it successfully kills the other process and starts the server.
But when the scheduler calls this script, flask gives me an 'error, socket in use'. Why does executing this from another terminal shutdown the server and free the socket, but executing it from inside the scheduler leave the socket in use?
How can I make the script my scheduler runs totally shutdown my server?
1 month later and I still haven't gotten an answer to this question :(
To clarify, I do not want a workaround that makes it so I don't need to restart the server. I want the server to have the ability to restart itself.
For now I am going to use the workaround of setting the server to start on boot, and then restarting the entire computer.
The task is to use python to run a remote process in background and immediately close the ssh session.
I have a remote script name 'start' under server:PATH/, the start script does nothing but lunch a long-live background program. 'start' script which has one line:
nohup PATH/Xprogram &
When I use python subprocess module to call my remote 'start' script, it does start OK. But the issue is: it seems the SSH connection is persist, meaning I am getting stdout from the remote Xprogram (since it is a long live program which has output to stdout). Does this indicating ssh connection is still there ?
All I need is call the start script without blocking and forget about it (leave the long-live program running, close ssh, release resources).
my python function call looks like this:
ret = subprocess.Popen(["ssh", "xxx#servername", "PATH/start"])
if I use ret.terminate() after the command, it then will kill the long-live program too.
I have also tried spur module. basically the same thing.
=====update====
#Dunes' answer solves the problem. Based on his answer, I did more digging and found this link very helpful.
My understanding of this is: basically, if any file descriptor is still held by your process (e.g. stdout held by my XProgram), then SSH session won't exit. However redirect stdout/stderr to NULL effectively close those file descriptor and let SSH session exit normally.
solution
ret = subprocess.Popen(["ssh", "xxx#servername", "PATH/start >dev/null 2>&1"])
After playing about a bit I found that nohup doesn't seem to be properly disconnecting the child process from the parent ssh session (as it should be). This means you have to manually close stdout or point it at a file, e.g.
Using bash:
ssh user#host "nohup PATH/XProgram >&- &"
Shell agnostic (as far as I know):
ssh user#host "nohup PATH/XProgram >/dev/null 2>&1 &"
In python:
from shlex import split
from subprocess import Popen
p = Popen(split('ssh user#host "nohup PATH/XProgram >&- &"'))
p.communicate() # returns (None, None)
Try
subprocess.Popen(["ssh", "xxx#servername", "nohup PATH/start & disown"])
For me,
subprocess.Popen(["ssh", "xxx#servername", "nohup sleep 1000 & disown"])
lets my script exit immediately while leaving sleep running on the server awhile.
When your script dies, an ssh process is left on your system, but killing it doesn't kill the remote process.
I have written a simple automation script for deploying and restarting my twisted application on remote Debian host. But I have an issue with starting using twistd.
I have a run.tac file and start my application as follows inside fabric task:
#task
def start():
run("twistd -y run.tac")
And then just fab -H host_name start. It works great on localhost but when I want to start application on remote host I get nothing. I can see in log file that application is actually launched, but factory is not started. I've also checked netstat -l - nothing is listening my port.
I've tried to run in non-daemon mode, like so twistd -ny run.tac, and, voila, factory started and I can see it in netstat -l on remote host. But that is not the way I want it to work cause it. Any help is appreciated.
There was an issue reported sometime back which is similar to this.
Init scripts frequently fail to start their daemons
init-scripts-dont-work
It also suggested that it seems to succeed with the option pty=False. Can you try and check that?
run("twistd -y run.tac", pty=False)
Some more pointers from the FaQ:
why-can-t-i-run-programs-in-the-background-with-it-makes-fabric-hang