Develop Reference

How to kill a process in window without killing current process

Hello I just wont to kill a process without killing current process suppose that i have a python code which write a string in txt file e.g
import os
while True:
with open('input.txt' , 'r') as re:
all_data = re.read()
if all_data == 'pause':
os.system('kill.bat')
else:
print("\nContinue\n")
here it will read input.txt and if it is equal to pause then it will run kill.bat here i would like to restart this code for doing this i will write another script kill.bat which restart this code but the problem is it is not restarting because it was killing kill.bat file but i wont to kill only python terminal and restart it how can i do it here is kill.bat file code
taskkill /IM cmd.exe
python main.py
main.py is my python file

Nice try #Pryanshu
You're doing it right. The problem with your current approach is python and the batch script will be under same process and get's killed at the same time.
So just change the approach a bit
here's the gist:
python
get current process id
while loop
read file
if matches condition
call kill.bat and pass this python process ID as separate process
batch
get first parameter (which is the python process to kill)
kill the process via taskkill
start the python script as seprate process
exit
Helpers
get current process ID - pid = os.getpid()
pass id to batch script - os.system("kill.bat %s" % pid)
batch script get arguments - %1
this %1 will contain the first argument passed.
by default %* will contain all the arguments passed to the script.
python start program as
separate process - use subprocess python package
cmd start program as
separate process - use Start command
kill process via taskkill - taskkill /F /PID <pid>
replace <pid> with your argument
I know you can handle the code part. You'll have to make use of the Helpers and combine them to do the task. Lemme know the output.
If you have any question do comment. I'll try to reply ASAP
Cheers

Run a python script from another python script running as a daemon

I have a small python file which just outputs a string:
#!/usr/bin/env python
print("This is a Test")
I can call this python script from another python script like so:
subprocess.call(['python', 'myPythonFile.py'])
And I can see the 'This is a Test' in my source python program.
But I want to call this script from a running Daemon as described here: https://gist.github.com/andreif/cbb71b0498589dac93cb
When I put the call to
subprocess.call(['python', 'myPythonFile.py'])
In MyDaemon.Run I DO NOT see the output.
How can I do this?

Try using the check_output function to see the actual output in your console
subprocess.check_output(['python', 'myPythonFile.py'])
You can find more info in the subprocess docs

subprocess.call can send its output to a file
tempfile = '/tmp/output.tmp' # better use mktemp
with open( tempfile, 'w') as output:
response = subprocess.call(
['python', 'myPythonFile.py'],
stdout=output,
stderr=output,
)
with open( tempfile, 'r') as output:
# read what happened from output, and decide what to do next
# or perhaps just feed it into your logging system

A daemon process is characterised by having no controlling terminal, because it is detached from whatever started the daemon. The daemon process is not connected to any console, by definition.
So, if that daemon runs another process:
I want to call this script from a running Daemon
then there is still no controlling terminal, and standard output is by default connected to the null device.
You will need to have the daemon process arrange to have its output somewhere. For example, a log file.
Try the python daemon library for a way to create daemons and nominate specific files (e.g. a log file you opened) to remain open in the daemon process.

How can I handle user input for subprocesses ran in parallel in Python?

I have a Python helper function to run grunt commands in parallel, using Popen to handle subprocesses. The purpose is communication over CLI. The problem starts when any user input is required for all those processes, e.g. file path, password, 'yes/no' decision:
Enter file path: Enter file path: Enter file path: Enter file path: Enter file path: Enter file path: Enter file path:
Everything up-to-date
Grunt task completed successfully.
User provides input once, one process completes successfully and all others never finish executing.
Code:
from subprocess import check_output, Popen
def run_grunt_parallel(grunt_commands):
return_code = 0
commands = []
for command in grunt_commands:
with tempfile.NamedTemporaryFile(delete=False) as f:
app = get_grunt_application_name(' '.join(command))
commands.append({'app': app, 'process': Popen(command, stdout=f)})
while len(commands):
sleep(5)
next_round = []
for command in commands:
rc = command['process'].poll()
if rc == None:
next_round.append(command)
else:
if rc == 0:
else:
return_code = rc
commands = next_round
return return_code
Is there a way to make sure that user can provide all necessary input for each process?

What you want is almost (if not entirely) impossible. But if you can recognize prompts in a prefix-free fashion (and, if it varies, know from them how many lines of input they expect), you should be able to manage it.
Run each process with two-way unbuffered pipes:
Popen(command, stdin=subprocess.PIPE,
stdout=f, stderr=subprocess.PIPE, bufsize=0)
(Well-behaved programs prompt on standard error. Yours seem to do so, since you showed the prompts despite the stdout=f; if they don’t do so reliably, you get to read that from a pipe as well, search for prompts in it, and copy it to a file yourself.)
Unix
Set all pipes non-blocking. Then use select to monitor the stderr pipes for all processes. (You might try selectors instead.) Buffer what you read separately for each process until you recognize a prompt from one. Then display the prompt (identifying the source process) and accept input from the user—if the output between prompts fits in the pipe buffers, this won’t slow the background work down. Put that user input in a buffer associated with that process, and add its stdin pipe to the select.
When a stdin pipe shows ready, write to it, and remove it from the set if you finish doing so. When a read from a pipe returns EOF, join the corresponding process (or do so in a SIGCHLD handler if you worry that a process might close its end early).
Windows
Unless you have a select emulation available that supports pipes, you’ll have to use threads—one for each process, or one for each pipe if a process might produce output after writing a prompt and before reading the response. Then use a Queue to post prompts as messages to the main thread, which can then use (for example) another per-process Queue to send the user input back to the thread (or its writing buddy).
This works on any threading-supporting platform and has the potential advantage of not relying on pipe buffers to avoid stalling talkative processes.

Python script doesn't restart itself properly

I have a Python script and I want to have it restart itself. I found the following lines Googling around:
def restart_program():
"""Restarts the current program.
Note: this function does not return. Any cleanup action (like
saving data) must be done before calling this function."""
python = sys.executable
os.execl(python, python, * sys.argv)
but problems became apparent right after trying this out. I'm running on a really small embedded system and I ran out of memory really quick (after 2 or three iterations of this function). Checking the process list, I can see a whole bunch of python processes.
Now, I realize, I could check the process list and kill all processes that have another PID than myself - is this what I have to do or is there a better Python solution?

This spawns a new child process using the same invocation that was used to spawn the first process, but it does not stop the existing process (more precisely: the existing process waits for the child to exit).
The easier way would be to refactor your program so you don't have to restart it. Why do you need to do this?

I rewrote my restart function as follows, it will kill every python process other than itself before launching the new sub process:
def restart_program():
"""Restarts the current program.
Note: this function does not return. Any cleanup action (like
saving data) must be done before calling this function."""
logger.info("RESTARTING SCRIPT")
# command to extract the PID from all the python processes
# in the process list
CMD="/bin/ps ax | grep python | grep -v grep | awk '{ print $1 }'"
#executing above command and redirecting the stdout int subprocess instance
p = subprocess.Popen(CMD, shell=True, stdout=subprocess.PIPE)
#reading output into a string
pidstr = p.communicate()[0]
#load pidstring into list by breaking at \n
pidlist = pidstr.split("\n")
#get pid of this current process
mypid = str(os.getpid())
#iterate through list killing all left over python processes other than this one
for pid in pidlist:
#find mypid
if mypid in pid:
logger.debug("THIS PID "+pid)
else:
#kill all others
logger.debug("KILL "+pid)
try:
pidint = int(pid)
os.kill(pidint, signal.SIGTERM)
except:
logger.error("CAN NOT KILL PID: "+pid)
python = sys.executable
os.execl(python, python, * sys.argv)
Not exactly sure if this is the best solution but it works for the interim anyways...

Python too many open files (subprocesses)

I seem to be having an issue with Python when I run a script that creates a large number of sub processes. The sub process creation code looks similar to:
Code:
def execute(cmd, stdout=None, stderr=subprocess.STDOUT, cwd=None):
proc = subprocess.Popen(cmd, shell=True, stdout=stdout, stderr=stderr, cwd=cwd)
atexit.register(lambda: __kill_proc(proc))
return proc
The error message I am receiving is:
OSError: [Errno 24] Too many open files
Once this error occurs, I am unable to create any further sub processes until kill the script and start it again. I am wondering if the following line could be responsible.
atexit.register(lambda: __kill_proc(proc))
Could it be that this line creates a reference to the sub process, resulting in a "file" remaining open until the script exits?

So it seems that the line:
atexit.register(lambda: __kill_proc(proc))
was indeed the culprit. This is probably because of the Popen reference being kept around so the process resources aren't free'd. When I removed that line the error went away. I have now changed the code as #Bakuriu suggested and am using the process' pid value rather than the Popen instance.

Firstly, run ulimit -a to find out how many the maximum open files are set in your Linux system.
Then edit the system configuration file /etc/security/limits.conf and add those code in the bottom.
* - nofile 204800
Then you can open more sub processes if you want.