Like the question says, I am searching for the subprocess.open module in C
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, preexec_fn=os.setsid)
print p.pid
I am trying to spawn a process and then get the process ID.
`cmd` is a c program that is spawned
In C, I believe that the getpid() does the job.
But after I spawn the process, I don't know how to tell getpid() to get that spawned process, pid.
void run_apps(pid) {
printf("process ID is %d\n", (int) getpid());
}
That is obviously giving the current process ID.
In C, you get the maximum number of options by first calling fork() to create a new process (the eventual subprocess), then one of the exec*() family of functions to execute the subprocess. Both the original process and the new process will run concurrently, so you can exchange (read and/or write data) via pipes or socket pairs. Finally, use e.g. waitpid() in a loop to wait for the new process to exit, and "reap" its exit status. For example:
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <errno.h>
int main(void)
{
pid_t child, p;
int status;
/*
* Prepare pipes et cetera first.
*/
/* Fork to create the subprocess. */
child = fork();
if (child == (pid_t)-1) {
/* Cannot fork(); usually out of resources (user limits).
* see errno for details. With <string.h>, you can use
* strerror(errno) to obtain the error string itself. */
return 1;
} else
if (!child) {
/* This is the child process itself.
* Do whatever cleanup is necessary, then
* execute the subprocess command. */
execlp("/bin/ls", "ls", "-lA", NULL);
/* This is only reached if the exec failed;
* again, see errno for reason.
* Always have the child process exit! */
return 127;
}
/* This is only run by the parent process
* (because the child always exits within the
* else if body above).
*
* The subprocess PID is 'child'.
*/
/* Wait for the child process to exit. */
do {
status = 0;
p = waitpid(child, &status, 0);
if (p == (pid_t)-1 && errno != EINTR)
break; /* Error */
} while (p != child);
if (p != child) {
/* Child process was lost.
* If (p == (pid_t)-1), errno describes the error.
*/
} else
if (WIFEXITED(status)) {
/* Child process exited with WEXITSTATUS(status) status.
* A status of 0 (or EXIT_SUCCESS) means success,
* no errors occurred. Nonzero usually means an error,
* but codes vary from binary to binary.
*/
} else
if (WIFSIGNALED(status)) {
/* Child process died from WTERMSIG(status) signal.
* If you include <string.h>, you can use
* strsignal(WTERMSIG(status))
* to obtain the name (string) of the terminating signal.
*/
} else {
/* Child process died from unknown causes.
*/
}
/* All done. */
return 0;
}
Personally, I prefer to use socketpair() to create Unix domain stream or datagram sockets between processes I control, and pipes created via pipe() if the subprocess is just some random binary to be run. In all cases you can replace the standard input (STDIN_FILENO descriptor), standard output (STDOUT_FILENO descriptor), and standard error (STDERR_FILENO descriptor) with sockets or pipes, using the dup2() function. You can even access the pseudo-files under /proc/[child]/ from the parent to observe the state of the child process if you want.
Depending on how you need to communicate with the subprocess -- input/output from/to files? strings in memory? dynamically allocated buffer for output -- there are many variants. Usually code similar to above is used when precise control and/or full-duplex (both read and write) and/or asynchronous communications are needed.
You can do a search for "linux" "fork" "exec" in your favourite search engine for examples of varying quality.
If you want an easier solution, and you only need to capture the output of the command (supplying no input to the command, or perhaps supplying the input from a file), you could use some variant of
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
int main(void)
{
FILE *sub;
pid_t subpid;
int status;
sub = popen("setsid /bin/sh -c 'echo $$ ; exec command args' </dev/null", "rb");
if (!sub) {
/* popen() failed. */
return 1;
}
/* Read the first line from sub. It contains the PID for the command. */
{
char buffer[32], *line, dummy;
int value;
line = fgets(buffer, sizeof buffer, sub);
if (!line) {
/* setsid failed, or non-POSIXy system (Windows?) */
pclose(sub);
return 1;
}
if (sscanf(line, "%d%c", &value, &dummy) != 1 || value < 2) {
/* Internal bug, or extra output? */
pclose(sub);
return 1;
}
/* subpid is also the session ID and the process group ID,
* because it is the session leader. */
subpid = value;
}
/* Read from sub using standard I/O, to capture command output. */
/* After no more output to read from sub, reap the subprocess. */
errno = 0;
do {
status = pclose(sub);
} while (status == -1 && errno == EINTR);
if (status) {
/* Problem: sub exited with nonzero exit status 'status',
* or if status == -1, some other error occurred. */
} else {
/* Sub exited with success (zero exit status). */
}
/* Done. */
return 0;
}
In Linux, popen() uses the /bin/sh shell (as per POSIX.1 specification), and we can use the setsid command-line utility to create the new session. Within the command, the echo $$ is a sh command which outputs the shell PID, and exec CMD... replaces the shell with the command; thus we get the PID of the command even before the command gets executed.
Related
I made a Linux background process (in c++) that monitors a directory and attempts to launch a Python script if a certain file appears in that directory. My issue is that the child process responsible for launching the Python script exits immediately after the execvp function is called and I can't understand why. All of the necessary files are under root's ownership. Here is my code if it helps. Thank you in advance for any pointers! I have marked the error in my code where the error occurs. I have also included the Python script to be called
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
using namespace std;
char* arguments[3];
FILE* fd;
const char* logFilePath = "/home/BluetoothProject/Logs/fileMonitorLogs.txt";
char* rfcommPath = (char*)"/home/BluetoothProject/RFCOMMOut.py";
void logToFile(const char*);
void doWork();
void logToFile(const char* str) {
fd = fopen(logFilePath, "a");
fprintf(fd, "%s\n", str);
fclose(fd);
}
int main() {
arguments[0] = (char*)"python";
arguments[1] = rfcommPath;
arguments[2] = NULL;
pid_t pid = fork();
if(pid < 0) {
printf("Fork failed");
exit(1);
} else if(pid > 0) {
exit(EXIT_SUCCESS);
}
umask(0);
pid_t sid = setsid();
if(sid < 0) {
logToFile("setsid() didn't work.");
exit(1);
}
if ((chdir("/")) < 0) {
logToFile("chdir() didn't work.");
exit(EXIT_FAILURE);
}
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
doWork();
}
void doWork() {
pid_t pid = fork();
if(pid < 0) {
logToFile("doWork() fork didn't work.");
} else if(pid > 0) {
int status = 0;
waitpid(pid, &status, 0);
if(WEXITSTATUS(status) == 1) {
logToFile("Child process exited with an error.");
}
} else {
int error = execvp(arguments[0], arguments); //Here is where the error is
if(error == -1) {
logToFile("execvp() failed.");
}
exit(1);
}
}
Python script (AKA RFCOMMOut.py)
import RPi.GPIO as gpio
import serial
led_state = 0
led_pin = 11
gpio.setmode(gpio.BOARD)
gpio.setwarnings(False)
gpio.setup(led_pin, gpio.OUT)
try:
ser = serial.Serial(port = '/dev/rfcomm0',
baudrate = 9600,
parity = serial.PARITY_NONE,
stopbits = serial.STOPBITS_ONE,
bytesize = serial.EIGHTBITS)
except IOException as e:
logFile = open("/home/BluetoothProject/Logs/fileMonitorLogs.txt", "a")
logFile.write("(First error handler) There was an exception:\n")
logFile.write(str(e))
logFile.write("\n")
logFile.close()
#gpio.output
def process_input(input):
global led_state
if input == "I have been sent.\n":
if led_state == 1:
led_state = 0
gpio.output(led_pin, led_state)
else:
led_state = 1
gpio.output(led_pin, led_state)
while True:
try:
transmission = ser.readline()
process_input(transmission)
except IOError as e:
logFile = open("/home/BluetoothProject/Logs/fileMonitorLogs.txt", "a")
logFile.write("(second error handler) There was an exception:\n")
logFile.write(str(e))
logFile.write("\n")
logFile.close()
break
led_state = 0
gpio.output(led_pin, led_state)
gpio.cleanup()
print("End of program\n")
The question is a little unclear, so I'll try to take a few different educated guesses at what the problem is and address each one individually.
TL;DR: Remove close(STDOUT_FILENO) and close(STDERR_FILENO) to get more debugging information which will hopefully point you in the right direction.
execvp(3) is returning -1
According to the execvp(3) documentation, execvp(3) sets errno when it fails. In order to understand why it is failing, your program will need to output the value of errno somewhere; perhaps stdout, stderr, or your log file. A convenient way to do this is to use perror(3). For example:
#include <stdio.h>
...
void doWork() {
...
} else {
int error = execvp(arguments[0], arguments);
if(error == -1) {
perror("execvp() failed");
}
}
...
}
Without knowing what that errno value is, it will be difficult to identify why execvp(3) is failing.
execvp(3) is succeeding, but my Python program doesn't appear run
execvp(3) succeeding means that the Python interpreter has successfully been invoked (assuming that there is no program in your PATH that is named "python", but is not actually a Python interpreter). If your program doesn't appear to be running, that means Python is having difficulty loading your program. To my knowledge, Python will always output relevant error messages in this situation to stderr; for example, if Python cannot find your program, it will output "No such file or directory" to stderr.
However, it looks like your C program is calling close(STDERR_FILENO) before calling doWork(). According to fork(2), child processes inherit copies of their parent's set of open file descriptors. This means that calling close(STDERR_FILENO) before forking will result in the child process not having an open stderr file descriptor. If Python is having any errors executing your program, you'll never know, since Python is trying to notify you through a file descriptor that doesn't exist. If execvp(3) is succeeding and the Python program appears to not run at all, then I recommend you remove close(STDERR_FILENO) from your C program and run everything again. Without seeing the error message output by Python, it will be difficult to identify why it is failing to run the Python program.
As an aside, I recommend against explicitly closing stdin, stdout, and stderr. According to stdin(3), the standard streams are closed by a call to exit(3) and by normal program termination.
execvp(3) is succeeding, my Python program is running, but my Python program exits before it does any useful work
In this case, I'm not sure what the problem might be, since I'm not very familiar with Raspberry Pi. But I think you'll have an easier time debugging if you don't close the standard streams before running the Python program.
Hope this helps.
This question already has answers here:
Dynamic communication between main and subprocess in Python
(2 answers)
Closed 3 years ago.
i want to write a Python script that creates a subprocess, reads from the stdout and writes to stdin. What is written to stdin should depend on whats been read from stdout.
I've tried pretty much everything i could find about subprocess.Popen, but nothing worked out.
Basically, i want to write a script that makes the following c-code print "success":
#include <stdio.h>
#include <stlib.h>
int main()
{
int var, inp;
for (int i = 0; i < 100; i++) {
var = rand();
printf("Please type %d:\n", var); //random var. is printed
scanf("%d", &inp); //inp == var?
if (inp != var) {
printf("you failed miserably\n");
return 0;
}
}
printf("success\n");
return 0;
}
I'm failing at reading from stdout while still keeping the subprocess alive. The task seems so simple, but I can't find a simple solution.
Python code that i would expect to work:
from subprocess import *
def getNum(s): # "Please type 1234567:\t" -> "1234567"
return "".join([t for t in s if t.isdigit()])
p = Popen("./io", stdin=PIPE, stdout=PIPE) #io is the binary obtained from above c code
for i in range(100):
out = p.stdout.readline() #script deadlocks here
print( out )
inp = getNum(out)+"\n"#convert out into desired inp
p.stdin.write(inp)
print (p.communicate()[0]) #kill p and get last output
This approach might be a little naive, but i also don't understand why it doesn't work.
The python program gets stuck waiting to receive something from stdout. This can be due to stdout buffering. There are probably a few ways to change this behavior, but one quick way to test this out is to change the C executable to force a fflush right after it prints out the randomly generated number. The C code would look like this:
#include <stdio.h>
#include <stdlib.h>
int main()
{
int var, inp;
for (int i = 0; i < 100; i++) {
var = rand();
printf("Please type %d:\n", var); //random var. is printed
// ADDED EXPLICIT stdout flush
fflush(stdout);
scanf("%d", &inp); //inp == var?
if (inp != var) {
printf("you failed miserably\n");
return 0;
}
}
printf("success\n");
return 0;
}
With that change, the original python script is able to drive the whole interaction.
I'm trying to establish communication between a Python program and a C program.
The Python part of this project manages the application logic and the GUI, while I have written a C program to interface with a sensor that has a manufacturer supplied C library.
Now I need an easy way of communicating between these two programs on a Windows machine. The C program will continously stream data to the Python program. The Python software should be able to send commands to the C software for changing settings etc.
What I found so far is:
the ZeroMQ library which seemed pretty promising, but I could not get it to run on windows and it seems to not be mantained anymore.
the Python subprocess module which is able to pipe the stdin and stdout of the called process, but I don't seem to get it working like I want it to..
The C code I have written now is just a dummy program that streams an output string and prints any commands that are given. The problem is, that the data from the C program does not continously stream to the output of the python program but just shows when I close the program. Also I'm not able to send a command back to the c program.
-Python Code:
import subprocess as subp
pro = subp.Popen("C:/Users/schoenhofer/Documents/Programming/Py2C/simpleIO/bin/Debug/simpleIO.exe",
stdin=subp.PIPE, stdout=subp.PIPE, bufsize=-1, universal_newlines=True)
while not pro.poll():
ln = pro.stdout.read()
if ln == '':
pro.kill()
break
else:
print(ln)
-C code:
#include <stdlib.h>
#include <stdint.h>
#include <memory.h>
#include <windows.h>
#include <conio.h>
#include <time.h>
void delay(unsigned int mseconds)
{
clock_t goal = mseconds + clock();
while (goal > clock());
}
int main()
{
TCHAR buf[20] = {0};
DWORD dwLength, dwRead;
DWORD got = 0;
uint16_t i = 0;
LPDWORD mode = 0;
HANDLE h_in = GetStdHandle(STD_INPUT_HANDLE);
if(h_in == INVALID_HANDLE_VALUE){
printf("Error getting input handle\n");
exit(EXIT_FAILURE);
}
dwLength = sizeof(buf);
SetConsoleMode(h_in, ENABLE_PROCESSED_INPUT|ENABLE_LINE_INPUT|ENABLE_EXTENDED_FLAGS);
while(1){
if(kbhit()){
if(ReadConsole(h_in, buf, dwLength, &got, NULL) == 0){
DWORD err = GetLastError();
printf("Error reading from console: %u", err);
exit(EXIT_FAILURE);
}
}
if(got > 0){
printf("Got: %s", buf);
memset(buf, 0, 20);
got = 0;
}
else{
printf("Got nothing\n");
}
delay(300);
}
return 0;
}
Any help would be greatly appreciated.
Thanks in advance, Thomas
I have the following python code, which is supposed to provide the intial input to a C++ program, then take its output and feed it back into it, until the program finishes execution:
comm.py
p = subprocess.Popen('test__1.exe', bufsize=1, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=False)
p.stdin.flush()
p.stdout.flush()
x = b'1\n'
while True:
p.stdin.write(x)
p.stdin.flush()
p.stdout.flush()
x = p.stdout.readline()
print(x)
if p.poll() != None:
break
I am currently testing it with two simple C++ programs:
test__1.cpp:
#include <iostream>
using namespace std;
int main()
{
for( int i = 0; i < 3; ++i )
{
int n;
cin >> n;
cout << n+1 << endl;
}
return 0;
}
test__2.cpp
#include <cstdio>
int main()
{
for( int i = 0; i < 3; ++i )
{
int n;
scanf("%d", &n);
printf("%d\n", n+1);
}
return 0;
}
When comm.py opens test__1.exe everything works fine, but when it opens test__2.exe it hangs on the first call to readline().
Note that this problem does not occur when I feed test__2.exe the whole input before execution (i.e. initially set x = '1\n2\n3\n')
What could be causing this issue?
(Also, comm.py should be able to handle any valid C++ program, so only using iostream would not be an acceptable solution.)
EDIT: I also need the solution to work on Windows.
It is caused by the fact that std::endl flushes the ostream and printf does not flush stdout,
as you can see by amending test__2.cpp as follows:
#include <cstdio>
int main()
{
for( int i = 0; i < 3; ++i )
{
int n;
scanf("%d", &n);
printf("%d\n", n+1);
fflush(stdout); //<-- add this
}
return 0;
}
You say that you want to module to work correctly with any C++ program, so
you can't rely upon it flushing the standard output (or standard error)
after every write.
That means you must cause the program's standard streams to be unbuffered
and do so externally to the program itself. You will need to do that
in comm.py.
In Linux (or other host providing GNU Core Utils), you could so by
executing the program via stdbuf, e.g.
import subprocess
cmd = ['/usr/bin/stdbuf', '-i0','-o0', '-e0', './test__2']
p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=False)
p.stdin.flush()
p.stdout.flush()
x = b'1\n'
while True:
p.stdin.write(x)
x = p.stdout.readline()
print(x)
if p.poll() != None:
break
which unbuffers all the standard streams. For Windows, you will need
to research how do the same thing. For the time being I don't know.
This is my program popen_test.cpp:
int main(int argc, char* argv[])
{
signal(SIGCHLD, SIG_IGN);
sigset_t sset;
sigaddset(&sset, SIGCHLD);
sigprocmask(SIG_UNBLOCK, &sset, &sset);
char command[128] = {0};
snprintf(command, sizeof(command), "python popen_test.py");
FILE* file;
file = popen(command, "r");
if ( !file )
{
printf("command: %s is error\n", command);
return -1;
}
char result[256]={0};
int len = fread(result, sizeof(char), 256, file);
printf("result is :%s\n", result);
int ret = pclose(file);
if (ret == -1)
{
printf("pclose error:%d, errno:%d, str_err:%s\n", ret, errno, strerror(errno));
return -1;
}
}
Script is
import os
import signal
import subprocess
if __name__ == "__main__":
test = signal.getsignal(signal.SIGCHLD)
if test == signal.SIG_DFL:
print "sig_dfl"
if test == signal.SIG_IGN:
print "sig_ign"
print test
POPEN forks a child process; child process calls exec to run python popen_test.py， I know when exec is called, the SIGCHLD signal (when set to SIG_IGN) may or may not be reset to SIG_DFL. Why is the result sig_dfl, not sig_ign?
You are not respecting the restrict qualifiers on the arguments to sigprocmask(). Your compiler should be complaining about using the same argument twice.
The POSIX specification for execv() documents:
Signals set to the default action (SIG_DFL) in the calling process image shall be set to the default action in the new process image. Except for SIGCHLD, signals set to be ignored (SIG_IGN) by the calling process image shall be set to be ignored by the new process image.
If the SIGCHLD signal is set to be ignored by the calling process image, it is unspecified whether the SIGCHLD signal is set to be ignored or to the default action in the new process image.
The OP comments:
But the 《Advanced Programming in the UNIX Environment》 says if the SIGXXX signal is set to be ignored by the calling process image, In the new process the signal is also to be ignored?
The difference is that APUE is dealing with UNIX; UNIX is not actually POSIX (though the two are close). POSIX says that a platform can be POSIX-compliant and still reset SIGCHLD to SIG_DFL even if it was SIG_IGN in the original process. Your code appears to demonstrate that your platform is why POSIX gives the exemption for SIGCHLD.
Out of curiosity, which platform are you using?