I want to call python script from C++ program as a child process.
I want to run this command: /home/minty99/.virtualenvs/venv_waveglow/bin/python /home/minty99/tacotron2/inference.py mps 1 4
which have commandline argument for python.
I tried these code:
string pt_arg = "mps " + to_string(i) + " " + to_string(fd[i][1]);
[1] execl("bash", "-c", "/home/minty99/.virtualenvs/venv_waveglow/bin/python /home/minty99/tacotron2/inference.py", pt_arg.c_str(), NULL);
[2] execl("/home/minty99/.virtualenvs/venv_waveglow/bin/python", "/home/minty99/tacotron2/inference.py", pt_arg.c_str(), NULL);
But it was not working.
First one: exec fails with "No such file or directory"
Second one: /home/minty99/tacotron2/inference.py: can't open file 'mps
1 4': [Errno 2] No such file or directory
How can I do this properly?
I haven't tried it, but reading the manpage of execl, it says
The first argument, by convention, should point to the file name
associated with the file being executed.
Where
int
execl(const char *file, const char arg0, ... /, (char *)0 */);
To me, this means your second version should likely read
execl("/home/minty99/.virtualenvs/venv_waveglow/bin/python", "python", "/home/minty99/tacotron2/inference.py", pt_arg.c_str(), NULL);
Which makes sense, since if you get the list of arguments in python, sys.argv[0] will be "python", not your first argument.
Related
I have a lua script configured to trigger once a subject's metadata is sent to a specific online Orthanc server. I want it to scrape the subject ID and then call a python script with the ID as an argument. When I put the command manually into my terminal, it works, but the lua script doesn't seem to be executing it.
There is a built-in Orthanc function that scrapes the ID from the subject once it is sent to the server.
The initial lua script had the following:
path = "/path/to/python_script.py"
os.execute("python " .. path .. " " .. subjectId)
But the script wasn't getting called.
I first wanted to see if it was even getting triggered, so I added in:
file = io.open("/path/to/textfile.txt", "a")
file:write("\nI am alive, subjectId is " .. subjectId)
file:close()
And that worked!
So then I wanted to see if there was something wrong with os.execute, so I did:
os.execute("touch /same/path/deleteme.txt")
which worked as well.
So it doesn't seem like os.execute isn't working.
Does anyone have any idea why the script isn't getting called?
EDIT: Does anyone know how to check the status of the os.execute command?
EDIT: I am using Python 3.5.6, Lua 5.1.4, and Linux.
Firstly, to address you question about checking the status from os.execute: this function returns a a status code, which is system dependent (https://www.lua.org/manual/5.1/manual.html#pdf-os.execute). I tried to handle an invalid command by recording this status code, but found it to be somewhat unhelpful; additionally, the shell itself printed an error message.
os.execute("hello") -- 'hello' is not a shell command.
> sh: 1: hello: not found
This error message from the shell was not being caught and read by my Lua script, but was instead being sent directly to stderr. (Good reference about that: https://www.jstorimer.com/blogs/workingwithcode/7766119-when-to-use-stderr-instead-of-stdout.)
I found an interesting solution for catching any error output using temp files.
tmpname = os.tmpname()
os.execute(string.format("hello 2> %s", tmpname))
for line in io.lines(tmpname) do
print("line = " .. line)
end
This prints: line = sh: 1: hello: not found, which is the error described earlier. os.execute should also return the status of the command like this:
a, b, c = os.execute("echo hello")
> hello
print(a, b, c)
> true exit 0
d, e, f = os.execute("hello") -- An invalid command
> sh: 1: hello: not found
print(d, e, f)
> nil exit 127
In this example, c and f are the exit statuses of their respective commands. If the previous command, i.e. executing your Python script, failed, then the exit status should be non-zero.
To address your primary question regarding Python, I would double-check the path to the script--always a good idea to start with a simple sanity check. Consider using string.format to assemble the command like this:
command = string.format("python %s %i", tostring(path), tonumber(subjectId))
os.execute(command)
Also, it would be helpful to know which version of Lua/Python you are using, and perhaps your system as well.
EDIT: depending on whether or not you need them to remain for a bit, you should remove any temp files generated by os.tmpname with os.remove. I also attempted to replicate your situation with a simple test, and I had no trouble executing the Python script with os.execute in a Lua script located in a different directory.
For reference, this is the Lua script, called test.lua, I created in a temp directory called /tmp/throwaway:
#!/usr/bin/lua
local PATH_TO_PY_FILE = "/tmp/py/foo.py"
local function fprintf(fil, formal_arg, ...)
fil:write(string.format(formal_arg, ...))
return
end
local function printf(formal_arg, ...)
io.stdout:write(string.format(formal_arg, ...))
return
end
local function foo(...)
local t = {...}
local cmd = string.format("python3 %s %s", PATH_TO_PY_FILE, table.concat(t, " "))
local filename = os.tmpname()
local a, b, status = os.execute(string.format("%s 2> %s", cmd, filename))
printf("status = %i\n", status)
local num = 1
for line in io.lines(filename) do
printf("line %i = %s\n", num line)
num = num + 1
end
os.remove(filename)
return
end
local function main(argc, argv)
foo()
foo("hello", "there,", "friend")
return 0
end
main(#arg, arg)
(Please forgive my C-style main function, haha.)
In a separate temp directory, called /tmp/py, I created a Python file that looks like this:
import sys
def main():
for arg in sys.argv:
print(arg)
if __name__ == '__main__':
main()
The Lua script's function foo takes a variable number of arguments and supplies them as command-line arguments to the Python script; the Python script then simply prints those arguments one-by-one. Again, this was just a simple test for proof of concept.
The temp file created by os.tmpname should be in /tmp; as for your files, that is, your Lua and Python scripts, you would make sure you know exactly where those files are located. Hopefully that can resolve your problem.
Also, you could supply the path to the Python script--or any other necessary files--to the Lua script as command-line arguments and then slightly modify the existing code.
$> ./test.lua path-to-python-file
Then simply modify foo in test.lua to accept the Python file's path as an argument:
local function foo(py_path, ...)
local t = {...}
local cmd = string.format("python3 %s %s", py_path, table.concat(t, " "))
-- Everything else should remain the same.
end
In my django python project I used to invoke my C++ script via g++:
os.system('g++ -std=c++0x mutualcepepe.cpp -D "threshold = ' + str(thresh) + '" -o mutualout')
"thresh" was a simple float variable. It worked but the idea of whole project changed a bit and now I want to pass a string containing different let's say "type" of characters.
I will show my problem on the example and in this case my macro "djangoname" (not a "threshold" anymore) is ">gi|111>gi|222>gi|333>gi|444".
Invocation:
os.system('g++ -std=c++0x mutualcepepe.cpp -D "djangoname = ' + str(filename2only) + '" -o mutualout')
Errors I get in the terminal:
mutualcepepe.cpp: In function ‘int main(int, char**)’:
<command-line>: 0:14: error: expected primary-expression before ‘>’ token
mutualcepepe.cpp:
99:30: note: in expansion of macro ‘djangoname’ string filename
to_string(djangoname);
^
<command-line>:0:15: error: ‘gi’ was not declared in this scope
mutualcepepe.cpp:99:30: note: in expansion of macro ‘djangoname’
string filename = to_string(djangoname);
I think the point is, that when g++ compilator "read" what the macro contains, it some kinda divides it, when it gets special character, or when after number it reads letter, because after that it treat it as a integer not a string data. So my question is, is it possible to pass in g++ the macro (or anyhow "string variable") containing "different type" of characters, in the way which g++ compiler will run without the problem?
I wondered about translation some "unconvinient" characters for other ones, and turned them back in c++ script, but I can't be sure what my macro will contain, that depend on users who will use my net app.
To be honest I have an idea to avoid it, but it is totally silly and connected with senseless opening new files and reading from them what take time.
Mabye I'm wrong and the problem has different nature, I hope You will be able to help me or give helpful advise.
You have to make the macro explicitly a string, like e.g.
os.system('g++ ... -Ddjangoname="' + ... + '" ...')
Note the placement of the double-quotes.
This problem has nothing to do with the shell or the compiler invocation, although I honestly think you would be well advised to use a different way to invoke the compiler from python, such as the [subprocess]1 module.
Somewhere in your C++ program you have:
string filename = to_string(djangoname);
You are using the -D option to effectively insert
#define djangoname >gi|111>gi|222>gi|333>gi|444
at the beginning of your program. That means that your declaration of filename will be:
string filename = to_string(>gi|111>gi|222>gi|333>gi|444);
which makes no sense at all, neither to C++ nor to me. Hence the error message, which is saying that you cannot start an expression with the > operator.
I don't think that is what you meant, but I have no idea what you really want to do.
os.system('g++ ...') doesn't directly starts the g++ process. It actually starts whatever is configured as the default shell (e.g. /bin/sh) and the command line is then interpreted by the shell.
To avoid this unnecessary hoop and its complications, you can directly execute g++ with Python's subprocess.Popen and its communicate() method. This allows you to pass command line arguments as an array.
For example :
import sys, subprocess
filename2only = '">gi|111>gi|222>gi|333>gi|444"'
args = [
'g++',
'-std=c++0x', 'mutualcepepe.cpp',
'-Ddjangoname=' + str(filename2only),
'-omutualout'
]
p = subprocess.Popen(args=args, bufsize=-1, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
sys.stdout.write(stdout)
sys.stderr.write(stderr)
if p.returncode != 0: sys.stderr.write('failed with exit code: ' + str(p.returncode))
This passes a -Ddjangoname=">gi|111>gi|222>gi|333>gi|444" option to the compiler, which is equivalent to #define djangoname ">gi|111>gi|222>gi|333>gi|444".
Note that str(filename2only) isn't strictly necessary in this case, but it has the nice property of supporting any type of value (string, int or float).
You could, e.g. do :
filename2only = 12.3 to pass -Ddjangoname=12.3, which is equivalent to #define djangoname 12.3
filename2only = 'a b c' to pass -Ddjangoname=a b c, which is equivalent to #define djangoname a b c
filename2only = '"a b c"' to pass -Ddjangoname="a b c", which is equivalent to #define djangoname "a b c"
filename2only = '"a \\"b\\" c"' to pass -Ddjangoname="a \"b\" c", which is equivalent to #define djangoname "a \"b\" c", i.e. djangoname is then the string literal a "b" c, which contains double quotes!
I'm currently working on a project that uses a C source file that has to interact with a python file (run the file and capture output) and im not exactly sure how to do it. currently the python file is run through terminal (linux) using:
python file arg1 arg2 arg3 arg4
and i am trying to embed python into the C code to just run the file first (no output capture) using the following code:
void python() {
FILE * file;
int argc;
char * argv[5];
argc=5;
argv[0]="pathtofile/file";
argv[1]="arg1";
argv[2]="arg2";
argv[3]="arg3";
argv[4]="arg4";
Py_SetProgramName(argv[0]);
Py_Initialize();
PySys_SetArgv(argc,argv);
file= fopen("pathtofile/file","r");
PyRun_SimpleFile(file,"pathtofile/file");
PyFinalize();
}
args1-2 are hard coded, and args3-4 are determined by the C code (just determines integer values), this is then passed to the python file where it then executes.
When running the above code i get a:
TypeError: unsupported operand type(s) for + :NoneType and 'str'
Any advice from here on what could be my issue is greatly appreciated.
EDIT:
I was using this as a guide as it seems to be similar to what im trying to acheive
Run a python script with arguments
Your argc is uninitialized - did you compile with warnings enabled and warnings made into errors (-Wall, -Werror on GCC?); and your argv is not properly null-terminated. Thus your code has undefined behaviour. Anything might happen including demons flying out of your nose. The argument to fopen mode must be a string yet you pass an int (character constant) - which has another UB.
Thus at least you must do:
int argc = 5;
char *argv[] = {
"pathtofile/file",
"arg1",
"arg2",
"arg3",
"arg4",
0
};
input = fopen(..., "r"); // "r", not 'r'!
Additionally you're not checking the return values of any of these functions. Any of them may fail and with Python you should expect them to fail - including your fopen! (Please tell that they're omitted for brevity).
I want to call an R program from a python script.
I wrote the following:
os.system("cat " + variableName + " | code.R")
It returns the error: sh: 1 : code.R: not found
cat: write error: Broken pipe
Yet, I am sure of the name of the R file.
Why is it not working?
So, if code.R is a script that has to be interpreted you must build the pipe to the interpreter not to the script. You receive a Broken PIPE error because code.R by it self don't know how to handle command line arguments.
On the other hand if what you want is store the variable value inside code.R you have to change | by >>.
os.system("cat " + variablename + ">> code.R")
EDIT: Since it's working from terminal, try this:
import subprocess
input = open(variableName, "r")
result = suprocess.call(["code.R"], stdin=input) # result is the return code for the command being called.
see subprocess.call for more details.
Is code.R in the current working directory? Is it executable? Can you run cat xxx | code.R from the shell and have it work properly, instead of running your python program?
I am writing a Go program. From this Go program, I would like to call a Python function defined in another file and receive the function's return value so I can use it in subsequent processing in my Go program. I am having trouble getting any returned data back in my Go program though. Below is a minimum example of what I thought would work, but apparently doesn't:
gofile.go
package main
import "os/exec"
import "fmt"
func main() {
fmt.Println("here we go...")
program := "python"
arg0 := "-c"
arg1 := fmt.Sprintf("'import pythonfile; print pythonfile.cat_strings(\"%s\", \"%s\")'", "foo", "bar")
cmd := exec.Command(program, arg0, arg1)
fmt.Println("command args:", cmd.Args)
out, err := cmd.CombinedOutput()
if err != nil {
fmt.Println("Concatenation failed with error:", err.Error())
return
}
fmt.Println("concatentation length: ", len(out))
fmt.Println("concatenation: ", string(out))
fmt.Println("...done")
}
pythonfile.py
def cat_strings(a, b):
return a + b
If I call go run gofile I get the following output:
here we go...
command args: [python -c 'import pythonfile; print pythonfile.cat_strings("foo", "bar")']
concatentation length: 0
concatenation:
...done
A few notes:
I'm using the -c flag in the Python invocation so I can call the function cat_strings directly. Assume cat_strings is part of a Python file full of utility functions that are used by other Python programs, hence why I don't have any if __name__ == __main__ business.
I don't want to modify the Python file to print a + b (instead of return a + b); see the prior point about the function being part of a set of utility functions that ought to be callable by other Python code.
The cat_strings function is fictional and for demonstration purposes; the real function is something I don't want to simply reimplement in Go. I really am interested in how I can call a Python function from Go and get the return value.
I managed to have some working code for this by simply removing the quote around the command itself:
package main
import "fmt"
import "os/exec"
func main() {
cmd := exec.Command("python", "-c", "import pythonfile; print pythonfile.cat_strings('foo', 'bar')")
fmt.Println(cmd.Args)
out, err := cmd.CombinedOutput()
if err != nil { fmt.Println(err); }
fmt.Println(string(out))
}
And sure enough, in the source, you have this function (for Windows, at least, I don't know if that works for other OSes):
// EscapeArg rewrites command line argument s as prescribed
// in http://msdn.microsoft.com/en-us/library/ms880421.
// This function returns "" (2 double quotes) if s is empty.
// Alternatively, these transformations are done:
// - every back slash (\) is doubled, but only if immediately
// followed by double quote (");
// - every double quote (") is escaped by back slash (\);
// - finally, s is wrapped with double quotes (arg -> "arg"),
// but only if there is space or tab inside s.
func EscapeArg(s string) string { ...
So your code is ending up passing the following command line call:
$ python -c "'import pythonfile; print pythonfile.cat_strings(\\"foo\\", \\"bar\\")'"
Which, if tested, evaluates to a string and returns nothing, hence the 0-length output.