I read similar posts but nothing specific to this. Or maybe my logic is incorrect and you can correct me please.
What I'm trying to do:
Write python code, that then calls an already compiled C or C++ code. But how would I pass an argument to that C/C++ script given that it's already been compiled?
I have a program in Python that manipulates an Excel file. And then at some point I need to search through, append, and create an array of 10,000 cells. I figured C/C++ would make this faster since python is taking some time to do that.
Help please.
Let's say we want to write a Python script that acts as a wrapper on top of a C binary and passes arguments from the terminal to the C binary. first, create a test.c C program as follows:
#include <stdio.h>
int main(int argc, char *argv[])
{
if(argc > 1)
{
int i;
printf("C binary: ");
for(i = 0; i < argc; i++)
printf("%s ", argv[i]);
printf("\n");
}
else
printf("%s: No argument is provided!\n", argv[0]);
return(0);
}
then compile it using:
gcc -o test test.c
and run it for two dummy arguments using:
./test arg1 arg2
Now, going back to your question. How I could pass arguments from Python to a C binary. First you need a Python script to read the arguments from the terminal. The test.py would do that for you:
import os
import sys
argc = len(sys.argv)
argv = sys.argv
if argc > 2:
cmd = './'
for i in range(1,argc):
cmd = cmd + argv[i] + ' '
if os.path.isfile(argv[1]):
print('Python script: ', cmd)
os.system(cmd)
else:
print('Binary file does not exist')
bin = 'gcc -o ' + argv[1] + ' '+ argv[1] + '.c'
print(bin)
os.system(bin)
if os.path.isfile(argv[1]):
os.system(cmd)
else:
print('Binary source does not exist')
exit(0)
else:
print('USAGE: python3.4', argv[0], " BINARY_FILE INPUT_ARGS");
exit(0)
Having test.c and test.py in the same directory. Now, you can pass arguments from the terminal to the test C binary using:
python3.4 test.py test arg1 arg2
Finally, the output would be:
Python script: ./test arg1 arg2
C binary: ./test arg1 arg2
Two last remarks:
Even if you don't compile the source code, the test.py will look for the test.c source file and try to compile it.
If you don't want to pass arguments from the terminal, you can always define the arguments in the Python script and pass them to the C binary.
Related
On Windows, when a Python script gets its argv, has parsing been done by the MS C Runtime?
Does the windows implementation of Python use the MS C Runtime to get its argv, then pass arguments to a script's argv, by inspecting its own argv, removing the first element that has the path to python.exe, and passing rest of argv on?
If not, then I'm wondering where is the documentation for the built in python library that feeds the command line to argv, in Windows programs?
I will include some example programs that give some background to what I am talking about
C:\blah>type w.c
#include <stdio.h>
#include <windows.h>
int main(int argc, char *argv[]) {
printf(GetCommandLine());
return 0;
}
C:\blah>w.exe "asdf" fff
w.exe "asdf" fff
C:\blah>
C:\blah>type w2.c
#include <stdio.h>
int main(int argc, char *argv[]) {
int i = 0;
while (argv[i]) {
printf("argv[%d] = %s\n", i, argv[i]);
i++;
}
return 0;
}
C:\blah>w2 "asdf" fff
argv[0] = w2
argv[1] = asdf
argv[2] = fff
C:\blah>
In C, the thing that converts the Command Line (maybe what is shown by GetCommandLine()), to argv, is called the MS C Runtime Library, and it is documented here
The C program that uses argv is using the MS C Runtime library. Parsing C command-line arguments that's implicitly imported/included and implicitly called.
And by the way here is the Python equivalent of those.
C:\blah>type wpyth.py
import win32api
# pip install pywin32 , pip install pypiwin32
print(win32api.GetCommandLine())
C:\blah>
C:\blah>wpyth.py asdf werwe<ENTER>
C:\Users\User\AppData\Local\Programs\Python\Python38\python.exe "C:\blah\wpyth.py" asdf werwe
C:\blah>
or, more normally.. it can use argv:
C:\blah>type w2pyth.py
import sys
for n, a in enumerate(sys.argv): print(f"argv[{n}] = {a}")
C:\blah>
C:\blah>w2pyth.py asdf werwe
argv[0] = C:\blah\w2pyth.py
argv[1] = asdf
argv[2] = werwe
C:\blah>
C:\blah>python w2pyth.py asdf werwe
argv[0] = w2pyth.py
argv[1] = asdf
argv[2] = werwe
C:\blah>
I would think Yes, because python scripts are run by a python executable which is probably written in C, and that seems like the easiest way it would go about things.
Though am not sure, so am asking.
According to realpython.com, CPython (the original Python runtime) directly inherits its command line arguments from its C base. Explained more in-depth at https://nullprogram.com/blog/2022/02/18, C splits the arguments, which are passed to the program as a single string, by whitespace and some quotation/backslash escape handling. However, it's also mentioned that implementations vary, even between Microsoft's own runtimes, and the language does not define an exact algorithm. After the arguments are split, they're passed to the program as an array of strings, which are then usable for whatever you like, in Python.
If working with both C and Python, you may notice the lack of argc in Python, and only having argv, due to it's high-level features like lengths from arrays and enumerables. C requires argc, just in case you need to use them, since it does not have the same level of abstraction as Python.
I've got an .exe which prompts the user for input at the command line interface for several numerical parameters and then generates data in a .txt. I would like to use Python in order to run the .exe repeatedly with different numerical parameters.
In Python, I've called the executable with:
subprocess.call(["executable.exe"])
How can I run the executable and specify input parameters (note: I am not referring to miscellaneous parameters such as -s, -t, etc but actual numerical parameters which are fed into the .exe)?
Thanks
EDIT: My .exe was created from a .cpp which doubles a integer given by the user when prompted at CLI.
#include <iostream>
#include <fstream>
#include <iomanip>
using namespace std;
int main() {
int ExampleNumber;
cout << "Please enter a number: ";
cin >> ExampleNumber;
ExampleNumber = ExampleNumber*2;
ofstream ExampleFile;
ExampleFile.open("ExampleFile.txt");
ExampleFile << ExampleNumber;
ExampleFile.close();
}
I tried running the .py with the an input of '3' as an example but it does not seem to be working still?
import subprocess
subprocess.call(["Executable.exe", '3'])
You can pass arguments as follows
subprocess.call(["executable.exe", '--parametername1', 'value1',
'--parameter2', 'value2'])
Edit: I mentioned this answer when the code was not given. I assumed that program can read parameters from CLI. My answer is valid only if executable.exe can use input arguments from command line which is not the case here.
I guess my problem is fairly straight-forward, however I can't find a way to solve it. My process is the following:
I run a Python script: Test.py.
Within that script, I am calling a c++ program.
Test.py:
RunCprogram = "./Example"
os.system(RunCprogram)
I want that ./Example executable to return a double that can be used later in my Python script. What is the best way to do that?
First of all, make sure Example outputs the desired data to stdout. If it's not, you can do nothing about it.
Then use the Python's subprocess module.
import subprocess
res=subprocess.check_output(["./Example"], universal_newlines=True)
If res contains a newline at the end, remove it with res.strip(). Finally, cast res to float with float(res).
Here's a small example based on #ForceBru answer:
example.cpp, compile with g++ example.cpp -o example
#include <iostream>
int main() {
std::cout << 3.14159 ;
return 0;
}
example.py
#!/usr/local/bin/python
import subprocess
res = subprocess.check_output(["./example"], universal_newlines=True)
print "The value of 2*" + u"\u03C0" + " is approximately " + str(2*float(res))
So I have a Python program that's finding .txt file directories and then passing those directories as a list(I believe) to my C++ program. The problem I am having is that I am not sure how to pass the list to C++ properly. I have used :
subprocess.call(["path for C++ executable"] + file_list)
where file_list is the [] of txt file directories.
My arguments that my C++ code accepts are:
int main (int argc, string argv[])
Is this correct or should I be using a vector? When I do use this as my argument and try to print out the list I get the directory of my executable, the list, and then smiley faces, symbols, and then the program crashes.
Any suggestions? My main point that I am trying to find out is the proper syntax of utilizing subprocess.call. Any help would be appreciated! thanks!
Another option is to use cython, (not a direct answer). Here is a simple complete example:
Suppose you have the following files:
cython_file.cpp
python_file.py
setup.py
sum_my_vector.cpp
sum_my_vector.h
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension(
name="cython_file",
sources=["cython_file.pyx", "sum_my_vector.cpp"],
extra_compile_args=["-std=c++11"],
language="c++",
)]
setup(
name = 'cython_file',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules,
)
cython_file.pyx
from libcpp.vector cimport vector
cdef extern from "sum_my_vector.h":
int sum_my_vector(vector[int] my_vector)
def sum_my_vector_cpp(my_list):
cdef vector[int] my_vector = my_list
return sum_my_vector(my_vector)
sum_my_vector.cpp
#include <iostream>
#include <vector>
#include "sum_my_vector.h"
using namespace::std;
int sum_my_vector(vector<int> my_vector)
{
int my_sum = 0;
for (auto iv = my_vector.begin(); iv != my_vector.end(); iv++)
my_sum += *iv;
return my_sum;
}
sum_my_vector.h
#ifndef SUM_MY_VECTOR
#define SUM_MY_VECTOR
using namespace::std;
int sum_my_vector(vector<int> my_vector);
#endif
python_file.py
from cython_file import sum_my_vector_cpp
print sum_my_vector_cpp([1,2,3,5])
Now run
python setup.py build_ext --inplace
and the you can run the python file
python python_file.py
11
"Passing a list through Python to C++"
An alternative approach would be to use Boost.Python, this may not answer your question directly, but still its worth pointing out another solution.
#include <boost/python.hpp>
#include <vector>
#include <string>
void get_dir_list( boost::python::list dir_list )
{
for (int i = 0; i < len(dir_list); ++i)
{
std::string x = boost::python::extract<std::string>(dir_list[i]);
// perform stuffs
std::cout << "This is " << x << std::endl ;
}
}
BOOST_PYTHON_MODULE(get_dir_list)
{
def("get_dir_list", get_dir_list);
}
Compiled Using :
g++ main.cpp -shared -fPIC -o get_dir_list.so -I/usr/include/python2.7 -lboost_python
Usage :
import get_dir_list
import os
get_dir_list.get_dir_list(os.listdir('.'))
Live Demo Here
I'll post this alternative solution since it would also work for other long lists of strings that needed to be passed.
In your Python script create a text file (I'll call it "masterFile") and write the file paths to the masterFile. You could give each file path a separate line. Then pass the masterFile's file path to your C++ program. This way you don't have to worry about the length of your command line arguments. Let your C++ program open and read the file for processing.
You can use something like os.remove() to get rid of the masterFile in your Python script once the C++ program has finished.
Also, you mentioned in the comments that you need to do different tasks dependent on different file paths: A suggestion would be to add a char at the beginning of each line in the masterFile to signal what needs to be done for the particular file. Example:
a Random/path/aFile.txt # a could mean do task 1
b Random2/path2/differentFile.c # b could mean do task 2
You pass a list to subprocess.call. subprocess.call converts this to what is needed for the system (which may vary, but certainly isn't a Python list). The system then arranges for this to be copied somewhere in the new process, and sets up the standard arguments to main, which are int, char**. In your C++ program, you must define main as int main( int argc, char** argv ); nothing else will work. (At least... a system could support int main( std::string const& ) or some such as an extension. But I've never heard of one that did.)
I am writing a Go program. From this Go program, I would like to call a Python function defined in another file and receive the function's return value so I can use it in subsequent processing in my Go program. I am having trouble getting any returned data back in my Go program though. Below is a minimum example of what I thought would work, but apparently doesn't:
gofile.go
package main
import "os/exec"
import "fmt"
func main() {
fmt.Println("here we go...")
program := "python"
arg0 := "-c"
arg1 := fmt.Sprintf("'import pythonfile; print pythonfile.cat_strings(\"%s\", \"%s\")'", "foo", "bar")
cmd := exec.Command(program, arg0, arg1)
fmt.Println("command args:", cmd.Args)
out, err := cmd.CombinedOutput()
if err != nil {
fmt.Println("Concatenation failed with error:", err.Error())
return
}
fmt.Println("concatentation length: ", len(out))
fmt.Println("concatenation: ", string(out))
fmt.Println("...done")
}
pythonfile.py
def cat_strings(a, b):
return a + b
If I call go run gofile I get the following output:
here we go...
command args: [python -c 'import pythonfile; print pythonfile.cat_strings("foo", "bar")']
concatentation length: 0
concatenation:
...done
A few notes:
I'm using the -c flag in the Python invocation so I can call the function cat_strings directly. Assume cat_strings is part of a Python file full of utility functions that are used by other Python programs, hence why I don't have any if __name__ == __main__ business.
I don't want to modify the Python file to print a + b (instead of return a + b); see the prior point about the function being part of a set of utility functions that ought to be callable by other Python code.
The cat_strings function is fictional and for demonstration purposes; the real function is something I don't want to simply reimplement in Go. I really am interested in how I can call a Python function from Go and get the return value.
I managed to have some working code for this by simply removing the quote around the command itself:
package main
import "fmt"
import "os/exec"
func main() {
cmd := exec.Command("python", "-c", "import pythonfile; print pythonfile.cat_strings('foo', 'bar')")
fmt.Println(cmd.Args)
out, err := cmd.CombinedOutput()
if err != nil { fmt.Println(err); }
fmt.Println(string(out))
}
And sure enough, in the source, you have this function (for Windows, at least, I don't know if that works for other OSes):
// EscapeArg rewrites command line argument s as prescribed
// in http://msdn.microsoft.com/en-us/library/ms880421.
// This function returns "" (2 double quotes) if s is empty.
// Alternatively, these transformations are done:
// - every back slash (\) is doubled, but only if immediately
// followed by double quote (");
// - every double quote (") is escaped by back slash (\);
// - finally, s is wrapped with double quotes (arg -> "arg"),
// but only if there is space or tab inside s.
func EscapeArg(s string) string { ...
So your code is ending up passing the following command line call:
$ python -c "'import pythonfile; print pythonfile.cat_strings(\\"foo\\", \\"bar\\")'"
Which, if tested, evaluates to a string and returns nothing, hence the 0-length output.