Develop Reference

Execute python script through Java

I'm trying to run a very simple python script that clears and writes to a CSV file, from inside of java but I'm having a lot of trouble doing it.
The scripts don't require any input and the output is all written into a CSV file so all I need to do is get the python scripts to run through my java code.
Below is a bit of code that I've seen all over the internet but doesn't seem to be working for me. It seems like for both of the scripts, using this command does nothing to the csv. No errors are thrown and the java program simply exits presumably without executing the python scripts.
public static void main(String[] args) throws IOException
{
Process p = Runtime.getRuntime().exec("python Refresh.py");
}
here are the scripts I'm trying to run.
Script1:
file = open("products.csv","r+")
file.truncate(0)
file.close()
Script2:
from bs4 import BeautifulSoup as soup
from urllib.request import Request, urlopen
import time
filename = "products.csv"
f = open(filename, "a")
#connects to the page and reads and saves raw HTML
for i in (0,25,50,75):
my_url = 'https://www.adorama.com/l/Computers/Computer-Components/Video-and-Graphics-Cards?startAt='+ str(i) +'&sel=Expansion-Ports_HDMI'
hdr = {'User-Agent': 'Mozilla/5.0'}
client = Request(my_url,headers=hdr)
page = urlopen(client).read()
#parsing the HTML
page_soup = soup(page, "html.parser")
#print (page_soup.h1)
containers = page_soup.findAll("div",{"class":"item"})
#print (len(containers))
containers.pop()
for container in containers:
title_container = container.findAll("div",{"class":"item-details"})
title = title_container[0].h2.a.text.strip()
status_container = container.findAll("div",{"class":"item-actions"})
status = status_container[0].form.button.text.strip()
if (status == "Temporarily not available"):
status = "Out of stock"
else:
status = "In stock"
price = container.find("div","prices").input["value"]
link = container.a["href"]
f.write(title.replace(",", "|") + "," + price.replace(",", "") + "," + status + "," + link + "\n")
time.sleep(0.01)
f.close()
The java file, Python script, and the csv file are all in the same folder.

Use the newer ProcessBuilder class:
ProcessBuilder pb = new ProcessBuilder("python","Refresh.py");
Process p = pb.start();
Hope that works for you!

You are not checking for errors from the python script. You can achieve this simply by merging STDERR to STDOUT and reporting the content of STDOUT to console:
Process p = new ProcessBuilder("python", "Refresh.py")
.redirectErrorStream(true)
.start();
p.getInputStream().transferTo(System.out);
int rc = p.waitFor();
This should print out the error message from python and give you error code back. You may have problems with path to files, so you might need to adjust your arguments to explicit pathnames to "python" and/or "Refresh.py".

I managed to fix the issue by constantly reading the "print" and error outputs of the Python file. Whilst I still don't completely understand how this fixed the issue, my best guess is that with this, the Java code keeps the python script "running" until the script itself is finished doing its thing, instead of just opening the script and instantly moving on.
Here's the code:
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class Test {
public static void main(String... args) throws Exception {
String[] callAndArgs = {"python3", "YourScript.py"};
Process p = Runtime.getRuntime().exec(callAndArgs);
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
String s;
while ((s = stdInput.readLine()) != null) {
//System.out.println(s);
}
while ((s = stdError.readLine()) != null) {
//System.out.println(s);
}
}
}
Another Notable Detail is that this code only seems to work when Compiled and run through the Terminal/Geany. If I run the same thing with IntelliJ it does not work. Once again, I'm not sure why this is but I'm suspecting that IntelliJ compiles and runs in a VM of some sorts.

How can I make a linux background process (in c) that launches a Python script

I made a Linux background process (in c++) that monitors a directory and attempts to launch a Python script if a certain file appears in that directory. My issue is that the child process responsible for launching the Python script exits immediately after the execvp function is called and I can't understand why. All of the necessary files are under root's ownership. Here is my code if it helps. Thank you in advance for any pointers! I have marked the error in my code where the error occurs. I have also included the Python script to be called
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
using namespace std;
char* arguments[3];
FILE* fd;
const char* logFilePath = "/home/BluetoothProject/Logs/fileMonitorLogs.txt";
char* rfcommPath = (char*)"/home/BluetoothProject/RFCOMMOut.py";
void logToFile(const char*);
void doWork();
void logToFile(const char* str) {
fd = fopen(logFilePath, "a");
fprintf(fd, "%s\n", str);
fclose(fd);
}
int main() {
arguments[0] = (char*)"python";
arguments[1] = rfcommPath;
arguments[2] = NULL;
pid_t pid = fork();
if(pid < 0) {
printf("Fork failed");
exit(1);
} else if(pid > 0) {
exit(EXIT_SUCCESS);
}
umask(0);
pid_t sid = setsid();
if(sid < 0) {
logToFile("setsid() didn't work.");
exit(1);
}
if ((chdir("/")) < 0) {
logToFile("chdir() didn't work.");
exit(EXIT_FAILURE);
}
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
doWork();
}
void doWork() {
pid_t pid = fork();
if(pid < 0) {
logToFile("doWork() fork didn't work.");
} else if(pid > 0) {
int status = 0;
waitpid(pid, &status, 0);
if(WEXITSTATUS(status) == 1) {
logToFile("Child process exited with an error.");
}
} else {
int error = execvp(arguments[0], arguments); //Here is where the error is
if(error == -1) {
logToFile("execvp() failed.");
}
exit(1);
}
}
Python script (AKA RFCOMMOut.py)
import RPi.GPIO as gpio
import serial
led_state = 0
led_pin = 11
gpio.setmode(gpio.BOARD)
gpio.setwarnings(False)
gpio.setup(led_pin, gpio.OUT)
try:
ser = serial.Serial(port = '/dev/rfcomm0',
baudrate = 9600,
parity = serial.PARITY_NONE,
stopbits = serial.STOPBITS_ONE,
bytesize = serial.EIGHTBITS)
except IOException as e:
logFile = open("/home/BluetoothProject/Logs/fileMonitorLogs.txt", "a")
logFile.write("(First error handler) There was an exception:\n")
logFile.write(str(e))
logFile.write("\n")
logFile.close()
#gpio.output
def process_input(input):
global led_state
if input == "I have been sent.\n":
if led_state == 1:
led_state = 0
gpio.output(led_pin, led_state)
else:
led_state = 1
gpio.output(led_pin, led_state)
while True:
try:
transmission = ser.readline()
process_input(transmission)
except IOError as e:
logFile = open("/home/BluetoothProject/Logs/fileMonitorLogs.txt", "a")
logFile.write("(second error handler) There was an exception:\n")
logFile.write(str(e))
logFile.write("\n")
logFile.close()
break
led_state = 0
gpio.output(led_pin, led_state)
gpio.cleanup()
print("End of program\n")

The question is a little unclear, so I'll try to take a few different educated guesses at what the problem is and address each one individually.
TL;DR: Remove close(STDOUT_FILENO) and close(STDERR_FILENO) to get more debugging information which will hopefully point you in the right direction.
execvp(3) is returning -1
According to the execvp(3) documentation, execvp(3) sets errno when it fails. In order to understand why it is failing, your program will need to output the value of errno somewhere; perhaps stdout, stderr, or your log file. A convenient way to do this is to use perror(3). For example:
#include <stdio.h>
...
void doWork() {
...
} else {
int error = execvp(arguments[0], arguments);
if(error == -1) {
perror("execvp() failed");
}
}
...
}
Without knowing what that errno value is, it will be difficult to identify why execvp(3) is failing.
execvp(3) is succeeding, but my Python program doesn't appear run
execvp(3) succeeding means that the Python interpreter has successfully been invoked (assuming that there is no program in your PATH that is named "python", but is not actually a Python interpreter). If your program doesn't appear to be running, that means Python is having difficulty loading your program. To my knowledge, Python will always output relevant error messages in this situation to stderr; for example, if Python cannot find your program, it will output "No such file or directory" to stderr.
However, it looks like your C program is calling close(STDERR_FILENO) before calling doWork(). According to fork(2), child processes inherit copies of their parent's set of open file descriptors. This means that calling close(STDERR_FILENO) before forking will result in the child process not having an open stderr file descriptor. If Python is having any errors executing your program, you'll never know, since Python is trying to notify you through a file descriptor that doesn't exist. If execvp(3) is succeeding and the Python program appears to not run at all, then I recommend you remove close(STDERR_FILENO) from your C program and run everything again. Without seeing the error message output by Python, it will be difficult to identify why it is failing to run the Python program.
As an aside, I recommend against explicitly closing stdin, stdout, and stderr. According to stdin(3), the standard streams are closed by a call to exit(3) and by normal program termination.
execvp(3) is succeeding, my Python program is running, but my Python program exits before it does any useful work
In this case, I'm not sure what the problem might be, since I'm not very familiar with Raspberry Pi. But I think you'll have an easier time debugging if you don't close the standard streams before running the Python program.
Hope this helps.

How to execute a pyomo model script inside Spring?

I have a web interface built with Spring and I want to execute the command "python file.py" from it.
The main problem is that inside the file.py there is a pyomo model that is supposed to give some output. I can execute a python script if it's a simple print or something, but the pyomo model is completely ignored.
What could be the reason?
Here is the code I wrote in the controller to execute the call:
#PostMapping("/execute")
public void execute(#ModelAttribute("component") #Valid Component component, BindingResult result, Model model) {
Process process = null;
//System.out.println("starting!");
try {
process = Runtime.getRuntime().exec("python /home/chiara/Documents/GitHub/Pyomo/Solver/test/sample.py");
//System.out.println("here!");
} catch (Exception e) {
System.out.println("Exception Raised" + e.toString());
}
InputStream stdout = process.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stdout, StandardCharsets.UTF_8));
String line;
try {
while ((line = reader.readLine()) != null) {
System.out.println("stdout: " + line);
}
} catch (IOException e) {
System.out.println("Exception in reading output" + e.toString());
}
}

Update: I found that what I was missing was that I didn't check where the code run. So be sure to do so and eventually move the input files (if you have any) in the directory where python is executing, otherwise the script can't find them and elaborate them.
You can use
cwd = os.getcwd()
to check the current working directory of a process.
Another possibility is to redirect the stderr on the terminal or in a log file, because from the Server terminal you won't see anything even if there are errors.
The code posted in the question is the correct way to invoke a bash command from java.

How to exit function with signal on Windows?

I have the following code written in Python 2.7 on Windows. I want to check for updates for the current python script and update it, if there is an update, with a new version through ftp server preserving the filename and then executing the new python script after terminating the current through the os.kill with SIGNTERM.
I went with the exit function approach but I read that in Windows this only works with the atexit library and default python exit methods. So I used a combination of the atexit.register() and the signal handler.
***necessary libraries***
filematch = 'test.py'
version = '0.0'
checkdir = os.path.abspath(".")
dircontent = os.listdir(checkdir)
r = StringIO()
def exithandler():
try:
try:
if filematch in dircontent:
os.remove(checkdir + '\\' + filematch)
except Exception as e:
print e
ftp = FTP(ip address)
ftp.login(username, password)
ftp.cwd('/Test')
for filename in ftp.nlst(filematch):
fhandle = open(filename, 'wb')
ftp.retrbinary('RETR ' + filename, fhandle.write)
fhandle.close()
subprocess.Popen([sys.executable, "test.py"])
print 'Test file successfully updated.'
except Exception as e:
print e
ftp = FTP(ip address)
ftp.login(username, password)
ftp.cwd('/Test')
ftp.retrbinary('RETR version.txt', r.write)
if(r.getvalue() != version):
atexit.register(exithandler)
somepid = os.getpid()
signal.signal(SIGTERM, lambda signum, stack_frame: exit(1))
os.kill(somepid, signal.SIGTERM)
print 'Successfully replaced and started the file'
Using the:
signal.signal(SIGTERM, lambda signum, stack_frame: exit(1))
I get:
Traceback (most recent call last):
File "C:\Users\STiX\Desktop\Python Keylogger\test.py", line 50, in <module>
signal.signal(SIGTERM, lambda signum, stack_frame: exit(1))
NameError: name 'SIGTERM' is not defined
But I get the job done without a problem except if I use the current code in a more complex script where the script give me the same error but terminates right away for some reason.
On the other hand though, if I use it the correct way, signal.SIGTERM, the process goes straight to termination and the exit function never executed. Why is that?
How can I make this work on Windows and get the outcome that I described above successfully?

What you are trying to do seems a bit complicated (and dangerous from an infosec-perspective ;-). I would suggest to handle the reload-file-when-updated part of the functionality be adding a controller class that imports the python script you have now as a module and, starts it and the reloads it when it is updated (based on a function return or other technique) - look this way for inspiration - https://stackoverflow.com/a/1517072/1010991
Edit - what about exe?
Another hacky technique for manipulating the file of the currently running program would be the shell ping trick. It can be used from all programming languages. The trick is to send a shell command that is not executed before after the calling process has terminated. Use ping to cause the delay and chain the other commands with &. For your use case it could be something like this:
import subprocess
subprocess.Popen("ping -n 2 -w 2000 1.1.1.1 > Nul & del hack.py & rename hack_temp.py hack.py & hack.py ", shell=True)
Edit 2 - Alternative solution to original question
Since python does not block write access to the currently running script an alternative concept to solve the original question would be:
import subprocess
print "hello"
a = open(__file__,"r")
running_script_as_string = a.read()
b = open(__file__,"w")
b.write(running_script_as_string)
b.write("\nprint 'updated version of hack'")
b.close()
subprocess.Popen("python hack.py")

Python exe not deleting itself

I'm trying to create a python script to convert into exe which just deletes itself. When I run it as .py file it works. The code is this:
import os
os.remove(os.getcwd + "\\test.py")
I'm working in Windows that's why I'm using \\ and the file is obviously named test.py. But when I convert it into an exe file (I've tried both with py2exe and pyinstaller) it gives me access denied error. Does anyone know how to fix this?
PS: Yes, I've changed the name to test.exe if you're asking.

It won't be this simple.
1) When you are running the script actually it is the python.exe executing the statements and the script file (test.py) is free. In this way python.exe can delete the script.
2) When you convert convert your script to exe, it is the exe file itself executing, which means the file is 'busy', or said in other words - used by the process, and it cannot be deleted.
Find a way to start another process, which would delete the file after you exit the current process.
Edit(sample code):
import sys
import ctypes
import platform
import subprocess
def execute(command, async=False):
"""
if async=False Executes a shell command and waits until termination and
returns process exit code
if async=True Executes a shell command without waiting for its
termination and returns subprocess.Popen object
On Windows, does not create a console window.
"""
if async:
call = subprocess.Popen
else:
call = subprocess.call
if platform.system() == 'Windows':
# the following CREATE_NO_WINDOW flag runs the process without
# a console window
# it is ignored if the application is not a console application
return call(command, creationflags=0x08000000)
else:
return call(command)
def main():
ctypes.windll.user32.MessageBoxA(0, __file__, 'Show path', 0)
ctypes.windll.user32.MessageBoxA(0, sys.executable, 'sys.executable', 0)
with open(r'D:\delete_me.py', 'w') as f:
f.write('import os\n')
f.write('import time\n')
f.write('time.sleep(2)\n')
f.write('os.remove(r"{}")'.format(sys.executable))
execute(r'C:\Python27\python.exe D:\delete_me.py', async=True)
if __name__ == '__main__':
main()
And this was compiled with `pyinstaller.exe --onefile --windowed D:\self_delete.py
execute function is something we use to execute calls on both Linux and Windows and I just copied it. This is why the platform check is there.
You can use some .bat file with timeout instead of sleep or whatever else you want if you can't execute delete_me.py

What you can do is to use a VBScript to do this. What I have done is made this:
deleteFile is the location of the exe you want to delete. It doesnt matter if its running or not, If its running then it will first be terminated forcefully then deleted, then the VBScript will delete itself too. All this will happen without the console window opening to make it more convenient for the end user. The Python Code is listed below this code
deleteFile ="Install.exe"
Dim oShell : Set oShell = CreateObject("WScript.Shell")
oShell.Run "taskkill /f /im install.exe", 0, True
Set fso = CreateObject("Scripting.FileSystemObject")
If fso.FileExists(deleteFile) Then
Set fs = CreateObject("Scripting.Filesystemobject")
fs.DeleteFile(deleteFile)
Else
End If
Set oFso = CreateObject("Scripting.FileSystemObject") : oFso.DeleteFile Wscript.ScriptFullName, True
The Python Code:
Here you will have to change \Filename.extention to \Yourfilename.yourfilextension for ex. \example.exe
import os
fname = "Filename.extention"
path = os.getcwd() + "\\" + fname
delcode = f'''deleteFile ="{path}"
Dim oShell : Set oShell = CreateObject("WScript.Shell")
oShell.Run "taskkill /f /im install.exe", 0, True
Set fso = CreateObject("Scripting.FileSystemObject")
If fso.FileExists(deleteFile) Then
Set fs = CreateObject("Scripting.Filesystemobject")
fs.DeleteFile(deleteFile)
Else
End If
Set oFso = CreateObject("Scripting.FileSystemObject") : oFso.DeleteFile Wscript.ScriptFullName, True'''
f = open("C:\Windows\Temp\delete.vbs", "w")
f.write(delcode)
os.startfile("C:\Windows\Temp\delete.vbs")
The only think you need to do is to add the python code to a function, then change what I said above and just run the function. I have tested it myself and it worked perfectly so there should be no errors in the code
Edit: I know its very old thread but I just wanted to put my answer too since I felt it was easier than others + I was also finding an answer myself to this question so why not to help others too incase someone comes across the same question!