The following code works fine in Windows:
subprocess.Popen([PATH_TO_G++]/g++ file.cpp -o file.exe)
However in Linux I get the following error:
OSError: [Errno 2] No such file or directory
After reading the documentation and several SO threads, I found out that subprocess.Popen works differently in Windows and nix systems. In windows it takes the string as the parameter and launches it just like you'd launch it in terminal.
In linux however it requires a list of strings if you have parameters. The first value is the program itself, then go the attributes. You can make it behave like Windows by passing the Shell=True argument, but that's not a good solution for me.
I tried the shlex.split function, but it still doesn't work.
Based on your example you'll need to make sure the command is properly quoted:
subprocess.Popen([PATH_TO_G++ + "/g++", "file.cpp", "-o", "file.exe"])
There is no way PATH_TO_G++ is a valid variable name, so I'm just going to assume that you provided that as an example.
Now, more importantly, what are you trying to do with the subprocess? Just launch it and have it be the primary operation? Launch it and capture the output? Launch it in the background?
The documentation for the subprocess module is pretty clear and provides a lot of examples on how you might use it.
Related
Using Java's ProcessBuilder, I try to run a python script with activating virtual environment but i'm getting error like '.\venv\Scripts\activate" "' is not recognized as an internal or external command.
In command window cmd, I can use ".\venv\Scripts\activate" directly with no problem but in my code at below, error occured.
ProcessBuilder processBuilder = new ProcessBuilder("cmd.exe", "/c","cd C:\\Users\\onurc\\PycharmProjects\\OCR", "& .\\venv\\Scripts\\activate", "& python main.py");
That command doesn't look valid to me. You may be confused about what ProcessBuilder does / how windows (or any OS) works.
When you type a command in a dosbox or terminal, you're sending it to the shell - on linux, probably /bin/bash, or zsh or whatnot. On windows, you're sending it to cmd.exe. That program (so, bash or cmd) then decodes what you typed and does all sorts of fancy things to it, and then it in turns asks the underlying kernel to actually go run that app.
Java's ProcessBuilder does not invoke cmd/bash/zsh, it is a sibling to cmd/bash/zsh - it also does some fancy things, but far fewer fancy things. On windows, even the OS (or rather, the tool) does some fancy things whereas on linux it does not.
In general, trying to even guess whether the tool you run does fancy things, and which fancy things java's ProcessBuilder does, is not a sound way to write dependable software. Especially considering the relative hardship of testing this, given that it involves invoking external stuff.
Hence, you should opt out. Of all the fancy things. Thus:
Always call the command with a complete and absolute path.
Always use the multi-string/list version, never rely on java to split on spaces.
Never use *, ? or any other glob. Write out each and every parameter in full.
Do not try to use > or & or any other 'symbol that means something specific'. An OS doesn't support any of this stuff - you just pass [A] the full path to an executable and [B] a bunch of strings that the kernel will directly pass to the process without doing any processing on it. If you want to do stuff to standard out, error out, or standard in - then use Process's methods to do this.
Those &s in your command sure look like shellisms to me.
To fix:
You can tell the process which directory to run in, so there is no need for a cd command.
You should write the absolute path to CMD.exe. You can ask the environment. You're now relying on a combination of a properly configured PATH + the undefined behaviour of the JVM that does a bare minimum of fancy things (turning relative paths to absolute ones by PATH-scanning is a 'fancy thing' you should not be relying on).
I don't know what that & thing is, but you don't want that.
If activate is a script (shouldn't that end in .bat or .js or .vbs or whatnot?), run that, then run python separately. If activate sets things in the local environment table, make a script file that runs both and run the script instead.
Running python is not a good idea. Run C:\whatever\python.exe.
You have the right idea for activate, more or less (you're in a known dir and then write an actual path. If python.exe is guaranteeed to be in that OCR dir, run .\\python.exe, not python.
Convert command as a single string instead of passing separate arguments to the ProcessBuilder.
ProcessBuilder processBuilder = new ProcessBuilder("cmd.exe", "/c","cd C:\\Users\\onurc\\PycharmProjects\\OCR & .\\venv\\Scripts\\activate & python main.py");
Another way could be, if all command has to run under same folder, then change ProcessBuilder working directory using directory method:
Path rootDir = Paths.get("C:\\Users\\onurc\\PycharmProjects\\OCR");
ProcessBuilder processBuilder = new ProcessBuilder("cmd", "/C", ".\\venv\\Scripts\\activate & python main.py");
processBuilder.directory(rootDir.toFile());
Process process = processBuilder.start();
process.waitFor();
I am on mac OSX.
I have a program where I am trying to call downloaded libraries from the terminal. This is not possible if I don't know where the libraries are. I will use pip as a common library example
>>> os.system("pip -h")
32512
>>> os.system("which pip")
256
I have read this response to the 256 error, however, I still don't understand why it appears here. It says it is "frequently used to indicate an argument parsing failure" however the exact command works because this does not seem to be an argument parsing error to me.
I would like to be able to do something to the effect of:
os.system(os.system("which pip") +" -h")
If there is another way of doing this, I would love to hear it
Don't use os.system like that (and don't use which, either). Try this to find a program:
import os
for bin_dir in os.environ.get("PATH").split(":"):
if 'my_program' in os.listdir(bin_dir):
executable_path = os.path.join(bin_dir, 'my_program')
break
Note that this does assume that PATH was properly set by whatever process started the script. If you are running it from a shell, that shouldn't be an issue.
In general, using os.system to call common *NIX utilities and trying to parse the results is unidiomatic-- it's writing python as if it was a shell script.
Then, instead of using system to run pip, use the solution describe in this answer.
I have the situation that I want the job done from eclipse so I use eclipse's .launch configuration. I tried to make it run python directly but got error: error 193 (%1 is not a valid Win32 app). where %1 is probably my python script.
I decided to create a simple batch script that calls this big wild python animal.
I did a lot of combinations and found this the best (batch outputs some strings, runs python, waits for it, the outputs some strings again):
start /b /wait "Python_script.py" "%1" "%2" "%3" "%4" "%5"
It worked until python itself started to run exe file.
Once again I tried a lot of combinations:
os.system([exe, arg1, arg2, ...]) and
subprocess.call(..) and subprocess.check_output(..)
-> I either didn't see the output in eclipse console, or the output was delayed or there was only python / or only exe's output in console.
finally I used subprocess.Popen(...) and it's nearly allright - the only defect is that the output from python script don't wait for exe's process to finish, and when I use subprocess.Popen(...).wait() exe passes output to console but the WHOLE output from python script is delayed until the 'exe' terminates. I want to delay only the part of pythons script output that is written after the exe is called.
how to achieve this 'partly console output delay' is the main topic
advices on python and eclipse .launch configuration will be appreciated
general advices on how does the communication between this processes(?) work will be appreciated
Thanks!
It sounds to me like you have three different processes you're trying to get to work together, you've tried a whole bunch of stuff to get it working, and the code is complex enough that you can't easily post it here. That makes it pretty hard to get a good answer (Stack Overflow works much better with focused questions), but here's the general approach I'd take:
Does your script run if you try to run Python_script.py directly from the command prompt?
If it doesn't, then look into registering the .py file type in Windows.
If it does, then maybe Eclipse launch configurations don't support or don't properly support Windows registered file types. There should be no need to mess with batch files and start; just replace Python_script.py in your launch configuration with c:\Python27\python.exe Python_script.py (or similar).
Get your script working from a command prompt - able to run, with proper Python and subprocess output, and waiting for everything to terminate.
If things work from the command prompt and still don't work from Eclipse, then post a new question with a small snippet of code showing what you're trying and a description of what's wrong. subprocess.call, subprocess.check_output, and Popen all have different uses, so it's hard to give general advice besides just referring to the documentation.
I am trying to open a .txt file in Windows. The code is as follows:
subprocess.Popen("C:\folder\file.txt", shell=True)
This works perfectly fine. The default editor is automatically opened and the file is loaded, however, I have read somewhere before that invoking calls through the shell (cmd.exe in Windows) is less safe. How can I do the same without it. Simply setting shell=False is giving me the error:
OSError: [WinError 193] %1 is not a valid Win32 application
Now, I can try this as a workaround:
subprocess.Popen("notepad C:\folder\file.txt")
but this only works if notepad is available, hence loses its generality.
If you are using windows then there is the (non portable) os.startfile command which will take a file path and open it in the default application for that filetype.
In your case you would do:
import os
os.startfile("C:\folder\file.txt")
Note that this method won't work on Linux and Mac OSX, you'll have to have to use their own utilities for this. (open for OSX and xdg-open on Linux) and start them with subprocess.
The feature you try to use is a builtin thing of the windows cmd.exe. Therefore you need to set the shell=True parameter. The cmd.exe knows what to do with the file you hand in.
If you use shell=False, you try to start the file like a programm and hence nothing happens, since .txt files have no exe-header.
Read more about it in the documentation.
Reading further, you can find why using the shell=True parameter can be a security flaw. If you consider to assemble the parameters by user inputs, you should not use this, otherwise nothing speaks against it.
Anyway, I recommend using your second example, because it is explicit. You decide what program to start.
subprocess.Popen("notepad C:\folder\file.txt")
I am getting the following error:
WindowsError: [Error 2] The system cannot find the file specified
My code is:
subprocess.call(["<<executable file found in PATH>>"])
Windows 7, 64 bit. Python 3.x latest, stable.
Any ideas?
Thanks,
When the command is a shell built-in, add a shell=True to the call.
E.g. for dir you would type:
import subprocess
subprocess.call('dir', shell=True)
To quote from the documentation:
The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
On Windows, I believe the subprocess module doesn't look in the PATH unless you pass shell=True because it use CreateProcess() behind the scenes. However, shell=True can be a security risk if you're passing arguments that may come from outside your program. To make subprocess nonetheless able to find the correct executable, you can use shutil.which. Suppose the executable in your PATH is named frob:
subprocess.call([shutil.which('frob'), arg1, arg2])
(This works on Python 3.3 and above.)
On Windows you have to call through cmd.exe. As Apalala mentioned, Windows commands are implemented in cmd.exe not as separate executables.
e.g.
subprocess.call(['cmd', '/c', 'dir'])
/c tells cmd to run the follow command
This is safer than using shell=True, which allows shell injections.
If you are using powershell, then in it will be subprocess.call(['powershell','-command','dir']). Powershell supports a large portion of POSIX commands
After much head scratching, I discovered that running a file that is located in C:\Windows\System32\ while running a 32bit version of python on a 64bit machine is a potential issue, due to Windows trying to out-smart the process, and redirect calls to C:\Windows\System32 to C:\Windows\SysWOW64.
I found an example of how to fix this here:
http://code.activestate.com/recipes/578035-disable-file-system-redirector/
To quote from the documentation:
"Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use run() in many cases, but lots of existing code calls these functions."
SO: instead of subprocess.call use subprocess.run for Python 3.5 and above
I met the same issue while I was calling a PHP. The reason is PHP isn't in PATH so the command PHP was not found. But PowerShell found it does exist in the current location and it suggests replacing the 'PHP' by the '.\PHP' if I trust this command. Then it runs well.