When I use Popen to start a subprocess, if shell=True isn't set I get the following error message:
Traceback (most recent call last):
File "C:\Users\Student01\PycharmProjects\yolov5-master\detect.py", line 37, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
I don't get this error message if shell=True.
I don't want to need to set shell=True because it causes problems when I then try to call communicate or stdout on the subprocess (It just runs the subprocess into the shell without executing anything bellow the communicate call).
Here is the code (In case it can help):
#!/usr/bin/env python
import subprocess
detectPath = "C:\\Users\\Student01\\PycharmProjects\\yolov5-master\\detect.py"
print("First Print Passed")
process = subprocess.Popen("python {} --source 0".format(detectPath), stdout=subprocess.PIPE, shell=False)
# output = str(process.communicate())
while process.poll() is None:
process.stdout.readline()
print("Poll is None")
I am using Pycharm
I tried adding the yolov5 project files into the included files of my pycharm project (since I'm working in a venv) but still get the error.
The only solution to the moduleNotFound error seems to be to set shell=True in the Popen but that creates other problems.
Related
I'm currently trying to understand the subprocess.run() api and to do this my goal is to create a detached subprocess that will source another python env, log it's python executable path and exit properly.
Right now, I have this main function that throws an NamedError. I'd really like to rely on the run api but I don't know how to pass the creationflags and even if it's the good way to achieve my goal.
Edit : the error is of course because I didn't import the constants. My question was mainly : how to create such a subprocess with subprocess.run() and without being able to use DETACHED_PROCESS nor CREATE_NEW_PROCESS_GROUP.
import subprocess
import logging
import shlex
import sys
# VENV is virtual env on the model /my/venv/path/bin
def main() -> None:
python_path = sys.executable
logger.debug("Base python executable : %(python_path)s")
tokens = shlex.split(
f"source {VENV} && python3 -c 'import sys;print(sys.executable)'"
)
proc = subprocess.run(
tokens,
shell=True,
text=True,
capture_output=True,
creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP,
)
logger.debug(
"In the detached subprocess, the python exec is: %(proc.stdout)s"
)
The error :
Traceback (most recent call last):
File "test.py", line 37, in main
creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP,
NameError: name 'DETACHED_PROCESS' is not defined
DETACHED_PROCESS is in the subprocess namespace and it is only there on Windows systems: https://github.com/python/cpython/blob/3.10/Lib/subprocess.py#L85
subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP
I'm having issues with the subprocess module. I'm trying to run a terminal command in Python, which works perfectly fine in the terminal. The command is:
hrun SomeAction LogFile
I've tried a variety of options, including call(), run(), check_output(), and Popen(). No matter which method I use, I get the error:
FileNotFoundError: [Errno 2] No such file or directory: 'hrun': 'hrun'
My code is:
output = Popen(["hrun", "SomeAction", log_file_name], stdout=PIPE, stderr=PIPE)
where "hrun" and "SomeAction" are strings and log_file_name is a string variable.
I found other SO issues and, most (if not all) were resolved with either shell=True (which I dont want), or because the issue was due to a string instead of a list argument.
Thanks!
If you are just trying to run a command from a prompt within a script why not use something like
import os
os.system("your command")
You should be able to just run it like
os.system("hrun SomeAction LogFile")
Have having issues with Popen in python. Code in question:
from subprocess import Popen
Popen(["nohup", "/usr/local/bin/python2.7 /somescript.py"])
With following error:
failed to run command `/usr/local/bin/python2.7 /somescript.py': No such file or directory
Thing is that when I run the same command in a terminal, it works and the file definitely exists.
You are missing 2 " and a , Popen takes a list of arguments. Try this:
from subprocess import Popen
Popen(["nohup", "/usr/local/bin/python2.7", "/somescript.py"])
I successfully ran this script in Python, but IPython notebook threw an exception.
What could be the problem?
import os
from subprocess import call, Popen, PIPE, STDOUT
command = "java -cp \"C:\mallet-2.0.7\class;C:\mallet-2.0.7\lib\mallet-deps.jar\" cc.mallet.fst.SimpleTagger --train true --model-file nouncrf train.txt"
p = Popen(command, stdout=PIPE, stderr=STDOUT, shell = True)
for line in p.stdout:
print line
Exception in thread "main" java.io.FileNotFoundException: train.txt (The system cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(Unknown Source)
at java.io.FileReader.<init>(Unknown Source)
at cc.mallet.fst.SimpleTagger.main(SimpleTagger.java:504)
Apparently the question is not related to Python at all, as it is a Java exception, not a Python one. Most probably, the file (train.txt) is looked for in a current directory. When you run your script in Python, the current directory may differ from what it is in IPython. Is the Java program able to accept absolute file path? If yes, you may specify the absolute path in command line for Java program.
I'm trying to start a python program as a subprocess using the following code but the subprocess outputs ImportErrors to stderr.
The code
import subprocess
import sys
import os
environment = os.environ
command = ["python", "-u", "/test/my_python_program.py"]
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=environment)
for line in iter(p.stdout.readline, ''):
line = line.replace('\r', '').replace('\n', '')
print line
sys.stdout.flush()
The error
Traceback (most recent call last):
File "/test/my_python_program.py", line 31, in <module>
from PySide import QtGui, QtCore, QtNetwork
ImportError: dlopen(/Applications/Autodesk/maya2014/Maya.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python2.7/site-packages/PySide/QtGui.so, 2): Library not loaded: #executable_path/libpyside-python2.7.1.1.dylib
Referenced from: /Applications/Autodesk/maya2014/Maya.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python2.7/site-packages/PySide/QtGui.so
Reason: image not found
I'm running my code from a version of python embedded into another program (Autodesk Maya). The module's that cause the ImportError are importable from the main processes python interpreter. My assumption was that I could supply the main process's python's env to the subprocess to give it access to the modules that the main python instance had access too.
Does the approach make sense? and if so what am I doing wrong?
Edit1
I also tried to use the embedded version of python used in the popen call and the errors remained. If I remove the Import the embedded version of python runs just fine.
Edit 2
These are the environment variables that os.environ returns
MAYA_MODULE_PATH
AUTOLOADER_LAPS
LOGNAME
USER
SUBSTANCES_LOCATION
PATH
HOME
MAYA_SCRIPT_BASE
MENTALRAY_INCLUDE_LOCATION
SHELL
MAYA_LICENSE_METHOD
MAYA_LICENSE
QT_MAC_NO_NATIVE_MENUBAR
MAYA_SCRIPT_PATH
MAYA_REVERSE_FILEFORMAT_EXT
WF_IMF_CIN_WHITE_POINT
MAYA_LOCATION
PYTHONPATH
SSH_AUTH_SOCK
MENTALRAY_SHADERS_LOCATION
Apple_PubSub_Socket_Render
MAYA_PRESET_PATH
XBMLANGPATH
MAYA_RENDER_DESC_PATH
MAYA_SHADER_LIBRARY_PATH
MENTALRAY_LOCATION
TMPDIR
__KMP_REGISTERED_LIB_5123
MAYA_PLUG_IN_PATH
MAYA_APP_DIR
PYTHONHOME
MAYA_PLUG_IN_RESOURCE_PATH
__CF_USER_TEXT_ENCODING
IMF_PLUG_IN_PATH
__CHECKFIX1436934
WF_IMF_CIN_CORRECTION
Edit 3
Here's the contents of PYHOME
/Applications/Autodesk/maya2014/Maya.app/Contents/Frameworks/Python.framework/Versions/Current
The Python running inside Maya is not completely standard...
When run from within Maya, you might need to use the mayapy command to launch Python. This bootstraps Python according to Autodesk's requirements and should ensure the environment required to run Python is correct. (Although I think this may be what you are referring to in Edit1).
Have you also tried not passing through the environment to the popen command? I think the default behaviour of popen is to inherit the current process' environment which is what you want. I would expect the environment to be setting DYLD_LIBRARY_PATH and/or DYLD_FRAMEWORK_PATH at some point to tell it where Qt is.