I have read way to many threads now and really lost.
Just trying to do something basic before I make it complicated.
so i have a script test.py
I want to call the script from within runme.py but without waiting so it will process the other chunk of code, but then when it gets to the end wait for test.py code to finish before continuing on.
I cant seem to figure out the correct syntax for the p = subprocess.Popen (I have tried so many)
and do I need the that to the test.py if its in the same directory?
here is what i have but cant get to work.
import subprocess
p = subprocess.Popen(['python test.py'])
#do some code
p.wait()
I cant seem to figure out the correct syntax for the p = subprocess.Popen (I have tried so many)
You want to pass it a list of arguments. The first argument is the program to run, python (although actually, you probably want sys.executable here); the second is the script that you want python to run. So:
p = subprocess.Popen(['python', 'test.py'])
and do I need the that to the test.py if its in the same directory?
This will work the same way as if you ran python test.py at the shell: it will just pass test.py as-is to python, and python will treat that as a path relative to the current working directory (CWD).
So, if test.py is in the CWD, this will just work.
If test.py is somewhere else, then you need to provide either an absolute path, or one relative to the CWD.
One common thing you want is that test.py is in not necessarily in the CWD, but instead it's in the same directory as the script/module that wants to launch it:
scriptpath = os.path.join(os.path.dirname(__file__), 'test.py')
… or in the same directory as the main script used to start your program:1
scriptpath = os.path.join(os.path.dirname(sys.argv[0]), 'test.py')
Either way, you just pass that as the argument:
p = subprocess.Popen(['python', scriptpath])
1. On some platforms, this may actually be a relative path. If you might have done an os.chdir since startup, it will now be wrong. If you need to handle that, you want to stash os.path.abspath(os.path.dirname(sys.argv[0])) in the main script at startup, then pass it down to other functions for them to use instead of calling dirname(argv[0]) themselves.
Related
My file structure looks like this:
runner.py
scripts/
something_a/
main.py
other_file.py
something_b/
main.py
anythingelse.py
something_c/
main.py
...
runner.py should look at all folders in scripts/ in run the main.py located there.
Right now I'm achieving this through subprocess.check_output. It works, but some of these scripts take a long time to run and I don't get to see any progress; it prints everything after the process has finished.
I'm hoping to find a solution that allows for 2 things to be done somewhat easily:
1) Stream the output instead of getting it all at the end
2) Doesn't
prohibit running multiple scripts at once
Is this possible? A lot of the solutions I've seen for running a Python script from another require knowledge of the other script's name/location. I can also enforce that all the main.py's have a specific function if that helps.
You could use Popen to loop through each file and write its content to multiple log files. Then, you could read from these files in real-time, while each one is populated. :)
How you would want to translate the output to a more readable format, is a little bit more tricky because of readability. You could create another script which reads these log files, with Popen, and decide on how you'd like this information read back in a understandable manner.
""" Use the same command as you would do for check_output """
cmd = ''
for filename in scriptList:
log = filename + ".log"
with io.open(filename, mode=log) as out:
subprocess.Popen(cmd, stdout=out, stderr=out)
I have a a file structure like the following (Windows):
D:\
dir_1\
batch_1.bat
dir_1a\
batch_2.bat
dir_2\
main.py
For the sake of this question, batch_1.bat simply calls batch_2.bat, and looks like:
cd dir_1a
start batch_2.bat %*
Opening batch_1.bat from a command prompt indeed opens batch_2.bat as it's supposed to, and from there on, everything is golden.
Now I want my Python file, D:\dir_2\main.py, to spawn a new process which starts batch_1.bat, which in turn should start batch_2.bat. So I figured the following Python code should work:
import subprocess
subprocess.Popen(['cd "D:/dir_1"', "start batch_1.bat"], shell=True)
This results in "The system cannot find the path specified" being printed to my Python console. (No error is raised, of course.) This is due to the first command. I get the same result even if I cut it down to:
subprocess.Popen(['cd "D:/"'], shell=True)
I also tried starting the batch file directly, like so:
subprocess.Popen("start D:/dir_1/batch_1.bat", shell=True)
For reasons that I don't entirely get, this seems to just open a windows command prompt, in dir_2.
If I forego the start part of this command, then my Python process is going to end up waiting for batch_1 to finish, which I don't want. But it does get a little further:
subprocess.Popen("D:/dir_1/batch_1.bat", shell=True)
This results in batch_1.bat successfully executing... in dir_2, the directory of the Python script, rather than the directory of batch_1.bat, which results in it not being able to find dir_1a\ and hence, batch_2.bat is not executed at all.
I am left highly confused. What am I doing wrong, and what should I be doing instead?
Your question is answered here: Python specify popen working directory via argument
In a nutshell, just pass an optional cwd argument to Popen:
subprocess.Popen(["batch_1.bat"], shell=True, cwd=r'd:\<your path>\dir1')
I am currently using subprocess to run a Python script inside of my current Python but it is keep giving me an error:
for dir in os.listdir(os.path.join(DIR2,dirname)):
temp = os.path.join(os.path.join(DIR2,dirname),dir)
files = [os.path.join(temp, f) for f in os.listdir(temp) if f.endswith("json")]
for lists in files:
subprocess.Popen(["python", DIR4, os.path.join(temp,lists)])
Above is what I am currently using.
DIR4 is the path of the python that I want to run.
Problem is, the python that I want to run can only take one file at a time.
However this subprocess looks like it tries to execute ALL at ONCE.
I want to run ONE at a time, instead of ALL at ONCE.
Because it is running ALL at ONCE, my python that I want to run does not work the way it is..
What do I need to do to change this?
If you want to wait first for the subprocess to terminate, before going ahead, I think you could use Popen.wait():
...
p = subprocess.Popen(["python", DIR4, os.path.join(temp,lists)])
p.wait()
...
To actually do what you're asking, and not hack it together through subprocess, you can use exec which allows you to run python code with your own provided globals and locals.
In older versions of Python (meaning pre-3), you can use execfile to achieve the same thing.
import os
import subprocess
import sys
import re
## fname_ext=sys.argv[1]
fname_ext=r"C:\mine\.cs\test.cs"
exe=os.path.splitext(fname_ext)[0]+".exe" # Executable
fdir=os.path.split(fname_ext)[0]
fcontent=open(fname_ext).read()
p_using=re.compile("\s*using\s+((\w+[.]*)+)")
p_namespace=re.compile("\s*namespace\s+(\w+)")
usings=p_using.findall(fcontent)
usings=[x[0] for x in usings]
references=[]
for i in os.listdir(fdir):
path=fdir+"\\"+i
try:
if os.path.isdir(path) or (not path.endswith('cs')):continue
with open(path) as fp:
content=fp.read()
namespaces=p_namespace.findall(content)
for n in namespaces:
if n in usings and 'System' not in n:
references+=[path]
except:
pass
command="csc /nologo "+" ".join(references)+" "+fname_ext
## command=" ".join(references)
#~ ---------------------------------------------------------
# Build:
option=1
if option==0:
# using os.system
print ">>",command
if os.system(command)==0:
os.system(exe)
else:
#~ Using subprocess module
## print type(references)
command=['csc']
## print command,references
command.extend(["/nologo","/out:"+exe])
command.extend(references)
command.append(fname_ext)
## print command
if subprocess.call(command,shell=True)==0:
## print "running %s"%exe
subprocess.call([exe],shell=True)
else:
pass
## print "Failed to run"
#~ ---------------------------------------------------------
I have this code above that is supposed to run a Csharp program from SciTE. It searches
every .cs file in the directory and finds the file with the namespace that the current
file has included. The command to run the file in SciTE is:
command.go.*.cs=python C:\mine\.py\csc.py $(FilePath)
command.go.subsystem.*.cs=0
That program logic part is okay.
The issue is that when hit F5 with sample Csharp code like this:
using System;
using System.Collections;
using MyNamespace;
class Test{
public static void Main(String[] args){
MyObject inst=new MyObject();
MyObject.self_destruct(inst);
}
}
it runs ok. But when I uncomment the second fname_ext and comment the first one
and run the csc.py file, a window opens and keeps running, printing command(this happens
using the os.system option). When you use the subprocess.call option, the same thing
happens but this time only when shell=True. It ran for only 15 seconds and there were 800+
cmd.exe and python.exe processes.I had to wait almost 5 minutes after killing cmd.exe
for the mouse to start responding and 2 minutes more for desktop peek to work.
When shell=False, it runs ok, the same way as when you hit the F5 key from the file.
What is happening here?
What is shell=True doing that makes it behave that way?
The problem is that your sys.argv looks something like this:
['python', r'C:\mine\.py\csc.py', 'whatever.cs']
So, with the fname_ext line uncommented, you set fname_ext to r'C:\mine\.py\csc.py'. Which means your script ends up just running itself—which again runs itself, etc., as fast as possible until your system chokes.
The reason it doesn't happen with shell=False is that you can't actually exec a Python script. Ultimately you end up calling CreateProcess with your script, which tries to interpret it as a .exe file, fails, and returns an error. But with shell=True, you pass your script to cmd.exe to run as a program, and it does the same thing an interactive prompt or Explorer would do: finds the right mapping to execute .py files and uses it. (And os.system does effectively the same thing as shell=True, but with a couple extra layers tossed in for good measure.)
Okay, I'll take a stab at this. If I understand the situation, this script is called csc.py and you want to call the csc c# compiler. When you run csc /nologo (etc...) through cmd.exe, it starts looking for something called 'csc' with a known extension. It finds csc.py in the current directory and since .py is a registered extension, that's what gets executed.
The solution is to rename your python file or call out 'csc.exe' explicitly.
When I run any Python script, I would like to see the script's filename appear in the Windows command line window's titlebar. For example, if I run a script called "mytest.py", I want to see "mytest" in the titlebar. I would like this to be automatic, so I don't have to add code to every one of my scripts.
Currently I'm attempting to do this with sitecustomize.py, because when Python is run, including from double-clicking a Python script, sitecustomize is imported before the script runs.
I've tried getting __main__'s __file__ and sys.argv, but sitecustomize doesn't see either:
file sitecustomize.py:
import __main__, sys
print "hasattr __main__.__file__:", hasattr(__main__, "__file__")
print "hasattr sys.argv:", hasattr(sys, "argv")
print "-" * 60
file mytest.py:
import sys
print "__file__ is:", __file__
print "sys.argv is:", sys.argv
raw_input() # don't end the script immediately
output:
hasattr __main__.__file__: False
hasattr sys.argv: False
------------------------------------------------------------
__file__ is: C:\Documents and Settings\Owner\Desktop\mytest.py
sys.argv is: ['C:\\Documents and Settings\\Owner\\Desktop\\mytest.py']
I'm glad you asked! I now have it working for my scripts, and it's pretty cool.
Here's the code:
import sys
import time
from ctypes import windll
class SetTitle(object):
def __del__(self):
time.sleep(1)
command = ' '.join(sys.argv)
windll.kernel32.SetConsoleTitleA(command)
sys.argv = SetTitle()
This is for Python 2.x -- for 3.x you need to change SetConsoleTitleA to SetConsoleTitleW (last letter changes from A to W).
How it works: since the sys.argv object does yet exist, I create an object and assign it to sys.argv; then, when Python assigns the actual argv to sys.argv, my object is tossed, and the __del__ method is called; the __del__ method is then able to access the real argv and set the title bar accordingly. I put the 1 second sleep in just to avoid any possible race conditions, but I'm not sure it's necessary. If you don't want to see all the command-line args, you can pre-process command any way you like.
My thanks to the folks on python-win32 mailing list, and Thomas Heller in particular, for helping with the 'set title' portion of this question.
When I run any Python script, I would
like to see the script's filename
appear in the Windows command line
window's titlebar. For example, if I
run a script called "mytest.py", I
want to see "mytest" in the titlebar.
I would like this to be automatic, so
I don't have to add code to every one
of my scripts.
I think you should add this functionality to all your scripts by a module and not by hacking it into sitecustomize.py. Also even if you still want to go the sitecustomize path you will need to pass __file__ from your script, which means you will not get around to add some code to all your scripts.
What you certainly can do is to put that code into a module and then import it in all your python scripts. Like I mentioned above, you need to pass __file__ from your main script otherwise you will get the modules filename. Also there is no need to import __main__ to retrieve __file__.