import os
import subprocess
import sys
import re
## fname_ext=sys.argv[1]
fname_ext=r"C:\mine\.cs\test.cs"
exe=os.path.splitext(fname_ext)[0]+".exe" # Executable
fdir=os.path.split(fname_ext)[0]
fcontent=open(fname_ext).read()
p_using=re.compile("\s*using\s+((\w+[.]*)+)")
p_namespace=re.compile("\s*namespace\s+(\w+)")
usings=p_using.findall(fcontent)
usings=[x[0] for x in usings]
references=[]
for i in os.listdir(fdir):
path=fdir+"\\"+i
try:
if os.path.isdir(path) or (not path.endswith('cs')):continue
with open(path) as fp:
content=fp.read()
namespaces=p_namespace.findall(content)
for n in namespaces:
if n in usings and 'System' not in n:
references+=[path]
except:
pass
command="csc /nologo "+" ".join(references)+" "+fname_ext
## command=" ".join(references)
#~ ---------------------------------------------------------
# Build:
option=1
if option==0:
# using os.system
print ">>",command
if os.system(command)==0:
os.system(exe)
else:
#~ Using subprocess module
## print type(references)
command=['csc']
## print command,references
command.extend(["/nologo","/out:"+exe])
command.extend(references)
command.append(fname_ext)
## print command
if subprocess.call(command,shell=True)==0:
## print "running %s"%exe
subprocess.call([exe],shell=True)
else:
pass
## print "Failed to run"
#~ ---------------------------------------------------------
I have this code above that is supposed to run a Csharp program from SciTE. It searches
every .cs file in the directory and finds the file with the namespace that the current
file has included. The command to run the file in SciTE is:
command.go.*.cs=python C:\mine\.py\csc.py $(FilePath)
command.go.subsystem.*.cs=0
That program logic part is okay.
The issue is that when hit F5 with sample Csharp code like this:
using System;
using System.Collections;
using MyNamespace;
class Test{
public static void Main(String[] args){
MyObject inst=new MyObject();
MyObject.self_destruct(inst);
}
}
it runs ok. But when I uncomment the second fname_ext and comment the first one
and run the csc.py file, a window opens and keeps running, printing command(this happens
using the os.system option). When you use the subprocess.call option, the same thing
happens but this time only when shell=True. It ran for only 15 seconds and there were 800+
cmd.exe and python.exe processes.I had to wait almost 5 minutes after killing cmd.exe
for the mouse to start responding and 2 minutes more for desktop peek to work.
When shell=False, it runs ok, the same way as when you hit the F5 key from the file.
What is happening here?
What is shell=True doing that makes it behave that way?
The problem is that your sys.argv looks something like this:
['python', r'C:\mine\.py\csc.py', 'whatever.cs']
So, with the fname_ext line uncommented, you set fname_ext to r'C:\mine\.py\csc.py'. Which means your script ends up just running itself—which again runs itself, etc., as fast as possible until your system chokes.
The reason it doesn't happen with shell=False is that you can't actually exec a Python script. Ultimately you end up calling CreateProcess with your script, which tries to interpret it as a .exe file, fails, and returns an error. But with shell=True, you pass your script to cmd.exe to run as a program, and it does the same thing an interactive prompt or Explorer would do: finds the right mapping to execute .py files and uses it. (And os.system does effectively the same thing as shell=True, but with a couple extra layers tossed in for good measure.)
Okay, I'll take a stab at this. If I understand the situation, this script is called csc.py and you want to call the csc c# compiler. When you run csc /nologo (etc...) through cmd.exe, it starts looking for something called 'csc' with a known extension. It finds csc.py in the current directory and since .py is a registered extension, that's what gets executed.
The solution is to rename your python file or call out 'csc.exe' explicitly.
Related
I am using a batch file to access my portable VLC executable to convert an mp4 to an mp3:
set arg1=%1 REM -> arg1={my_mp4_full_path}
set arg2=%2 REM -> arg2={my_mp3_full_path}
echo %arg1%
echo %arg2%
REM batch file is in the same directory as "VLCPlayer" folder
"%~dp0\VLCPlayer\VLCPortable.exe" -I dummy %arg1% --sout=#transcode{acodec=mp3,ab=128,vcodec=dummy}:std{access="file",mux="raw",dst=%arg2%} vlc://quit
When I run this script the first time, vlc crashes and I get an unplayable mp3 file, however when I run the script again the script works and I get a playable mp3. Is there a way to remedy this, or make it consistent? I don't see why running it twice would yield different outcomes.
No I don't have ffmpeg on my computer it is unrecognizable internal or external command.
Note that I face the same problem when using powershell to perform the same task, when I import my function from a .psm1 script:
function ConvertToMp3(
[switch] $inputObject,
[string] $vlc = '{PAth_TO_PORTABLE_VLC}\VLCPortable.exe')
{
PROCESS {
$codec = 'mp3';
$oldFile = $_;
$newFile = $oldFile.FullName.Replace($oldFile.Extension, ".$codec").Replace("'","");
&"$vlc" -I dummy "$oldFile" ":sout=#transcode{acodec=$codec,
vcodec=dummy}:standard{access=file,mux=raw,dst=`'$newFile`'}" vlc://quit | out-null;
# delete the original file
Remove-Item $oldFile;
}
}
I get the same random output that sometimes works, sometimes crashes.
Update:
I feel like I should add more info of how I use the batch file:
I have a python script Convert.py and I call my batch file inside using os.system():
mp4_to_convert = arguments.file
full_path_mp4 = os.path.join(outdir,mp4_to_convert)
mp3_to_convert_to = mp4_to_convert.replace(".mp4",".mp3")
full_path_mp3 = os.path.join(outdir,mp3_to_convert_to)
command_string = """Convert_Script.bat \"{}\" \"{}\"""".format(full_path_mp4, full_path_mp3)
os.system(command_string)
This is the documentation of os.system():
os.system(command)
Execute the command (a string) in a subshell. This
is implemented by calling the Standard C function system(), and has
the same limitations. Changes to sys.stdin, etc. are not reflected in
the environment of the executed command. If command generates any
output, it will be sent to the interpreter standard output stream.
On Unix, the return value is the exit status of the process encoded in
the format specified for wait(). Note that POSIX does not specify the
meaning of the return value of the C system() function, so the return
value of the Python function is system-dependent.
On Windows, the return value is that returned by the system shell
after running command. The shell is given by the Windows environment
variable COMSPEC: it is usually cmd.exe, which returns the exit status
of the command run; on systems using a non-native shell, consult your
shell documentation.
Any pointers or suggestions would be helpful, thank you in advance for your help.
I have read way to many threads now and really lost.
Just trying to do something basic before I make it complicated.
so i have a script test.py
I want to call the script from within runme.py but without waiting so it will process the other chunk of code, but then when it gets to the end wait for test.py code to finish before continuing on.
I cant seem to figure out the correct syntax for the p = subprocess.Popen (I have tried so many)
and do I need the that to the test.py if its in the same directory?
here is what i have but cant get to work.
import subprocess
p = subprocess.Popen(['python test.py'])
#do some code
p.wait()
I cant seem to figure out the correct syntax for the p = subprocess.Popen (I have tried so many)
You want to pass it a list of arguments. The first argument is the program to run, python (although actually, you probably want sys.executable here); the second is the script that you want python to run. So:
p = subprocess.Popen(['python', 'test.py'])
and do I need the that to the test.py if its in the same directory?
This will work the same way as if you ran python test.py at the shell: it will just pass test.py as-is to python, and python will treat that as a path relative to the current working directory (CWD).
So, if test.py is in the CWD, this will just work.
If test.py is somewhere else, then you need to provide either an absolute path, or one relative to the CWD.
One common thing you want is that test.py is in not necessarily in the CWD, but instead it's in the same directory as the script/module that wants to launch it:
scriptpath = os.path.join(os.path.dirname(__file__), 'test.py')
… or in the same directory as the main script used to start your program:1
scriptpath = os.path.join(os.path.dirname(sys.argv[0]), 'test.py')
Either way, you just pass that as the argument:
p = subprocess.Popen(['python', scriptpath])
1. On some platforms, this may actually be a relative path. If you might have done an os.chdir since startup, it will now be wrong. If you need to handle that, you want to stash os.path.abspath(os.path.dirname(sys.argv[0])) in the main script at startup, then pass it down to other functions for them to use instead of calling dirname(argv[0]) themselves.
I have a a file structure like the following (Windows):
D:\
dir_1\
batch_1.bat
dir_1a\
batch_2.bat
dir_2\
main.py
For the sake of this question, batch_1.bat simply calls batch_2.bat, and looks like:
cd dir_1a
start batch_2.bat %*
Opening batch_1.bat from a command prompt indeed opens batch_2.bat as it's supposed to, and from there on, everything is golden.
Now I want my Python file, D:\dir_2\main.py, to spawn a new process which starts batch_1.bat, which in turn should start batch_2.bat. So I figured the following Python code should work:
import subprocess
subprocess.Popen(['cd "D:/dir_1"', "start batch_1.bat"], shell=True)
This results in "The system cannot find the path specified" being printed to my Python console. (No error is raised, of course.) This is due to the first command. I get the same result even if I cut it down to:
subprocess.Popen(['cd "D:/"'], shell=True)
I also tried starting the batch file directly, like so:
subprocess.Popen("start D:/dir_1/batch_1.bat", shell=True)
For reasons that I don't entirely get, this seems to just open a windows command prompt, in dir_2.
If I forego the start part of this command, then my Python process is going to end up waiting for batch_1 to finish, which I don't want. But it does get a little further:
subprocess.Popen("D:/dir_1/batch_1.bat", shell=True)
This results in batch_1.bat successfully executing... in dir_2, the directory of the Python script, rather than the directory of batch_1.bat, which results in it not being able to find dir_1a\ and hence, batch_2.bat is not executed at all.
I am left highly confused. What am I doing wrong, and what should I be doing instead?
Your question is answered here: Python specify popen working directory via argument
In a nutshell, just pass an optional cwd argument to Popen:
subprocess.Popen(["batch_1.bat"], shell=True, cwd=r'd:\<your path>\dir1')
Idea
Basically, what my script does is checking C:/SOURCE for .txt files and add a timestamp to it. To replicate it you can basically make that folder and put some txt files in there. Then, it's supposed to run a .vbs file, which then runs a .bat files with some rclone commands which don't matter here. I did it like this because there wont be a CMD window opening when running the rclone command through the .vbs file.
Python code
import time, os, subprocess
while True:
print("Beginning checkup")
print("=================")
timestamp = time.strftime('%d_%m_%H_%M') # only underscores: no naming issues
the_dir = "C:/SOURCE"
for fname in os.listdir(the_dir):
if fname.lower().endswith(".txt"):
print("found " + fname)
time.sleep(0.1)
new_name = "{}-{}.txt".format(os.path.splitext(fname)[0], timestamp)
os.rename(os.path.join(the_dir, fname), os.path.join(the_dir, new_name))
time.sleep(0.5)
else:
subprocess.call(['cscript.exe', "copy.vbs"])
time.sleep(60)
VBScript code
Set WshShell = CreateObject("WScript.Shell" )
WshShell.Run Chr(34) & "copy.bat" & Chr(34), 0
Set WshShell = Nothing
The only important part for the Python script is below the very last else, where the subprocess.call() is supposed to run the .vbs file. What happens when running the script is it shows the first two lines that always come up when running CMD, but then nothing.
How could I fix that? I tried:
subprocess.call("cscript copy.vbs")
subprocess.call("cmd /c copy.vbs")
both with the same outcome, it doesn't do anything.
Anyone have an idea?
Why are you invoking a VBScript to invoke a batch script from Python? You should be able to simple run whatever the batch script is doing directly from your Python code. But even if you wanted to keep the batch script, something like this should do just fine without VBScript as an intermediary.
subprocess.call(['cmd', '/c', 'copy.bat'])
You may want to give the full path of the batch file, though, to avoid issues like the working directory not being what you think it is.
If your batch script resides in the same directory as the Python script, you can build the path with something like this:
import os
import subprocess
scriptdir = os.path.dirname(__file__)
batchfile = os.path.join(scriptdir, 'copy.bat')
subprocess.call(['cmd', '/c', os.path.realpath(batchfile)])
It seems there is no such an operation that could not be done using plain Python. Scan a directory, copy a file -- Python has it all in the standard library. See os.path and shutil modules.
Adding VB scripts and launching subprocesses make your code complex and difficult to debug.
In a linux terminal typing
python script.py
Will run run script.py and exit the python console, but what if I just want to run a part of the script and leave the console open? For example, run script.py until line 15 and leave the console open for further scripting. How would I do this?
Let's say it's possible, then with the console still open and script.py run until line 15, can I then from inside the console call line fragments from other py files?
...something like
python script.py 15 #(opens script and runs lines 1-15 and leaves console open)
Then having the console open, I would like to run lines 25-42 from anotherscript.py
>15 lines of python code run from script.py
> run('anotherscript.py', lines = 25-42)
> print "I'm so happy the console is still open so I can script some more")
I'm so happy the console is still open so I can script some more
>
Your best bet might be pdb, the Python debugger. You can start you script under pdb, set a breakpoint on line 15, and then run your script.
python -m pdb script.py
b 15 # <-- Set breakpoint on line 15
c # "continue" -> run your program
# will break on line 15
You can then inspect your variables and call functions. Since Python 3.2, you can also use the interact command inside pdb to get a regular Python shell at the current execution point!
If that fits your bill and you also like IPython, you can check out IPdb, which is a bit nicer than normal pdb, and drops you into an IPython shell with interact.
if you want to run script.py from line a to line b, simply use this bash snippet:
cat script.py|head -{a+b}|tail -{b-a}|python -i
replace {a+b} and {b-a} with their values
You could use the python -i option to leave the console open at the end of the script.
It lets your script run until it exits, and you can then examine variables, call any function and any Python code, including importing and using other modules.
Of course your script needs to exit first, either at the end or, if your goal is to debug that part of the script, you could add a sys.exit() or os._exit() call where you want it to stop (such as your line 15).
For instance:
import os
print "Script starting"
a=1
def f(x):
return x
print "Exiting on line 8"
os._exit(0) # to avoid the standard SystemExit exception
print "Code continuing"
Usage example:
python -i test_exit.py
Scrit starting
Exiting on line 8
>>> print a
1
>>> f(4)
4
>>>
You cannot do that directly but you can do something similar from inside Python console (or IDLE) with exec :
just open you favorite Python console
load wanted lines into a string and exec them :
script = 'script.py'
txt = ''
with open(script) as sc:
for i, line in enumerate(sc):
if i >= begline and i<= endline:
txt = txt + line
exec(txt)
You can even write your own partial script runner based on that code ...
EDIT
I must admit that above answer alone really deserved to be downvoted. It is technically correct and probably the one that most closely meet what you asked for. But I should have warned you that it is bad pratice. Relying on line numbers to load pieces of source files is error prone and you should avoid it unless you really know what you are doing and why you do it that way. Python debugger at least allows you to control what are the lines you are about to execute.
If you really have to use this solution be sure to always print and double check the lines that you are about to execute. IMHO it is always both simpler and safer to copy and past lines in an IDE like IDLE that is packaged into any standard Python installation.