Develop Reference

Execute batch file in different directory

I have a a file structure like the following (Windows):
D:\
dir_1\
batch_1.bat
dir_1a\
batch_2.bat
dir_2\
main.py
For the sake of this question, batch_1.bat simply calls batch_2.bat, and looks like:
cd dir_1a
start batch_2.bat %*
Opening batch_1.bat from a command prompt indeed opens batch_2.bat as it's supposed to, and from there on, everything is golden.
Now I want my Python file, D:\dir_2\main.py, to spawn a new process which starts batch_1.bat, which in turn should start batch_2.bat. So I figured the following Python code should work:
import subprocess
subprocess.Popen(['cd "D:/dir_1"', "start batch_1.bat"], shell=True)
This results in "The system cannot find the path specified" being printed to my Python console. (No error is raised, of course.) This is due to the first command. I get the same result even if I cut it down to:
subprocess.Popen(['cd "D:/"'], shell=True)
I also tried starting the batch file directly, like so:
subprocess.Popen("start D:/dir_1/batch_1.bat", shell=True)
For reasons that I don't entirely get, this seems to just open a windows command prompt, in dir_2.
If I forego the start part of this command, then my Python process is going to end up waiting for batch_1 to finish, which I don't want. But it does get a little further:
subprocess.Popen("D:/dir_1/batch_1.bat", shell=True)
This results in batch_1.bat successfully executing... in dir_2, the directory of the Python script, rather than the directory of batch_1.bat, which results in it not being able to find dir_1a\ and hence, batch_2.bat is not executed at all.
I am left highly confused. What am I doing wrong, and what should I be doing instead?

Your question is answered here: Python specify popen working directory via argument
In a nutshell, just pass an optional cwd argument to Popen:
subprocess.Popen(["batch_1.bat"], shell=True, cwd=r'd:\<your path>\dir1')

the system was unable to find the specified registry key

I run this dos command:
reg query HKLM\SOFTWARE\Classes\CLSID\{824293FD-32E2-4DAA-BC28-166C140543BE}\InprocServer32 /ve
and successfullly get a value.
In the same terminal, when I use python as below:
cmd = "reg query HKLM\\SOFTWARE\\Classes\\CLSID\\{824293FD-32E2-4DAA-BC28-166C140543BE}\\InprocServer32 /ve"
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
output = proc.communicate()[0]
it fails with this error:
ERROR: The system was unable to find the specified registry key or value.
I'm using the same terminal which is opened as admin.
I even tried puting this in a x.bat file and called x.bat in a python subprocess...still the same error.
The reg key has read permission for all users and full control for admin.
This is happening on windows server 2008R2 with python 2.7.
Also I've had no luck with _winreg so any ideas how to solve this problem by running the batch command within python?
To make it clear: it works in a dos terminal, but fails in python!
Moreover, I tried another key:
reg query HKLM\SOFTWARE\Classes\XML
and this one works. One thing I've noticed is that the key that has this problem has Read permission checked and disabled while this latter key has read permission checked and enabled (can be changed).

I realized that I've seen this problem before...
solution: disabling wow64 redirect
change reg to %Windir%/sysnative/reg.exe
cmd = "%Windir%/sysnative/reg.exe query HKLM\\SOFTWARE\\Classes\\CLSID\\{824293FD-32E2-4DAA-BC28-166C140543BE}\\InprocServer32 /ve"
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
output = proc.communicate()[0]
what's more, I managed to do it with _winreg:
import _winreg
key = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, r'SOFTWARE\Classes\CLSID\{824293FD-32E2-4DAA-BC28-166C140543BE}\InprocServer32', 0, _winreg.KEY_WOW64_64KEY + _winreg.KEY_ALL_ACCESS)
print _winreg.QueryValueEx(key, "")

Django Python Script Doesn't Output Files

I have a script that executes the following code through a Django view:
# generate dag file
try:
commandString = [
'python',
os.path.join('/srv/nfsshare/transcode50', userFolder, directory, 'condor_execute.py')
]
subprocess.check_call(commandString,
stdout=open('/srv/nfsshare/transcode50/output2.txt', 'w'),
stderr=subprocess.STDOUT)
except Exception, e:
open('/tmp/test_exception.txt', 'w').write(str(e))
raise
What that is supposed to do is execute a Python file I have generated. If I run that exact command from the server's command prompt (i.e., python /srv/nfsshare/transcode50/cogden/algorithms/condor_execute.py), it works just fine and produces the needed files it's supposed to within that directory. However, if I run it from Django, it produces no error messages, produces the correct console output (a message that basically says "Created file blah at blah"), but produces no files whatsoever, when it normally generates two. I'm getting no permissions errors or anything and have ensured the directory is chmodded appropriately.

I suspect your process is never getting around to closing the file (as you can't do it explicitly as you have no reference to it - while run as a script - the interpreter would have stopped - but Django works different with loaded processes etc...) and it's not big enough to be flushed - try moving out your open.
with open('/srv/nfsshare/transcode50/output2.txt', 'w') as stdout:
subprocess.check_call(commandString, stdout=stdout, stderr=subprocess.STDOUT)
Also, make sure that when you make changes to make sure the Django server is restarted/similar to make sure code is reloaded and changes are taken into effect.

Persistent WinSCP connection for batch copy in Python

I'm trying to copy thousands files to a remote server. These files are generated in real-time within the script. I'm working on a Windows system and need to copy the files to a Linux server (hence the escaping).
I currently have:
import os
os.system("winscp.exe /console /command \"option batch on\" \"option confirm off\" \"open user:pass#host\" \"put f1.txt /remote/dest/\"")
I'm using Python to generate the files but need a way to persist the remote connection so that I can copy each file, to the server, as it is generated (as opposed to creating a new connection each time). That way, I'll only need to change the field in the put option thus:
"put f2 /remote/dest"
"put f3 /remote/dest"
etc.

I needed to do this and found that code similar to this worked well:
from subprocess import Popen, PIPE
WINSCP = r'c:\<path to>\winscp.com'
class UploadFailed(Exception):
pass
def upload_files(host, user, passwd, files):
cmds = ['option batch abort', 'option confirm off']
cmds.append('open sftp://{user}:{passwd}#{host}/'.format(host=host, user=user, passwd=passwd))
cmds.append('put {} ./'.format(' '.join(files)))
cmds.append('exit\n')
with Popen(WINSCP, stdin=PIPE, stdout=PIPE, stderr=PIPE,
universal_newlines=True) as winscp: #might need shell = True here
stdout, stderr = winscp.communicate('\n'.join(cmds))
if winscp.returncode:
# WinSCP returns 0 for success, so upload failed
raise UploadFailed
This is simplified (and using Python 3), but you get the idea.

Instead of using an external program (winscp) you could also use an python ssh-library like pyssh.

You would have to start persistent WinSCP sub-process in Python and feed the put commands to its standard input continuously.
I do not have Python example for this, but there's an equivalent JScript example:
https://winscp.net/eng/docs/guide_automation_advanced#inout
or C# example:
https://winscp.net/eng/docs/guide_dotnet#input
Though using WinSCP .NET assembly via its COM interface for Python would be a way easier:
https://winscp.net/eng/docs/library

Runing bcdedit from python in Windows 2008 SP2

I do not know windows well, so that may explain my dilemma ...
I am trying to run bcdedit in Windows 2008R2 from Python 2.6.
My Python routine to run a command looks like this:
def run_program(cmd_str):
"""Run the specified command, returning its output as an array of lines"""
dprint("run_program(%s): entering" % cmd_str)
cmd_args = cmd_str.split()
subproc = subprocess.Popen(cmd_args, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
(outf, errf) = (subproc.stdout, subproc.stderr)
olines = outf.readlines()
elines = errf.readlines()
if Options.debug:
if elines:
dprint('Error output:')
for line in elines:
dprint(line.rstrip())
if olines:
dprint('Normal output:')
for line in olines:
dprint(line.rstrip())
errf.close()
outf.close()
res = subproc.wait()
dprint('wait result=', res)
return (res, olines)
I call this function thusly:
(res, o) = run_program('bcdedit /set {current} MSI forcedisable')
This command works when I type it from a cmd window, and it works when I put it in a batch file and run it from a command window (as Administrator, of course).
But when I run it from Python (as Administrator), Python claims it can't find the command, returning:
bcdedit is not recognized as an internal or external command,
operable program or batch file
Also, if I trying running my batch file from Python (which works from the command line), it also fails. I've also tried it with the full path to bcdedit, with the same results.
What is it about calling bcdedit from Python that makes it not found?
Note that I can call other EXE files from Python, so I have some level of confidence that my Python code is sane ... but who knows.
Any help would be most appreciated.

Windows 2008 R2 is 64-bit-only, yes? Python's a 32-bit process. When a 32-bit app runs something from C:\Windows\System32, Windows actually looks in C:\Windows\SysWOW64. Use C:\Windows\SysNative.

Perhaps the path to bcdedit.exe isn't in your system path when Python is running for some reason (a different user account, for example). You can find this out by printing:
os.environ.get("PATH")
It's semicolon-delimited, so os.environ.get("PATH").split(';') might be more useful.
I can't see any reason why it wouldn't be there, but just in case, you should be looking for C:\Windows\System32, where C is your Windows drive letter.

Check your PATH variable and see if C:\windows\system32 is there. (use set in DOS)

For some reason I experiment the same trouble from c#. If I list the files it was not here, but when I was looking from Explorer it was there. maybe it is some kind of protected file. To call bcdedit.exe, I manually copied it from system32 to my application folder and it worked. There is also another one in windows\winsxs folder. I can start it from my application, but I`m not sure it is the same path on all computers.
Hope it helps!