Develop Reference

How to redirect python OS system call to a file?

I have no idea why the below code is not working. The file arch_list does not get created or anything written to it. The commands work fine when run in the terminal alone.
from yum.plugins import PluginYumExit , TYPE_CORE, TYPE_INTERACTIVE
import os
requires_api_version = '2.3'
plugin_type = (TYPE_CORE, TYPE_INTERACTIVE)
ip_vm = ['192.168.239.133']
def get_arch():
global ip_vm
os.system("uname -p > ~/arch_list")
for i in ip_vm:
cmd = "ssh thejdeep#"+i+" 'uname -p' >> ~/arch_list"
print cmd
os.system(cmd)
def init_hook(conduit):
conduit.info(2,'Hello World !')
get_arch()

I don't think os.system() will return to stdout in that case. You may try using subprocess.call() with the appropriate parameters.
Edit: Actually I think I remember seeing similar behaviour with ssh when running in a standard bash loop. You might try adding a -n to your ssh call.. I think that is the solution I used years ago in bash.
I just ran your code and it works fine for me, writing to the local arch file. I suspect adding more than one host to your list is where you start having problems. What version of python are you running? I'm on 2.7.6.

os.system() will not redirect stdout and stderr.
You can use subprocess modules Popen to set the stdout and stderr to a file descriptor or a pipe.
For example:
>>> import subprocess
>>> child1 = subprocess.Popen(["ls","-l"], stdout=subprocess.PIPE)
>>> print child1.stdout.readlines()
You can replace subprocess.PIPE to any valid file descriptor you opened for write. or you could pick up some lines to the file. It's your call.

Python IDLE script does not show output of subprocess but cmd.exe does

I'm using Python 2.7.6 and IDLE on Windows 7.
I have 2 Python scripts:
script.py:
import subprocess, os, sys
print("hello 1")
mypath = os.path.abspath(__file__)
mydir = os.path.dirname(mypath)
start = os.path.join(mydir, "script2.py")
subprocess.call([sys.executable, start, "param"])
print("bye 1")
and script2.py that is being called by the previous script:
import sys
print "hello 2"
print (sys.argv[1])
print "bye 2"
If I run script.py with cmd.exe shell I get the expected result:
C:\tests>python ./script.py
hello 1
hello 2
param
bye 2
bye 1
But if I open script.py with the IDLE editor and run it with F5 I get this result:
>>> ================================ RESTART ================================
>>>
hello 1
bye 1
>>>
Why is the sub script not writing to the IDLE Python shell?

You're running the subprocess without providing any stdout or stderr.
When run in a terminal, the subprocess will inherit your stdout and stderr, so anything it prints will show up intermingled with your output.
When run in IDLE, the subprocess will also inherit your stdout and stderr, but those don't go anywhere. IDLE intercepts the Python-level wrappers sys.stdout and sys.stderr,* so anything you print to them from within Python will end up in the GUI window, but anything that goes to real stdout or stderr—like the output of any subprocess you run that inherits your streams—just goes nowhere.**
The simplest fix is to capture the stdout and stderr from the subprocess and print them yourself. For example:
out = subprocess.check_output([sys.executable, start, "param"],
stderr=subprocess.STDOUT)
print out
* IDLE is more complicated than it looks. It's actually running separate processes for the GUI window and for running your code, communicating over a socket. The sys.stdout (and likewise for the others) that IDLE provides for your script isn't a file object, it's a custom file-like object that redirects each write to the GUI process via remote procedure call over the socket.
** Actually, if you launched IDLE from a terminal rather than by double-clicking its icon, the subprocess's output may end up there. I'm not sure how it works on Windows. But at any rate, that isn't helpful to you.

I verified that abamert's change works in 2.7, on Win7, with Idle started normally from the icon. The slight glitch is that 'print out' inserts an extra blank line. This is easily changed by making print a function with a future import and use of the end parameter.
from __future__ import print_function
...
print(out, end='')
With Python 3, there is an additional issue that 'out' is bytes instead of str, so that it prints as
b'hello 2\r\nparam\r\nbye 2\r\n'
Since your output is all ascii, this can be fixed by changing the print call to
print(out.decode(), end='')
The resulting program works identically in 2.7 and 3.x.

how can I silence all of the output from a particular python command?

Autodesk Maya 2012 provides "mayapy" - a modded build of python filled with the necessary packages to load Maya files and act as a headless 3D editor for batch work. I'm calling it from a bash script. If that script opens a scene file in it with cmds.file(filepath, open=True), it spews pages of warnings, errors, and other info I don't want. I want to turn all of that off only while the cmds.file command is running.
I've tried redirecting from inside of the Python commands I'm sending into mayapy inside the shell script, but that doesn't work. I can silence everything by redirecting stdout/err to /dev/null in the call to the bash script. Is there any way to silence it in the call to the shell, but still allow my passed-in command inside the script to print out information?
test.sh:
#!/bin/bash
/usr/autodesk/maya/bin/mayapy -c "
cmds.file('filepath', open=True);
print 'hello'
"
calling it:
$ ./test.sh # spews info, then prints 'hello'
$ ./test.sh > /dev/null 2>&1 # completely silent

Basically, I think the best way to solve this is to implement a wrapper that will execute test.sh and sanitize the output to the shell. To sanitize the output, I would simply prepend some string to notify your wrapper that this text is good for output. My inspiration for the wrapper file came from this: https://stackoverflow.com/a/4760274/2030274
The contents are as follows:
import subprocess
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
retcode = p.poll() #returns None while subprocess is running
line = p.stdout.readline()
yield line
if(retcode is not None):
break
for line in runProcess(['./test.sh']):
if line.startswith('GARYFIXLER:'):
print line,
Now you could imagine test.sh being something along the lines of
#!/bin/bash
/usr/autodesk/maya/bin/mayapy -c "
cmds.file('filepath', open=True);
print 'GARYFIXLER:hello'
"
and this will only print the hello line. Since we are wrapping the python call in a subprocess, all output typically displayed to the shell should get captured and you should intercept the lines that you don't want.
Of course, to call test.sh from a python script, you need to make sure you have the correct permissions.

I knew I was just getting twisted around with pipes. Maya is indeed sending all batch output to stderror. This frees stdout entirely once you properly pipe stderr away. Here's an all-bash one-liner that works.
# load file in batch; divert Maya's output to /dev/null
# then print listing of things in file with cmds.ls()
/usr/autodesk/maya/bin/mayapy -c "import maya.standalone;maya.standalone.initialize(name='python');cmds.file('mayafile.ma', open=True);print cmds.ls()" 2>/dev/null

Reading from flushed vs unflushed buffers

I've got a script parent.py trying to to read stdout from a subprocess sub.py in Python.
The parent parent.py:
#!/usr/bin/python
import subprocess
p = subprocess.Popen("sub.py", stdout=subprocess.PIPE)
print p.stdout.read(1)
And the subprocess, sub.py:
#!/usr/bin/python
print raw_input( "hello world!" )
I would expect running parent.py to print the 'h' from "hello world!". Actually, it hangs. I can only get my expected behaviour by adding -u to sub.py's she-bang line.
This confuses me because the -u switch makes no difference when sub.py is run directly from a shell; the shell is somehow privy to the un-flushed output stream, unlike parent.py.
My goal is to run a C program as the subprocess, so I won't be able to control whether or not it flushes stdout. How is it that a shell has better access to a process's stdout than Python running the same thing from subprocess.Popen? Am I going to be able to read such a stdout stream from a C program that doesn't flush its buffers?
EDIT:
Here is an updated example based on korylprince's comment...
## capitalize.sh ##
#!/bin/sh
while [ 1 ]; do
read s
echo $s | tr '[:lower:]' '[:upper:]'
done
########################################
## parent.py ##
#!/usr/bin/python
from subprocess import Popen, PIPE
# cmd = [ 'capitalize.sh' ] # This would work
cmd = [ 'script', '-q', '-f', '-c', 'capitalize.sh', '/dev/null']
p = Popen(cmd, stdin=PIPE)
p.stdin.write("some string\n")
p.wait()
When running through script, I get steady printing of newlines (and if this were a Python, subprocess, it'd raise an EOFerror).

An alternative is
p = subprocess.Popen(["python", "-u", "sub.py"], stdout=subprocess.PIPE)
or the suggestions here.
My experience is that yes, you will be able to read from most C programs without any extra effort.
The Python interpreter takes extra steps to buffer its output which is why it needs the -u switch to disable output buffering. Your typical C program won't do this.
I haven't run into any program (C or otherwise) other than the Python interpreter that I expected to work and didn't within a subshell.

The reason the shell can read output immediately, regardless of "-u" is because the program you're launching from the shell has its output connected to a TTY. When the stdout is connected to a TTY, it is unbuffered (because it is up to the TTY to buffer). When you launch the python subprocess from within python, you're connecting stdout to a pipe, which means you're at the mercy of the subprocess to flush its output when it feels like it.
If you're looking to do complicated interactions with a subprocess, look into this tutorial.

How to execute a command prompt command from python

I tried something like this, but with no effect:
command = "cmd.exe"
proc = subprocess.Popen(command, stdin = subprocess.PIPE, stdout = subprocess.PIPE)
proc.stdin.write("dir c:\\")

how about simply:
import os
os.system('dir c:\\')

You probably want to try something like this:
command = "cmd.exe /C dir C:\\"
I don't think you can pipe into cmd.exe... If you are coming from a unix background, well, cmd.exe has some ugly warts!
EDIT: According to Sven Marnach, you can pipe to cmd.exe. I tried following in a python shell:
>>> import subprocess
>>> proc = subprocess.Popen('cmd.exe', stdin = subprocess.PIPE, stdout = subprocess.PIPE)
>>> stdout, stderr = proc.communicate('dir c:\\')
>>> stdout
'Microsoft Windows [Version 6.1.7600]\r\nCopyright (c) 2009 Microsoft Corporatio
n. All rights reserved.\r\n\r\nC:\\Python25>More? '
As you can see, you still have a bit of work to do (only the first line is returned), but you might be able to get this to work...

Try:
import os
os.popen("Your command here")

Using ' and " at the same time works great for me (Windows 10, python 3)
import os
os.system('"some cmd command here"')
for example to open my web browser I can use this:
os.system(r'"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"')
(Edit)
for an easier way to open your browser I can use this:
import webbrowser
webbrowser.open('website or leave it alone if you only want to open the
browser')

Try adding a call to proc.stdin.flush() after writing to the pipe and see if things start behaving more as you expect. Explicitly flushing the pipe means you don't need to worry about exactly how the buffering is set up.
Also, don't forget to include a "\n" at the end of your command or your child shell will sit there at the prompt waiting for completion of the command entry.
I wrote about using Popen to manipulate an external shell instance in more detail at: Running three commands in the same process with Python
As was the case in that question, this trick can be valuable if you need to maintain shell state across multiple out-of-process invocations on a Windows machine.

Taking some inspiration from Daren Thomas's answer (and edit), try this:
proc = subprocess.Popen('dir C:\\', shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = proc.communicate()
out will now contain the text output.
They key nugget here is that the subprocess module already provides you shell integration with shell=True, so you don't need to call cmd.exe directly.
As a reminder, if you're in Python 3, this is going to be bytes, so you may want to do out.decode() to convert to a string.

Why do you want to call cmd.exe ? cmd.exe is a command line (shell). If you want to change directory, use os.chdir("C:\\"). Try not to call external commands if Python can provide it. In fact, most operating system commands are provide through the os module (and sys). I suggest you take a look at os module documentation to see the various methods available.

It's very simple. You need just two lines of code with just using the built-in function and also it takes the input and runs forever until you stop it. Also that 'cmd' in quotes, leave it and don't change it. Here is the code:
import os
os.system('cmd')
Now just run this code and see the whole windows command prompt in your python project!

Here's a way to just execute a command line command and get its output using the subprocess module:
import subprocess
# You can put the parts of your command in the list below or just use a string directly.
command_to_execute = ["echo", "Test"]
run = subprocess.run(command_to_execute, capture_output=True)
print(run.stdout) # the output "Test"
print(run.stderr) # the error part of the output
Just don't forget the capture_output=True argument and you're fine. Also, you will get the output as a binary string (b"something" in Python), but you can easily convert it using run.stdout.decode().

In Python, you can use CMD commands using these lines :
import os
os.system("YOUR_COMMAND_HERE")
Just replace YOUR_COMMAND_HERE with the command you like.

From Python you can do directly using below code
import subprocess
proc = subprocess.check_output('C:\Windows\System32\cmd.exe /k %windir%\System32\\reg.exe ADD HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System /v EnableLUA /t REG_DWORD /d 0 /f' ,stderr=subprocess.STDOUT,shell=True)
print(str(proc))
in first parameter just executed User Account setting you may customize with yours.