Develop Reference

Running and piping text into a .exe file from Python

I am trying to a run a .exe file from python and pipe a string into it. The .exe itself opens a command box and requires a series of string inputs that can be entered in one go on a series of lines (as below)
In bash the solution would be:
printf "test.dat\nMoreinput\nMoreinput" | ~/Desktop/Median_filt_exes/ascxyz.exe
To recreate this in python I have tried:
from subprocess import Popen, PIPE
p = Popen(r"./ascxyz.exe", stdin=PIPE,text=True)
p.communicate("test.dat\nMoreinput/nMoreinput")
There's no error however it doesn't seem to be working (the .exe should create a new file when run successfully). Any help into what I could do to figure out why the exe isnt running properly would be very appreciated!

The immediate problem is probably that you are not terminating the input with a newline. But you really also don't want to do the Popen plumbing yourself.
from subprocess import run
run(['./ascxyz.exe'], text=True,
input="test.dat\nMoreInput\nMoreInput\n")
Notice also how we pass in a list as the first argument, to avoid the complications of shell=True.

python3 for win and cygwin - line endings in buffer

Setup: python3.6 for windows in Cygwin (have to use Win version because of functionalities introduced in 3.5 and Cygwin is stuck at 3.4)
How to get \n new lines in buffer (stdout) output from a python script (instead of \r\n)? The output is a list of paths and I want to get one per line for further processing by other Cygwin/Windows tools.
All answers I've found so far are dealing with file writing and I just want to modify what is written to stdout. So far the only sure way to get rid of \r is piping results through sed 's/\\10//' which is awkward.
Weird thing is that even Windows applications fed with script output don't accept it with messages like:
Can't find file <asdf.txt
>
(note newline before >)
Supposedly sys.stdout.write is doing pure output but when doing:
sys.stdout.write(line)
I get a list of paths without any separation. If I introduce anything which resembles NL (\n, \012, etc.) it is automatically converted to CRLF (\r\n). How to stop this conversion?

You need to write to stdout in binary mode; the default is text mode, which translates everything you write.
According to Issue4571 you can do this by writing directly to the internal buffer used by stdout.
sys.stdout.buffer.write(line)
Note that if you're writing Unicode strings you'll need to encode them to byte strings first.
sys.stdout.buffer.write(line.encode('utf-8')) # or 'mbcs'

Making a Block of Text into a list

I am writing a Python script that enumerates all processes running on the computer. My current code does this but prints this out in a large block of text that is hard to read. How can I improve my script to have the output text in a vertical list for each process and all?
import subprocess
print(subprocess.check_output('set',shell=True)
*Edit: Here is the output text from the above script

set is an internal command that displays cmd.exe environment variables in your case.
To get environment variables in Python, use os.environ instead.
If you want to get the output of set command as a list of strings (not tested):
#!/usr/bin/env python3
import os
from subprocess import check_output
lines = check_output('cmd.exe /U /c set').decode('utf-16').split(os.linesep)

set should already print with newlines, so if they're not showing up, something is more wrong than you're telling us. You could always double up the newlines if you want to split the settings apart, e.g.:
import subprocess
print(subprocess.check_output('set',shell=True).replace('\n', '\n\n'))
If the problem is that you're running on Python 3 and the bytes object is a big blob, you can make subprocess decode it to a friendly printable string for you:
print(subprocess.check_output('set',shell=True, universal_newlines=True))
# Yes, the name of the keyword is dumb; it sounds like it handles different
# line ending conventions, but on Python 3, it also decodes from `bytes`
# to `str` for you.
For the general case of line wrapping nicely (though it does nothing for paragraphs of text that are just "too big"), you might want to look at the textwrap module; it splits a block of text up into a list of lines wrapped nicely at word boundaries so you don't have words split across lines.

Disclaimer: I have not done what you are doing before but this might work.
import subprocess
processes = subprocess.check_output('set',shell=True)
processes = processes.decode('UTF-8').split('\n') # convert bytes to unicodes and split
for process in processes:
print(process)

How to run a command line in Python

I am trying to execute
"C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe" -runscriptandexit "C:/Python27/simula_SIR_Phyton.py"
that is a to run a script in a program and I am not able to do it. I have succeed to run a single file like:
os.startfile("C:/Users/amrodri.UPVNET/Desktop/Scripts/SIR_europea_script.adsn")
But I have not succeed with the other problem. Can anyone help?
I have tried among others:
os.system("C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe" -runscriptandexit "C:/Python27/simula_SIR_Phyton.py")

os.system takes a single string as an argument. In order to have double quotes within a Python string (without terminating the string), you need to escape them using a backslash, like this:
os.system("\"C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe\" -runscriptandexit \"C:/Python27/simula_SIR_Phyton.py\"")
Or, alternatively, use single quotes instead:
os.system("'C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe' -runscriptandexit 'C:/Python27/simula_SIR_Phyton.py'")
See:
os.system()
Using quotes at the command line (This is Unix-specific, but should also apply to Windows if you're using something like PowerShell)

the culprit here is the space between Program and files. In windows, when you want to execute an address with an space in it, you need to put it between "", which is going to get mixed with Python's quotations. An easy solution would be to use raw '' in Python. For example:
import os
ansysedt_exe = r'"C:\Program Files\AnsysEM\AnsysEM16.0\Win64\ansysedt.exe" -runscriptandexit C:\automation\test_1.py'
print ansysedt_exe
os.system(ansysedt_exe)
Please notice that the designer address was put between "c:\...\designer.exe" because of the space in program files folder name, but we don't have to do the same for the script address, because there is no space there. Also just a heads up, in R16, designer.exe is going to be merged with AnsysEDT.exe.

Using textfile as stdin in python under windows 7

I'm a win7-user.
I accidentally read about redirections (like command1 < infile > outfile) in *nix systems, and then I discovered that something similar can be done in Windows (link). And python is also can do something like this with pipes(?) or stdin/stdout(?).
I do not understand how this happens in Windows, so I have a question.
I use some kind of proprietary windows-program (.exe). This program is able to append data to a file.
For simplicity, let's assume that it is the equivalent of something like
while True:
f = open('textfile.txt','a')
f.write(repr(ctime()) + '\n')
f.close()
sleep(100)
The question:
Can I use this file (textfile.txt) as stdin?
I mean that the script (while it runs) should always (not once) handle all new data, ie
In the "never-ending cycle":
The program (.exe) writes something.
Python script captures the data and processes.
Could you please write how to do this in python, or maybe in win cmd/.bat or somehow else.
This is insanely cool thing. I want to learn how to do it! :D

If I am reading your question correctly then you want to pipe output from one command to another.
This is normally done as such:
cmd1 | cmd2
However, you say that your program only writes to files. I would double check the documentation to see if their isn't a way to get the command to write to stdout instead of a file.
If this is not possible then you can create what is known as a named pipe. It appears as a file on your filesystem, but is really just a buffer of data that can be written to and read from (the data is a stream and can only be read once). Meaning your program reading it will not finish until the program writing to the pipe stops writing and closes the "file". I don't have experience with named pipes on windows so you'll need to ask a new question for that. One down side of pipes is that they have a limited buffer size. So if there isn't a program reading data from the pipe then once the buffer is full the writing program won't be able to continue and just wait indefinitely until a program starts reading from the pipe.
An alternative is that on Unix there is a program called tail which can be set up to continuously monitor a file for changes and output any data as it is appended to the file (with a short delay.
tail --follow=textfile.txt --retry | mycmd
# wait for data to be appended to the file and output new data to mycmd
cmd1 >> textfile.txt # append output to file
One thing to note about this is that tail won't stop just because the first command has stopped writing to the file. tail will continue to listen to changes on that file forever or until mycmd stops listening to tail, or until tail is killed (or "sigint-ed").
This question has various answers on how to get a version of tail onto a windows machine.

import sys
sys.stdin = open('textfile.txt', 'r')
for line in sys.stdin:
process(line)

If the program writes to textfile.txt, you can't change that to redirect to stdin of your Python script unless you recompile the program to do so.
If you were to edit the program, you'd need to make it write to stdout, rather than a file on the filesystem. That way you can use the redirection operators to feed it into your Python script (in your case the | operator).
Assuming you can't do that, you could write a program that polls for changes on the text file, and consumes only the newly written data, by keeping track of how much it read the last time it was updated.

When you use < to direct the output of a file to a python script, that script receives that data on it's stdin stream.
Simply read from sys.stdin to get that data:
import sys
for line in sys.stdin:
# do something with line