Develop Reference

Running grep through Python - doesn't work

I have some code like this:
f = open("words.txt", "w")
subprocess.call(["grep", p, "/usr/share/dict/words"], stdout=f)
f.close()
I want to grep the MacOs dictionary for a certain pattern and write the results to words.txt. For example, if I want to do something like grep '\<a.\>' /usr/share/dict/words, I'd run the above code with p = "'\<a.\>'". However, the subprocess call doesn't seem to work properly and words.txt remains empty. Any thoughts on why that is? Also, is there a way to apply regex to /usr/share/dict/words without calling a grep-subprocess?
edit:
When I run grep '\<a.\>' /usr/share/dict/words in my terminal, I get words like: aa
ad
ae
ah
ai
ak
al
am
an
ar
as
at
aw
ax
ay as results in the terminal (or a file if I redirect them there). This is what I expect words.txt to have after I run the subprocess call.

Like #woockashek already commented, you are not getting any results because there are no hits on '\<a.\>' in your input file. You are probably actually hoping to find hits for \<a.\> but then obviously you need to omit the single quotes, which are messing you up.
Of course, Python knows full well how to look for a regex in a file.
import re
rx = re.compile(r'\ba.\b')
with open('/usr/share/dict/words', 'Ur') as reader, open('words.txt', 'w') as writer:
for line in reader:
if rx.search(line):
print(line, file=writer, end='')
The single quotes here are part of Python's string syntax, just like the single quotes on the command line are part of the shell's syntax. In neither case are they part of the actual regular expression you are searching for.
The subprocess.Popen documentation vaguely alludes to the frequently overlooked fact that the shell's quoting is not necessary or useful when you don't have shell=True (which usually you should avoid anyway, for this and other reasons).
Python unfortunately doesn't support \< and \> as word boundary operators, so we have to use (the functionally equivalent) \b instead.

The standard input and output channels for the process started by call() are bound to the parent’s input and output. That means the calling programm cannot capture the output of the command. Use check_output() to capture the output for later processing:
import subprocess
f = open("words.txt", "w")
output = subprocess.check_output(['grep', p ,'-1'])
file.write(output)
print output
f.close()
PD: I hope it works, i cant check the answer because i have not MacOS to try it.

Making a Block of Text into a list

I am writing a Python script that enumerates all processes running on the computer. My current code does this but prints this out in a large block of text that is hard to read. How can I improve my script to have the output text in a vertical list for each process and all?
import subprocess
print(subprocess.check_output('set',shell=True)
*Edit: Here is the output text from the above script

set is an internal command that displays cmd.exe environment variables in your case.
To get environment variables in Python, use os.environ instead.
If you want to get the output of set command as a list of strings (not tested):
#!/usr/bin/env python3
import os
from subprocess import check_output
lines = check_output('cmd.exe /U /c set').decode('utf-16').split(os.linesep)

set should already print with newlines, so if they're not showing up, something is more wrong than you're telling us. You could always double up the newlines if you want to split the settings apart, e.g.:
import subprocess
print(subprocess.check_output('set',shell=True).replace('\n', '\n\n'))
If the problem is that you're running on Python 3 and the bytes object is a big blob, you can make subprocess decode it to a friendly printable string for you:
print(subprocess.check_output('set',shell=True, universal_newlines=True))
# Yes, the name of the keyword is dumb; it sounds like it handles different
# line ending conventions, but on Python 3, it also decodes from `bytes`
# to `str` for you.
For the general case of line wrapping nicely (though it does nothing for paragraphs of text that are just "too big"), you might want to look at the textwrap module; it splits a block of text up into a list of lines wrapped nicely at word boundaries so you don't have words split across lines.

Disclaimer: I have not done what you are doing before but this might work.
import subprocess
processes = subprocess.check_output('set',shell=True)
processes = processes.decode('UTF-8').split('\n') # convert bytes to unicodes and split
for process in processes:
print(process)

How to run a command line in Python

I am trying to execute
"C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe" -runscriptandexit "C:/Python27/simula_SIR_Phyton.py"
that is a to run a script in a program and I am not able to do it. I have succeed to run a single file like:
os.startfile("C:/Users/amrodri.UPVNET/Desktop/Scripts/SIR_europea_script.adsn")
But I have not succeed with the other problem. Can anyone help?
I have tried among others:
os.system("C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe" -runscriptandexit "C:/Python27/simula_SIR_Phyton.py")

os.system takes a single string as an argument. In order to have double quotes within a Python string (without terminating the string), you need to escape them using a backslash, like this:
os.system("\"C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe\" -runscriptandexit \"C:/Python27/simula_SIR_Phyton.py\"")
Or, alternatively, use single quotes instead:
os.system("'C:/Program Files/AnsysEM/AnsysEM15.0/Win64/Designer.exe' -runscriptandexit 'C:/Python27/simula_SIR_Phyton.py'")
See:
os.system()
Using quotes at the command line (This is Unix-specific, but should also apply to Windows if you're using something like PowerShell)

the culprit here is the space between Program and files. In windows, when you want to execute an address with an space in it, you need to put it between "", which is going to get mixed with Python's quotations. An easy solution would be to use raw '' in Python. For example:
import os
ansysedt_exe = r'"C:\Program Files\AnsysEM\AnsysEM16.0\Win64\ansysedt.exe" -runscriptandexit C:\automation\test_1.py'
print ansysedt_exe
os.system(ansysedt_exe)
Please notice that the designer address was put between "c:\...\designer.exe" because of the space in program files folder name, but we don't have to do the same for the script address, because there is no space there. Also just a heads up, in R16, designer.exe is going to be merged with AnsysEDT.exe.

Bash Variable Contains '\r' - Carriage Return Causing Problems in my Script

I have a bash script (rsync.sh) that works fine and has this line in it:
python /path/to/rsync_script.py $EMAIL "$RSYNC $PATH1 $PATH1_BACKUP"
I want to break the command (it's actually much longer than shown here because my variables have longer names) in two and use something like this:
python /path/to/rsync_script.py \
$EMAIL "$RSYNC $PATH1 $PATH1_BACKUP"
But when I do this I get the error:
scripts/rsync.sh: line 32: $'admin#mydomain.com\r': command not found
It puts the carriage return, \r in there.
How can I break this line up and not include the carriage return?

The problem looks like Windows line endings.
Here's how you can check in Python.
repr(open('rsync.sh', 'rb').read())
# If you see any \\r\\n, it's windows
Here's how you can fix it:
text = open('rsync.sh', 'r').read().replace('\r\n', '\n')
open('rsync.sh', 'wb').write(text)
Edit
Here's some code that shows the problem.
# Python:
open('abc-n.sh', 'wb').write('echo abc \\' + '\n' + 'def')
open('abc-r-n.sh', 'wb').write('echo abc \\' + '\r\n' + 'def')
And then run the files we made...
$ sh abc-n.sh
abc def
$ sh abc-r-n.sh
abc
abc-r-n.sh: 2: def: not found

If you can chnage the python script, maybe it will be easier to pass it the variable names thenselves, instead of their content.
From within the Python code you w=have better and more consistent tools to deal with whitespace characters (like \r) than from within bash.
To do that, just change your .sh line to
python /path/to/rsync_script.py EMAIL "RSYNC PATH1 PATH1_BACKUP"
And on your rsync_script.py, use os.environ to read the contents of the shell variables (and clear the \r's in them) - something like:
import os, sys
paths = []
for var_name in sys.argv(2).split(" "):
paths.append(os.environ[var_name].strip())

So I figured it out... I made a mistake in this question and I got so much awesome help but it was me doing a dumb thing that caused the problem. As I mentioned above, I may have copied and pasted from Windows at some point (I had forgotten since I did most of the edits in vim). I went back and wrote a short script with the essentials of the original in vim and then added in the '\' for line break and the script worked just fine. I feel bad accepting my own answer since it was so stupid. I made sure to up-vote everyone who helped me. Thanks again.

Python: calling Fortran with subprocess and giving commands via communicate

I want to call a Fortran program from python. I use the Popen statement from subprocess like this:
p = Popen(['./finput'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
I then want to send some file names to the fortran program. The fortran program reads them from stdin and then opens the files.
If I use something like:
p_stdout = p.communicate(input='file1.dat\nfile2.dat\n')[0]
everything is fine and the fortran program works as expected.
However I want to give the file names as a variable from within the python program.
So if I use
p_stdout = p.communicate(input=file1+'\n'+file2+'\n')[0]
my fortran program can not open the file names. The problem is that the string that fortran reads looks like this
f i l e 1 . d a t
with a blank character as a first character and some strange character inbetween every correct character. Unfortunately this only shows up if you print every character of the string individually. If you just print the file name with
print*,file1
you get
file1.dat
So my question is, why is python putting in these strange characters into the communication with the child process and, more important, how do I get rid of the?
many thanks

Sounds like your Fortran might be getting Unicode, are you using Python 3? If so, then construct the string to be passed then use string.encode()