Develop Reference

Launching Python script from zsh and staying in Python shell

I would like to simplify running Python scripts from within the Python shell. In Python 2, you could just use execfile(path). But in Python 3 it's harder to remember:
exec(open(path).read())
So I want a function to run a script, as simple as run(path). I can do this from the Python shell:
def run(filename):
source = open(filename).read()
code = compile(source, filename, 'exec')
exec(code)
Then I can just type in run(path). This works great, and now I want to simplify things by defining the run function every time I launch Python 3.
I'd like to configure my ~/.zshenv with a zsh alias or function (say, py) that launches Python and tells it to define the run function. So that's where I'm stumped. What would a such a zsh command look like? I've tried and failed with things like:
py () {
python -c "\
def run(filename): \
source = open(filename).read() \
code = compile(source, filename, 'exec') \
exec(code)" \
}
But that fails miserably:
% py
File "<string>", line 1
def run(filename): source = open(filename).read() code = compile(source, filename, 'exec') exec(code)
IndentationError: unexpected indent
%
And even if it were to work, it would drop back out of the Python shell once the function was defined. Obviously I don't know what I'm doing here. Any pointers?
Also… please don't assume I have asked the right question. Usually on StackOverflow we try to avoid second-guessing posters' assumptions. But go ahead and second-guess mine if there's a better way to get Python to always define a run function when it is launched.

If you need this function only for interactive shells, you can write it in a file and then run python -i file_with_function.py. The -i option will tell the interpreter to drop into an interactive session after whatever is in the file_with_function.py file runs.
If you want it for any .py file that you will run non-interactively then you can do one of the following:
Create a package that contains your run function and install your package on your interpreter. There is a detailed guide on the Python docs (https://packaging.python.org/tutorials/packaging-projects/).
Add the directory that contains a .py file with your function on the PYTHONPATH environmental variable and import it from there.

In the command which you are passing to Python (using python -c), you start the function definition with a couple of spaces. Spaces at the start of a line are significant in Python. You would get the same error, if you would open a Python shell and write
def foo:
with several spaces in front: Python responds with IndentationError: unexpected indent.
In addition, your use of backslash characters makes all the linefeeds disappear, with the effect that you are going to define the complete function in a single line. This is also invalid in Python, so even if you would fix the initial spaces, you would still get SyntaxError: invalid syntax.
Note that you can use the -m option of Python to load initial definition together with starting Python. You can do a
python -h
to get a list of the valid command line options.

Running python script from perl, with argument to stdin and saving stdout output

My perl script is at path:
a/perl/perlScript.pl
my python script is at path:
a/python/pythonScript.py
pythonScript.py gets an argument from stdin, and returns result to stdout. From perlScript.pl , I want to run pythonScript.py with the argument hi to stdin, and save the results in some variable. That's what I tried:
my $ret = `../python/pythonScript.py < hi`;
but I got the following error:
The system cannot find the path specified.
Can you explain the path can't be found?

The qx operator (backticks) starts a shell (sh), in which prog < input syntax expects a file named input from which it will read lines and feed them to the program prog. But you want the python script to receive on its STDIN the string hi instead, not lines of a file named hi.
One way is to directly do that, my $ret = qx(echo "hi" | python_script).
But I'd suggest to consider using modules for this. Here is a simple example with IPC::Run3
use warnings;
use strict;
use feature 'say';
use IPC::Run3;
my #cmd = ('program', 'arg1', 'arg2');
my $in = "hi";
run3 \#cmd, \$in, \my $out;
say "script's stdout: $out";
The program is the path to your script if it is executable, or perhaps python script.py. This will be run by system so the output is obtained once that completes, what is consistent with the attempt in the question. See documentation for module's operation.
This module is intended to be simple while "satisfy 99% of the need for using system, qx, and open3 [...]. For far more power and control see IPC::Run.

You're getting this error because you're using shell redirection instead of just passing an argument
../python/pythonScript.py < hi
tells your shell to read input from a file called hi in the current directory, rather than using it as an argument. What you mean to do is
my $ret = `../python/pythonScript.py hi`;
Which correctly executes your python script with the hi argument, and returns the result to the variable $ret.

The Some of the other answers assume that hi must be passed as a command line parameter to the Python script but the asker says it comes from stdin.
Thus:
my $ret = `echo "hi" | ../python/pythonScript.py`;

To launch your external script you can do
system "python ../python/pythonScript.py hi";
and then in your python script
import sys
def yourFct(a, b):
...
if __name__== "__main__":
yourFct(sys.argv[1])
you can have more informations on the python part here

Python not getting raw binary from subprocess.check_call

How can I get subprocess.check_call to give me the raw binary output of a command, it seems to be encoding it incorrectly somewhere.
Details:
I have a command that returns text like this:
some output text “quote” ...
(Those quotes are unicode e2809d)
Here's how I'm calling the command:
f_output = SpooledTemporaryFile()
subprocess.check_call(cmd, shell=True, stdout=f_output)
f_output.seek(0)
output = f_output.read()
The problem is I get this:
>>> repr(output)
some output text ?quote? ...
>>> type(output)
<str>
(And if I call 'ord' the '?' I get 63.)
I'm on Python 2.7 on Linux.
Note: Running the same code on OSX works correctly to me. The problem is when I run it on a Linux server.

Wow, this was the weirdest issue ever but I've fixed it!
It turns out that the program it was calling (a java program) was returning different encoding depending on where it was called from!
Dev osx machine, returns the characters fine, Linux server from command line, returns them fine, called from a Django app, nope turns into "?"s.
To fix this I ended up adding this argument to the command:
-Dfile.encoding=utf-8
I got that idea here, and it seems to work. There's also a way to modify the Java program internally to do that.
Sorry I blamed Python! You guys had the right idea.

The redirection (stdout=file) happens at the file descriptor level. Python has nothing to do with what is written to the file if you see ? instead of “ in the file itself (not in a REPL).
If it work on OS X and it "doesn't work" on Linux server then the likely reason is the difference in the environment, check LC_ALL, LC_CTYPE, LANG envvars—python, /bin/sh (due to shell=True), and the cmd may use your locale encoding that is ASCII if the environment is not set (C, POSIX locale).
To get "raw binary" from a subprocess:
#!/usr/bin/env python
import subprocess
raw_binary = subprocess.check_output(['cmd', 'arg 1', 'arg 2'])
print(repr(raw_binary))
Note:
no shell=True—don't use it unless it is necessary
many programs may change their behavior if they detect that the output is not a tty, example.

Python equivalent/emulator of Bash "parameter expansion"

I have a bash script that I use to update several computers in my house. It makes use of the deborphan program which identifies programs that are no longer required on my system (Linux obviously).
The bash script makes use of bash's parameter expansion which enables me to pass deborphan's results to my package manager (in this case aptitude):
aptitude purge $(deborphan --guess-all) -y
deborphan's results are:
python-pip
python3-all
I would like to convert my bash script into python (partly as a learning opportunity, as I am new to python), but I have run into a significant snag. My obvious start for the python script is
subprocess.call(["aptitude", "purge", <how do I put the deborphan results here?>, "-y"])
I have tried a separate subprocess.call for a parameter inside the above subprocess.call just for deborphan and that fails.
Interestingly enough I cannot seem to capture the deborphan results with:
deb = subprocess.call(["deborphan", "--guess-all"])
to pass deborphan's results as a variable for the parameter either.
Is there anyway to emulate Bash's parameter expansion in python?

You can use + to concatenate lists:
import subprocess as sp
deborphan_results = sp.check_output(…)
deborphan_results = deborphan_results.splitlines()
subprocess.call(["aptitude", "purge"] + deborphan_results + ["-y"])
(if you're using a Python version below 2.7, you can use proc = sp.Popen(…, stdout=sp.PIPE); deborphan_results, _ = proc.communicate())

Write to a FIFO from a Python program

I a trying to control the volume of mplayer from a python program. The mplayer program gets started from a bash script:
#!/bin/bash
mkfifo /home/administrator/files/mplayer-control.pipe
/usr/bin/mplayer -slave -input file=/home/administrator/files/mplayer-control.pipe /home/administrator/music/file.mp3
Then I have a GUI written in Python that is supposed to be able to control the volume of the instance of mplayer that is being played. I have tried the following:
os.system('echo "set_property volume $musicvol" > /home/administrator/files/mplayer-control.pipe')
That works if i substitute $musicvol with the numeric value instead, but that is unfortunately of no use. I need to be able to pass the variable.
I would also be able to solve it by invoking a bash script from the Python application, but I can not get that to work either:
subprocess.call("/home/administrator/files/setvolume.sh", executable="bash", shell=True)

You don't need to call os.system and invoke a shell to write that line to the FIFO from your Python script- you can just do:
new_volume = 50
with open("/home/administrator/files/mplayer-control.pipe","w") as fp:
fp.write("set_property volume %d\n" % (new_volume,))
It's not clear to me what you expect to happen in your original python, though - is musicvol set in the environment? If instead it's a Python variable that you want to insert into the string that you're passing, the easiest way is to use the string interpolation operator (%) as I've done in the example above.
In your example of using subprocess.call you don't need the executable or shell keyword arguments if setvolume.sh is executable and has a #! line - you could just do:
subprocess.call("/home/administrator/files/setvolume.sh")
However, it's better to just use open and write in Python as above, I think.