Develop Reference

How can I export MATLABPATH and call matlab function through the same python subprocess?

I need to call matlab with a defined MATLABPATH from a python script and I try to do this with the following code (snippet) in python:
addMatlabPath = os.path.join(<validPath>,'src') + ":" + \
os.path.join(<someOtherValidPaht>,'src') + ":"
matlabPathCommand = "export MATLABPATH="+addMatlabPath+"$MATLABPATH"
commandLine = matlabPathCommand+" && echo $MATLABPATH && "+"/Applications/MATLAB_R2015a.app/bin/matlab -nodisplay -nosplash -r \"my_matlab_script\"".format(os.path.realpath(os.path.dirname(__file__)),output_dir)
I try to execute the commandLine through subprocess:
process = subprocess.check_call(commandLine, stdout=out_buffer, stderr=subprocess.STDOUT,shell=True)
with which I can call matlab perfectly without the matlabPathCommand and the echo part in front.
The paths I use for the variable addMatlabPath are valid. I tested the command matlabPathCommand+ && echo $MATLABPPATH and this works correctly.
So, both parts of the command work individually as expected but not together. Python seems to hang in the check_call command and doesn't return even after a multiple of time the call to matlab normally takes.
Does anyone have a hint where my error could be?

The python code snippet above is correct. There was a problem in the matlab code which I didn't recognise while testing. This question and answer can be deleted.

Using cdrecord through popen won't eject

So, I'm making a cd burning app and I need to eject the drive to let the user put the disk in. It's a little more complicated, but simplest case I run into is this; I can use cdrecord via the command line to eject the cd tray using this command:
cdrecord --eject dev='/dev/sg1'
which should mean that I can do the same thing with subprocess.call, like this:
subprocess.call(["cdrecord", "--eject", "dev='/dev/sg1'"])
however, when I do that, I get this error:
wodim: No such file or directory.
Cannot open SCSI driver!
For possible targets try 'wodim --devices' or 'wodim -scanbus'.
For possible transport specifiers try 'wodim dev=help'.
For IDE/ATAPI devices configuration, see the file README.ATAPI.setup from
the wodim documentation.
and the tray doesn't open.
This is a very similar error to one I got before when trying to run it form the command line, but I fixed that error by loading the sg kernel module.
If I just run:
subprocess.call(["cdrecord", "--eject"])
it opens the tray just fine. However, this needs to work with possibly multiple cd trays, so that won't work.
How can I get this to eject the cd correctly?

Try this:
subprocess.call(["cdrecord", "--eject", "dev=/dev/sg1"])
The shell will take care of interpreting the quotes, but cdrecord will not.
The only reason you need the quotes in the first place is that the dev path might have spaces in it, causing the shell to split things into separate arguments. For example, if you type this:
cdrecord --eject dev=/dev/my silly cd name
The arguments to cdrecord will be --eject, dev=/dev/my, silly, cd, name. But if you do this:
cdrecord --eject dev='/dev/my silly cd name'
The arguments to cdrecord will be --eject, dev=/dev/my silly cd name.
When you're using subprocess.call, there's no shell to pull the arguments apart; you're passing them explicitly. So, if you do this:
subprocess.call(["cdrecord", "--eject", "dev=/dev/my silly cd name"])
The arguments to cdrecord will be --eject, dev=/dev/my silly cd name.
In some cases—e.g., because you get things in a hopelessly confused state in the first place (e.g., you're reading a config file that's meant to be used by your program or executed by the shell)—you really have no recourse but to run through the shell. If that happens, do this:
subprocess.call("cdrecord --eject dev='/dev/sg1'", shell=True)
But this generally isn't what you want, and it isn't what you want in this case.

You are not using cdrecord but a buggy fork called "wodim" that
might be the reason for your problems.
I recommend you to use recent original software from:
ftp://ftp.berlios.de/pub/cdrecord/alpha/

OS X & Python 3: Behavior while executing bash command in new terminal?

I've seen similar questions (e.g. Running a command in a new Mac OS X Terminal window ) but I need to confirm this command and its expected behavior in a mac (which I don't have). If anyone can run the following in Python 3 Mac:
import subprocess, os
def runcom(bashCommand):
sp = subprocess.Popen(['osascript'], stdin=subprocess.PIPE, stderr=subprocess.PIPE)
sp.communicate('''tell application "Terminal"\nactivate\ndo script with command "{0} $EXIT"\nend tell'''.format(bashCommand))
runcom('''echo \\"This is a test\\n\\nThis should come two lines later; press any key\\";read throwaway''')
runcom('''echo \\"This is a test\\"\n\necho \\"This should come one line later; press any key\\";read throwaway''')
runcom('''echo \\"This is testing whether I can have you enter your sudo pw on separate terminal\\";sudo ls;\necho \\"You should see your current directory; press any key\\";read throwaway''')
Firstly, and most basically, is the "spawn new terminal and execute" command correct? (For reference, this version of the runcom function came from this answer below, and is much cleaner than my original.)
As for the actual tests: the first one tests that internal double escaped \\n characters really work. The second tests that we can put (unescaped) newlines into the "script" and still have it work just like semicolon. Finally, the last one tests whether you can call a sudo process in a separate terminal (my ultimate goal).
In all cases, the new terminal should disappear as soon as you "press any key". Please also confirm this.
If one of these doesn't work, a correction/diagnosis would be most appreciated. Also appreciated: is there a more pythonic way of spawning a terminal on Mac then executing a (sudo, extended) bash commands on it?
Thanks!

[...] its expected behavior [...]
This is hard to answer, since those commands do what I expect them to do, which might not be what you expect them to do.
As for the actual tests: the first one tests that internal double escaped \n characters really work.
The \\n with the doubled backslash does indeed work correctly in that it causes echo to emit a newline character. However, no double quotes are emitted by echo.
The second tests that we can put (unescaped) newlines into the "script" and still have it work just like semicolon.
That works also.
Finally, the last one tests whether you can call a sudo process in a separate terminal (my ultimate goal).
There is no reason why this should not work also, and indeed it does.
In all cases, the new terminal should disappear as soon as you "press any key". Please also confirm this.
That will not work because of several reasons:
read in bash will by default read a whole line, not just one character
after the script you supply is executed, there is no reason for the shell within the terminal to exit
even if the shell would exit, the user can configure Terminal.app not to close a window after the shell exits (this is even the default setting)
Other problems:
the script you supply to osascript will appear in the terminal window before it is executed. in the examples above, the user will see every "This is a test [...]" twice.
I cannot figure out what $EXIT is supposed to do
The ls command will show the user "the current directory" only in the sense that the current working directory in a new terminal window will always be the user's home directory
throwaway will not be available after the script bashCommand exits
Finally, this script will not work at all under Python 3, because it crashes with a TypeError: communicate() takes a byte string as argument, not a string.
Also appreciated: is there a more pythonic way of spawning a terminal on Mac [...]
You should look into PyObjC! It's not necessarily more pythonic, but at least you would eliminate some layers of indirection.

I don't have Python 3, but I edited your runcom function a little and it should work:
def runcom(bashCommand):
sp = subprocess.Popen(['osascript'], stdin=subprocess.PIPE, stderr=subprocess.PIPE)
sp.communicate('''tell application "Terminal"\nactivate\ndo script with command "{0} $EXIT"\nend tell'''.format(bashCommand))

Running Another program from python

I want to call a program multiple times from a python code, and save the output of that program in a text file. My first problem right now is just calling the other code. I have to redirect to a different directory and call ./rank on output.txt. This is how Im trying to do it:
TheCommand = "~/src/rank-8-9-2011/rank output.txt"
os.system(TheCommand)
but im getting a parsing error.
[Parsing error on line ]Unknown error: 0
Im running python2.7 on Mac OS 10.5.8. Im not sure what the problem is. I also tried using subprocess:
subprocess.call(["~/src/rank-8-9-2011/rank", "output.txt"])
This does not find the directory (I have a feeling Im using the subprocess incorrectly), but I dont know what is wrong with the os.system.

the name of the program in the first argument to subprocess.Popen must not contain ~ as it doesn't pass the string to the shell for processing (which like always using parameterized queries in sql, protects one from string injection attacks, e.g. if instead of output.text one had ;rm -rf /, the system version would run rank and then run rm -rf . but the subprocess.Popen would only have rank open a file named ;rm -rf .), so one should expand it by calling os.path.expanduser:
subprocess.Popen([os.path.expanduser('~/src/rank-8-9-2011/rank'), "output.txt"])
although it is possible to turn shell processing on by passing shell=True, it is not recommended for the aforementioned reason.

you should try http://docs.python.org/library/os.path.html#os.path.expanduser
import os.path
subprocess.call([os.path.expanduser("~/src/rank-8-9-2011/rank"), "output.txt"])

I'm fairly certain your parsing error is coming from rank, not from your os.system command, as nothing there looks weird. What happens if you run rank manually?
subprocess seems to have a problem with '~', although I'm not immediately sure why. Put the full path and it should work (although you'll likely get that parsing error if it is indeed a problem with rank).

Save and run at the same time in Vim

I do a lot of Python quick simulation stuff and I'm constantly saving (:w) and then running (:!!). Is there a way to combine these actions?
Maybe a "save and run" command.

Okay, the simplest form of what you're looking for is the pipe command. It allows you to run multiple cmdline commands on the same line. In your case, the two commands are write `w` and execute current file `! %:p`. If you have a specific command you run for you current file, the second command becomes, e.g. `!python %:p`. So, the simplest answer to you question becomes:
:w | ! %:p
^ ^ ^
| | |--Execute current file
| |--Chain two commands
|--Save current file
One last thing to note is that not all commands can be chained. According to the Vim docs, certain commands accept a pipe as an argument, and thus break the chain...

Option 1:
Write a function similar to this and place it in your startup settings:
function myex()
execute ':w'
execute ':!!'
endfunction
You could even map a key combination to it -- look at the documentation.
Option 2 (better):
Look at the documentation for remapping keystrokes - you may be able to accomplish it through a simple key remap. The following works, but has "filename.py" hardcoded. Perhaps you can dig in and figure out how to replace that with the current file?
:map <F2> <Esc>:w<CR>:!filename.py<CR>
After mapping that, you can just press F2 in command mode.
imap, vmap, etc... are mappings in different modes. The above only applies to command mode. The following should work in insert mode also:
:imap <F2> <Esc>:w<CR>:!filename.py<CR>a
Section 40.1 of the Vim manual is very helpful.

Use the autowrite option:
:set autowrite
Write the contents of the file, if it has been modified, on each :next, :rewind, :last, :first, :previous, :stop, :suspend, :tag, :!, :make, CTRL-] and CTRL-^ command [...]

Here you go:
:nmap <F1> :w<cr>:!%<cr>
Save and run (you have to be in n mode though - just add esc and a for i mode).

Command combination seems to work through the | character, so perhaps something like aliasing :w|!your-command-here to a distinct key combination.

Another possibility:
au BufWriteCmd *.py write | !!
Though this will run every time you save, which might not be what you want.

In Vim, you could simply redirect any range of your current buffer to an external command (be it Bash, the Python interpreter, or you own Python script).
# Redirect whole buffer to 'python'
:%w !python
Suppose your current buffer contains two lines as below,
import numpy as np
print np.arange(12).reshape(3, 4)
then :%w !python will run it, be it saved or not. And print something like below on your terminal,
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
Of course, you could make something persistent, for example, some keymaps.
nnoremap <F8> :.w !python<CR>
vnoremap <F8> :w !python<CR>
The first one, run the current line. The second one, run the visual selection, via the Python interpreter.
#!! Be careful, in Vim ':w!python' and ':.w !python' are very different. The
first write (create or overwrite) a file named 'python' with contents of
current buffer, and the second redirects the selected cmdline range (here dot .,
which mean current line) to external command (here 'python').
For cmdline range, see
:h cmdline-ranges
Not the below one, which concerning normal command, not cmdline one.
:h command-range
This was inspired by Execute current line in Bash from Vim.

This is what I put in my .vimrc file and works like a charm:
nnoremap <leader>r :w<CR>:!!<CR>
Of course you need to run your shell command once before this, so it knows what command to execute.
Example:
:!node ./test.js

I got the following from the Vim tips wiki:
command! -complete=file -nargs=+ shell call s:runshellcommand(<q-args>)
function! s:runshellcommand(cmdline)
botright vnew
setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
call setline(1,a:cmdline)
call setline(2,substitute(a:cmdline,'.','=','g'))
execute 'silent $read !'.escape(a:cmdline,'%#')
setlocal nomodifiable
1
endfunction
But I changed new to vnew on the third line, and then for Python I have the following:
map <F9> :w:Shell python %<cr><c-w>
Hitting F9 saves, runs, and dumps the output into a new vertically split scratch buffer, for easy yanking, saving, etc. It also hits c-w so I only have to press h/c to close it and move back to my code.

Try making it inside the Bash.
In case of a C file called t.c, this is very convenient:
vi t.c && cc t.c -o t && ./t
The and symbols (&&) ensure that one error message breaks the command chain.
For Python this might be even easier:
vi t.py && python t.py

This will work in insert mode too:
" F5 => Save & Run python3 "
nnoremap <F5> :w <CR> :!sh -c 'python3 %' <CR>
inoremap <F5> <Esc> :w <CR> :!sh -c 'python3 %' <CR>
I use it all the time to test stuff that is just too long to retype in the interpreter.

Consider switching to IDLE. F5 does everything.
Consider switching to Komodo IDE. You can define a command so that F5 does everything.