Develop Reference

When to use the terms python shell and python interpreter?

1) I am new to Python and love to learn its core. I have downloaded the python software package and discovered the python.exe application inside. I double clicked it and a balck and white window popped up.
Should I call it a python Interpreter or python Shell?
2) I am learning python online. I came across the terms python tty, python shell and python interpreter. I am satisfied by calling that screen inside the window as a tty(TeleTYpewriter) because we could use only keyboard to work inside and no mouse. But actually that screen has got some intelligence responding to our request. Is python tty an apt term for it?
3) In UNIX, shell is an user interface and command line interpreter, so does python interpreter and python shell are the same.

Python shell lets you use the Python interpreter in interactive mode, just as an OS shell, such as bash, lets you use the OS in interactive mode. You can use the Python interpreter in script mode or batch mode wherein you let the interpreter execute all lines of code in one sequence. It is comparable to writing shell scripts (or batch files in Microsoft Windows).
In your case the screenshot is of a python "shell".
You shouldn't really pay attention to this distinction because in the end everything runs through the python interpreter be it in interactive mode or not.

It is both the python shell and the python interpreter. The shell is where you write your code directly in the CLI, whereas the interpreter is the program that will interpret your code and execute it. Therefore, the interpreter is called in the shell when you write some code, it may also be called when you execute some python code directly from a file.

The customary term for the interactive Python shell is the Python REPL. Many modern interpreters enter a Read-Eval-Print-Loop when you run them interactively, and this term has stuck.
The program which interprets and executes your Python code is the Python interpreter; it can act as a shell, as described above, or run silently and just execute your Python code without any visible user interface of its own, like when you run a script of yours with
python scriptname.py

launch external shell python instance in shell from python

I'd like to call a separate non-child python program from a python script and have it run externally in a new shell instance. The original python script doesn't need to be aware of the instance it launches, it shouldn't block when the launched process is running and shouldn't care if it dies. This is what I have tried which returns no error but seems to do nothing...
import subprocess
python_path = '/usr/bin/python'
args = [python_path, '&']
p = subprocess.Popen(args, shell=True)
What should I be doing differently
EDIT
The reason for doing this is I have an application with a built in version of python, I have written some python tools that should be run separately alongside this application but there is no assurance that the user will have python installed on their system outside the application with the builtin version I'm using. Because of this I can get the python binary path from the built in version programatically and I'd like to launch an external version of the built in python. This eliminates the need for the user to install python themselves. So in essence I need a simple way to call an external python script using my current running version of python programatically.
I don't need to catch any output into the original program, in fact once launched I'd like it to have nothing to do with the original program
EDIT 2
So it seems that my original question was very unclear so here are more details, I think I was trying to over simplify the question:
I'm running OSX but the code should also work on windows machines.
The main application that has a built in version of CPython is a compiled c++ application that ships with a python framework that it uses at runtime. You can launch the embedded version of this version of python by doing this in a Terminal window on OSX
/my_main_app/Contents/Frameworks/Python.framework/Versions/2.7/bin/python
From my main application I'd like to be able to run a command in the version of python embedded in the main app that launches an external copy of a python script using the above python version just like I would if I did the following command in a Terminal window. The new launched orphan process should have its own Terminal window so the user can interact with it.
/my_main_app/Contents/Frameworks/Python.framework/Versions/2.7/bin/python my_python_script
I would like the child python instance not to block the main application and I'd like it to have its own terminal window so the user can interact with it. The main application doesn't need to be aware of the child once its launched in any way. The only reason I would do this is to automate launching an external application using a Terminal for the user

If you're trying to launch a new terminal window to run a new Python in (which isn't what your question asks for, but from a comment it sounds like it's what you actually want):
You can't. At least not in a general-purpose, cross-platform way.
Python is just a command-line program that runs with whatever stdin/stdout/stderr it's given. If those happen to be from a terminal, then it's running in a terminal. It doesn't know anything about the terminal beyond that.
If you need to do this for some specific platform and some specific terminal program—e.g., Terminal.app on OS X, iTerm on OS X, the "DOS prompt" on Windows, gnome-terminal on any X11 system, etc.—that's generally doable, but the way to do it is by launching or scripting the terminal program and telling it to open a new window and run Python in that window. And, needless to say, they all have completely different ways of doing that.
And even then, it's not going to be possible in all cases. For example, if you ssh in to a remote machine and run Python on that machine, there is no way it can reach back to your machine and open a new terminal window.
On most platforms that have multiple possible terminals, you can write some heuristic code that figures out which terminal you're currently running under by just walking os.getppid() until you find something that looks like a terminal you know how to deal with (and if you get to init/launchd/etc. without finding one, then you weren't running in a terminal).

The problem is that you're running Python with the argument &. Python has no idea what to do with that. It's like typing this at the shell:
/usr/bin/python '&'
In fact, if you pay attention, you're almost certainly getting something like this through your stderr:
python: can't open file '&': [Errno 2] No such file or directory
… which is exactly what you'd get from doing the equivalent at the shell.
What you presumably wanted was the equivalent of this shell command:
/usr/bin/python &
But the & there isn't an argument at all, it's part of sh syntax. The subprocess module doesn't know anything about sh syntax, and you're telling it not to use a shell, so there's nobody to interpret that &.
You could tell subprocess to use a shell, so it can do this for you:
cmdline = '{} &'.format(python_path)
p = subprocess.Popen(cmdline, shell=True)
But really, there's no good reason to. Just opening a subprocess and not calling communicate or wait on it already effectively "puts it in the background", just like & does on the shell. So:
args = [python_path]
p = subprocess.Popen(args)
This will start a new Python interpreter that sits there running in the background, trying to use the same stdin/stdout/stderr as your parent. I'm not sure why you want that, but it's the same thing that using & in the shell would have done.

Actually I think there might be a solution to your problem, I found a useful solution at another question here.
This way subprocess.popen starts a new python shell instance and runs the second script from there. It worked perfectly for me on Windows 10.

You can try using screen command
with this command a new shell instance created and the current instance runs in the background.
# screen; python script1.py
After running above command, a new shell prompt will be seen where we can run another script and script1.py will be running in the background.
Hope it helps.

How can I make my program utilize tab completion?

I've noticed that some programs (e.g. hg) allow the user to tab-complete specific parts of the command. For example, if, in an hg repository working directory, I type:
hg qpush --move b8<TAB>
It will try to complete the command with any mercurial patches in my patch queue that start with "b8".
What I'd like to do is imitate this behavior in my program. That is, I have a series of commands that depend on files within a certain directory, and I'd like to be able to provide tab completion in the shell. Is there an API for providing this on Ubuntu Linux (preferably using python, as that's what my script is written in)?

To do this, you need to write tab-completion modules for your shell. The default shell in most Linux distributions is bash, so you should write a completion script (typically a shell script). Once you've written your script, add it to /etc/bash_completion.d/. This should be distributed with your program (for Linux distributions, included in the package).
Debian Administration has a guide for writing your completion scripts. For using completion on a Mac, see https://trac.macports.org/wiki/howto/bash-completion.
For examples of completion files, take a look at the bash-completion project from Debian (also on Github). See also https://unix.stackexchange.com/questions/4738/an-easy-bash-completion-tutorial.
If you use zsh, hack.augusto linked to the documentation for writing completions.

This is what the readline module is for.
Actually, readline is a common C library so it has bindings in many languages. And I can say, I've had tons of fun with it.
Enjoy B)

You might want to try the zsh shell, it has a great completion system with support for tons of applications.
The completion system is written with the shell language, but if you really want to use python you can run the interpreter from inside your completion function. The down side it that if you want to write completion for your own software, you will need to do some reading (user manual and the manpage for instance).

Take a look at the source of the 'cmd' module in the Python library. It supports command completion.

python programming query

I'm new to Python programming.
My question is where to write python programs in linux--
in terminal or
any writing pad like gedit etc
Should we write one program again and again for running or we should call the program and run it.

After you install Python (it's installed by default on most Linux-es), you have two options:
Terminal. Just type python <ENTER> and you will be taken to the Python interpreter interactive prompt. For anything but the most basic stuff, however, you may want to intall IPython - an alternative interactive prompt that's much better than the default one.
In a file. Save your code into a .py file and run it with python myfile.py
But first and foremost, start by learning Python - the official tutorial is a great place to start. Among other useful information, its chapter 2 discusses how to use the Python interpreter for beginners.
Stackoverflow also has a lot of great resources for learning Python. This question, for example, and many others.

If you are learning, or you are evaluating expressions, you could run Python in terminal, or use IDLE.
But if you are writing large chunks of code, then you should consider using an IDE.
You could either use Geany, or use Eclipse with PyDev. I prefer Eclipse myself.
For running, you can run it using the command python program.py, or just add the line
#!/bin/python
to the beginning of your program, grant it execution permission using chmod, and run it with ./program.py command.

Python IDE on Linux Console

This may sound strange, but I need a better way to build python scripts than opening a file with nano/vi, change something, quit the editor, and type in python script.py, over and over again.
I need to build the script on a webserver without any gui. Any ideas how can I improve my workflow?

put this line in your .vimrc file:
:map <F2> :w\|!python %<CR>
now hitting <F2> will save and run your python script

You should give the screen utility a look. While it's not an IDE it is some kind of window manager on the terminal -- i.e. you can have multiple windows and switch between them, which makes especially tasks like this much easier.

You can execute shell commands from within vim.

Using emacs with python-mode you can execute the script with C-c C-c

you can try ipython. using its edit command, it will bring up your editor (nano/vim/etc), you write your script, and then on exiting you're returned to the ipython prompt and the script is automatically executed.

When working with Vim on the console, I have found that using "tabs" in Vim, instead of having multiple Vim instances suspended in the background, makes handling multiple files in Vim more efficient. It takes a bit of getting used to, but it works really well.

You could run XVNC over ssh, which is actually passably responsive for doing this sort of thing and gets you a windowing GUI. I've done this quite effectively over really asthmatic Jetstart DSL services in New Zealand (128K up/ 128K down =8^P) and it's certainly responsive enough for gvim and xterm windows. Another option would be screen, which lets you have multiple textual sessions open and switch between them.

There are actually 2 questions. First is polling for a console IDE for python and the second is a better dev/test/deploy workflow.
For while there are many ways you can write python code in the console, I find a combination of screen, vim and python/ipython is the best as they are usually available on most servers. If you are doing long sessions, I find emacs + python-mode typically involves less typing.
For a better workflow, I would suggest setting up a development environment. You can easily setup a Linux VM on your desktop/laptop easily these days - there isn't a excuse not to even if it's for hobby projects. That opens up a much larger selection of IDEs available to you, such as:
GUI versions of VI and friends
Remote file editing with tramp and testing locally with python-mode inside Emacs
http://www.netbeans.org
and of course http://eclipse.org with the PyDev plugin
I would also setup a SCM to keep track of changes so that you do
better QA and use it to deploy tested changes onto the server.
For example I use Mercurial for my pet projects and I simply tag my repo when it's ready and update the production server to the tag when I deploy. On the devbox, I do:
(hack hack hack, test test test)
hg ci -m 'comment'
hg tag
hg push
Then I jump onto the server and do the following when I deploy:
hg update
restart service/webserver as needed

Well, apart from using one of the more capable console editors (Emacs or vi would come to mind), why do you have to edit it on the web server itself? Just edit it remotely if constant FTP/WebDAV transfer would seem to cumbersome.
Emacs has Tramp Mode, gedit on Linux and bbedit on the Mac support remote editing, too. Probably quite a large number of other editors. In that case you would just edit in on a more capable desktop and restart the script from a shell window.

For what it's worth, VIM alone can do the same tasks as previously posted. I have had the same problem with testing Python from the command line.
My solution was to use the screen command. I split screens vertically, I run Python in one instance of a shell, and on the second screen, I usually edit Python code with VIM.
Command to install screen:
sudo apt-get install screen
The screen package has a bit of a learning curve but there isn't any mystery if you can remember the "Ctrl-Alt ?" command that contains all knowledge.
No GUI is required!