Develop Reference

Why is IDLE's Shell color coding keywords in user response to input(prompt)?

I'm working on a Python 3 program that takes input from the user via the Python Shell. For some reason when I enter the information into the shell (once the input asks for info...), it's coloring keywords & functions with specific colors.
For example, if I type in "John is blue". It will color the word "is" as a keyword (Which it technically is but this is string input).
I haven't been able to bring up anything relevant on google so I'm bring the question here. Thanks.
Here is the code that runs the input.
if __name__ == '__main__':
global string
string = str(input('Enter info: '))
string = bytes(string.encode("utf-8"))
c = cont.key_gen_01()
c.func_01()
run_a = obf_01()
run_a.func_02()
#run_a.func_03()
run_a.func_04()
run_a.func_05(string)
Screen shot:

Colorizing in IDLE editor and Shell windows is done by the IDLE syntax colorizer. In Shell, it also colors console prompts ('>>> '), internal IDLE errors (now extremely rare), user code tracebacks, and user code output. (Colors can all be customized on the Highlights tab of the Settings dialog.) So the colorizer should not be turned off between code entries.
Unless your program is requesting entry of python code, I consider colorizing input() responses to be a minor bug. But it is not obvious how to tell the colorizer to ignore them. To a display, input() prompts are normal output. Besides which, responses can be entered before the prompt. Try the following with or without hitting ENTER before the prompt.
import time; time.sleep(5); s = input('what??? '); print(s)
The above also works in python, but at least in Windows Console, I don't see the entry until the prompt is displayed.

The Python input() function doesn't and can't do anything with how text you enter is displayed. It merely receives text after it has been typed and you hit enter. Instead, the color changes are applied by the IDLE shell window, which implements the input and output environment that the Python interpreter is connected to.
The IDLE Python shell treats all user input as Python source code when it comes to highlighting. This is just a change in how the text is rendered on your display, it has no influence on what string value is being returned from the input() function.
Other environments (IDE consoles, terminal windows, notebook UIs, etc) that can act as front-end displays for a Python interactive interpreter each can have their own specific ways of treating text.

You can print syntax highlighted text using python pygments
A good way to get started with is the prompt_toolkit moudle, comes with support for pygments and ALOT of command line features
Then you can create an prompt() which is input, and set the pygments lexer
check the prompt_toolkit docs or https://replit.com/#CodinUser/PyShell-15 to see a real life use of the syntax highlighting input pygments prompt_toolkit

Is there a way to replace a previously outputted text in Python?

There is something I want to do, but unfortunately I have absolutely no idea on how to do it, and I doubt someone else has asked. So, basically, what I want is the program to output text in an previous area. I'll try to explain.
Let's say I outputted the following text to the screen:
===========================================================================================
===========================================================================================
My question is, is there a way, without using pygame, to replace the blank lines between the lines with a certain text WITHOUT printing the lines again? Is this possible using for example Python IDLE or directly in the terminal, or is it only possible using pygame or something "like" that?
It would look like this, for example:
===========================================================================================
Hello World!
===========================================================================================

You can use the curses library for this kind of text-only UI. Here's a simple example, first printing the horizontal lines, and then, after some time, adding a string on a previous line.
import time, curses
scr = curses.initscr()
scr.addstr(0, 0, "#" * 80)
scr.addstr(2, 0, "#" * 80)
scr.refresh()
time.sleep(1)
scr.addstr(1, 35, "Hello World")
scr.refresh()

While there should be multiple libraries to handle this task, you should check out curses and colorama. I'm unsure about curses but this is definitely possible with colorama (install it via pip).
Here's an example:
from colorama import *
def pos(x, y):
return '\x1b['+str(y)+';'+str(x)+'H'
def display():
init() #just for safety here; needed in Windows
print(Fore.RED+pos(30, 10)+ 'This string is a different place!')
display()

Given your output, say, for example that is in a file test.dat:
===========================================================================================
===========================================================================================
a 1 simple line in vim would produce your desired output:
:3s/^$/Hello World!
If you want to automate this in a script: hello.sh
#!/bin/bash
ex test.dat <<-EOF
:3s/^$/Hello World!
wq " Update changes and quit.
EOF
Output:
===========================================================================================
Hello World!
===========================================================================================

What do the three arrow (">>>") signs mean?

I haven't been able to figure out what the >>> does, even though I often see it often in source code.

You won't see it in source code, it's probably documentation. It indicates an interactive session, and things typed into the 'interpreter' are marked with this. Output is shown without the arrows.
In fact, the python documentation often has a button >>>at the top right of example code to be able to hide the arrows (and output) so that you can copy and paste the code.
Shown:
Hidden:

'>>>' is the prompt of the interactive Python interpreter, meaning that the interpreter is ready to get Python statements typed in. It's occuring quite often in examples within the documentation of a Python program, in order to show which commands can be used and what will be the result of giving these commands to the interactive interpreter. For example, in a documentation of the print statement, one could give this example:
>>> print "Hello world."
Hello world.
This would be an actual snippet of a session with the interactive Python interpreter.
An interesting feature in IPython is that it ignores leading >>>, meaning that you can copy and paste code from such documentation without needing to remove the leading >>>:
In [1]: >>> print "Hello world."
Hello world.
(The prompt in IPython is In [n]:, where n is counting the interactive commands issued.)

Here are some of my findings on >>> and consequently ... complementing the previous answers.
You only see >>> when you are running Python in interactive mode prompting/asking the user for the "next command". Technical details here.
>>> and ... are not written in stone. These are stored in sys.ps1 and sys.ps2, and therefore can be changed. Further elaborated here.
>>> import sys
>>> sys.ps1 = "$ "
$
Every standard Python has this prompt unless you compile your own Python after changing >>> and ... to what you (sanely) wish to. Apart from that there seems to be a way to change it for all future interactive sessions by changing /usr/lib/python2.7/code.py but I couldn't find any success with it.

The >>> prompt is the Python interpreter’s way of asking you, “What do you want
me to do next?”, and it is called "chevron" prompt

In case you are trying to figure out how to exit a session, run this:
quit()

I found it to be called ' REPL'

How can I add a command to the Python interactive shell?

I'm trying to save myself just a few keystrokes for a command I type fairly regularly in Python.
In my python startup script, I define a function called load which is similar to import, but adds some functionality. It takes a single string:
def load(s):
# Do some stuff
return something
In order to call this function I have to type
>>> load('something')
I would rather be able to simply type:
>>> load something
I am running Python with readline support, so I know there exists some programmability there, but I don't know if this sort of thing is possible using it.
I attempted to get around this by using the InteractivConsole and creating an instance of it in my startup file, like so:
import code, re, traceback
class LoadingInteractiveConsole(code.InteractiveConsole):
def raw_input(self, prompt = ""):
s = raw_input(prompt)
match = re.match('^load\s+(.+)', s)
if match:
module = match.group(1)
try:
load(module)
print "Loaded " + module
except ImportError:
traceback.print_exc()
return ''
else:
return s
console = LoadingInteractiveConsole()
console.interact("")
This works with the caveat that I have to hit Ctrl-D twice to exit the python interpreter: once to get out of my custom console, once to get out of the real one.
Is there a way to do this without writing a custom C program and embedding the interpreter into it?
Edit
Out of channel, I had the suggestion of appending this to the end of my startup file:
import sys
sys.exit()
It works well enough, but I'm still interested in alternative solutions.

You could try ipython - which gives a python shell which does allow many things including automatic parentheses which gives you the function call as you requested.

I think you want the cmd module.
See a tutorial here:
http://wiki.python.org/moin/CmdModule

Hate to answer my own question, but there hasn't been an answer that works for all the versions of Python I use. Aside from the solution I posted in my question edit (which is what I'm now using), here's another:
Edit .bashrc to contain the following lines:
alias python3='python3 ~/py/shellreplace.py'
alias python='python ~/py/shellreplace.py'
alias python27='python27 ~/py/shellreplace.py'
Then simply move all of the LoadingInteractiveConsole code into the file ~/py/shellreplace.py Once the script finishes executing, python will cease executing, and the improved interactive session will be seamless.

How to make a python, command-line program autocomplete arbitrary things NOT interpreter

I am aware of how to setup autocompletion of python objects in the python interpreter (on unix).
Google shows many hits for explanations on how to do this.
Unfortunately, there are so many references to that it is difficult to find what I need to do, which is slightly different.
I need to know how to enable, tab/auto completion of arbitrary items in a command-line program written in python.
My specific use case is a command-line python program that needs to send emails. I want to be able to autocomplete email addresses (I have the addresses on disk) when the user types part of it (and optionally presses the TAB key).
I do not need it to work on windows or mac, just linux.

Use Python's readline bindings. For example,
import readline
def completer(text, state):
options = [i for i in commands if i.startswith(text)]
if state < len(options):
return options[state]
else:
return None
readline.parse_and_bind("tab: complete")
readline.set_completer(completer)
The official module docs aren't much more detailed, see the readline docs for more info.

Follow the cmd documentation and you'll be fine
import cmd
addresses = [
'here#blubb.com',
'foo#bar.com',
'whatever#wherever.org',
]
class MyCmd(cmd.Cmd):
def do_send(self, line):
pass
def complete_send(self, text, line, start_index, end_index):
if text:
return [
address for address in addresses
if address.startswith(text)
]
else:
return addresses
if __name__ == '__main__':
my_cmd = MyCmd()
my_cmd.cmdloop()
Output for tab -> tab -> send -> tab -> tab -> f -> tab
(Cmd)
help send
(Cmd) send
foo#bar.com here#blubb.com whatever#wherever.org
(Cmd) send foo#bar.com
(Cmd)

Since you say "NOT interpreter" in your question, I guess you don't want answers involving python readline and suchlike. (edit: in hindsight, that's obviously not the case. Ho hum. I think this info is interesting anyway, so I'll leave it here.)
I think you might be after this.
It's about adding shell-level completion to arbitrary commands, extending bash's own tab-completion.
In a nutshell, you'll create a file containing a shell-function that will generate possible completions, save it into /etc/bash_completion.d/ and register it with the command complete. Here's a snippet from the linked page:
_foo()
{
local cur prev opts
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
prev="${COMP_WORDS[COMP_CWORD-1]}"
opts="--help --verbose --version"
if [[ ${cur} == -* ]] ; then
COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
return 0
fi
}
complete -F _foo foo
In this case, the typing foo --[TAB] will give you the values in the variable opts, i.e. --help, --verbose and --version. For your purposes, you'll essentially want to customise the values that are put into opts.
Do have a look at the example on the linked page, it's all pretty straightforward.

I am surprised that nobody has mentioned argcomplete, here is an example from the docs:
from argcomplete.completers import ChoicesCompleter
parser.add_argument("--protocol", choices=('http', 'https', 'ssh', 'rsync', 'wss'))
parser.add_argument("--proto").completer=ChoicesCompleter(('http', 'https', 'ssh', 'rsync', 'wss'))

Here is a full-working version of the code that was very supplied by ephemient here (thank you).
import readline
addrs = ['angela#domain.com', 'michael#domain.com', 'david#test.com']
def completer(text, state):
options = [x for x in addrs if x.startswith(text)]
try:
return options[state]
except IndexError:
return None
readline.set_completer(completer)
readline.parse_and_bind("tab: complete")
while 1:
a = raw_input("> ")
print "You entered", a

# ~/.pythonrc
import rlcompleter, readline
readline.parse_and_bind('tab:complete')
# ~/.bashrc
export PYTHONSTARTUP=~/.pythonrc

You can try using the Python Prompt Toolkit, a library for building interactive command line applications in Python.
The library makes it easy to add interactive autocomplete functionality, allowing the user to use the Tab key to visually cycle through the available choices. The library is cross-platform (Linux, OS X, FreeBSD, OpenBSD, Windows). Example:
(Image source: pcgli)

The posted answers work fine but I have open sourced an autocomplete library that I wrote at work. We have been using it for a while in production and it is fast, stable and easy to use. It even has a demo mode so you can quickly test what you would get as you type words.
To install it, simply run: pip install fast-autocomplete
Here is an example:
>>> from fast_autocomplete import AutoComplete
>>> words = {'book': {}, 'burrito': {}, 'pizza': {}, 'pasta':{}}
>>> autocomplete = AutoComplete(words=words)
>>> autocomplete.search(word='b', max_cost=3, size=3)
[['book'], ['burrito']]
>>> autocomplete.search(word='bu', max_cost=3, size=3)
[['burrito']]
>>> autocomplete.search(word='barrito', max_cost=3, size=3) # mis-spelling
[['burrito']]
Checkout: https://github.com/seperman/fast-autocomplete for the source code.
And here is an explanation of how it works: http://zepworks.com/posts/you-autocomplete-me/
It deals with mis-spellings and optionally sorting by the weight of the word. (let's say burrito is more important than book, then you give burrito a higher "count" and it will show up first before book in the results.
Words is a dictionary and each word can have a context. For example the "count", how to display the word, some other context around the word etc. In this example words didn't have any context.

This works well.
#!/usr/bin/python3
import readline
readline.parse_and_bind("tab: complete")
def complete(text,state):
volcab = ['dog','cat','rabbit','bird','slug','snail']
results = [x for x in volcab if x.startswith(text)] + [None]
return results[state]
readline.set_completer(complete)
line = input('prompt> ')