I'm trying to use the curses textpad.Textbox() function for text input. Everything is working fine so far, however, some keys don't get recognized, including the section sign (§) and all German umlauts (ä/ö/ü). I guess it's somehow related to the text encoding, but I have no idea how to fix this. My German keyboard layout works perfectly fine with input().
Here is some minimal example:
import curses
import curses.textpad as textpad
try:
stdtscr = curses.initscr()
curses.cbreak()
stdtscr.keypad(1)
curses.noecho()
textpad.Textbox(stdtscr).edit()
finally:
curses.nocbreak()
stdtscr.keypad(0)
curses.echo()
curses.endwin()
Just as in C, you should initialize the locale. It's spelled out in both the Python documentation:
Since version 5.4, the ncurses library decides how to interpret non-ASCII data using the nl_langinfo function. That means that you have to call locale.setlocale() in the application and encode Unicode strings using one of the system’s available encodings.
and the ncurses manual page:
The library uses the locale which the calling program has
initialized. That is normally done with setlocale:
setlocale(LC_ALL, "");
If the locale is not initialized, the library assumes that
characters are printable as in ISO-8859-1, to work with cer-
tain legacy programs. You should initialize the locale and
not rely on specific details of the library when the locale
has not been setup.
Addressing the followup comment, textpad.py does not expect UTF-8 input in any case. Essentially it "validates" its input, decides it isn't ASCII and ignores it when it's not.
Python's curses binding provides an interface to wgetch, which (with ncurses) gives the individual bytes for the UTF-8. (X/Open Curses specifies a different function wget_wch, for which Python has no binding).
textpad.py could be modified to work around the curses binding by assembling the bytes into a Unicode value, but you'd need the setlocale as the first step.
Related
This may be a question better suited for UNIX stack exchange but thought I would post here as I do not know what exactly the problem is. I have a python script that uses curses and displays characters from font awesome 5 free. This script worked fine, but just recently I reinstalled my os and now it is not displaying these characters. The terminal I am using is capable of displaying the character on the cli, python is able to print the character, it is only when it is echoed through curses that it does not display. Any help would be greatly appreciated.
More information:
Here is a mock up of the script:
import curses
from curses import wrapper
def main(stdscr):
while True:
stdscr.addstr(0,0,STRING)
stdscr.refresh()
wrapper(main)
when STRING is something like "a", "b", or "cat" it shows up fine. However when the character is something like "" or "" it does not display anything.
I use standard tips for customizing interactive Python session:
$ cat ~/.bashrc
export PYTHONSTARTUP=~/.pystartup
$ cat ~/.pystartup
import os
import sys
import atexit
import readline
import rlcompleter
historyPath = os.path.expanduser("~/.pyhistory")
def save_history(historyPath=historyPath):
import readline
readline.write_history_file(historyPath)
if os.path.exists(historyPath):
readline.read_history_file(historyPath)
term_with_colors = ['xterm', 'xterm-color', 'xterm-256color', 'linux', 'screen', 'screen-256color', 'screen-bce']
if os.environ.get('TERM') in term_with_colors:
green='\033[32m'
red='\033[31m'
reset='\033[0m'
sys.ps1 = red + '>>> ' + reset
sys.ps2 = green + '... ' + reset
del term_with_colors
atexit.register(save_history)
del os, sys, atexit, readline, rlcompleter, save_history, historyPath
Now I get context sensitive completion and color prompt.
The problem comes from color prompt - when I invoke history-search-backward (by pressing UP) in an interactive Python session, Readline takes in account terminal escape sequences, so the cursor position is wrongly calculated and the text is wrongly displayed.
In Bash man page this problem is mentioned and fixed by special markers:
\[ begin a sequence of non-printing characters,
which could be used to embed a
terminal control sequence into the prompt
\] end a sequence of non-printing characters
How to fix this issue for Python prompt?
I open info readline and found:
-- Function: int rl_expand_prompt (char *prompt)
Expand any special character sequences in PROMPT and set up the
local Readline prompt redisplay variables. This function is
called by `readline()'. It may also be called to expand the
primary prompt if the `rl_on_new_line_with_prompt()' function or
`rl_already_prompted' variable is used. It returns the number of
visible characters on the last line of the (possibly multi-line)
prompt. Applications may indicate that the prompt contains
characters that take up no physical screen space when displayed by
bracketing a sequence of such characters with the special markers
`RL_PROMPT_START_IGNORE' and `RL_PROMPT_END_IGNORE' (declared in
`readline.h'. This may be used to embed terminal-specific escape
sequences in prompts.
As text suggested I searched for RL_PROMPT_START_IGNORE and RL_PROMPT_END_IGNORE definition in readline.h and found next:
/* Definitions available for use by readline clients. */
#define RL_PROMPT_START_IGNORE '\001'
#define RL_PROMPT_END_IGNORE '\002'
So I put appropriate changes to my ~/.pystartup:
green='\001\033[32m\002'
red='\001\033[31m\002'
reset='\001\033[0m\002'
and now all work fine!!!
For a better python shell experience, I'd recommend you to use either ipython or bpython.
If you happen to come here because of recent Python 3.10+ REPL (CLI) problems in Win10, then please have a look here:
https://github.com/pyreadline/pyreadline/issues/73
Change Python interactive prompt ">>>"
Some of these issues has now been fixed in a better maintained repo:
https://github.com/pyreadline3/pyreadline3/
I'm trying to draw Unicode characters using PyCDC.DrawText(), but it seems to draw two ASCII characters instead. For example, when trying to draw 'Я' (\u042F), I get: http://i.stack.imgur.com/hh9RJ.png
My string is defined as a Unicode string:
text = u'Я'
And the file starts with:
# -*- coding:utf-8 -*-
I also tried printing the string (to the console) and it comes out fine, so the problem is probably lying within the implementation of DrawText().
Thanks!
To output Unicode text on Windows you need to encode it in UTF-16 and call the wide character version of the DrawText() or TextOut() Win32 functions. In case you aren't familiar, the Windows API is natively UTF-16 and has parallel 8 bit ANSI versions for legacy support.
I know nothing of the Win32 wrapper you are using but rather suspect that PyCDC.DrawText() is calling the ANSI version of whichever one of these Win32 functions is doing the work. Your solution will likely involve finding a way to invoke DrawTextW() or TextOutW(). You could do it with ctypes, and these functions must surely be available through PyWin32 also.
However, I would probably opt for something higher level, like PyQt.
David: 'these functions must surely be available through PyWin32 ' actually, they are not.
After dozens of hours of searching, trying to figure out where within win32ui, win32gui, etc. there might be a hidden TextOutW, writing my own C extension that had other flaws so couldn't use it, writing an external prog. called from within python only to find out that HDC handles cannot be transfered to other processes, I have finally stumbled upon this single-line elegant pre-programmed solution, based on ctypes as suggested above:
you need the TextOutW or similar function as you were used to from windows gdi c. Although there is a function called win32gdi.DrawTextW which works exactly like the windows counterpart, yet sometimes you need to specifically use e.g. TextOut, ExtTextOut etc., which are not available in the unicode W-suffixed versions in pywin32's win32gdi
to achieve this, instead of using the limited win32gui functions, use windll.gdi32.TextOutW available from ctypes:
from ctypes import *
import win32gdi
# init HDC
# set hdc to something like
hdc = win32gdi.CreateDC(print_processor, printername, devmode)
# here comes the ctypes function that does your deal
text = u'Working! \u4e00\u4e01'
windll.gdi32.TextOutW(hdc, x, y, text, len(text))
# ... continue your prog ...
Have fun with this
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Changing default encoding of python?
I am reading dive in python and it mentions setting python's default encoding scheme in the XML parsing chapter.
The setdefaultencoding is used in python-installed-dir/site-packages/pyanaconda/sitecustomize.py
import sys
sys.setdefaultencoding('utf-8')
But when I run the script, it raises:
AttributeError: 'module' object has no attribute 'setdefaultencoding'
How to set the default encoding,anyway?
I am using python 2.7
Solution:
find the site.py in the python installation.
Edit the setencoding function
def setencoding():
encoding = "ascii"
if 0:
import locale
loc = locale.getdefaultlocale()
if loc[1]:
encoding = loc[1]
if 0: #changes comes here, change 0 to 1
encoding = "undefined" #the encoding you want
if encoding != "ascii":
sys.setdefaultencoding(encoding)
I am using python 2.7
Python's sys module has had a setdefaultencoding function since Python 2.0. However,
This function is only intended to be used by the site module implementation and, where needed, by sitecustomize. Once used by the site module, it is removed from the sys module’s namespace.
The docs back to at least Python 2.1 indicate this happens, so it was never appropriate for PyAnaconda to use this method, and I'm not sure why it ever worked.
How to set the default encoding,anyway?
Run sys.setdefaultencoding in the file sitecustomize.py, which needs to be in sys.path (e.g. lib/site-packages) when Python starts up. You can verify the change with sys.getdefaultencoding.
Edit for anonymous downvoter:
Whoever downvoted this answer, would you care to explain? This question is for Python 2.x only. There is no sys.setdefaultencoding in Python 3 if that's your problem. I stand by my explanation of how to use this function if one wants to in Python 2. I wasn't defending its use or recommending its use. A library should never touch it, which is why it's removed from the sys namespace after site.py and sitecustomize.py have a chance to call it. A library should also never assume the default encoding is ASCII in 2.x. It's up to the system. Personally I leave it as ASCII.
I've written a simple shell-like program that uses readline in order to provide smart completion of arguments. I would like the mechanism to support arguments that have spaces and are quoted to signify as one argument (as with providing the shell with such).
I've seen that shlex.split() knows how to parse quoted arguments, but in case a user wants to complete mid-typing it fails (for example: 'complete "Hello ' would cause an exception to be thrown when passed to shlex, because of unbalanced quotes).
Is there code for doing this?
Thanks!
I don't know of any existing code for the task, but if I were to do this I'd catch the exception, try adding a fake trailing quote, and see how shlex.split does with the string thus modified.
GNU Readline allows for that scenario with the variable rl_completer_quote_characters. Unfortunatelly, Python does not export that option on the standard library's readline module (even on 3.7.1, the latest as of this writing).
I found a way of doing that with ctypes, though:
import ctypes
libreadline = ctypes.CDLL ("libreadline.so.6")
rl_completer_quote_characters = ctypes.c_char_p.in_dll (
libreadline,
"rl_completer_quote_characters"
)
rl_completer_quote_characters.value = '"'
Note this is clearly not portable (possibly even between Linux distros, as the libreadline version is hardcoded, but I didn't have plain libreadline.so on my computer), so you may have to adapt it for your environment.
Also, in my case, I set only double quotes as special for the completion feature, as that was my use case.
References
https://robots.thoughtbot.com/tab-completion-in-gnu-readline#adding-quoting-support
#eryksun's comment on how to set data to a global variable in a shared library using python
To make #caxcaxcoatl answer a little bit more portable, readline hardcoded version can be replaces with readline.__file__ and it will be:
import ctypes
import readline
libreadline = ctypes.CDLL (readline.__file__)
rl_completer_quote_characters = ctypes.c_char_p.in_dll (
libreadline,
"rl_completer_quote_characters"
)
rl_completer_quote_characters.value = '"'