Is there a method to print terminal formatted output to a variable?
print 'a\bb'
--> 'b'
I want that string 'b' to a variable - so how to do it?
I am working with a text string from telnet. Thus I want to work with the string that would be printed to screen.
So what I am looking for is something like this:
simplify_string('a\bb') ==> 'b'
Another example with a carriage return:
simplify_string('aaaaaaa\rbb') ==> 'bbaaaaa'
This turns out to be quite tricky because there are a lot of terminal formatting commands (including e.g. cursor up/down/left/right commands, terminal colour codes, vertical and horizontal tabs, etc.).
So, if you want to emulate a terminal properly, get a terminal emulator! pyte (pip install pyte) implements a VT102-compatible in-memory virtual terminal. So, you can feed it some text, and then get the formatted text from it:
import pyte
screen = pyte.Screen(80, 24)
stream = pyte.ByteStream(screen)
stream.feed(b'xyzzz\by\rfoo')
# print the first line of text ('foozy')
print(screen.display[0].rstrip())
To handle multiple lines, just join all of the lines in the text (e.g. '\n'.join(row.rstrip() for row in screen.display).rstrip()).
Note that this doesn't handle trailing spaces, but those would be indistinguishable on a real terminal anyway.
Related
I am wondering if I can have print() outputs such as
in a terminal and/or an IPython/Jupyter Notebook. I want to develop a library working with toleranced dimensions and these types of pretty-printed outputs will come quite handy during development and testing.
What I know so far:
There are escape characters such as Carriage Return \r that goes to the beginning of the line without erasing the existing characters and the Backspace \b that deletes the last character. For example print("some text\bsome other text \rbingo", end="") should give me bingotexsome other text. Anyway, when printing a new character the previous one is erased.
I also know how to use Unicode characters to have superscripted/subscripted digits and plus/minus signs. For example, the print('1.23\u207a\u2074\u2027\u2075\u2076') will give me something like 1.23+4.56 and print('1.23\u208b\u2087.\u2088\u2089') outputs close to 1.23-7.89. Although what unicode characters should be used for superscript/subscript decimal delimiters (in this case period/dot/point) is still debatable. There are multiple options for superscipted dot including also \u0387 and \u22c5. However, AFIK there are no unicode characters suitable for subscripted dot. (more info here)
what I don't know
if there is an escape character or Unicode one that replicates the left arrow ← key on the keyboard?
how to print without erasing the pixels in the terminal? Is there a way to print/display characters on top of each other?
and if none of the above is possible in a terminal, if/how I can control the HTTP/CSS outputs in a Jupyter Notebook to print both superscript and subscript at the same time?
In Jupyter Notebook/Lab this should work:
from IPython.display import Math
Math(r"1.23^{+4.56}_{-7.89}")
For convenience, you can package it in a class:
from IPython.display import Math
class PPrint:
def __init__(self, base, sub, sup):
self.base = base
self.sub = sub
self.sup = sup
def _ipython_display_(self):
display(Math(f"{{{self.base}}}^{{{self.sub}}}_{{{self.sup}}}"))
Then you can create an instance e.g.:
x = PPrint("1.23", "+4.56", "-7.89")
and if you execute in a notebook either x or display(x), it should appear as in your example.
A python module I am using provides a hook that allows capturing user keyboard input before it is sent to a shell terminal. The problem I am facing is that it captures input character-by-character, which makes capturing the input commands difficult when the user performs such things as backspacing or moving the cursor.
For example, given the string exit\x1b[4D\x1b[Jshow myself out, the following takes place:
>>> a = exit\x1b[4D\x1b[Jshow myself out
>>> print(a)
show myself out
>>> with open('file.txt', 'w+') as f:
>>> f.write(a)
>>> exit()
less abc.txt
The less command shows the raw command (exit\x1b[4D\x1b[Jshow myself out), when in fact I would like it to be stored 'cleanly' as it is displayed when using the print function (show myself out).
Printing the result, or 'cat'ing the file shows exactly what I would want to be displayed, but I am guessing here that the terminal is transforming the output.
Is there a way to achieve a 'clean' write to file, either using some python module, or some bash utility? Surely there must be some module out there that can do this for me?
less is interpreting the control characters.
You can get around this with the -r command line option:
$ less -r file.txt
show myself out
From the manual:
-r or --raw-control-chars
Causes "raw" control characters to be displayed. The default is
to display control characters using the caret notation; for
example, a control-A (octal 001) is displayed as "^A". Warning:
when the -r option is used, less cannot keep track of the actual
appearance of the screen (since this depends on how the screen
responds to each type of control character). Thus, various dis‐
play problems may result, such as long lines being split in the
wrong place.
The raw control characters are sent to the terminal, which then interprets them as cat would.
As others have stated, you would need to interpret the characters yourself before writing them to a file.
I am running a console application that takes data coming from various sensors around the house. Sometimes the transmission is interrupted and thus the packet does not make sense. When that happens, the contents of the packet is output to a terminal session.
However, what has happened is that while outputting the erroneous packet, it has contained characters that changed character set of the current terminal window, rendering any text (apart from numbers) as unreadable gibberish.
What would be the best way to filter the erroneous packets before their display while retaining most of the special characters? What exactly are sequences that can change behaviour of the terminal?
I would also like to add that apart from scrambled output, the application still works as it should.
Your terminal may be responding to ANSI escape codes.
To prevent the data from inadvertently affecting the terminal, you could print the repr of the data, rather than the data itself:
For example,
good = 'hello'
bad = '\033[41mRED'
print('{!r}'.format(good))
print('{!r}'.format(bad))
yields
'hello'
'\x1b[41mRED'
whereas
print(good)
print(bad)
yields
(Btw, typing reset will reset the terminal.)
You can convert all characters outside of the ASCII range which should eliminate any stray escape sequences that will change the state of your terminal.
print s.encode('string-escape')
When you get a packet check it for validity before outputting it. One possibility is to check that each character in the packet is printable, that is, in the range of 32-127 decimal, before output. Or add checksums to the packets and reject any with bad checksums.
If introduce a for loop in iPython, or any multi-line command, how do I go back and add lines to it? I ran this:
for row in table.find_all('tr'):
cells = row.find_all('td')
for c,cell in enumerate(cells):
print c,":",cell.get_text().strip()
try:
this = cells[0]
that = cells[1]
the_docket = cells[2]
other_thign = cells[3]
jumble = re.sub('\s+',' ',str(cells[5])).strip()
except:
"Nope"
And realized I need to add a line to it, but I can't just hit "enter" in iPython, b/c that runs the command. So can I edit that multi-line command w/in iPython?
Been suffering this problem for a while. I just found that when using Ctrl-qCtrl-j (That's lowercase Q, J, no need to hold the shift key) will add a linefeed to an existing IPython edit session.
for li in some_list: print(li)
Moving the cursor after the colon and pressing Ctrl-qCtrl-j
for li in some_list:
print(li)
IPython: 5.2.1, iTerm2: 3.0.15, macOS: 10.12.6
The %edit magic function in iPython lets you edit code in your favorite editor and will then execute it as if it was typed directly. You can also edit code you've already typed into the repl since it's stored in a special variable, for example:
In [1]: def foo(x):
...: print x
...:
In [2]: %edit _i1
There is also a way to add a newline directly in the repl: ctrl-v, ctrl-j
The ctrl-v basically lets you send a control code and then the ctrl-j is the code for a newline (line-feed). It's a bit awkward to type but has the advantage of also working in the regular Python shell as well as in Bash itself.
Edit: At least in iTerm2, you can assign it to a single hotkey as well. I set ctrl-enter to "Send hex codes" of 0x16 0x0a. Could also use cmd-enter or whatever else.
You can use ctrl+on (i.e. press the control button and enter the characters 'o', 'n' while holding it). You can also do it in two steps - ctrl+o ctrl+n but I find the former easier.
ctrl-o - enter the multiline mode
ctrl-n - access command history going forward.
But since there is no forward history, cursor just moves to the next line.
I verified that it works with both IPython 5.3.0 and IPython 7.3.0 on a machine running git bash 2.18.0 + windows 7.
A easy way of doing it is using the ;\ operator. ; to signal that its the end of a command and \ to indicate the the command follows in a new line as follows:
In[4]: password = "123";\
username = "alpha"
This will let you have multiple line commands in ipython without invoking the editor
For completeness: newer IPython versions (0.11+) have a very nice graphical console which allows you to navigate your code with the arrows and even reposition the cursor with the mouse.
In multi-line statements, some keys take on a special function (arrows to navigate, Enter to insert a line break and others). You'll have to position the cursor at the end of the last line of the statement in order to get default behaviour, e.g. get the Up arrow ↑ to mean "previous statement" instead of "move the cursor to the previous line". Or get Enter to mean "execute code" instead of "insert line break in the middle of code".
The documentation on it is a bit sparse and fragmented in different pages, so here are the essential three links for getting started with it:
Intro to the Qt Console
Configuring IPython using nice per-user profile files instead of command line arguments
You are interested in the ipython_qtconsole_config.py
How to get an ipython graphical console on Windows 7?
Type Ctrl+q then Enter. As other pointed out, Ctrl+q lets you send a character code, but hitting Ctrl+q then Enter may be more natural than Ctrl+q then Ctrl+j.
Another option is to type ( then Enter, write, and delete the ( afterwards. This solution is similar to the one where you to type ;\ to say the statement is not completed.
Bit late to the party! But I noticed many people are talking about using Ctrl combinations to add extra lines. This didn't work for me until I saw a comment about it being the Emacs binding. I have set my in line editor to be Vi, if you have too then pressing Esc to go into "Normal" mode on the line before you want to add an extra line then press o (or O if you are after the line). Just like in normal Vi(m). Works for me!
I'm writing a little python utility to help move our shell -help documentation to searchable webpages, but I hit a weird block :
output = subprocess.Popen([sys.argv[1], '--help'],stdout=subprocess.PIPE).communicate()[0]
output = output.split('\n')
print output[4]
#NAME
for l in output[4]:
print l
#N
#A
#
#A
#M
#
#M
#E
#
#E
#or when written, n?na?am?me?e
It does this for any heading/subheading in the documentation, which makes it near unusable.
Any tips on getting correct formatting? Where did I screw up?
Thanks
The documentation contains overstruck characters done in the ancient line-printer way: print each character, followed by a backspace (\b aka \x08), followed by the same character again. So "NAME" becomes "N\bNA\bAM\bME\bE". If you can convince the program not to output that way, it would be the best; otherwise, you can clean it up with something like output = re.sub(r'\x08.', '', output)
A common way to mark a character as bold in a terminal is to print the character, followed by a backspace characters, followed by the character itself again (just like you would do it on a mechanical typewriter). Terminal emulators like xterm detect such sequences and turn them into bold characters. Programs shouldn't be printing such sequences if stdout is not a terminal, but if your tool does, you will have to clean up the mess yourself.