There appears to be some concept of lines vs rows in terminal emulators, about which I'd like to know more.
Demonstration of what I mean by rows vs lines
The Python script below displays three lines of 'a' and waits, then with three lines of 'b'.
import sys, struct, fcntl, termios
write = sys.stdout.write
def clear_screen(): write('\x1b[2J')
def move_cursor(row, col): write('\x1b['+str(row)+';'+str(col)+'H')
def current_width(): #taken from blessings so this example doesn't have dependencies
return struct.unpack('hhhh', fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, '\000' * 8))[1]
clear_screen()
for c in 'ab':
#clear_screen between loops changes this behavior
width = current_width()
move_cursor(5, 1)
write(c*width+'\n')
move_cursor(6, 1)
write(c*width+'\n')
move_cursor(7, 1)
write(c*width+'\n')
sys.stdout.flush()
try: input() # pause and wait for ENTER in python 2 and 3
except: pass
If you narrow the terminal window width by one character during this break, you see
That seems pretty reasonable - each line has been separately wrapped. When we hit enter again to print bs,
Everything works as expected. I've used absolute cursor positioning, and written to the same rows I wrote to previously - which of course doesn't overwrite all of the a's, because many of them are on other rows.
However, when we narrow the window by one more character, the wrapping works differently:
Why did the second and third rows of b wrap together, and why did last line of a's merge with the first line of b's? A hint of why is in the top visible row above - we see two a's because theyse two rows are still linked - of course if we move the window again, that one line will continue to wrap the same way. This seems to be happening even for lines which we replaced a whole row of.
It turns out that the rows that had wrapped before are now linked to their corresponding parent rows; it's more obvious that they belong to the same logical line once we widen the terminal a lot:
My question
Practically, my question is how to prevent or predict this rows-being-combined-into-lines. Clearing the whole screen eliminates the behavior, but it would be nice to do this only for individual lines that need it if possible so I can keep the caching by line that is significantly speeding up my application. Clearing to the end of a row unlinks that row from the row below it, but clearing to the beginning of a row does not unlink that row from the one above it.
I'm curious - what are these line things? Where can I read about them? Can I find out which rows are part of the same line?
I've observed this behavior with terminal.app and iterm, with and w/o tmux. I imagine source-diving into any of these would yield an answer even if there's no spec - but I imagine there's a spec somewhere!
Background: I'd like to make a terminal user interface that can predict the way terminal wrapping will occur if the user decreases the window width. I'm aware of things like fullscreen mode (tput smcup, or python -c 'print "\x1b[?1049h"', which are what ncurses uses) which would work for preventing line wrap, but don't want to use it here.
Edit: made it more clear that I understand the overwriting behavior of the script already and want an explanation of the wrapping behavior.
OK. Let's start with the causes for the behavior you are seeing:
I tested your code and noticed that it only happened when you resized the window. When the window was left alone, it would write out the a's, and upon pressing enter would over-write them with b's (I assume that's the intended behavior).
What appears to be happening is that when you resize the window partway through, the line indices change, so that on your next iteration, you can't trust the same coordinates when you call move_cursor().
Interestingly, when you resize the window, the word wrapping pushes the text before the cursor upwards. I assume this is part of the terminal emulator's code (since we almost always want to retain focus on the cursor and if the cursor is at the bottom of the screen, resizing might obscure it beyond the window's height if the word-wrapping pushed it downwards).
You'll notice that after a resize when you hit enter, only two lines of a's remain visible (and not all 3). Here's what appears to be happening:
First we begin with the initial output. (line numbers added for clarity)
1
2
3
4
5 aaaaaaaaaaaaaaa\n
6 aaaaaaaaaaaaaaa\n
7 aaaaaaaaaaaaaaa\n
8
Note that there is a new line character at the end of each of these lines (which is why your cursor appears below the last despite your not having moved the cursor again)
When you shrink the window by one character, this happens:
1
2 aaaaaaaaaaaaaa
3 a\n
4 aaaaaaaaaaaaaa
5 a\n
6 aaaaaaaaaaaaaa
7 a\n
8
You'll notice what I mean by "pushing the text upwards"
Now when you hit enter and your loop reiterates, the cursor is sent to row 5 col 1 (as specified by your code) and is placed directly over the last a of the second line. When it starts writing b's it overwrites the last a of the second line with b's and the subsequent line as well.
1
2 aaaaaaaaaaaaaa
3 a\n
4 aaaaaaaaaaaaaa
5 bbbbbbbbbbbbbb\n
6 bbbbbbbbbbbbbb
7 bbbbbbbbbbbbbb\n
8
Importantly, this also overwrites the new-line character at the end of the second line of a's. This means that there is now no new-line dividing the second line of a's and the first line of b's, so when you expand the window: they appear as a single line.
1
2
3
4
5 aaaaaaaaaaaaaaa\n
6 aaaaaaaaaaaaaabbbbbbbbbbbbbb\n
7 bbbbbbbbbbbbbbbbbbbbbbbbbbbb\n
8
I'm not totally sure why this second line of b's also gets put together but it appears to likely have something to do with the fact that the line of a's which the first one overwrites is now missing it's own new-line termination. However, that's just a guess.
The reason why you get two characters of line-wrap if you try to shrink the window by yet another character is because now you are shrinking two halves of the same line of text, which means that one pushes on the other, causing two characters instead of one at the end.
For example: in these test windows I've shown, the width begins at 15 characters, I then shrink it to 14 and print out the b's. There is still one line of a's which is 15 chars long, and now a line of 14 a's & 14 b's which is line-wrapped at 14 chars. The same (for some reason) is true of the last two rows of b's (they are one line of 28 chars, wrapped at 14). So when you shrink the window by one more character (down to 13): the first line of 15 a's now has two trailing characters (15 - 13 = 2); the next line of 28 chars now has to fit in a 13 character-wide window (28 / 13 = 2 R2), and the same applies to the last b's as well.
0 aaaaaaaaaaaaa
1 aa\n
2 aaaaaaaaaaaaa
3 abbbbbbbbbbbb
4 bb\n
5 bbbbbbbbbbbbb
6 bbbbbbbbbbbbb
7 bb\n
8
Why does it work like this?:
This sort of stuff is the difficulty you run into when you are trying to run your program within another program that has the power to reposition the text as it sees fit. In the event of a resize your indices become unreliable. Your terminal emulator is trying to handle the realignment for you and is pushing the text before your prompt (which is fixed at row 8) up and down in the scroll-back to ensure you can always see your active prompt.
Rows and columns are something defined by the terminal/terminal emulator and it is up to it to interpret their location accordingly. When the appropriate control sequences are given it is the terminal which interprets them accordingly for proper display.
Note that SOME terminals do behave differently and in an emulated terminal there is often a setting to change what sort of terminal it is emulating, which may also affect how certain escape sequences respond. This is why a UNIX environment usually has a setting or environment variable ($TERM) which tells it which type of terminal it is communicating with so that it knows what control sequences to send.
Most Terminals use standard ANSI compliant control sequences, or systems based on the DEC VT series of Hardware Terminals.
In the Terminal.app preferences under Preferences->Settings->Advanced you can actually see (or change) which type of Terminal is being emulated by your window in the drop-down menu next to "Declare terminal as:"
How to overcome this:
You might be able to mitigate this by storing the last known width and checking to see if there has been a change. In which case you can change your cursor logic to compensate for the changes.
Alternately you might consider using escape sequences designed for relative cursor movement (as opposed to absolute) to avoid accidentally overwriting previous lines after a resize. There is also the ability to save and restore specific cursor locations using only escape sequences.
Esc[<value>A Up
Esc[<value>B Down
Esc[<value>C Forward
Esc[<value>D Backward
Esc[s Save Current Position
Esc[u Restore Last Saved Position
Esc[K Erase from cursor position to end of line
However you have no real guarantee that all Terminal emulators will deal with window resizes the same way (that's not really part of any terminal standard, AFAIK), or that it won't change in the future. If you are hoping to make a true terminal emulator, I suggest first getting your GUI window setup so that you can be in control of all resizing logic.
However if you want to run in a terminal-emulator window and deal with mitigating window resizes for a given command-line utility that you're writing. I'd suggest looking at the curses library for python. This is the sort of functionality used by all window-resize aware programs that I know of off the top of my head (vim, yum, irssi), and can deal with this sort of changes. Though I don't personally have any experience using it.
It's available for python via the curses module.
(and please, if you plan on redistributing your program, consider writing it in Python3. Do it for the children :D)
Resources:
These links might be helpful:
ANSI Escape Sequences
VT100 Escape Sequences
I hope that helps!
As 0x783czar pointed out, the key difference is whether an explicit newline was printed which caused the terminal to begin a new row, or there was an implicit overflow because there was no room left on the right to print the desired characters.
It's important to remember this at the end of each line for copy-pasting purposes (whether there'll be a newline character in the buffer or not), for triple-click highlight behavior in many terminals, and for rewrapping the contents when the window is resized (in those terminals that support it).
Applications running inside terminals hardly ever care about this difference, and they use the words "line" and "row" interchangeably. Hence, when we implemented rewrapping the contents on resize in gnome-terminal, we preferred the words "row" or "line" for one single visual line of the terminal, and the word "paragraph" for the contents between two adjacent newline characters. A paragraph wraps into multiple lines if it's wider than the terminal. (This is not by any means an official terminology, but IMO is quite reasonable and helps talk about these concepts.)
Related
I am using an embedded Python program to take an average of a large number of spectra in GNU radio Companion.
I want to continuously print the number of averaged spectra so that output happens on a same line, always overwriting the previous number. So not printing each number on a separate line.
In Python you can do it in the following way:
for i in range(100000):
print('Number of spectra averaged ', i, end='\r')
However, that does not work from within an embedded Python program, each number is on its own line, which is very time consuming.
I thought that the escape character '\r' causes the problem, but probably not because the escape character '\n' works as it should be when printing from within a Python program.
I have tried all the tricks I have found for printing the output on a single line, but none of them works for an embedded Python program.
The printing from the embedded Python program goes both to the terminal where I started gnuradio-companion, and to the console of the gnuradio-companion. In both cases the printing produces several lines, not a single line which would be always overwritten.
How to print on a single line, overwriting the previous output ?
I am using GRC 3.9.5.0 (Python 3.8.10) on Ubuntu 22.04.4
Cheers, Kimmo
Thanks Marcus
Yes, I was referring to the 'mini-terminal' or 'log-terminal' in GRC.
Using QT GUI Number Sink works fine.
I just added one more output (of the type float because Number Sink cannot receive integer value inputs) to the Python block, and directed it to Number Sink.
In the Number Sink I am using 'Update period = 0.5' because if using a value of zero the CPU load is increasing by about 10%
let's take the really simple case of two nested for loops
after pressing return and tab the first time, I get an indentation level where 1 tab equals 4 spaces
after pressing return and tab the second time, I get an indentation level where 1 tab equals 9 spaces
this is rather annoying when I want to copy a piece of code from a script and run it on the interpreter, as it returns the
TabError: inconsistent use of tabs and spaces in indentation
I did some googling but you can imagine what the keywords "python", "interpreter", "indentation" returns: nothing useful
Is there a way to correct this? Thanks
EDIT
just try yourselves:
open a terminal
run python command
then type for i in range(3):
press Enter
then press Tab once, to get to the right indentation level
then type for j in range(3):
press Enter again
then press Tab twice, to get to the (expected) indentation level
if, like me, you have a Mac, chances are that the second Tab hit in the last step above, will yield 8 spaces, instead of 4
This causes problems when copying lines of code written with an IDE
(I'm using Sublime 3, with the option translate_tabs_to_spaces set to True)
I was hoping someone could identify some flag/options/setting of my interpreter which could solve this issue
Title question :
Because you are at 2nd indentation level since you have a loop inside a loop.
Tabulation/space error :
Python uses only indentation to interpret which block belongs to the code. No "BEGIN...END" , no brackets. So indentation IS information.
The rule is "indentation must be uniform", so either everything is spaces (1,2,4.. as you want) or everything is tabulations.
For more information, refer to the Python style guideline (PEP8) here :
https://www.python.org/dev/peps/pep-0008/
By using an IDE ( such as Spyder, for example) tabs will automatically be replaced by spaces. I recommend you to use one, especially if you are a new to python.
example output estimate of pi:
3.13658765
then a new better estimate comes along. Say, 3.1416232.
So I find the first character in the old estimate that doesn't match the new. In this case, it is the 4th character.
Question: Is there a way to delete the 4th character in console(and then repeat this until all characters after 3.1 are gone) so that I can then print the new, better values for each of those characters?
Note: I don't want to delete everything and console and then reprint, as this would get considerably slower as the number of digits increases.
It sounds like you want something like:
best_est = None
while True:
est = some_estimate_method() # assuming this is a 'str'
if not best_est:
best_est = est
else:
if (float(est) > float(best_est)):
best_est = est
print(best_est, end = '\r')''
Depending on the directional relation of the estimate (i.e. is less better or is more better), you would have to change the inequality to accommodate.
However, if we look at your example:
# initial print
3.13658765
# get new estimate (3.1416232)
# overwrite the print
3.1416232
You originally wanted to "re-print" 3.1 then print all the new characters in the new estimate. That begs to ask the question why not just print the whole new estimate?
Since the entire value of the new estimate is greater, that means the comparison you are looking for (starting at index 4 and beyond) has already be checked when using < or > for inequality. Simply calling a print(est, end = '\r') returns the console cursor to the beginning of the line, effectively overwriting the value each time.
If you just need to overwrite your last line, you can do something like this. The "\r" means instead of a newline, a return carriage will be printed, moving you back to the start of the same line to overwrite
print("3.1515", end="\r")
print("3.1415")
Furthermore, if you want to do tricky things, you could look into ANSI escape codes, these will let you print to any arbitrary location, although they aren't the most portable. These will also let you do fun things like setting the colour.
For example print("\033[31;1HX") would print an X to the 31's column and 1st row. To delete a character, you could simply print a space over top (print("\033[31;1H ")).
This may not work on all terminals
http://ascii-table.com/ansi-escape-sequences.php
Is there a way to get way to get an integer to a valid Tkinter text widget position. For example, 0 would get changed to '1.0' and if there were no characters on line 1, 1 would go to '2.0'. I am aware of the widget's index method however only to get the cursor position (in case this is the answer).
Many thanks
EDIT
I believe (having now done more research on the matter) that the Tk Text widget lazy loads the text (for performance). Therefore, without you getting the entire text contents and performing string actions on that, I don't think its possible (the Tk Text only has a minor understanding of what is above and below the currently viewed text - hence why a horizontal scrollbar will constantly adjust when scrolling vertically).
The text widget supports many modifiers to an index. For example, you can take an existing index and compute a new index by adding or subtracting characters or lines. For example, if you want to get the first 10 characters on line 2 you can use "2.0 + 10 chars" (or the more compact "2.0+10c"). If you want the 100th character in the text widget you can use "1.0+100c", etc.
In addition to characters, you can add or subtract lines (eg: "end-1line"), and you can get the start or end of a word or line (eg: "end-1c linestart" represents the first character of the last line in the widget).
You can read about all of the forms an index can take in the canonical tcl/tk documentation: http://tcl.tk/man/tcl8.5/TkCmd/text.htm#M7
I am writing a script that runs in the terminal and that displays a (long) multiple line string. My problem is that, when the string is printed, the terminal automatically places the cursor at the end of the string.
The string being longer than the number of lines in the terminal, I only see the last 72 lines of my string (my terminal window has 72 lines), so it forces to scroll up to the beginning of the string every time I run that script, and it turns out to be pretty annoying.
Is there a way to go back to the beginning of the string once it's printed?
End of string, the cursor is at the bottom:
Beginning of the string, ~200 lines above, where I want to be after the script runs:
I thought of using curses, but that seems to be overkill for what I am looking for.
Also, I'm on Mac OS and I don't particularly care about portability
While curses is the portable solution, try printing the sequence ESC [ H. It will likely work on all of the terminals you care about.
print "\033[H"
Reference:
https://en.wikipedia.org/wiki/ANSI_escape_code
Re your comment you made on Aug 28, 2015 at 1:51:
I think what you are looking for is printing this escape code:
print("\033[F")