How APT does its fancy progress bar?

[aum]

Today while running an apt full-upgrade I asked myself how apt does this nice progress bar stuck at the bottom line while still writing scrolling text.

 

We needed no more with a coworker to try to reproduce it!

 

 

Fortunately, while being very bored in the tube a few years back, I wrote a Headless VT100 emulator, so I remembered some things and had a few hints on how it could be done, so I started poking around with some Python code.

 

I remembered a few instructions like DECSTBM to set the margins, and the various commands to move the cursor up and down so I started with this.

 

After some trials-and-errors bottom margin reservation and log printing worked but the log were displayed on the line near the bottom margin, not where the command started: notice you can run an apt upgrade on the top of your terminal, it displays the progress bar at the bottom, but the logs will start from the top, as if there were no progress bar.

 

While trying to solve this with my coworker, we were discussing about my implementation, and looking at a random function:

static void DECSC(struct lw_terminal *term_emul)
{
    /*TODO: Save graphic rendition and charset.*/
    struct lw_terminal_vt100 *vt100;

    vt100 = (struct lw_terminal_vt100 *)term_emul->user_data;
    vt100->saved_x = vt100->x;
    vt100->saved_y = vt100->y;
}

and soon we realized it was the missing piece! Saving the cursor position to restore it later!!

It soon started to look like an ugly undocumented, but almost working, ... thing?

print(f"\0337\033[0;{lines-1}r\0338")
try:
    for i in range(250):
        time.sleep(0.2)
        print("Hello world", i)
        print(
            f"\0337\033[{lines};0f", datetime.now().isoformat(), "\0338", sep="", end=""
        )
except KeyboardInterrupt:
    pass
finally:
    print(f"\0337\033[0;{lines}r\033[{lines};0f\033[0K\0338")

But wow, f"\0337\033[0;{lines}r\033[{lines};0f\033[0K\0338" should really be made more readable, it start to hurt my eyes. I had to go and let it as is for a night.

 

Today I'm back at it again, and tried to add some comments, and delay to actually see how it behave step by step:

import time
import os
from datetime import datetime

columns, lines = os.get_terminal_size()


def write(s):
    print(s, end="")
    time.sleep(1)


write("\n")  # Ensure the last line is available.
write("\0337")  # Save cursor position
write(f"\033[0;{lines-1}r")  # Reserve the bottom line
write("\0338")  # Restore the cursor position
write("\033[1A")  # Move up one line

try:
    for i in range(250):
        time.sleep(0.2)
        write(f"Hello {i}")
        write("\0337")  # Save cursor position
        write(f"\033[{lines};0f")  # Move cursor to the bottom margin
        write(datetime.now().isoformat())  # Write the date
        write("\0338")  # Restore cursor position
        write("\n")
except KeyboardInterrupt:
    pass
finally:
    write("\0337")  # Save cursor position
    write(f"\033[0;{lines}r")  # Drop margin reservation
    write(f"\033[{lines};0f")  # Move the cursor to the bottom line
    write("\033[0K")  # Clean that line
    write("\0338")  # Restore cursor position

For the record here's what's used (\033 is ESC😞

ESC 7 is DECSC (Save Cursor)
ESC 8 is DECRC (Restore Cursor)
ESC [ Pn ; Pn r is DECSTBM (Set Top and Bottom Margins)
ESC [ Pn A is CUU (Cursor Up)
ESC [ Pn ; Pn f is HVP (Horizontal and Vertical Position)
ESC [ Ps K is EL (Erase In Line)

Don't forget the -u (unbuffered) Python flag if you want to see it step by step:

 

 

Started from the bottom so we see it scroll:

 

 

Notice how on interruption (or normal exit) it cleans the progress bar before exiting, restoring the cursor at the right place.

 

But hey, we could have read the apt code!

 

Yes, it was less challenging, but now that it works, we have to take a look at it!

 

So I apt-get source apt and found, in install-progress.cc:

// scroll down a bit to avoid visual glitch when the screen
// area shrinks by one row
std::cout << "\n";

// save cursor
std::cout << "\0337";

// set scroll region (this will place the cursor in the top left)
std::cout << "\033[0;" << std::to_string(nr_rows - 1) << "r";

// restore cursor but ensure its inside the scrolling area
std::cout << "\0338";
static const char *move_cursor_up = "\033[1A";
std::cout << move_cursor_up;

Already looks familiar to you?

 

Source