DTS/TUI integration

[m-iwanicki]

List of tasks that might have to be done before releasing DTS with new TUI.

References

• DTS changes 3mdeb/tui-sh#1
• Add tui-sh dts-scripts#121

Priorities

• 🟢 - needs to be done before release
• 🟡 - should be done before release
• 🔴 - might be done before release

Tasks

🟢

• Move tui-sh_git.bb to different meta-layer, so we can use it in non-DTS projects. Can be done as last step, it's currently in meta-dts to simplify testing during development
• TUI: Fix delay during refresh (tui-sh) #1701
• Fix OSFV test or modify integration in such a way that all currently passing tests will still pass
• Do a review of code in tui-lib.sh
• Run a workflow on real machine and compare with output from before TUI changes, check if everything is printed correctly

🟡

• Review whole https://github.com/3mdeb/tui-sh repository, tests, documents, examples...
• Remove examples where we override TUI shell functions (whole dts-integration), I don't think that's a good way to do this and it might encourage others to use library like this. Maybe create simple example based on current DTS implementation
• Separate TUI library into core (tui_run and all internal functions used to display or configure menu) and utility, used for printing, colors, input and everything else that we might want to use in scripts
• Add pre-commit to tui-sh (shellcheck, shfmt, markdownlint, maybe more). Run tests as a part of pre-commit.
• Format strings before displaying them, e.g. section or option labels, entry values. Either disallow printing too long string, e.g. by trimming them and adding ... to the end so it looks better, or in case of section information such as RAM values wrap and display multiple lines correctly indented and with borders drawn
• Allow usage of commands in callbacks, currently we have to pass whole path to executable
• Allow usage of commands (with arguments) in conditions, currently we have to use shell variables. e.g. allow usage of systemctl is-active sshd, hide any possible output this might've produced, and display option based on return value of this command
• Remove hidden Q option that quits menu, this might surprise both developer and user. Allow adding explicit exit option e.g. in footer.
• Add more utility functions to be used by scripts, name them correctly and document how they should be used. Some of those might already be implemented
  • Colored printing - variables for changing color (exported, or in separate lib to be included), documented usage
  • tui_echo_* prints whole line, maybe change name to tui_print_line. Do we want echo -n version?
  • Various user input:
    • ask_for_choice - Allows us to specify prompt, and list of pairs, <choice_opt> and <choice_msg>.
    • Similar to above, but only prompt + list of choices, opt should be generated automatically (numerically, from 1) by function. Function should echo choice.
    • ask_for_confirmation - yes/no choice
    • Silent read (for passwords), or with * displayed for each key

🔴

• In tui-lib.sh, use yq directly instead of first converting to JSON
• Simplify (if possible) and comment config parsing as currently it's very hard to know what's happening (at least for casual jq user)
• Don't export anything, if you want to use it, source library
• Add right border - optional, might look better that way.
• Allow using arguments for callbacks - Current implementation might require us to create useless wrappers that do only one thing, call target command with arguments.
• Resolve TUI: Fix DTS extensions submenu #1684

Optional features

• Add option to force usage of tui_* functions to print on screen. Normal commands e.g. echo shouldn't display anything.

• Do we want to format output of main/footer commands? Without doing this we might have results like this

  ************************************************************
  **     1) Dasharo HCL report
  **     2) Update Dasharo Firmware
  **     4) Edit your DPP keys
  **     6) Transition Dasharo Firmware
  **     7) Fuse platform
  ************************************************************
  R to reboot  P to poweroff  S to enter shell
  K to launch SSH server  L to enable sending DTS logs
  C to display DPP credentials
  
  Enter an option:
  Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum
  
  Press Enter to continue...
  

• We might want to have some kind of tui_cat_* function to use if we want to display command output e.g.: ls | tui_cat_*

[macpijan]

Addressed most of the lib-related comments in the meantime: 3mdeb/tui-sh#2
You can just run demo.sh locally to quickly test the look&feel.

[3mkusiak]

tui_echo_* prints whole line, maybe change name to tui_print_line. Do we want echo -n version?

How bout we rip off how Arduino does it? println for new line and print without new line.

Add right border - optional, might look better that way.

We'd need to implement wrapping text, but if we want to do it, I +1 that idea.

Additionally from me:

• Let's yeet all "_normal" suffixes from the function names. The default should be tui_print not tui_print_normal.
• tui_render... functions not only "render" stuff but also print it at the same time. Normally, you'd want to render the "frame" and display it separately, but for DTS i think we can make that shortcut.

I looked through the newest library code, and it looks Fine™ to me. I mean, we're printing bunch of # and * in patterns. There is nothing rocket science about it.

[DaniilKl]

Regarding

Fix OSFV test or modify integration in such a way that all currently passing tests will still pass

I think we should consider change our approach of sharing global variables by sourcing the dts-environment.sh in a lot of files, for example in the callbacks:

https://github.com/Dasharo/dts-scripts/blob/508faf9e4e9f95d8f715adbd7d06866f4d9a8637/tui/tui_callbacks/reboot.sh#L4

Because dts-environment.sh not only exports the variables, but executes dmidecode commands from DTS HAL:

https://github.com/Dasharo/dts-scripts/blob/main/include/dts-environment.sh#L64-L73

that are being logged to profile. Sourcing the file in every script will cause uncontrolled execution of the DTS HAL commands, that will break our profile approach, where we are not only recording what commands were executed but when commands were executed as well.

Another two ideas on how to solve it are:

• Moving dmidecode commands execution somewhere else.
• Recording to profile only the commands, that do modifications to firmware/hardware state, not the commands, that read some data from hardware or firmware.

[DaniilKl]

Do we want to format output of main/footer commands? Without doing this we might have results like this

I do not think that it is an optional feature, it should be implemented, maybe not as 🟢 , but as 🟡.

[DaniilKl]

Just a reminder that we should adhere to creating primitives to sdandartise our menu design, smth like I described here: Dasharo/dts-scripts#116 (comment) .