Rework the graphics handling
-
Move text console support out of vesad to allow other graphics drivers to reuse it. (drivers!128 (merged)) -
Better IPC serialization/deserialization. The current ad-hoc code is error prone and hard to keep in sync between components as you need to change everything in lockstep. Something like Cap'n Proto (Rust implementation) may be a better fit. It should allow keeping compatibility with older programs even when we add and remove commands. -
Move the console multiplexing out of the individual graphics drivers. Instead have a protocol that allows swapping the framebuffer to be rendered by the graphics driver and then have an fbmuxd daemon which saves the current framebuffer for every virtual terminal and tells the graphics driver to swap it out whenever either switching between vt's or when the client requests the framebuffer to be swapped. The vga/vesa driver can then write it to the actual framebuffer while the virtio-gpu driver in the future could request the virtio-gpu device to change which framebuffer to use for scanout once it becomes possible for users to directly create buffers on the gpu. Fbmuxd would also handle multiplexing of input devices. -
Support for hardware backed cursor. This both decreases latency and improves energy efficiency by offloading rendering of the cursor to the display controller (on mobile devices this is an entirely separate part of the SoC by the way). Not every gpu and gpu interfaces support this though. VGA doesn't support it, but virtio-gpu does and so do all desktop gpu's. -
Allow delegating control over a virtual terminal to a different program. This would allow for example the login screen to delegate control to an instance of orbital running as the logged in user, reducing attack surface. Or for a VR game to get full control over page flipping and mode setting. It should also be possible for the delegating program to revoke access again for all the (in)directly delegated programs. This delegation should happen for both the display and input devices. Preferrably with an option for the delegating program to keep accepting input such that it can for example listen for Super+L to lock the screen and show a password prompt for the current user. -
Support mode setting. That is changing display resolutions, display mirroring, selecting which part of a buffer to show, using multiple planes aside from the hardware backed cursor. -
Allow creating buffers on the gpu and running commands like shader invocations on them. Initially just virgl (OpenGL) or venus (Vulkan) support for virtio-gpu would work. This final step would allow actual hardware acceleration.