diff --git a/content/news/rsoc-2023-eny-1.md b/content/news/rsoc-2023-eny-1.md index 1528b220d0cbbeda205ce022cbfd0a7cdf5c01d6..48fe6dc5e2db6232908ddafc1322031276ff6423 100644 --- a/content/news/rsoc-2023-eny-1.md +++ b/content/news/rsoc-2023-eny-1.md @@ -4,7 +4,8 @@ author = "Enygmator" date = "2023-07-15T07:09:07Z" +++ -Hey there everyone! I'm Eny a.k.a enygmator! +Hey there, everyone! +I'm Eny a.k.a enygmator! I'm back this year as part of **RSoC (Redox Summer of Code) 2023** to share with you a non-technical post on the **work I'm doing towards improving apps and driver support on Redox OS**! @@ -14,7 +15,7 @@ Feel free to discuss this post on social media. (links are in the Epilogue) The availability of support for various apps and drivers (for various hardware and software) is crucial for the general adoption of any *general purpose operating system* like Redox OS. Some of us developers are working on improving the *core* of Redox OS (like the Kernel), which should create a solid base on which high quality native drivers and apps can be created with ease. Some others are working on porting (and adapting) various open source drivers and apps (written for other OSes) such that they can work with Redox OS. This work is super important and helps Redox OS progress forward. -But in the meanwhile, there's a **potential shortcut to enabling _wide driver and app support_ for Redox OS**, without having to manually port and adapt drivers to Redox OS. (which can be helpful, both today and in the future). The shortcut, in simple words, is to use our Host machine running Redox OS, to run a Virtual Machine with another OS (Linux/Windows) as the guest, and then **cleverly use the drivers and apps that can run on that guest OS to help coverup the missing drivers and apps on Redox OS**. +But in the meanwhile, there's a **potential shortcut to enabling _wide driver and app support_ for Redox OS**, without having to manually port and adapt drivers to Redox OS. (which can be helpful, both today and in the future). The shortcut, in simple words, is to use our _host_ machine running Redox OS, to run a _virtual machine (VM)_ that will run another OS (like Linux or Windows) as the _guest_, and we then **cleverly use the drivers and apps that can run on that guest OS to help coverup for the missing drivers and apps on Redox OS**. Let's look into how we would go about doing that... @@ -22,16 +23,17 @@ Let's look into how we would go about doing that... ### Drivers on various OSes -In a "regular" case, we usually have native drivers running either in kernel-space (like in Linux) or user-space (like in Redox OS), and then there are apps which run on the OS, which utilize these drivers to accomplish tasks. Now, let's look into the following figure, which is an example of **how to use the driver of another OS in order to run your apps**. +In a "regular" case, we usually have native drivers running either in kernel-space (like most drivers in Linux) or user-space (like most drivers in Redox OS), and then there are apps which run on the OS and utilize these drivers to accomplish tasks. Now, let's look into the following figure, which is an example of **how to use the driver of another OS in order to run your apps on Redox OS**. <center> <img class="img-responsive" src="/img/rsoc-2023-eny/single-driver-vm.svg" alt="A Linux driver being used by app in Redox OS" /> Fig 1. A Linux driver being used by app in Redox OS </center> <br/> In the above figure, you can see that we are running a 3D game which is using a 3D game engine. This game is a userspace app running on Redox OS. -The audio driver here is an example of a native driver running on Redox OS, which is used by the game engine to generate sounds from the speaker. -Now let's look at the video Drivers. Assume that we have an Nvidia graphics card that uses the proprietary driver. This makes it hard, if not impossible for us to port it to Redox OS. So, what we do is create a VM running Linux, and then we give control of the GPU to the Linux VM. Now, the Nvidia driver running on Linux can control the GPU for us. All there is left for us to do is make it usable for applications running on Redox OS. This is where **the bridge** comes in. As you can see, the bridge establishes a connection between Redox OS and Linux such that the game engine in Redox OS can use the video driver running in Linux. +Look at the audio driver; it is an example of a native driver running on Redox OS, which is used by the game engine to generate sounds from the speaker. + +Now let's look at the video drivers. Assume that we have an Nvidia graphics card that uses the proprietary driver. This makes it hard, if not impossible for us to port it to Redox OS. So, what we do is create a VM running Linux, and then we give control of the GPU to the Linux VM. Now, the Nvidia driver running on Linux can control the GPU for us. All there is left for us to do is make it usable for applications running on Redox OS. This is where **the bridge** comes in. As you can see, the bridge establishes a connection between Redox OS and Linux such that the game engine in Redox OS can use the video driver running in Linux. <br/> @@ -45,23 +47,23 @@ Now let's look at another situation, where we would like to run an app that is b The most simple observation one can make is the fact that there are two devices, a speaker and a display, both of which are controlled by native device drivers running on Redox OS, and there's even an app on Redox OS which is using those two drivers. This is already happening today. -Now let's look at the VM; we can see that it is running Linux, and there's an app compiled for Linux (like a GTK/Qt app) running on it. While a Virtual Machine Manager (VMM) like QEMU can definitely emulate the speaker and display of the VM, it prevents us from harnessing the full potential of the actual hardware of the Host, as that is being controlled by native (host) drivers. The solution to that is to create **a "bridge" that enables the app to run on Linux as usual, but effectively use the video and audio drivers of the host**, as if the app were running on Redox OS itself. This also enables a seamless experience for the user, as they're unaware of where the app is actually being run. +Now let's look at the VM; we can see that it is running Linux, and there's an app compiled for Linux (like a GTK/Qt app) running on it. While a Virtual Machine Manager (VMM) like QEMU can definitely emulate the speaker and display of the VM, it prevents us from harnessing the full potential of the actual hardware of the Host, as host hardware is being controlled by host drivers. The solution to that is to create **a "bridge" that enables the app to run on Linux as usual, but effectively use the video and audio drivers of the host**, as if the app were running on Redox OS itself. This also enables a seamless experience for the user, as they're unaware of where the app is actually being run. <br/> ### A matrix of apps and drivers on various OSes -If you look at the above two solutions, you'll notice that there's a "hidden" feature that comes as a side effect of implementing the above solutions - we can use multiple OSes to run various drivers and multiple OSes to run various apps. The immediate idea that comes to mind is that of **security through isolation**. Have a look to the figure below: +If you look at the two solutions above, you'll notice that there's a "hidden" feature that comes as a side effect of implementing the above solutions - we can use multiple OSes to run various drivers and multiple OSes to run various apps. The immediate idea that comes to mind is that of **security through isolation**. Have a look to the figure below: <center> <img class="img-responsive" src="/img/rsoc-2023-eny/multi-app-driver-vm.svg" alt="Apps and drivers of various OSes executing in various VMs and interacting with each other seamlessly" />Fig 3. Apps and drivers of various OSes executing in various VMs and interacting with each other seamlessly: </center> <br/> -What you see in the above figure is a system where the "host" is a Hypervisor (known as a Type-1 hypervisor) that only manages virtual machines, their scheduling, permission, security and inter-VM communication. And on top of that, you see a "DOM_P" virtual machine, which is has the privilege required to control the entire system and all the VMs on it, and then there are many other non-privileged VMs, where each VM is either running an app or a driver, where each driver controls some hardware, and each apps uses these drivers to accomplish its task. This is essentially a "matrix" where everything is isolated. +What you see in the above figure is a system where the "host" is a Hypervisor (known as a Type-1 hypervisor) that only manages virtual machines, their scheduling, permission, security and inter-VM communication. And on top of that, you see a "DOM_P" virtual machine, which has the privilege required to control the entire system and all the VMs on it, and then there are many other non-privileged VMs, where each VM is either running an app or a driver, where each driver controls some hardware, and each app uses one or more drivers to accomplish its task. This is essentially a "matrix" of VMs with apps and drivers, where everything is isolated. ## The implementation -You may have noticed that I didn't go into any highly technical details in here. That's because I wanted to keep this simple and non-technical, so as _most_ technology enthusiasts can get a grasp on what's happening without having to sit through all the technical (and not so interesting) aspects of it. +You may have noticed that I didn't go into any highly technical details in here. That's because I wanted to keep this simple and non-technical, so that _most_ technology enthusiasts can get a grasp on what's happening without having to sift through all the technical aspects of it. But fear not! if you're interested in peeking behind the curtain to have a look at how this is going to be implemented, what techniques are being considered, and wanna keep tabs on the unravelling of a thrilling adventure of creating something cool, then I'll be creating noe or more _**INSIDER's**_ post in the coming weeks, where we get down to the nitty-gritty of things. @@ -73,33 +75,33 @@ I'll make a few important points here: - I'd like to point out that this is _not_ a unique solution. It is inspired by other technologies, especially 'Qubes OS' and 'Virtio', which, in some cases, do very similar things. - While we'll be implementing things differently in many cases, it will achieve some of the same effects, and we might even utilize Virtio to implement parts of the design, as it may be the perfect way to do this. - We're not looking to make something novel; the idea is to simply build something that can solve a problem that we're facing, and to build it well. -- The implementation of the 'simple' idea outlined above isn't as straightforward as it looks, as it involves a lot of parts coming to work together. -- This design will most definitely have some overhead, as we're having multiple OSes in VM interact with each other to perform a task which is generally supposed to be done completely on one system. -- The feasibility of this design also heavily depends on the ability to implement the various _frontends_ and _backends_ for various drivers and operating systems, which might be difficult to do (and further maintain), thus limiting the scope of how useful this endeavor will turn out to be. +- The implementation of the 'simple' idea outlined above isn't as straightforward as it looks, as it involves a lot of parts coming to work together. +- This design will most definitely have some overhead, as we're having multiple OSes in VMs interact with each other to perform a task which is generally supposed to be done completely on one system. +- The feasibility of this design also heavily depends on the ability to implement the various _frontends_ and _backends_ for various drivers and operating systems, which might be difficult to do (and further maintain), thus limiting the scope of how useful this endeavor will turn out to be. <br/> ### Current status The implementation of the above idea will require work on the following features: -1. evolving the revirt interface (which is like the kvm interface) -2. Revirt-U : integrating QEMU with revirt (similar to how QEMU integrates with KVM) -3. creating a flexible framework that can be used to create "bridges" between Redox-host and Linux-guest. (virtio might come in handy) -4. _using_ the above framework to provide support for well-known hardware which have Linux driver support (like the Nvidia proprietary driver) - this is the main goal -5. Revirt-K - the type-1 hypervisor which will provide the ability to create the "matrix" configuration above - +1. evolving the revirt interface (which is like the kvm interface) +2. Revirt-U : integrating QEMU with revirt (similar to how QEMU integrates with KVM) +3. creating a flexible framework that can be used to create "bridges" between Redox-host and Linux-guest. (virtio might come in handy) +4. _using_ the above framework to provide support for well-known hardware which have Linux driver support (like the Nvidia proprietary driver) - this is the main goal +5. Revirt-K - the type-1 hypervisor which will provide the ability to create the "matrix" configuration above + The above tasks cannot exactly be completed in a step-by-step manner, as many of the tasks are dependent on the other tasks. So, I have been making progress on various features up to various extents. I keep switching tasks and working on things simultaneously, which can slow me down a bit. These features not only involve coding, but also the design and architecture and the decision on which components to use, which unfortunately takes time. I'll be back with the **INSIDER** post with some demos and details of implementation and progress made. <br/> ### Expected outcome -I'd like to bring this post to an end by summarizing the key expectations, some of which are more feasible than the others: -- Broader driver and app support for Redox -- Ability to execute drivers and apps in isolation (on VMs, secure, resilient) -- seamless integration of apps and drivers across OSes with "one-click-deploy" of Linux or Windows apps, drivers, etc. with good integration support +I'd like to bring this post to an end by summarizing the key expectations, some of which are more feasible than the others: +- Broader driver and app support for Redox +- Ability to execute drivers and apps in isolation (on VMs) making them secure and the entire system resilient +- seamless integration of apps and drivers across OSes with "one-click-deploy" of Linux or Windows apps, drivers, etc. with good (seamless) integration with Redox OS -A personal deliverable that I want to keep in sight - Redox as a host, running linux (pop-os nvidia version) as a guest on QEMU (with revirt), where apps on Redox that use opengl graphics will be able to use the nvidia graphics card on my machine. +A personal deliverable that I want to keep in sight - Using Redox OS as a host, running Linux (Pop!_OS - Nvidia version) as a guest on QEMU virtual machine (backed by Revirt-U), where apps on Redox (that use OpenGL graphics) will be able to use the Nvidia graphics card on my machine by using the Nvidia drivers that are running on the Linux VM (and controlling the Nvidia GPU). ## Epilogue