redox issueshttps://gitlab.redox-os.org/redox-os/redox/-/issues2024-03-26T06:13:11Zhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1435Add driver customization2024-03-26T06:13:11ZRibbonAdd driver customizationDrivers are the most big part of an operating system and the average computer only use some of them.
This proposal is to add or remove individual drivers from a filesystem image (`config/*.toml`) by using this syntax:
```toml
[drivers]...Drivers are the most big part of an operating system and the average computer only use some of them.
This proposal is to add or remove individual drivers from a filesystem image (`config/*.toml`) by using this syntax:
```toml
[drivers] # A new section for driver entries
driver1 = {}
driver2 = {}
#driver3 = {} # Disabled driver
driver4 = "none" # Another way to disable a driver (if necessary)
```https://gitlab.redox-os.org/redox-os/redox/-/issues/1428Rework the graphics handling2024-02-13T17:44:36Zbjorn3Rework the graphics handling* [x] Move text console support out of vesad to allow other graphics drivers to reuse it. (https://gitlab.redox-os.org/redox-os/drivers/-/merge_requests/128)
* [ ] Better IPC serialization/deserialization. The current ad-hoc code is erro...* [x] Move text console support out of vesad to allow other graphics drivers to reuse it. (https://gitlab.redox-os.org/redox-os/drivers/-/merge_requests/128)
* [ ] Better IPC serialization/deserialization. The current ad-hoc code is error prone and hard to keep in sync between components as you need to change everything in lockstep. Something like [Cap'n Proto](https://capnproto.org/) ([Rust implementation](https://github.com/capnproto/capnproto-rust)) may be a better fit. It should allow keeping compatibility with older programs even when we add and remove commands.
* [ ] Move the console multiplexing out of the individual graphics drivers. Instead have a protocol that allows swapping the framebuffer to be rendered by the graphics driver and then have an fbmuxd daemon which saves the current framebuffer for every virtual terminal and tells the graphics driver to swap it out whenever either switching between vt's or when the client requests the framebuffer to be swapped. The vga/vesa driver can then write it to the actual framebuffer while the virtio-gpu driver in the future could request the virtio-gpu device to change which framebuffer to use for scanout once it becomes possible for users to directly create buffers on the gpu. Fbmuxd would also handle multiplexing of input devices.
* [ ] Support for hardware backed cursor. This both decreases latency and improves energy efficiency by offloading rendering of the cursor to the display controller (on mobile devices this is an entirely separate part of the SoC by the way). Not every gpu and gpu interfaces support this though. VGA doesn't support it, but virtio-gpu does and so do all desktop gpu's.
* [ ] Allow delegating control over a virtual terminal to a different program. This would allow for example the login screen to delegate control to an instance of orbital running as the logged in user, reducing attack surface. Or for a VR game to get full control over page flipping and mode setting. It should also be possible for the delegating program to revoke access again for all the (in)directly delegated programs. This delegation should happen for both the display and input devices. Preferrably with an option for the delegating program to keep accepting input such that it can for example listen for Super+L to lock the screen and show a password prompt for the current user.
* [ ] Support mode setting. That is changing display resolutions, display mirroring, selecting which part of a buffer to show, using multiple planes aside from the hardware backed cursor.
* [ ] Allow creating buffers on the gpu and running commands like shader invocations on them. Initially just virgl (OpenGL) or venus (Vulkan) support for virtio-gpu would work. This final step would allow actual hardware acceleration.https://gitlab.redox-os.org/redox-os/redox/-/issues/1427Wayland porting2024-01-23T20:45:03ZRibbonWayland portingThis issue will track how and what is needed to port Wayland to Redox.
The main difference between X11 and Wayland is the architecture and communication, on X11 you have a server and clients, on Wayland the server and client is one prog...This issue will track how and what is needed to port Wayland to Redox.
The main difference between X11 and Wayland is the architecture and communication, on X11 you have a server and clients, on Wayland the server and client is one program (the compositor), each Wayland compositor has the freedom to implement its rendering and input interfaces.
(In both display servers the user programs are clients)
### Porting Items
- [ ] Implement the Unix domain sockets
- [ ] Support for passing memfd or posix shared memory fd's over the socket
- [ ] Port libwayland
- [ ] Port cosmic-comp from COSMIC Desktop (Jeremy is doing that)
- [ ] Port Smithay
- [ ] Port wlroots (Sway and others)
- [ ] Port KWin from KDE Plasma
- [ ] Port Mutter from GNOMEhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1419Implement an out-of-memory killer daemon2023-12-02T16:44:20ZRibbonImplement an out-of-memory killer daemonOn Linux you have two types of OOM killers, kernel and user-space, as the kernel OOM killer can be slow to act, most Linux distributions tend to use an user-space OOM killer, in most cases the systemd-oom daemon.
Our OOM killer daemon c...On Linux you have two types of OOM killers, kernel and user-space, as the kernel OOM killer can be slow to act, most Linux distributions tend to use an user-space OOM killer, in most cases the systemd-oom daemon.
Our OOM killer daemon could be called `oomkd` or `oomd`.
### References
- [system-oom](https://www.man7.org/linux/man-pages/man8/systemd-oomd.8.html) - Used in most Linux distributions.
- [earlyoom](https://github.com/rfjakob/earlyoom) - Proposed for Fedora.
- [bustd](https://github.com/vrmiguel/bustd) - An user-space OOM killer written in Rust.https://gitlab.redox-os.org/redox-os/redox/-/issues/1418Multiseat support2024-01-12T12:57:21ZRibbonMultiseat supportA [multiseat](https://en.wikipedia.org/wiki/Multiseat_configuration) device is a computer with multiple ports for multiple peripherals linked to independent users of the operating system.
This issue will cover the implementation of this...A [multiseat](https://en.wikipedia.org/wiki/Multiseat_configuration) device is a computer with multiple ports for multiple peripherals linked to independent users of the operating system.
This issue will cover the implementation of this use case.
### Conditions
- Each user files, processes and devices must be isolated from other users.
- An user can't read the framebuffer of other user.
- An user can't use a wrong device.
### Proposal
(Needs more brainstorming)
- Each "seat" is a workspace for a monitor and HID devices connected to a USB hub.
- Each monitor will have a TTY with a login screen (start `orblogin` if available).
- Follow the USB hub ports position to avoid wrong device allocation.
- Use the TTY number of the monitor to assign devices.https://gitlab.redox-os.org/redox-os/redox/-/issues/1417Implement the "make prefix compiler" command2024-01-13T07:07:30ZRibbonImplement the "make prefix compiler" commandThe [proposal](https://gitlab.redox-os.org/redox-os/redox/-/issues/1389) to move the rustc source fetch to the `make prefix` command is redundant because it's called with `make all`, you would clone the rustc sources later.
My solution ...The [proposal](https://gitlab.redox-os.org/redox-os/redox/-/issues/1389) to move the rustc source fetch to the `make prefix` command is redundant because it's called with `make all`, you would clone the rustc sources later.
My solution is to add a specific command to fetch and build from sources (the CI server need this), see the current behavior:
- `bootstrap.sh` - rustc is a submodule, thus it clone the sources on each build system copy.
- `make prefix` - It download the toolchain binaries from the CI server and build relibc.
My proposed behavior:
- `make prefix compiler` - Active the `PREFIX_BINARY=0`, download the rustc/GCC sources and build the toolchain (`bootstrap.sh` don't fetch the rustc sources anymore).https://gitlab.redox-os.org/redox-os/redox/-/issues/1412Kernel and userspace separation policy2024-01-13T08:34:40ZRibbonKernel and userspace separation policyThis issue covers which components may be moved to userspace, or should continue stay in the kernel.
# Kernel
- CPU drivers - performance
- Memory manager - performance, and moving that to userspace is hard
- Context switch - core part...This issue covers which components may be moved to userspace, or should continue stay in the kernel.
# Kernel
- CPU drivers - performance
- Memory manager - performance, and moving that to userspace is hard
- Context switch - core part of the kernel
- Capability system - the kernel needs to implement at least the bare minimum to be able to isolate processes from each other
- Scheduler - performance
# Userspace currently
- cwd handling
- initfs
- fork and execv
- most drivers
- filesystem
- everything else
# Possible to move to userspace
- RTC driver
- path parsing (would require openat)
- the current namespace implementation (would also require openat)
- (part of) the file descriptor table (descriptors, not descriptions) - allows optimizing lseek, fcntl, POSIX pipes, etc.
- process management, including uid/gid/(not yet implemented:) supplementary groups
- application processor startup
- file descriptor allocation (where the lowest available number must be picked)
- virtual memory range allocationhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1405(Open to discussion) Filesystem sandbox system2023-10-16T10:38:16ZRibbon(Open to discussion) Filesystem sandbox systemFilesystem sandbox is very important to avoid data attacks/leaks and user-space rootkits, each process has its own namespace with schemes (use `ls :` to verify the current namespace).
The best security model is to let the user decide wh...Filesystem sandbox is very important to avoid data attacks/leaks and user-space rootkits, each process has its own namespace with schemes (use `ls :` to verify the current namespace).
The best security model is to let the user decide which folders/files some program can access (like recent Android versions), it can be done in two ways:
- When the user open a new program, the windowing system will ask which folders/files the program can access (similar to the xdg-portal window for file selection on XDG-compatible programs).
- The user can specify which folders/files some program can access using a TOML configuration (GUI/terminal).
## Terminal
In most cases the folders/files on terminal programs are specified on the command, thus it's a form of permission.
But the attacker can install an user-space rootkit using commands or a script, to mitigate this a permission prompt could be implemented for file operations on the terminal.
## Implementation
This will be done with scheme filters, the process can use a directory scheme (`home:` for `/home`) or the `file:` scheme directly.
Example:
- `home:path/to/folder`
- `file:/path/to/folder`https://gitlab.redox-os.org/redox-os/redox/-/issues/1404(Open to discussion) Permission system2024-03-26T21:38:17ZRibbon(Open to discussion) Permission systemA good security system has program/filesystem sandbox, this issue will cover the program sandbox implementation.
The permission configuration will require user or admin password (like any OS) but without `sudo`, because it's prone to ab...A good security system has program/filesystem sandbox, this issue will cover the program sandbox implementation.
The permission configuration will require user or admin password (like any OS) but without `sudo`, because it's prone to abuses (unsafe).
### Schemes
Each kernel subsystem or user-space daemon implement schemes for communication, programs will use these schemes to work with different areas of the system.
### Security models
- Pre-configured permissions
The Redox developers will decide which permissions (schemes) a program package will have by default.
- Manual permissions
The user will give permissions to the program.
### Terminal
Most terminals use `sudo` for privileged tasks, it's prone to abuses and won't be used.
### GUI
If Orbital is available a permission window will pop up (like Windows and Android) for some program.
### Implementations
- Group-based
The program user will need to be a member of some resource group (`audio` and `video` for example), these groups use schemes.
- Configuration-based
A global TOML file will decide which schemes the programs can use (it needs to be read-only to avoid abuses).https://gitlab.redox-os.org/redox-os/redox/-/issues/1394CI testing for packages2023-10-27T13:44:24ZRibbonCI testing for packagesUsing CI tests for packages we can have a stable rolling release system, Void Linux does this, being one of the most stable rolling release Linux distributions of the world.
- [Void Linux continuous integration system](https://build.voi...Using CI tests for packages we can have a stable rolling release system, Void Linux does this, being one of the most stable rolling release Linux distributions of the world.
- [Void Linux continuous integration system](https://build.voidlinux.org/)https://gitlab.redox-os.org/redox-os/redox/-/issues/1393Exploit mitigations2023-10-27T11:07:14ZRibbonExploit mitigationsThis issue will cover the mandatory and optional exploit mitigations for Redox.
As Redox is written in Rust there's memory-safety though the compiler, thus it might make sense to disable most exploit mitigations, at least by default.
B...This issue will cover the mandatory and optional exploit mitigations for Redox.
As Redox is written in Rust there's memory-safety though the compiler, thus it might make sense to disable most exploit mitigations, at least by default.
But Redox is not fully safe Rust because many crates rely on less thoroughly checked unsafe, such as when heavily using FFI. Therefore, it might make more sense to enable mitigations for such programs, as well as for C/C++ programs, even though unsafe Rust is generally still safer than C/C++.
Exploit mitigations specific to C/C++ memory errors are unnecessary for safe Rust code.
### Criteria
The exploit mitigations on this issue must follow some criteria.
1. The mitigation is needed for microkernel-based systems?
2. If the mitigation is cheap (low performance penault) and the security benefit is considerable, it can be enabled by default.
3. Classify if it's a compiler, manual system-wide/some programs or x86-specific mitigation.
### Mitigations
This list will filter the exploit mitigations.
- Address Space Layout Randomization - userspace
- ~~Kernel Address Space Layout Randomization~~ probably overkill for a microkernel; seL4 for example uses `-static -fno-pie -fno-pic`
- Position-Independent Executables - compiler-based
- RELRO
- BIND_NOW
- SEGVGUARD (ASLR brute force protection)
- W^X (memory mappings and switching pages)
- SROP
- Trusted Path Execution
- SafeStack
- Non-Cross-DSO Control-Flow Integrity
- Retpoline - Spectre mitigation for monolithic kernels?
### Implementations
This list will track the exploit mitigations implementation.
- [x] SMAP - x86-specific, manual system-wide
- [x] SMEP - x86-specific, manual system-wide
- [x] Zero-initialized userspace stack - manual system-wide
- [ ] Read-only pages where necessary - manual?
- [ ] IP ID randomization - netstack or smoltcp? manual.
- [ ] Temporary IPv6 addresses - netstack or smoltcp? manual.https://gitlab.redox-os.org/redox-os/redox/-/issues/1392Dev room2024-03-18T00:02:32ZRibbonDev roomThis issue cover what is necessary for the dev variant of Redox.
- [x] Port Rust
- [x] Port GCC
- [x] Port LLVM
- [ ] Port Python
- [ ] Port Go
- [x] Port Vim
- [ ] Port Neovim
- [ ] Port Emacs
- [ ] Port zsh
- [ ] Port fish
- [ ] Port ...This issue cover what is necessary for the dev variant of Redox.
- [x] Port Rust
- [x] Port GCC
- [x] Port LLVM
- [ ] Port Python
- [ ] Port Go
- [x] Port Vim
- [ ] Port Neovim
- [ ] Port Emacs
- [ ] Port zsh
- [ ] Port fish
- [ ] Port Nushell (WIP)
- [ ] Port GDB
- [ ] Port NodeJS
- [ ] Port OpenSSHhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1391Desktop room2024-03-17T23:59:15ZRibbonDesktop roomThis issue cover what is necessary for the desktop variant of Redox.
- [ ] Port GTK
- [ ] Port Qt
- [ ] Port Firefox
- [ ] Port VLC
- [ ] Port Servo
- [ ] Port Chromium (hard)
- [ ] Port WebAssembly
- [ ] Implement GPU acceleration (thr...This issue cover what is necessary for the desktop variant of Redox.
- [ ] Port GTK
- [ ] Port Qt
- [ ] Port Firefox
- [ ] Port VLC
- [ ] Port Servo
- [ ] Port Chromium (hard)
- [ ] Port WebAssembly
- [ ] Implement GPU acceleration (through Linux driver VMs on QEMU)
- [ ] Port COSMIC Desktop
- [ ] Port GNOME (time-consuming)
- [ ] Port KDE Plasma (time-consuming)
- [ ] Port Swayhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1387Feature requests index2024-02-10T18:35:35ZRibbonFeature requests indexThis issue will cover the open feature requests of Redox GitLab, send a comment if you want to see your feature request on the list.
- [ ] [Recipe categories on the Cookbook configuration](https://gitlab.redox-os.org/redox-os/redox/-/is...This issue will cover the open feature requests of Redox GitLab, send a comment if you want to see your feature request on the list.
- [ ] [Recipe categories on the Cookbook configuration](https://gitlab.redox-os.org/redox-os/redox/-/issues/1395) - build system
- [x] [Add an option for make to delete the recipe source](https://gitlab.redox-os.org/redox-os/redox/-/issues/1386) - build system
- [x] [Add an option to ignore recipe errors](https://gitlab.redox-os.org/redox-os/redox/-/issues/1416) - build system
- [x] [Enable the recipe binary syntax by default](https://gitlab.redox-os.org/redox-os/redox/-/issues/1401) - build system
- [x] [Recipe override list](https://gitlab.redox-os.org/redox-os/redox/-/issues/1402) - build system
- [x] [Dark theme by default](https://gitlab.redox-os.org/redox-os/website/-/issues/192) - website
- [ ] [Provide VM images for new versions](https://gitlab.redox-os.org/redox-os/redox/-/issues/1388) - CI server
- [ ] [CI testing for packages](https://gitlab.redox-os.org/redox-os/redox/-/issues/1394) - CI server
- [ ] [memtest86+ for the Redox bootloader](https://gitlab.redox-os.org/redox-os/redox/-/issues/1397) - recipe
- [ ] [Automatic operating system detection on boot loader](https://gitlab.redox-os.org/redox-os/redox/-/issues/1407) - boot loaderhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1384Tracking issues index2024-03-28T11:00:41ZRibbonTracking issues indexThis issue cover the current tracking issues of Redox.
- [ ] [Implement the missing POSIX APIs on relibc](https://gitlab.redox-os.org/redox-os/relibc/-/issues/173) - high priority
- [ ] [Kernel/userspace separation policy](https://gitla...This issue cover the current tracking issues of Redox.
- [ ] [Implement the missing POSIX APIs on relibc](https://gitlab.redox-os.org/redox-os/relibc/-/issues/173) - high priority
- [ ] [Kernel/userspace separation policy](https://gitlab.redox-os.org/redox-os/redox/-/issues/1412) - medium priority
- [ ] [Repositories with missing GitLab CI](https://gitlab.redox-os.org/redox-os/redox/-/issues/1377) - high priority
- [ ] [Self-hosting status](https://gitlab.redox-os.org/redox-os/redox/-/issues/1390) - medium priority
- [ ] [Community hardware device porting](https://gitlab.redox-os.org/redox-os/drivers/-/issues/40) - medium priority
- [ ] [Port the most used crates of the Rust ecossystem](https://gitlab.redox-os.org/redox-os/redox/-/issues/1378) - medium priority
- [ ] [Rustify relibc](https://gitlab.redox-os.org/redox-os/relibc/-/issues/178) - medium priority
- [ ] [Forks status](https://gitlab.redox-os.org/redox-os/redox/-/issues/1380) - low priority
- [ ] [Linux driver VM optimizations](https://gitlab.redox-os.org/redox-os/redox/-/issues/1382) - low priority
- [ ] [Linux app VMs](https://gitlab.redox-os.org/redox-os/redox/-/issues/1383) - low priority
- [ ] [RedoxFS tooling](https://gitlab.redox-os.org/redox-os/redoxfs/-/issues/42) - low priority
- [ ] [Convert all legacy recipes to TOML](https://gitlab.redox-os.org/redox-os/cookbook/-/issues/174) - low priority
- [ ] [Desktop improvements](https://gitlab.redox-os.org/redox-os/redox/-/issues/1391) - low priority
- [ ] [Server improvements](https://gitlab.redox-os.org/redox-os/redox/-/issues/1375) - medium priority
- [ ] [Dev improvements](https://gitlab.redox-os.org/redox-os/redox/-/issues/1392) - medium priority
- [ ] [Exploit mitigations](https://gitlab.redox-os.org/redox-os/redox/-/issues/1393) - low priority
- [ ] [Orbital improvements](https://gitlab.redox-os.org/redox-os/redox/-/issues/1430) - low priority
- [ ] [Feature requests index](https://gitlab.redox-os.org/redox-os/redox/-/issues/1387) - low priorityhttps://gitlab.redox-os.org/redox-os/redox/-/issues/1383Linux app VMs2023-07-08T02:30:22ZRibbonLinux app VMsSimilar to the Linux driver VMs [issue](https://gitlab.redox-os.org/redox-os/redox/-/issues/1382), the Linux app VMs will make Linux-only programs run on Redox.
Why? some programs rely on Linux kernel technologies/interfaces to work, li...Similar to the Linux driver VMs [issue](https://gitlab.redox-os.org/redox-os/redox/-/issues/1382), the Linux app VMs will make Linux-only programs run on Redox.
Why? some programs rely on Linux kernel technologies/interfaces to work, like containers and filesystems.
FreeBSD and NetBSD have a Linux compatibility layer to run unmodified Linux binaries using their ported Linux kernel API, it's possible because the BSDs design is similar to Linux, thus a kernel module is developed without massive changes.
Even with that kernel module, some Linux programs don't work, this kind of implementation can't be done on Redox without massive changes, because it's a microkernel with much more simplicity.
Filesystems rely more on the kernel APIs than other programs, all development is focused on these APIs, even if you port a filesystem to other OS, it will be hard/time-consuming to adapt the new changes from other OS if it's a big/complex filesystem.
To mitigate this you can use a FUSE driver or a VM, FUSE is an universal API but lack some filesystems and the drivers can be unmaintained easily.
A VM will bring many filesystems and their improvements over the time with a small configuration effort, at the cost of some overhead (can be optimized).
Like the Linux driver VMs proposal, the Linux app VMs can be simple and made for a specific use-case, we can have a specific daemon VM for Flatpak, other for Docker, other for Snappy, etc.
Each daemon VM will be optimized/built for his specific necessities, that way we reduce memory usage, CPU time, bugs and complexity.
### Daemon proposals
- [ ] flatpakd (a VM for Flatpak applications)
- [ ] snappyd (a VM for Snappy applications)
- [ ] dockerd (a VM for Docker applications)
- [ ] podmand (a VM for Podman applications)
- [ ] genericd (a VM for any application)
### Suggestions
- Flatpak, Snappy, Docker and Podman binaries can be static linked to relibc.
- Remove the GNU/Linux userland libraries/tools from VM images.
- We could have an option to have one VM per app.
Using one VM per app we will have the VM crash isolated on the program, but it will use more memory and CPU.https://gitlab.redox-os.org/redox-os/redox/-/issues/1382Linux driver VM optimizations2023-07-12T05:10:33ZRibbonLinux driver VM optimizationsThis issue will cover the optimizations that can be done to the Linux driver VMs.
Linux driver VMs are VMs created to offer driver support for Redox, see how it works:
- The Linux VM access the device though PCI passthrough (the host s...This issue will cover the optimizations that can be done to the Linux driver VMs.
Linux driver VMs are VMs created to offer driver support for Redox, see how it works:
- The Linux VM access the device though PCI passthrough (the host system doesn't need a driver, similar to Qubes OS).
- A Redox bridge program will run on Linux userspace, this bridge will communicate with some Redox host scheme based on the driver type (audio: for sound cards, video: for GPUs).
- The Linux VM will communicate with the bridge and the Redox host will control the device without native drivers.
This is a guest-to-host communication using VirtIO interfaces, because of the virtualization some overhead will exist.
Most operating systems speed up their VMs with a type-2 hypervisor running on the kernel (KVM and Hyper-V, for example), we will use Revirt-U to do that, but more optimizations can be done to reduce CPU cycles and memory usage.
- [ ] Build a separated bridge for each device type or Linux device system.
- `drmd` - GPUs.
- `netd` - network devices (Ethernet).
- `fsd` - filesystems.
- `wifid` - Wi-Fi adapters.
- `audiod` - sound devices.
- `inputd` - mouse/keyboards/gamepads/joysticks.
- `sensord` - sensor drivers.
The userspace part can be minimal or even empty, depending on whether a kernel module is needed/beneficial for communication with the bridged device. Userspace can be empty besides init, which might not have to do anything useful either depending on much how much of the bridging is done by a kernel module.
- [ ] Use a real-time scheduler.
Linux CFS have a multitasking design in mind, as our Linux driver VMs only run the Redox bridge, as well as a few background kthreads, so we can use a monotasking low-latency scheduler to have maximum performance.https://gitlab.redox-os.org/redox-os/redox/-/issues/1242Better differentiation for relative vs absolute file paths2023-10-27T13:59:50ZAudrey Yeena ToskinBetter differentiation for relative vs absolute file pathsI guess I'm not sure if this is an issue for Ion specifically or Redox more generally... But I feel like Redox expanding the "Everything is a file" concept to be "Everything is a URL" means that it's in a good position to fix a very comm...I guess I'm not sure if this is an issue for Ion specifically or Redox more generally... But I feel like Redox expanding the "Everything is a file" concept to be "Everything is a URL" means that it's in a good position to fix a very common user mistake.
The relative file paths `./file` and just `file` are both only a single character away from the absolute file path `/file`. So in basically every existing shell language, accidentally deleting or overwriting a file under root instead of the current working directory, or vice versa, is a fairly common mistake. The typo becomes more dangerous when combined with globs -- accidentally writing `./fiel` instead of `./file` will probably produce a "no such file" error, but `rm -rf /*` and `rm -rf ./*` will both always work. Obviously, it is up to the terminal user and script writer to proofread their own commands, but since this is such a common *and dangerous* mistake, it seems like it would have been a good idea to make sure relative and absolute paths have a greater difference than the absence or presence of a small single dot. Redox and Rust care a lot about code "safety" -- trying to avoid inevitably common typos maybe isn't quite what they meant, but I do feel like improving the safety of shell code in this area is at least a closely related idea.
I'm imagining file prefixes like `here:`, `relative:`, or `cwd:` for relative paths, and `root:`, `absolute:`, or `file:` for absolute paths, but you may think of something better. *Removing* the `./` and `/` file path prefixes might be too controversial, I dunno, but having something like `cwd:` and `root:` or similar, at least as an option, would be nice.https://gitlab.redox-os.org/redox-os/redox/-/issues/1241Lint and test the codebase with gitlab CI2023-06-13T04:10:27ZThomas LaferriereLint and test the codebase with gitlab CIAdding a `gitlab-ci` job for linting the code at every commit and running automated tests using `gitlab-ci` would be a great improvement for developer productivity and would ensure constant quality assurance. At the moment the `.gitlab-c...Adding a `gitlab-ci` job for linting the code at every commit and running automated tests using `gitlab-ci` would be a great improvement for developer productivity and would ensure constant quality assurance. At the moment the `.gitlab-ci.yml` file is only used to generate the `.iso` images. Setting up runners would probably be necessary for cross-platform testing. It could use clippy for this purpose relating this to #351 or simply run `rustfmt` as suggested in the [docs' style page](https://doc.redox-os.org/book/contributing/best_practices/style.html). I've created this issue in response to that page.https://gitlab.redox-os.org/redox-os/redox/-/issues/927dynamic linking support2023-06-13T04:42:46ZJeremy Sollerdynamic linking support*Created by: m4b*
Currently redox requires statically linked ELF binaries.
So first question, are there any plans to support dynamic linking?
If so, would you consider using/adopting [dryad](https://github.com/m4b/dryad) ? In short, ...*Created by: m4b*
Currently redox requires statically linked ELF binaries.
So first question, are there any plans to support dynamic linking?
If so, would you consider using/adopting [dryad](https://github.com/m4b/dryad) ? In short, I would love to see a home/use for it, considering all the work that went into it, and would be willing to help port it over to Redox/mentor anyone who wants to help on that front, once a direction is decided.
The long is I'll give some motivations for adopting it as the dynamic linker if there are such plans, and some difficulties/considerations to be made about creating a dynamic linking platform.
Dryad started as an experiment to write a parallel dynamic linker (I still thinks this is possible) and also just because I was interested in their design. I originally targeted only x86-64, but since have added support for 32-bit arm (which I've successfully tested on my arm phone), as well as 64-bit arm (unfortunately I cannot currently link it properly (missing symbols), but I suspect it will work with some tweaking). And by successful, I mean it segfaults ;)
Along the way, I encountered serious technical issues deep in GNU's libc/ld-linux-so-x86_64 which made development difficult (and uninteresting) - as I would essentially be required to copy the unspecified ABI of glibc's internal structs, in particular, `rtld_global_ro` - let alone any other libc. The common theme here appears that most libc's cache results from locales, `auxv` accesses, and sometimes environment variable pointers in a secret, non-standardized struct that the dynamic linker has access to. But this pervasiveness goes even further - in recent versions of my glibc, (2.25), the `GNU_IFUNC` resolver for `__exp_finite` accesses the dynamic linker struct (which has not been initialized because the dynamic linker isn't the glibc dynamic linker), and hence segfaults. There's really no way to know that any arbitrary function in glibc (which can access the `rtld_*` structs because it's built in the same compilation unit as the dynamic linker) is going to rely on this private, practically uninitializable struct.
Consequently, I largely stopped working on `dryad`, except for keeping it up to date and still compiling. If you do anything, _do not make this mistake_ that glibc made. At the very least, specify the structure and expected runtime values required to initialize and load any libraries, similar to the PLT/GOT specification. They fused their libc implementation with their dynamic linking implementation, for seemingly for no particularly good reason. (musl libc's intentionally does this - musl libc binary _is_ the dynamic linker too). Here is a more in depth, fairly sarcastic and tongue-in-cheek investigation of this issue: https://github.com/m4b/dryad/issues/5
Nevertheless, in terms of features, I think it is quite robust:
0. Correctly relocates and can run itself (e.g., `./dryad.so.1`)
1. multiple architectures (someone needs to write `AARCH64` assembly stubs for runtime resolution, as well asm stubs for `i386`)
2. gdb support (this was particularly vexing as there was no documentation on how to do this)
3. runtime (aka lazy) symbol resolution
4. partial (read currently non-working) TLS initializer/implementation - largely adopted from glibc/musl. I just use the internal musl implementation for now
5. Partial support for dynamic linking environment variables, e.g., `LD_DEBUG`, `LD_BIND_NOW`, etc.
6. All major relocation directives are supported, including `GNU_IFUNC` (more exotic x86 relocations are ignored, but most of these aren't emitted by any major toolchain anymore)
7. Fairly painless rustup + makefile compilation (this is substantially more work than you would think, see the `Makefile` if you don't believe me)
8. Uses gnu hashes/bloom filter for symbol resolution which makes it extremely fast (probably need to also support regular hashes, but I wasn't that interested since every linux linker emits GNU_HASH dynamic entries)
If this sounds good I would be interested in helping get dryad ported to work on a redox system; this will be some work, as there are a few linux assumptions made, but I don't think that in particular will be the difficult part.
The first issue will be hammering down the TLS implementation, as this is potentially the most OS-based assumption (besides `auxv` accesses, or how the so-called "kernel block" is setup). Ideally, I would like the implementation in `tls.rs` to have essentially a single entry-point across multiple systems, where the implementation is switched at compile time depending on the OS target. This particular area was/still is a WIP, and currently I just default to using the statically linked musl initializer.
I haven't looked at redox, so I don't know what requirements are here, how it sets up the the TLV, TP, etc. But since it seems able to run x86 ELF binaries (which I assume must be accessing the re-appropriated `fs` or `gs` segmented registers for TLS), it is perhaps the same?
The second issue is essentially what kind of runtime system library you want (e.g., a libc), and in the case of reusing glibc, what to do about the `rtld_*` problem. If you do end up just using glibc, then you are likely committed to using their dynamic linker (unless someone feels like cloning the ABI of libc and having dryad pretend it is `ld-linux`)
This last issue is the hard part I believe, and it's really more of an infrastructure issue/planning/what kind of system do you want to build problem, and all the pros and cons associated with any decision thereof.
Anyway, I've typed enough; let me know what you think. My feelings won't be hurt if you do want to implement dynamic linking, but don't want to use dryad as your dynamic linker. I'm just looking for a home for all the code that was written at this point, but if not, that's ok too :)