Commit 9abfdf23 authored by Elijah Caine's avatar Elijah Caine Committed by Elijah C. Voigt
Browse files

Many tweaks to the Redox Book

The changes include:
- Editing for flow and understandability.
- Unifying how links are included in the source files.
- Adding an `About this Book` page to the introduction.
- Adding a few examples or new pieces of information I felt were
  lacking when I read the book.
- Unifying 'user space' and 'kernel space' spelling.
- Added myself to the authors list in `overview/welcome.md`.
parent a66b46ac
......@@ -16,6 +16,7 @@
- [How Redox compares to other operating systems](./introduction/how_redox_compares_to_other_operating_systems.md)
- [The target of Redox]()
- [Will Redox replace Linux?](./introduction/will_redox_replace_linux.md)
- [About this Book](./introduction/about_this_book.md)
- [Getting started]()
- [Preparing Redox](./getting_started/preparing_the_build.md)
......@@ -87,8 +88,8 @@
- [What it is]()
- [Understanding the design]()
- [Using Sodium efficiently]()
- [Development in userspace]()
- [What userspace is]()
- [Development in user space]()
- [What user space is]()
- [Writing an application for Redox]()
- [Platform independence]()
- [What is available and what is not]()
......
The design of Redox
===================
This chapter will go over the design of Redox: the kernel, the userspace, the ecosystem, the trade-offs and much more.
This chapter will go over the design of Redox: the kernel, the user space, the ecosystem, the trade-offs and much more.
Advantages of microkernels
==========================
Arguably, there are quite a lot advantages (and disadvantages too!) of microkernels. These will briefly be discussed here:
There are quite a lot advantages (and disadvantages!) to microkernels, a few of which will be covered here.
Modularity and customizability
------------------------------
Monolithic kernels are, well, monolithic. They do not allow as fine-grained control as microkernels. This is due to many essential components being "hard-coded" into the kernel, and thus requiring modifications to the kernel itself.
Monolithic kernels are, well, monolithic. They do not allow as fine-grained control as microkernels. This is due to many essential components being "hard-coded" into the kernel, and thus requiring modifications to the kernel itself (e.g., device drivers).
Microkernels are very modular by nature. You can replace, reload, modify, change, and remove modules, on runtime, without even touching the kernel.
Modern monolithic kernels try to solve this issue, using kernel modules, but do often still require the system to reboot.
Modern monolithic kernels try to solve this issue using kernel modules but still often require the system to reboot.
Security
--------
......@@ -24,8 +24,10 @@ In other words: **drivers can do whatever, without restrictions, when running in
Less crashes
------------
Monolithic kernels are, when compared to microkernels, crash-prone. Simple logic bugs can result in a crashed driver, which can, for a kernel space driver, crash the whole system.
When compared to microkernels, Monolithic kernels tend to be crash-prone. A crashed driver in a Monolithic kernel can crash the whole system whereas with a microkernel there is a separation of concerns which allows the system to handle any crashes safely.
In Linux, this is often seen by errors with drivers dereferencing bad pointers, ultimately resulting in kernel panics.
In Linux we often see errors with drivers dereferencing bad pointers which ultimately results in kernel panics.
[There is very good documentation in MINIX about how this can be addressed by a microkernel](http://wiki.minix3.org/doku.php?id=www:documentation:reliability)
[There is very good documentation in MINIX about how this can be addressed by a microkernel.]
[There is very good documentation in MINIX about how this can be addressed by a microkernel.]: http://wiki.minix3.org/doku.php?id=www:documentation:reliability
......@@ -4,13 +4,19 @@ Disadvantages of microkernels
Performance
-----------
Any modern operating system needs basic security mechanisms such as virtualization and segmentation of memory. Furthermore any process (including the kernel) has its own stack and variables stored in registers. On [context switch](https://en.wikipedia.org/wiki/Context_switch), that is each time a syscall is invoked or any other inter-process communication (IPC) is done, some tasks have to be done, including:
Any modern operating system needs basic security mechanisms such as virtualization and segmentation of memory. Furthermore any process (including the kernel) has its own stack and variables stored in registers. On [context switch], that is each time a syscall is invoked or any other inter-process communication (IPC) is done, some tasks have to be done, including:
* saving caller registers, especially the program counter (caller: process invoking syscall or IPC)
* reprogramming the [MMU](https://en.wikipedia.org/wiki/Memory_management_unit)'s page table (aka [TLB](https://en.wikipedia.org/wiki/Translation_lookaside_buffer))
* putting CPU in another mode (kernel mode, user mode)
* restoring callee registers (callee: process invoked by syscall or IPC)
* Saving caller registers, especially the program counter (caller: process invoking syscall or IPC)
* Reprogramming the [MMU]'s page table (aka [TLB])
* Putting CPU in another mode (kernel mode, user mode)
* Restoring callee registers (callee: process invoked by syscall or IPC)
These are not necessarily slower on microkernels, instead microkernels suffers from having the need to perform these operations more frequently, due to many of the system functionality is performed by userspace processes, often requiring additional context switches.
These are not inherently slower on microkernels, but microkernels suffer from having to perform these operations more frequently. Many of the system functionality is performed by user space processes, requiring additional context switches.
To this date, microkernels have marginalized the performance difference between monolithic and microkernels, making the performance comparable. This is partly due to a smaller surface area, which can be more manageable to optimize. Unfortunately, Redox isn't quite there yet, we do still have a relatively slow kernel, since not much time has been spent on optimizing it.
The performance difference between monolithic and microkernels has been marginalized over time, making their performance comparable. This is partly due to a smaller surface area which can be easier to optimize.
Unfortunately, Redox isn't quite there yet. We still have a relatively slow kernel since not much time has been spent on optimizing it.
[context switch]: https://en.wikipedia.org/wiki/Context_switch
[MMU]: https://en.wikipedia.org/wiki/Memory_management_unit
[TLB]: https://en.wikipedia.org/wiki/Translation_lookaside_buffer
The kernel of Redox
===================
The kernel of Redox largely derives from the concept of microkernels, particularly with inspiration from MINIX. This chapter will discuss the design of the Redox kernel.
The Redox kernel largely derives from the concept of microkernels, with particular inspiration from [MINIX]. This chapter will discuss the design of the Redox kernel.
[MINIX]: https://en.wikipedia.org/wiki/MINIX
Microkernels
============
As noted previously, Redox' kernel is a microkernel. Microkernels stands out in their design by providing minimal abstractions in kernel space. Microkernels do have, in contrary to monolithic kernel, great emphasis on userspace.
Redox's kernel is a microkernel. Microkernels stands out in their design by providing minimal abstractions in kernel-space. Microkernels have an emphasis on user space, unlike Monolithic kernels which have an emphasis on .
The philosophy of microkernels is essentially, that any components, which can run in user space, should run in user space. Kernel space should only be utilized for the most essential components, that is: system calls, process separation, resource management, IPC, thread management, and so on.
The basic philosophy of microkernels is that any component which *can* run in user space *should* run in user space. Kernel-space should only be utilized for the most essential components (e.g., system calls, process separation, resource management, IPC, thread management, etc).
The kernel's main task is to act as a medium for communication and segregation of processes. The kernel should provide minimal abstraction over the hardware (that is, drivers which can, should run in user mode).
The kernel's main task is to act as a medium for communication and segregation of processes. The kernel should provide minimal abstraction over the hardware (that is, drivers which can and should run in user mode).
Microkernels are more secure and less prone to crashes than monolithic kernels. This is due to drivers and other abstraction being less privileged, and thus cannot do damage to the system. Furthermore, microkernels are extremely maintainable, due to their small code size, this can potentially reduce the number of bugs in the kernel.
......@@ -16,13 +16,18 @@ Versus monolithic kernels
Monolithic kernels provide a lot more abstractions than microkernels.
![An illustration](https://upload.wikimedia.org/wikipedia/commons/6/67/OS-structure.svg)
![An illustration]
The above illustration ([from Wikimedia](https://commons.wikimedia.org/wiki/File:OS-structure.svg), by Wooptoo, License: Public domain) shows how they differ.
The above illustration ([from Wikimedia], by Wooptoo, License: Public domain) shows how they differ.
> TODO
A note on the current state
---------------------------
Currently, Redox has a 16,000 lines kernel. We would like to move certain things to userspace to get an even smaller kernel. For comparison, Minix has a 6,000 lines kernel.
Redox has ~16,000 lines of kernel code. For comparison the Minix has ~6,000 lines of kernel code.
We would like to move parts of Redox to user space to get an even smaller kernel.
[An illustration]: https://upload.wikimedia.org/wikipedia/commons/6/67/OS-structure.svg
[from Wikipedia]: https://commons.wikimedia.org/wiki/File:OS-structure.svg
......@@ -3,7 +3,7 @@ Resources
Resources are opened schemes. You can think of them like an established connection between the scheme provider and the client.
Resources are closely connected to schemes, and are sometimes intertwined with schemes. The difference between them is subtle, yet crucial.
Resources are closely connected to schemes and are sometimes intertwined with schemes. The difference between schemes and resources is subtle but important.
Resource operations
-------------------
......@@ -15,7 +15,7 @@ A resource can be defined as a data type with following methods defined on it:
3. `seek` - seek the resource. That is, move the "cursor" without writing. Many resources do not support this operation. Defaults to `EBADF`.
4. `close` - close the resource, potentially releasing a lock. Defaults to `EBADF`.
> TODO add F-operations
> TODO add F-operations.
The resource type
-----------------
......@@ -25,4 +25,4 @@ There are two types of resources:
1. File-like resources. These behave a lot like files. They act in a blocking manner; reads and writes are "buffer-like".
2. Socket-like resources. These behave like sockets. They act in a non-blocking manner; reads and writes are more "stream-like".
I will expand on this later.
> TODO Expand on this.
Schemes
=======
Schemes are the natural counter-part to URLs. As described before, URLs are opened to schemes, which can then be opened, yielding a resource.
Schemes are the natural counter-part to URLs. URLs are opened to schemes, which can then be opened to yield a resource.
Schemes are named so that the kernel is able to identify them. This name is used in the `scheme` part of the URL.
Schemes are named such that the kernel is able to uniquely identify them. This name is used in the `scheme` part of the URL.
Schemes are a generalization of file systems. It should be noted that schemes do not necessarily represent normal files; often they are a "virtual file", that is, an abstract unit with certain operations defined on it.
Schemes are a generalization of file systems. It should be noted that schemes do not necessarily represent normal files; they are often a "virtual file" (i.e., an abstract unit with certain operations defined on it).
Throughout the whole ecosystem of Redox, schemes are used as the main communication primitive, because they are a really powerful abstraction. Namely, we have one unified interface.
Throughout the whole ecosystem of Redox schemes are used as the main communication primitive because they are a powerful abstraction. With schemes Redox can have one unified I/O interface.
Schemes can be defined both in userspace and in kernelspace, although, when possible, userspace is preferred.
Schemes can be defined both in user space and in kernel space but when possible user space is preferred.
Scheme operations
-----------------
......@@ -22,7 +22,7 @@ A scheme is just a data structure with certain functions defined on it:
2. `mkdir` - make a new sub-structure. Note that the name is a little misleading (and it might even be renamed in the future), since in many schemes `mkdir` won't make a `directory`, but instead perform some form of substructure creation.
Less important optional methods are:
Optional methods include:
1. `unlink` - remove a link (that is a binding from one substructure to another).
......
The root scheme
The Root Scheme
===============
The root scheme is the kernel scheme, which is used for registering and retrieving information about schemes. The root schemes name is simply an empty string.
The root scheme is the kernel scheme which is used for registering and retrieving information about schemes. The root scheme's name is simply an empty string ("").
Registering a scheme
Registering a Scheme
====================
Registering a scheme is done by opening the name of the scheme with the `CREATE` flag, in the root scheme.
......
Writing a Scheme
================
> TODO
How do URLs work under the hood?
==============================
================================
The representation
------------------
Representation
--------------
Since it is impossible to go from userspace to ring 0 in a typed manner, we have to use some weakly typed representation (that is, we can't use an enum, unless we want to do transmutations and friends). Therefore, we use a string-like representation when moving to kernel space. This is basically just a raw pointer to a C-like, null-terminating string. To avoid further overhead, we use more efficient representations:
Since it is impossible to go from user space to ring 0 in a typed manner we have to use some weakly typed representation (that is, we can't use an enum, unless we want to do transmutations and friends). Therefore we use a string-like representation when moving to kernel space. This is basically just a raw pointer to a C-like, null-terminating string. To avoid further overhead, we use more efficient representations:
# `Url<'a>`
......
URLs
====
The URL _itself_ is a relatively uninteresting, yet very important, notion for the design of Redox. The interesting part is what it represents.
The URL _itself_ is a relatively uninteresting (yet very important) notion for the design of Redox. The interesting part is what it represents.
The URL
-------
In short, a URL is an identifier of a resource. They contain two parts:
1. The scheme part. This part represents, the "receiver", i.e. what scheme will handle the (F)OPEN call. This can be any arbitrary UTF-8 string, and will often simply be the name of your protocol.
1. The scheme part. This represents the "receiver", i.e. what scheme will handle the (F)OPEN call. This can be any arbitrary UTF-8 string, and will often simply be the name of your protocol.
2. The reference part. This part represents the "payload" of the URL, namely what the URL refers to. Consider `file`, as an example. A URL starting with `file:` simply has a reference which is a path to a file. The reference can be any arbitrary byte string. The parsing, interpretation, and storage of the reference is left to the scheme. For this reason, it is not required to be a tree-like structure.
2. The reference part. This represents the "payload" of the URL, namely what the URL refers to. Consider `file`, as an example. A URL starting with `file:` simply has a reference which is a path to a file. The reference can be any arbitrary byte string. The parsing, interpretation, and storage of the reference is left to the scheme. For this reason, it is not required to be a tree-like structure.
So, the string representation of an URL looks like:
......@@ -18,12 +18,18 @@ So, the string representation of an URL looks like:
[scheme]:[reference]
```
For example:
```
file:/path/to/myfile
```
Note that `//` is not required, for convenience.
Opening a URL
-------------
URLs can be opened, yielding _schemes_, which can be opened to resources, which can be read, written and (for some resources) seeked (there are some more operations; these are described later on).
URLs can be opened, yielding _schemes_, which can be opened to resources, which can be read, written and (for some resources) seeked (there are more operations which are described later on).
For compatibility reasons, we use a file API similar to the Rust standard library's for opening URLs:
......
"Everything is an URL"
======================
"Everything is an URL" is an important principle in the design of Redox. Roughly, it means that the API, design, and ecosystem is centered around URLs, schemes, and resources as the main communication primitive. In other words, applications communicate with each other, the system, daemons, and so on, using URLs. As such, programs do not have to create their own constructions for communicating.
"Everything is an URL" is an important principle in the design of Redox. Roughly speaking it means that the API, design, and ecosystem is centered around URLs, schemes, and resources as the main communication primitive. Applications communicate with each other, the system, daemons, etc, using URLs. As such, programs do not have to create their own constructs for communication.
By unifying the API in this way, you get an extremely consistent, clean, and flexible interface.
By unifying the API in this way, you are able to have a consistent, clean, and flexible interface.
We can't really claim credits of this concept (except for the implementation and exact design). The idea is not a new one: The concept is very similar to _9P_ from _Plan 9_ by Bell Labs and a similiar approach has been taken in Unix and its successors.
We can't really claim credits of this concept (beyond our exact design and implementation). The idea is not a new one and is very similar to _9P_ from _Plan 9_ by Bell Lab; a similar approach has been taken in Unix and its successors.
How it differs from "Everything is a file"
------------------------------------------
With "Everything is a file", all sorts of devices, processes and kernel parameters can be accessed as files in a regular filesystem. This leads to absurd situations like the hard disk containing the root filesystem `/` contains a folder `dev` with device files including `sda` which in turn contains the root filesystem. This situation is just missing any logic. Furthermore, many file properties don't make sense on these 'special files': What's the size of `/dev/null` or a configuration option in sysfs?
With "Everything is a file" all sorts of devices, processes, and kernel parameters can be accessed as files in a regular filesystem. This leads to absurd situations like the hard disk containing the root filesystem `/` contains a folder named `dev` with device files including `sda` which contains the root filesystem. Situations like this are missing any logic. Furthermore many file properties don't make sense on these 'special files': What is the size of `/dev/null` or a configuration option in sysfs?
In contrast to "Everything is a file", Redox doesn't enforce a common tree node for all kinds of resources. Instead, resources are distinguished by protocol. This way, USB devices don't end up in a "filesystem", but a protocol-based scheme like `EHCI`. Real files are accessible through a scheme called `file`, which is widely used and specified in [RFC 1630](https://tools.ietf.org/html/rfc1630) and [RFC 1738](https://tools.ietf.org/html/rfc1738).
In contrast to "Everything is a file", Redox does not enforce a common tree node for all kinds of resources. Instead resources are distinguished by protocol. This way USB devices don't end up in a "filesystem", but a protocol-based scheme like `EHCI`. Real files are accessible through a scheme called `file`, which is widely used and specified in [RFC 1630] and [RFC 1738].
[RFC 1630]: https://tools.ietf.org/html/rfc1630
[RFC 1738]: https://tools.ietf.org/html/rfc1738
......@@ -5,7 +5,7 @@ Enough theory! Time for an example.
We will implement a scheme which holds a vector, and push elements when you write, and pop them when you read. Let's call it `vec:`.
Let's get goin':
Let's get going:
The code
--------
......@@ -103,4 +103,4 @@ Now, go ahead and test it!
Exercise for the reader
------------------------
Write a scheme with can run Brainfuck code.
Write a scheme that can run `Brainfuck` code.
Stiching it all together
Stiching it All Together
========================
The "URL, scheme, resource" model is simply a unified interface for efficient inter-process communication. URLs are simply resource descriptors. Schemes are simply resource "entries", which can be opened. You can think of a scheme as a closed book. It cannot itself be read or written, but you can open it to an open book: a resource. Resources are simply primitives for communications. They can behave either socket-like (as a stream of bytes, e.g. TCP and Orbital) or file-like (as an on-demand byte buffer, e.g. file systems and stdin).
......@@ -12,7 +12,7 @@ A quick, ugly diagram would look like this:
| +=========+
| +--------------------------------------+ ^ | write
| | | | |
Userspace < +----- URL -----+ | read | v
User space < +----- URL -----+ | read | v
| | +-----------+ | open +---------+ open | +----------+
| | | Scheme |-|---+ +------->| Scheme |------------>| Resource |
| | +-----------+ | | | +---------+ +----------+
......@@ -23,7 +23,7 @@ A quick, ugly diagram would look like this:
resolve | |
/ v |
| +=========+
Kernelspace < | Resolve |
Kernel space < | Resolve |
| +=========+
\
......
URLs, schemes, and resources
URLs, Schemes, and Resources
============================
This is one of the most important design choices Redox makes. These three essential concepts are very entangled.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment