diff --git a/content/news/rsoc-fat32-1.md b/content/news/rsoc-fat32-1.md index 65099e2aac2197aed586ee3113344bb361b38b7e..84367f229bb51b2ec7529a75c1af1cf9729a541e 100644 --- a/content/news/rsoc-fat32-1.md +++ b/content/news/rsoc-fat32-1.md @@ -27,7 +27,7 @@ The RedoxFS filesystem is an [extent based filesystem](https://en.wikipedia.org/ * Develop a 64-bit stub in Rust which parses the MBR/GPT and loads a module for the specific filesystem where the kernel resides * The module loads the kernel into memory and jumps to it -Currently I am using the `-D KERNEL` to insert the stub into the bootloader image. It is based on an old version of Philip Opperman's blog OS which has been hacked to mimic the Redox kernel's linker addresses as well as page size. The stub repo which I am currently using can be found [here](https://github.com/deepaksirone/pamb-os/tree/loader_stub). qemu does not boot the disk when using the `-machine q35` flag if the size of the disk image is too small. So the image is padded with about 8MB worth of zeroes, aligned to 512 bytes. Currently the stub does not support printing functions and hence the only was to verify that it is running is by using qemu's gdb debug target. +Currently I am using the `-D KERNEL` to insert the stub into the bootloader image. It is based on an old version of Philip Opperman's blog OS which has been hacked to mimic the Redox kernel's linker addresses as well as page size. The stub repo which I am currently using can be found [here](https://github.com/deepaksirone/redox-loader). qemu does not boot the disk when using the `-machine q35` flag if the size of the disk image is too small. So the image is padded with about 8MB worth of zeroes, aligned to 512 bytes. Currently the stub does not support printing functions and hence the only was to verify that it is running is by using qemu's gdb debug target. #### Future Work * Add printing/debug capability to the stub diff --git a/content/news/rsoc-fat32-2.md b/content/news/rsoc-fat32-2.md index c012b6712f999a86a435a7489f7b89ba8115f5c4..419a1b1e61d488a71958a9ebbd9a0d6c290de0c9 100644 --- a/content/news/rsoc-fat32-2.md +++ b/content/news/rsoc-fat32-2.md @@ -17,12 +17,12 @@ Initially the idea was to write a stub from the ground up and the reason behind Stripping down the kernel started with the serial console debug support. Qemu maps the serial console to stdio with the `-serial mon:stdio` option. The serial console code depended on the [arch](https://gitlab.redox-os.org/redox-os/kernel/tree/master/src/arch/x86_64) submodule and the [syscall](https://gitlab.redox-os.org/redox-os/syscall) crate and they were pulled in. After the arch submodule was pulled in I realized that all the subsystems initialized in the kstart function in [start.rs](https://gitlab.redox-os.org/redox-os/kernel/blob/master/src/arch/x86_64/start.rs#L51) would be useful at some point. So I began to get each of the subsystems working. -Getting the paging and the GDT initializations to work was straightforward. Setting up the IDT required that all the external interrupt handlers be disabled. Syscalls are completely disabled as the [INT 80h handler](https://github.com/deepaksirone/pamb-os/blob/loader_stub/src/arch/x86_64/interrupt/syscall.rs#L5) is mostly commented out. +Getting the paging and the GDT initializations to work was straightforward. Setting up the IDT required that all the external interrupt handlers be disabled. Syscalls are completely disabled as the [INT 80h handler](https://github.com/deepaksirone/redox-loader/blob/master/src/arch/x86_64/interrupt/syscall.rs#L5) is mostly commented out. With the base stub ready I started working on getting disk reads working. Initially the idea was to port the ahci driver such that it dosen't make use of Redox schemes. Again that meant a near complete rewrite of the driver and after discussions with [@jackpot51](https://github.com/jackpot51/) it was decided that the disks should be read after dropping to real mode, using BIOS interrupts. -To do this I referred to the OsDev article [here](https://forum.osdev.org/viewtopic.php?f=1&t=23125). The code should be loaded to a location which can be addressed in real mode i.e. a usable chunk of memory below the 1MB mark. The pages where the code is loaded should be both executable and writable as there is an interleaving of code and data. The stub adaptation of this code is [here](https://github.com/deepaksirone/pamb-os/blob/loader_stub/bootloader/x86_64/real.asm). +To do this I referred to the OsDev article [here](https://forum.osdev.org/viewtopic.php?f=1&t=23125). The code should be loaded to a location which can be addressed in real mode i.e. a usable chunk of memory below the 1MB mark. The pages where the code is loaded should be both executable and writable as there is an interleaving of code and data. The stub adaptation of this code is [here](https://github.com/deepaksirone/redox-loader/blob/master/bootloader/x86_64/real.asm). #### Roughly the code works as follows: * Disable interrupts as we are going to alter the IDT pointer @@ -38,9 +38,9 @@ To do this I referred to the OsDev article [here](https://forum.osdev.org/viewto * A far jmp is used to get back to kernel mode code * The original GDT is restored and all the data segments are restored with their appropriate descriptors -The real mode is loaded at address `0xb000` during booting. The pages starting from `0x9000` and `0xa000` are used for the real mode stack. The pages starting from `0xc000` upto `0x70000` can be used to read in disk data. This gives a total of 11 pages(each page being 4096 bytes) worth of disk data per drop into real mode. After some more testing the code can be moved further towards the bootsector ending address of `0x7e00` giving more disk data per drop. +The real mode is loaded at address `0xb000` during booting. The pages starting from `0x9000` and `0xa000` are used for the real mode stack. The pages starting from `0xc000` upto `0x70000` can be used to read in disk data. This gives a total of 100 pages(each page being 4096 bytes) worth of disk data per drop into real mode. After some more testing the code can be moved further towards the bootsector ending address of `0x7e00` giving more disk data per drop. -Before dropping to real mode from the stub, all the pages from `0x9000` upto `0x70000` are [identity mapped](https://wiki.osdev.org/Identity_Paging) and given write permission in the [init_real_mode](https://github.com/deepaksirone/pamb-os/blob/loader_stub/src/partition/mod.rs#L66) function. The function which drops to real mode can be found [here](https://github.com/deepaksirone/pamb-os/blob/loader_stub/src/partition/mod.rs#L79) +Before dropping to real mode from the stub, all the pages from `0x9000` upto `0x70000` are [identity mapped](https://wiki.osdev.org/Identity_Paging) and given write permission in the [init_real_mode](https://github.com/deepaksirone/redox-loader/blob/master/src/fs/mod.rs#L37) function. The function which drops to real mode can be found [here](https://github.com/deepaksirone/redox-loader/blob/master/src/fs/mod.rs#L69) #### The drop works as follows: * Cast the address `0xb000` as an extern "C" function using the `mem::transmute()` function @@ -60,6 +60,6 @@ The stub is currently about 250K in size. There is still a lot of code which can * Cleanup unnecessary code in the stub to make the code even smaller #### Miscellaneous Stuff -* The MBR parsing code can be found [here](https://github.com/deepaksirone/pamb-os/blob/loader_stub/src/partition/mod.rs#L20) +* The MBR parsing code can be found [here](https://github.com/deepaksirone/redox-loader/blob/master/src/fs/mbr.rs) * IMHO the Redox kernel is much easier to read and understand than the Linux kernel * Build fixes for the kernel: [graphical_debug](https://gitlab.redox-os.org/redox-os/kernel/merge_requests/91) , [version_fix](https://gitlab.redox-os.org/redox-os/kernel/merge_requests/92) diff --git a/content/news/rsoc-fat32-3.md b/content/news/rsoc-fat32-3.md new file mode 100644 index 0000000000000000000000000000000000000000..9bad5d532e34c999ef8b4acc2deb32d22b37f882 --- /dev/null +++ b/content/news/rsoc-fat32-3.md @@ -0,0 +1,41 @@ ++++ +title = "RSoC: FAT32 Filesystem in Redox - 3" +author = "Deepak Sirone" +date = "2018-08-09T22:39:51+05:30" ++++ + +This is the third blog post about implementing a FAT32 filesystem in Redox. + +#### The previous blog posts in this series are +* [RSoC: FAT32 Filesystem in Redox-1](https://www.redox-os.org/news/rsoc-fat32-1/) +* [RSoC: FAT32 Filesystem in Redox-2](https://www.redox-os.org/news/rsoc-fat32-2/) + +In the previous blog post the future work was detailed as follows: + +#### TODO +* Write a clean read() and if possible a write() API [done] +* Decide on a FAT32 implementation [done] +* Write adapters/wrappers over the read/write API to support the FAT32 implementation [done] +* Decide on where to put the kernel in the filesystem [done] +* Read in the kernel and transfer execution to it [done] +* Cleanup unnecessary code in the stub to make the code even smaller [done] + +The previous blog post discusses how raw disk reads were implemented in the loader stub. The next step was to implement a clean read API which can be used by different filesystem libraries in order to read their respective filesystems. Since the raw reads from the BIOS interrupt had a granularity in terms of sectors(each sector being 512 bytes), the reads had to be translated in order to provide byte level granularity. The `clone_from_slice` function ensures that a direct call to `memcopy` is not required. The refined read function is [here](https://github.com/deepaksirone/redox-loader/blob/master/src/fs/mod.rs#L69). + +The next step was to write code to parse [MBR](https://en.wikipedia.org/wiki/Master_boot_record) partitions. This was relatively straightforward and the code can be found [here](https://github.com/deepaksirone/redox-loader/blob/master/src/fs/disk.rs). + +Moving further, the next step was to decide on a FAT32 implementation. Rust provides [read](https://doc.rust-lang.org/std/io/trait.Read.html), [write](https://doc.rust-lang.org/std/io/trait.Write.html) and [seek](https://doc.rust-lang.org/std/io/trait.Seek.html) traits which when implemented for a struct, allows the read, write and seek functions to be called on it. So implementing the three traits for the `MBR` partition was the next goal. Unfortunately the loader is written with no support for the standard library. Instead it is wrtten using a baremetal library called [libcore](https://doc.rust-lang.org/beta/core/) which supports the traits only through the [core_io_crate](https://github.com/jethrogb/rust-core_io). The `core_io` crate requires a specific version of the Rust compiler to compile which breaks the rest of the loader. At first we considered [fatfs](https://github.com/rafalh/rust-fatfs) but it turned out that it depended on the `core_io` crate. Finally we decided on [fatrs](https://gitlab.com/susurrus/fat-rs) as it had it's own trait called a `StorageDevice` which did not depend on any `core_io` traits. The implementation of the trait for the `Partition` struct can be found [here](https://github.com/deepaksirone/redox-loader/blob/master/src/fs/fat32/mod.rs). The `fatrs` library did have a few bugs of it's own which was fixed in this [fork](https://gitlab.com/deepaksirone/fat-rs). + +With the FAT32 implementation running smoothly, the final lap had begun which was to load the kernel at address `0x100000` that is the first megabyte of memory, setup the environment including the page tables for the kernel and then transfer control to it. The problem was that the loader stub was currently loaded at the exact same location and the page tables could not be setup exactly as the kernel would have liked because some pages were used by the loader. The fix was inspired by a discussion with [@jackpot51](https://github.com/jackpot51) where we decided that we should initially copy the kernel at address `0x400000` and then drop into real mode as in the previous blog post, to copy the kernel at `0x100000`. Real mode does not have paging and hence all the page tables could be setup exactly like the original [asm bootloader](url_here) does. After this the real mode code jumps to the kernel and begins execution. A primary change which was made to the [real mode code](https://github.com/deepaksirone/redox-loader/blob/master/bootloader/x86_64/kernel_copy.asm) was the introduction of [unreal mode](https://wiki.osdev.org/Unreal_Mode) which ensures that data can be both read and written above the 1MB mark. + +Currently the kernel is loaded from a FAT32 partition and the rest of userspace i.e. init and friends are still loaded from a RedoxFS partition. + + +#### Future Work +* Write a FAT32 userspace daemon which can access a FAT32 partition from userspace +* Add code to read the UUID from a RedoxFS parition +* Do some more code cleanup + +#### Miscellaneous Stuff +* The release build of the loader stub is about 164KB in size +* The kernel read in time from the disk is < 3 seconds