...
 
Commits (105)
[submodule "syscall"]
path = syscall
url = https://github.com/redox-os/syscall.git
url = https://gitlab.redox-os.org/redox-os/syscall.git
[submodule "slab_allocator"]
path = slab_allocator
url = https://github.com/redox-os/slab_allocator
url = https://gitlab.redox-os.org/redox-os/slab_allocator
# Porting the core Redox kernel to arm AArch64: An outline
## Intro
This document is [my](https://github.com/raw-bin) attempt at:
* Capturing thinking on the work needed for a core Redox kernel port
* Sharing progress with the community as things evolve
* Creating a template that can be used for ports to other architectures
Core Redox kernel means everything needed to get to a non-graphical console-only multi-user shell.
Only the 64-bit execution state (AArch64) with the 64-bit instruction set architecture (A64) shall be supported for the moment. For more background/context read [this](https://developer.arm.com/products/architecture/a-profile/docs/den0024/latest/introduction).
This document is intended to be kept *live*. It will be updated to reflect the current state of work and any feedback received.
It is hard~futile to come up with a strict sequence of work for such ports but this document is a reasonable template to follow.
## Intended target platform
The primary focus is on [qemu's virt machine platform emulation for the AArch64 architecture](https://github.com/qemu/qemu/blob/master/hw/arm/virt.c#L127).
Targeting a virtual platform is a convenient way to bring up the mechanics of architectural support and makes the jump to silicon easier. The preferred boot chain for AArch64 (explained later) is well supported on this platform and boot-over-tftp from localhost makes the debug cycle very efficient.
Once the core kernel port is complete a similar follow on document will be created that is dedicated to silicon bring-up.
## Boot protocol elements
Item | Notes
-----|-------
[Linux kernel boot protocol for AArch64](https://www.kernel.org/doc/Documentation/arm64/booting.txt) | The linked document describes assumptions made from the bootloader which are field tested and worthwhile to have for Redox an AArch64. <br/> The intent is to consider most of the document except anything tied to the Linux kernel itself.
[Flattened Device Tree](https://elinux.org/Device_Tree_Reference) | FDT binary blobs supplied by the bootloader shall provide the Redox kernel with misc platform \{memory, interrupt, devicemem} maps. Qemu's virt machine platform synthetically creates an FDT blob at a specific address which is very handy.
## Boot flow elements
The following table lists the boot flow in order.
Item | Notes
-----|-------
[ARM Trusted Firmware (TF-A)](https://github.com/ARM-software/arm-trusted-firmware) | TF-A is a de-facto standard reference firmware implementation and proven in the field. <br/> TF-A runs post power-on on Armv8-A implementations and eventually hands off to further stages of the boot flow.<br />For qemu's virt machine platform, it is essentially absent but I mean to rely on it heavily for silicon bring up hence mentioning it here.
[u-boot](https://www.denx.de/wiki/U-Boot) | u-boot will handle early console access, media access for fetching redox kernel images from non-volatile storage/misc disk subsystems/off the network. <br /> u-boot supports loading EFI applications. If EFI support to AArch64 Redox is added in the future that should essentially work out of the box. <br /> u-boot will load redox and FDT binary blobs into RAM and jump to the redox kernel.
Redox early-init stub | For AArch64, the redox kernel will contain an A64 assembly stub that will setup the MMU from scratch. This is akin to the [x86_64 redox bootloader](https://github.com/redox-os/bootloader/blob/master/x86_64/startup-x86_64.asm). <br /> This stub sets up identity maps for MMU initialization, maps the kernel image itself as well as the device memory for the UART console. At present this stub shall be a part of the kernel itself for simplicity.
Redox kstart entry | The early init stub hands off here. kstart will then re-init the MMU more comprehensively.
## Supported devices
The following devices shall be supported. All necessary information specific to these devices will be provided to the redox kernel by the platform specific FDT binary blob.
Device | Notes
-------|-------
[Generic Interrupt Controller v2](https://developer.arm.com/products/architecture/a-profile/docs/ihi0048/b/arm-generic-interrupt-controller-architecture-version-20-architecture-specification) | The GIC is an Arm-v8A architectural element and is supported by all architecturally compliant processor implementations. GICv2 is supported by qemu's virt machine emulation and most subsequent GIC implementations are backward compatible to GICv2.
[Generic Timer](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0500d/BGBBIJCB.html) | The Generic Timer Architecture is an Arm-v8A architectural element and is implemented by all compliant processor implementations. It is supported by qemu.
[PrimeCell UART PL011](http://infocenter.arm.com/help/topic/com.arm.doc.ddi0183f/DDI0183.pdf) | The PL011 UART is supported by qemu and most ARM systems.
## Intended development sequence and status
Item | Description | Status | Notes
-----|-------|-----|-----
Redox AArch64 toolchain | Create an usable redox AArch64 toolchain specification | Done | Using this JSON spec in isolated tests produces valid AArch64 soft float code
Stubbed kernel image | Stub out AArch64 kernel support using the existing x86_64 arch code as a template <br /> Modify redox kernel build glue and work iteratively to get a linkable (non-functional) image | Not done yet |
Boot flow | Create a self hosted u-boot -> redox kernel workflow <br /> Should obtain the stubbed image from a local TFTP server, load it into RAM and jump to it | Not done yet |
GDB Debug flow | Create a debug workflow centered around qemu's GDB stub <br /> This should allow connecting to qemu's GDB stub and debug u-boot/redox stub via a GDB client and single stepping through code | Not done yet |
Verify Redox entry | Verify that control reaches the redox kernel from u-boot | Not done yet |
AArch64 early init stub | Add support for raw asm code for early AArch64 init in the redox kernel <br /> Verify that this code is located appropriately in the link map and that control reaches this code from u-boot | Not done yet |
Basic DTB support | Integrate the [device_tree crate](https://mbr.github.io/device_tree-rs/device_tree/) <br /> Use the crate to access the qemu supplied DTB image and extract the memory map | Not done yet |
Basic UART support | Use the device_tree crate to get the UART address from the DTB image and set up the initial console <br /> This is a polling mode only setup | Not done yet |
Initial MMU support | Implement initial MMU support in the early init stub <br /> This forces the MMU into a clean state overriding any bootloader specific setup <br /> Create an identity map for MMU init <br /> Create a mapping for the kernel image <br /> Create a mapping for any devices needed at this stage (UART)| Not done yet |
kmain entry | Verify that kmain entry works post early MMU init | Not done yet |
Basic Redox MMU support | Get Redox to create a final set of mappings for everything <br /> Verify that this works as expected| Not done yet |
Basic libc support | Flesh out a basic set of libc calls as required for simple user-land apps | Not done yet |
userspace_init entry | Verify user-space entry and /sbin/init invocation | Not done yet |
Basic Interrupt controller support | Add a GIC driver <br /> Verify functionality | Not done yet |
Basic Timer support | Add a Generic Timer driver <br /> Verify functionality | Not done yet |
UART interrupt support | Add support for UART interrupts | Not done yet |
Task context switch support | Add context switching support <br /> Verify functionality | Not done yet |
Login shell | Iteratively add and verify multi-user login shell support | Not done yet |
Publish development branch on github | Work with the community to post work done after employer approval | Not done yet |
Break out the Bubbly | Drink copious quantities of alcohol to celebrate | Not done yet |
Silicon bring-up | Plan silicon bring-up | Not done yet |
This diff is collapsed.
[package]
name = "kernel"
version = "0.1.33"
version = "0.1.51"
build = "build.rs"
[lib]
......@@ -9,29 +9,35 @@ path = "src/lib.rs"
crate-type = ["staticlib"]
[dependencies]
bitflags = "1"
clippy = { version = "*", optional = true }
linked_list_allocator = "0.5"
raw-cpuid = "3.0"
bitflags = "1.0.3"
clippy = { version = "0.0.209", optional = true }
linked_list_allocator = "0.6.2"
raw-cpuid = "4.0.0"
redox_syscall = { path = "syscall" }
slab_allocator = { path = "slab_allocator" }
spin = "0.4"
slab_allocator = { path = "slab_allocator", optional = true }
spin = "0.4.8"
[dependencies.goblin]
version = "0.0.10"
version = "0.0.15"
default-features = false
features = ["elf32", "elf64"]
[dependencies.rustc-demangle]
version = "0.1.13"
default-features = false
[dependencies.x86]
version = "0.7"
version = "0.9.0"
default-features = false
[features]
default = ["acpi", "graphical_debug"]
default = ["serial_debug"]
acpi = []
doc = []
graphical_debug = []
live = []
multi_core = []
pti = []
slab = []
qemu_debug = []
serial_debug = []
slab = ["slab_allocator"]
......@@ -88,6 +88,7 @@ fn fill_from_location(f: &mut fs::File, loc: &Path ) -> Result<(), (Error)> {
fn main() {
println!("cargo:rustc-env=TARGET={}", env::var("TARGET").unwrap());
println!("cargo:rerun-if-env-changed=INITFS_FOLDER");
let out_dir = env::var("OUT_DIR").unwrap();
let dest_path = Path::new(&out_dir).join("gen.rs");
......@@ -97,13 +98,16 @@ fn main() {
// Write header
f.write_all(b"
mod gen {
use alloc::BTreeMap;
use alloc::collections::BTreeMap;
pub fn gen() -> BTreeMap<&'static [u8], (&'static [u8], bool)> {
let mut files: BTreeMap<&'static [u8], (&'static [u8], bool)> = BTreeMap::new();
").unwrap();
match src {
Ok(v) => fill_from_location(&mut f, Path::new(&v)).unwrap(),
Ok(v) => {
println!("cargo:rerun-if-changed={}", v);
fill_from_location(&mut f, Path::new(&v)).unwrap()
},
Err(e) => {
f.write_all(
b" files.clear();" // Silence mutability warning
......
Subproject commit 0a53a0bc40020ed69f6e1a058165a82771217938
Subproject commit a9e2bb08a842752d44598a7b188760c21f15c335
use alloc::heap::{Alloc, AllocErr, Layout};
use core::alloc::{AllocErr, GlobalAlloc, Layout};
use core::ptr::NonNull;
use linked_list_allocator::Heap;
use spin::Mutex;
......@@ -14,8 +15,8 @@ impl Allocator {
}
}
unsafe impl<'a> Alloc for &'a Allocator {
unsafe fn alloc(&mut self, mut layout: Layout) -> Result<*mut u8, AllocErr> {
unsafe impl GlobalAlloc for Allocator {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
loop {
let res = if let Some(ref mut heap) = *HEAP.lock() {
heap.allocate_first_fit(layout)
......@@ -24,9 +25,7 @@ unsafe impl<'a> Alloc for &'a Allocator {
};
match res {
Err(AllocErr::Exhausted { request }) => {
layout = request;
Err(AllocErr) => {
let size = if let Some(ref heap) = *HEAP.lock() {
heap.size()
} else {
......@@ -41,28 +40,16 @@ unsafe impl<'a> Alloc for &'a Allocator {
panic!("__rust_allocate: heap not initialized");
}
},
other => return other,
other => return other.ok().map_or(0 as *mut u8, |allocation| allocation.as_ptr()),
}
}
}
unsafe fn dealloc(&mut self, ptr: *mut u8, layout: Layout) {
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
if let Some(ref mut heap) = *HEAP.lock() {
heap.deallocate(ptr, layout)
heap.deallocate(NonNull::new_unchecked(ptr), layout)
} else {
panic!("__rust_deallocate: heap not initialized");
}
}
fn oom(&mut self, error: AllocErr) -> ! {
panic!("Out of memory: {:?}", error);
}
fn usable_size(&self, layout: &Layout) -> (usize, usize) {
if let Some(ref mut heap) = *HEAP.lock() {
heap.usable_size(layout)
} else {
panic!("__rust_usable_size: heap not initialized");
}
}
}
......@@ -8,7 +8,10 @@ pub use self::linked_list::Allocator;
#[cfg(feature="slab")]
pub use self::slab::Allocator;
#[cfg(not(feature="slab"))]
mod linked_list;
#[cfg(feature="slab")]
mod slab;
unsafe fn map_heap(active_table: &mut ActivePageTable, offset: usize, size: usize) {
......
use alloc::heap::{Alloc, AllocErr, Layout};
use core::alloc::{Alloc, AllocErr, Layout};
use spin::Mutex;
use slab_allocator::Heap;
......
use core::fmt;
#[cfg(feature = "qemu_debug")]
use spin::Mutex;
use spin::MutexGuard;
use devices::uart_16550::SerialPort;
use log::{LOG, Log};
#[cfg(feature = "qemu_debug")]
use syscall::io::Io;
use syscall::io::Pio;
#[cfg(feature = "serial_debug")]
use devices::uart_16550::SerialPort;
use super::device::serial::COM1;
#[cfg(feature = "graphical_debug")]
use super::graphical_debug::{DEBUG_DISPLAY, DebugDisplay};
#[cfg(feature = "serial_debug")]
use super::device::serial::COM1;
#[cfg(feature = "qemu_debug")]
pub static QEMU: Mutex<Pio<u8>> = Mutex::new(Pio::<u8>::new(0x402));
pub struct Writer<'a> {
serial: MutexGuard<'a, SerialPort<Pio<u8>>>,
log: MutexGuard<'a, Option<Log>>,
#[cfg(feature = "graphical_debug")]
display: MutexGuard<'a, Option<DebugDisplay>>
display: MutexGuard<'a, Option<DebugDisplay>>,
#[cfg(feature = "qemu_debug")]
qemu: MutexGuard<'a, Pio<u8>>,
#[cfg(feature = "serial_debug")]
serial: MutexGuard<'a, SerialPort<Pio<u8>>>,
}
impl<'a> Writer<'a> {
pub fn new() -> Writer<'a> {
Writer {
serial: COM1.lock(),
log: LOG.lock(),
#[cfg(feature = "graphical_debug")]
display: DEBUG_DISPLAY.lock(),
#[cfg(feature = "qemu_debug")]
qemu: QEMU.lock(),
#[cfg(feature = "serial_debug")]
serial: COM1.lock(),
}
}
}
impl<'a> fmt::Write for Writer<'a> {
#[cfg(not(feature = "graphical_debug"))]
fn write_str(&mut self, s: &str) -> Result<(), fmt::Error> {
self.serial.write_str(s)
}
pub fn write(&mut self, buf: &[u8]) {
{
if let Some(ref mut log) = *self.log {
log.write(buf);
}
}
#[cfg(feature = "graphical_debug")]
fn write_str(&mut self, s: &str) -> Result<(), fmt::Error> {
if let Some(ref mut display) = *self.display {
let _ = display.write_str(s);
#[cfg(feature = "graphical_debug")]
{
if let Some(ref mut display) = *self.display {
let _ = display.write(buf);
}
}
#[cfg(feature = "qemu_debug")]
{
for &b in buf {
self.qemu.write(b);
}
}
self.serial.write_str(s)
#[cfg(feature = "serial_debug")]
{
self.serial.write(buf);
}
}
}
impl<'a> fmt::Write for Writer<'a> {
fn write_str(&mut self, s: &str) -> Result<(), fmt::Error> {
self.write(s.as_bytes());
Ok(())
}
}
......@@ -115,7 +115,7 @@ pub fn cpu_info<W: Write>(w: &mut W) -> Result {
if info.has_rep_movsb_stosb() { write!(w, " erms")? };
if info.has_invpcid() { write!(w, " invpcid")? };
if info.has_rtm() { write!(w, " rtm")? };
if info.has_qm() { write!(w, " qm")? };
//if info.has_qm() { write!(w, " qm")? };
if info.has_fpu_cs_ds_deprecated() { write!(w, " fpu_seg")? };
if info.has_mpx() { write!(w, " mpx")? };
}
......
use core::intrinsics::{volatile_load, volatile_store};
use x86::cpuid::CpuId;
use x86::msr::*;
use x86::shared::cpuid::CpuId;
use x86::shared::msr::*;
use memory::Frame;
use paging::{ActivePageTable, PhysicalAddress, Page, VirtualAddress};
......
//! Global descriptor table
use core::mem;
use x86::dtables::{self, DescriptorTablePointer};
use x86::segmentation::{self, SegmentSelector};
use x86::task::{self, TaskStateSegment};
use x86::current::segmentation::set_cs;
use x86::current::task::TaskStateSegment;
use x86::shared::PrivilegeLevel;
use x86::shared::dtables::{self, DescriptorTablePointer};
use x86::shared::segmentation::{self, SegmentDescriptor, SegmentSelector};
use x86::shared::task;
pub const GDT_NULL: usize = 0;
pub const GDT_KERNEL_CODE: usize = 1;
......@@ -33,9 +36,9 @@ pub const GDT_F_PAGE_SIZE: u8 = 1 << 7;
pub const GDT_F_PROTECTED_MODE: u8 = 1 << 6;
pub const GDT_F_LONG_MODE: u8 = 1 << 5;
static mut INIT_GDTR: DescriptorTablePointer = DescriptorTablePointer {
static mut INIT_GDTR: DescriptorTablePointer<SegmentDescriptor> = DescriptorTablePointer {
limit: 0,
base: 0
base: 0 as *const SegmentDescriptor
};
static mut INIT_GDT: [GdtEntry; 4] = [
......@@ -50,9 +53,9 @@ static mut INIT_GDT: [GdtEntry; 4] = [
];
#[thread_local]
pub static mut GDTR: DescriptorTablePointer = DescriptorTablePointer {
pub static mut GDTR: DescriptorTablePointer<SegmentDescriptor> = DescriptorTablePointer {
limit: 0,
base: 0
base: 0 as *const SegmentDescriptor
};
#[thread_local]
......@@ -100,13 +103,27 @@ pub unsafe fn set_tss_stack(stack: usize) {
TSS.rsp[0] = stack as u64;
}
/// Initialize GDT
pub unsafe fn init(tcb_offset: usize, stack_offset: usize) {
// Initialize GDT
pub unsafe fn init() {
// Setup the initial GDT with TLS, so we can setup the TLS GDT (a little confusing)
// This means that each CPU will have its own GDT, but we only need to define it once as a thread local
INIT_GDTR.limit = (INIT_GDT.len() * mem::size_of::<GdtEntry>() - 1) as u16;
INIT_GDTR.base = INIT_GDT.as_ptr() as u64;
INIT_GDTR.base = INIT_GDT.as_ptr() as *const SegmentDescriptor;
// Load the initial GDT, before we have access to thread locals
dtables::lgdt(&INIT_GDTR);
// Load the segment descriptors
set_cs(SegmentSelector::new(GDT_KERNEL_CODE as u16, PrivilegeLevel::Ring0));
segmentation::load_ds(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_es(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_fs(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_gs(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_ss(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
}
/// Initialize GDT with TLS
pub unsafe fn init_paging(tcb_offset: usize, stack_offset: usize) {
// Set the TLS segment to the offset of the Thread Control Block
INIT_GDT[GDT_KERNEL_TLS].set_offset(tcb_offset as u32);
......@@ -114,16 +131,11 @@ pub unsafe fn init(tcb_offset: usize, stack_offset: usize) {
dtables::lgdt(&INIT_GDTR);
// Load the segment descriptors
segmentation::load_cs(SegmentSelector::new(GDT_KERNEL_CODE as u16));
segmentation::load_ds(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_es(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_fs(SegmentSelector::new(GDT_KERNEL_TLS as u16));
segmentation::load_gs(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_ss(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_fs(SegmentSelector::new(GDT_KERNEL_TLS as u16, PrivilegeLevel::Ring0));
// Now that we have access to thread locals, setup the AP's individual GDT
GDTR.limit = (GDT.len() * mem::size_of::<GdtEntry>() - 1) as u16;
GDTR.base = GDT.as_ptr() as u64;
GDTR.base = GDT.as_ptr() as *const SegmentDescriptor;
// Set the TLS segment to the offset of the Thread Control Block
GDT[GDT_KERNEL_TLS].set_offset(tcb_offset as u32);
......@@ -142,15 +154,15 @@ pub unsafe fn init(tcb_offset: usize, stack_offset: usize) {
dtables::lgdt(&GDTR);
// Reload the segment descriptors
segmentation::load_cs(SegmentSelector::new(GDT_KERNEL_CODE as u16));
segmentation::load_ds(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_es(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_fs(SegmentSelector::new(GDT_KERNEL_TLS as u16));
segmentation::load_gs(SegmentSelector::new(GDT_KERNEL_DATA as u16));
segmentation::load_ss(SegmentSelector::new(GDT_KERNEL_DATA as u16));
set_cs(SegmentSelector::new(GDT_KERNEL_CODE as u16, PrivilegeLevel::Ring0));
segmentation::load_ds(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_es(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_fs(SegmentSelector::new(GDT_KERNEL_TLS as u16, PrivilegeLevel::Ring0));
segmentation::load_gs(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
segmentation::load_ss(SegmentSelector::new(GDT_KERNEL_DATA as u16, PrivilegeLevel::Ring0));
// Load the task register
task::load_ltr(SegmentSelector::new(GDT_TSS as u16));
task::load_tr(SegmentSelector::new(GDT_TSS as u16, PrivilegeLevel::Ring0));
}
#[derive(Copy, Clone, Debug)]
......
......@@ -27,7 +27,7 @@ impl DebugDisplay {
self.display
}
pub fn write(&mut self, c: char) {
pub fn write_char(&mut self, c: char) {
if self.x >= self.w || c == '\n' {
self.x = 0;
self.y += 1;
......@@ -74,14 +74,10 @@ impl DebugDisplay {
self.x += 1;
}
}
}
impl fmt::Write for DebugDisplay {
fn write_str(&mut self, s: &str) -> Result<(), fmt::Error> {
for c in s.chars() {
self.write(c);
pub fn write(&mut self, buf: &[u8]) {
for &b in buf {
self.write_char(b as char);
}
Ok(())
}
}
use alloc::allocator::{Alloc, Layout};
use alloc::heap::Heap;
use core::alloc::{GlobalAlloc, Layout};
use core::{cmp, slice};
use super::FONT;
......@@ -16,7 +15,7 @@ pub struct Display {
impl Display {
pub fn new(width: usize, height: usize, onscreen: usize) -> Display {
let size = width * height;
let offscreen = unsafe { Heap.alloc(Layout::from_size_align_unchecked(size * 4, 4096)).unwrap() };
let offscreen = unsafe { ::ALLOCATOR.alloc(Layout::from_size_align_unchecked(size * 4, 4096)) };
unsafe { fast_set64(offscreen as *mut u64, 0, size/2) };
Display {
width: width,
......@@ -145,6 +144,6 @@ impl Display {
impl Drop for Display {
fn drop(&mut self) {
unsafe { Heap.dealloc(self.offscreen.as_mut_ptr() as *mut u8, Layout::from_size_align_unchecked(self.offscreen.len() * 4, 4096)) };
unsafe { ::ALLOCATOR.dealloc(self.offscreen.as_mut_ptr() as *mut u8, Layout::from_size_align_unchecked(self.offscreen.len() * 4, 4096)) };
}
}
use core::mem;
use x86::dtables::{self, DescriptorTablePointer};
use x86::current::irq::IdtEntry as X86IdtEntry;
use x86::shared::dtables::{self, DescriptorTablePointer};
use interrupt::*;
use ipi::IpiKind;
pub static mut IDTR: DescriptorTablePointer = DescriptorTablePointer {
pub static mut INIT_IDTR: DescriptorTablePointer<X86IdtEntry> = DescriptorTablePointer {
limit: 0,
base: 0
base: 0 as *const X86IdtEntry
};
pub static mut IDTR: DescriptorTablePointer<X86IdtEntry> = DescriptorTablePointer {
limit: 0,
base: 0 as *const X86IdtEntry
};
pub static mut IDT: [IdtEntry; 256] = [IdtEntry::new(); 256];
pub unsafe fn init() {
dtables::lidt(&INIT_IDTR);
}
pub unsafe fn init_paging() {
IDTR.limit = (IDT.len() * mem::size_of::<IdtEntry>() - 1) as u16;
IDTR.base = IDT.as_ptr() as u64;
IDTR.base = IDT.as_ptr() as *const X86IdtEntry;
// Set up exceptions
IDT[0].set_func(exception::divide_by_zero);
......@@ -58,9 +69,11 @@ pub unsafe fn init() {
IDT[46].set_func(irq::ata1);
IDT[47].set_func(irq::ata2);
// Set IPI handler (null)
IDT[0x40].set_func(ipi::ipi);
IDT[0x41].set_func(ipi::pit);
// Set IPI handlers
IDT[IpiKind::Wakeup as usize].set_func(ipi::wakeup);
IDT[IpiKind::Switch as usize].set_func(ipi::switch);
IDT[IpiKind::Tlb as usize].set_func(ipi::tlb);
IDT[IpiKind::Pit as usize].set_func(ipi::pit);
// Set syscall function
IDT[0x80].set_func(syscall::syscall);
......
use core::sync::atomic::Ordering;
use x86::shared::tlb;
use context;
use device::local_apic::LOCAL_APIC;
use super::irq::PIT_TICKS;
interrupt!(ipi, {
interrupt!(wakeup, {
LOCAL_APIC.eoi();
});
interrupt!(tlb, {
LOCAL_APIC.eoi();
tlb::flush_all();
});
interrupt!(switch, {
LOCAL_APIC.eoi();
let _ = context::switch();
});
interrupt!(pit, {
LOCAL_APIC.eoi();
......
......@@ -2,8 +2,10 @@ use core::sync::atomic::{AtomicUsize, Ordering, ATOMIC_USIZE_INIT};
use context;
use context::timeout;
use device::{local_apic, pic};
use device::pic;
use device::serial::{COM1, COM2};
use ipi::{ipi, IpiKind, IpiTarget};
use scheme::debug::debug_input;
use time;
//resets to 0 in context::switch()
......@@ -39,11 +41,6 @@ pub unsafe fn acknowledge(irq: usize) {
}
interrupt!(pit, {
// Wake up other CPUs
if cfg!(feature = "multi_core") {
local_apic::LOCAL_APIC.set_icr(3 << 18 | 1 << 14 | 0x41);
}
// Saves CPU time by not sending IRQ event irq_trigger(0);
const PIT_RATE: u64 = 2_250_286;
......@@ -57,6 +54,9 @@ interrupt!(pit, {
pic::MASTER.ack();
// Wake up other CPUs
ipi(IpiKind::Pit, IpiTarget::Other);
// Any better way of doing this?
timeout::trigger();
......@@ -75,12 +75,16 @@ interrupt!(cascade, {
});
interrupt!(com2, {
COM2.lock().receive();
while let Some(c) = COM2.lock().receive() {
debug_input(c);
}
pic::MASTER.ack();
});
interrupt!(com1, {
COM1.lock().receive();
while let Some(c) = COM1.lock().receive() {
debug_input(c);
}
pic::MASTER.ack();
});
......
use arch::x86_64::pti;
use arch::{gdt, pti};
use syscall;
use x86::shared::msr;
pub unsafe fn init() {
msr::wrmsr(msr::IA32_STAR, ((gdt::GDT_KERNEL_CODE as u64) << 3) << 32);
msr::wrmsr(msr::IA32_LSTAR, syscall_instruction as u64);
msr::wrmsr(msr::IA32_FMASK, 1 << 9);
msr::wrmsr(msr::IA32_KERNEL_GS_BASE, &gdt::TSS as *const _ as u64);
let efer = msr::rdmsr(msr::IA32_EFER);
msr::wrmsr(msr::IA32_EFER, efer | 1);
}
#[naked]
pub unsafe extern fn syscall_instruction() {
#[inline(never)]
unsafe fn inner(stack: &mut SyscallStack) -> usize {
let rbp;
asm!("" : "={rbp}"(rbp) : : : "intel", "volatile");
syscall::syscall(stack.rax, stack.rdi, stack.rsi, stack.rdx, stack.r10, stack.r8, rbp, stack)
}
// Yes, this is magic. No, you don't need to understand
asm!("xchg bx, bx
swapgs // Set gs segment to TSS
mov gs:[28], rsp // Save userspace rsp
mov rsp, gs:[4] // Load kernel rsp
push 5 * 8 + 3 // Push userspace data segment
push qword ptr gs:[28] // Push userspace rsp
mov qword ptr gs:[28], 0 // Clear userspace rsp
push r11 // Push rflags
push 4 * 8 + 3 // Push userspace code segment
push rcx // Push userspace return pointer
swapgs // Restore gs
"
:
:
:
: "intel", "volatile");
// Push scratch registers
asm!("push rax
push rbx
push rcx
push rdx
push rdi
push rsi
push r8
push r9
push r10
push r11
push fs
mov r11, 0x18
mov fs, r11"
: : : : "intel", "volatile");
// Get reference to stack variables
let rsp: usize;
asm!("" : "={rsp}"(rsp) : : : "intel", "volatile");
// Map kernel
pti::map();
let a = inner(&mut *(rsp as *mut SyscallStack));
// Unmap kernel
pti::unmap();
asm!("" : : "{rax}"(a) : : "intel", "volatile");
// Interrupt return
asm!("pop fs
pop r11
pop r10
pop r9
pop r8
pop rsi
pop rdi
pop rdx
pop rcx
pop rbx
add rsp, 8
iretq"
: : : : "intel", "volatile");
}
#[naked]
pub unsafe extern fn syscall() {
......
use core::mem;
use core::{mem, str};
use goblin::elf::sym;
use rustc_demangle::demangle;
use paging::{ActivePageTable, VirtualAddress};
......@@ -30,6 +31,7 @@ pub unsafe fn stack_trace() {
}
} else {
println!(" {:>016X}: RBP OVERFLOW", rbp);
break;
}
}
}
......@@ -75,54 +77,10 @@ pub unsafe fn symbol_trace(addr: usize) {
}
if end > start {
let sym_name = &elf.data[start .. end];
print!(" ");
if sym_name.starts_with(b"_ZN") {
// Skip _ZN
let mut i = 3;
let mut first = true;
while i < sym_name.len() {
// E is the end character
if sym_name[i] == b'E' {
break;
}
// Parse length string
let mut len = 0;
while i < sym_name.len() {
let b = sym_name[i];
if b >= b'0' && b <= b'9' {
i += 1;
len *= 10;
len += (b - b'0') as usize;
} else {
break;
}
}
// Print namespace seperator, if required
if first {
first = false;
} else {
print!("::");
}
// Print name string
let end = i + len;
while i < sym_name.len() && i < end {
print!("{}", sym_name[i] as char);
i += 1;
}
}
} else {
for &b in sym_name.iter() {
print!("{}", b as char);
}
let sym_slice = &elf.data[start .. end - 1];
if let Ok(sym_name) = str::from_utf8(sym_slice) {
println!(" {:#}", demangle(sym_name));
}
println!("");
}
}
}
......
#[derive(Clone, Copy, Debug)]
#[repr(u8)]
pub enum IpiKind {
Wakeup = 0x40,
Tlb = 0x41,
Switch = 0x42,
Pit = 0x43,
}
#[derive(Clone, Copy, Debug)]
#[repr(u8)]
pub enum IpiTarget {
Current = 1,
All = 2,
Other = 3,
}
#[cfg(not(feature = "multi_core"))]
#[inline(always)]
pub fn ipi(_kind: IpiKind, _target: IpiTarget) {}
#[cfg(feature = "multi_core")]
#[inline(always)]
pub fn ipi(kind: IpiKind, target: IpiTarget) {
use device::local_apic::LOCAL_APIC;
let icr = (target as u64) << 18 | 1 << 14 | (kind as u64);
unsafe { LOCAL_APIC.set_icr(icr) };
}
......@@ -20,6 +20,9 @@ pub mod idt;
/// Interrupt instructions
pub mod interrupt;
/// Inter-processor interrupts
pub mod ipi;
/// Paging
pub mod paging;
......
......@@ -3,7 +3,7 @@
use core::{mem, ptr};
use core::ops::{Deref, DerefMut};
use x86::{msr, tlb};
use x86::shared::{control_regs, msr, tlb};
use memory::{allocate_frames, Frame};
......@@ -126,7 +126,7 @@ pub unsafe fn init(cpu_id: usize, kernel_start: usize, kernel_end: usize, stack_
let page = Page::containing_address(VirtualAddress::new(frame.start_address().get() + ::KERNEL_OFFSET));
let result = mapper.map_to(page, frame, EntryFlags::PRESENT | EntryFlags::GLOBAL | EntryFlags::NO_EXECUTE | EntryFlags::WRITABLE);
// The flush can be ignored as this is not the active table. See later active_table.switch
unsafe { result.ignore(); }
/* unsafe */ { result.ignore(); }
}
}
......@@ -168,7 +168,7 @@ pub unsafe fn init(cpu_id: usize, kernel_start: usize, kernel_end: usize, stack_
let page = Page::containing_address(VirtualAddress::new(virt_addr));
let result = mapper.map_to(page, frame, flags);
// The flush can be ignored as this is not the active table. See later active_table.switch
unsafe { result.ignore(); }
/* unsafe */ { result.ignore(); }
}
}
......@@ -283,15 +283,13 @@ impl ActivePageTable {
}
pub fn switch(&mut self, new_table: InactivePageTable) -> InactivePageTable {
use x86::controlregs;
let old_table = InactivePageTable {
p4_frame: Frame::containing_address(
PhysicalAddress::new(unsafe { controlregs::cr3() } as usize)
PhysicalAddress::new(unsafe { control_regs::cr3() } as usize)
),
};
unsafe {
controlregs::cr3_write(new_table.p4_frame.start_address().get() as u64);
control_regs::cr3_write(new_table.p4_frame.start_address().get() as u64);
}
old_table
}
......@@ -307,10 +305,8 @@ impl ActivePageTable {
pub fn with<F>(&mut self, table: &mut InactivePageTable, temporary_page: &mut TemporaryPage, f: F)
where F: FnOnce(&mut Mapper)
{
use x86::controlregs;
{
let backup = Frame::containing_address(PhysicalAddress::new(unsafe { controlregs::cr3() as usize }));
let backup = Frame::containing_address(PhysicalAddress::new(unsafe { control_regs::cr3() as usize }));
// map temporary_page to current p4 table
let p4_table = temporary_page.map_table_frame(backup.clone(), EntryFlags::PRESENT | EntryFlags::WRITABLE | EntryFlags::NO_EXECUTE, self);
......@@ -331,8 +327,7 @@ impl ActivePageTable {
}
pub unsafe fn address(&self) -> usize {
use x86::controlregs;
controlregs::cr3() as usize
control_regs::cr3() as usize
}
}
......
......@@ -9,7 +9,7 @@ use memory::allocate_frames;
use super::entry::{EntryFlags, Entry};
use super::ENTRY_COUNT;
pub const P4: *mut Table<Level4> = 0xffff_ffff_ffff_f000 as *mut _;
pub const P4: *mut Table<Level4> = (::RECURSIVE_PAGE_OFFSET | 0x7ffffff000) as *mut _;
pub trait TableLevel {}
......
......@@ -16,6 +16,7 @@ use device;
use gdt;
use idt;
use interrupt;
use log;
use memory;
use paging;
......@@ -72,17 +73,26 @@ pub unsafe extern fn kstart(args_ptr: *const KernelArgs) -> ! {
println!("Stack: {:X}:{:X}", stack_base, stack_base + stack_size);
println!("Env: {:X}:{:X}", env_base, env_base + env_size);
// Set up GDT before paging
gdt::init();
// Set up IDT before paging
idt::init();
// Initialize memory management
memory::init(0, kernel_base + ((kernel_size + 4095)/4096) * 4096);
// Initialize paging
let (mut active_table, tcb_offset) = paging::init(0, kernel_base, kernel_base + kernel_size, stack_base, stack_base + stack_size);
// Set up GDT
gdt::init(tcb_offset, stack_base + stack_size);
// Set up GDT after paging with TLS
gdt::init_paging(tcb_offset, stack_base + stack_size);
// Set up IDT
idt::init();
idt::init_paging();
// Set up syscall instruction
interrupt::syscall::init();
// Test tdata and tbss
{
......@@ -102,6 +112,9 @@ pub unsafe extern fn kstart(args_ptr: *const KernelArgs) -> ! {
// Setup kernel heap
allocator::init(&mut active_table);
// Activate memory logging
log::init();
// Use graphical debug
#[cfg(feature="graphical_debug")]
graphical_debug::init(&mut active_table);
......@@ -151,14 +164,23 @@ pub unsafe extern fn kstart_ap(args_ptr: *const KernelArgsAp) -> ! {
assert_eq!(BSS_TEST_ZERO, 0);
assert_eq!(DATA_TEST_NONZERO, 0xFFFF_FFFF_FFFF_FFFF);
// Set up GDT before paging
gdt::init();
// Set up IDT before paging
idt::init();
// Initialize paging
let tcb_offset = paging::init_ap(cpu_id, bsp_table, stack_start, stack_end);
// Set up GDT for AP
gdt::init(tcb_offset, stack_end);
// Set up GDT with TLS
gdt::init_paging(tcb_offset, stack_end);
// Set up IDT for AP
idt::init();
idt::init_paging();
// Set up syscall instruction
interrupt::syscall::init();
// Test tdata and tbss
{
......
......@@ -14,7 +14,10 @@ pub unsafe extern fn kreset() -> ! {
port.write(0xFE);
}
// TODO: Use triple fault to guarantee reset
// Use triple fault to guarantee reset
asm!("cli" : : : : "intel", "volatile");
asm!("lidt cs:0" : : : : "intel", "volatile");
asm!("int $$3" : : : : "intel", "volatile");
unreachable!();
}
......@@ -33,9 +36,17 @@ pub unsafe extern fn kstop() -> ! {
Pio::<u8>::new(port).write(c);
}
// Magic shutdown using qemu default ACPI method
{
let port = 0x604;
let data = 0x2000;
println!("Shutdown with outb(0x{:X}, 0x{:X})", port, data);
Pio::<u16>::new(port).write(data);
}
// Magic code for VMWare. Also a hard lock.
println!("Shutdown with cli hlt");
asm!("cli; hlt" : : : : "intel", "volatile");
unreachable!();
loop {
asm!("cli; hlt" : : : : "intel", "volatile");
}
}
use alloc::arc::Arc;
use alloc::sync::Arc;
use alloc::boxed::Box;
use alloc::{BTreeMap, Vec, VecDeque};
use alloc::vec::Vec;
use alloc::collections::VecDeque;
use core::alloc::{GlobalAlloc, Layout};
use core::cmp::Ordering;
use core::mem;
use spin::Mutex;
use arch::paging::PAGE_SIZE;
use context::arch;
use context::file::FileDescriptor;
use context::memory::{Grant, Memory, SharedMemory, Tls};
use device;
use ipi::{ipi, IpiKind, IpiTarget};
use scheme::{SchemeNamespace, FileHandle};
use syscall::data::{Event, SigAction};
use syscall::data::SigAction;
use syscall::flag::SIG_DFL;
use sync::{WaitMap, WaitQueue};
use sync::WaitMap;
/// Unique identifier for a context (i.e. `pid`).
use ::core::sync::atomic::AtomicUsize;
......@@ -108,6 +111,8 @@ pub struct Context {
pub egid: u32,
/// The effective namespace id
pub ens: SchemeNamespace,
/// Process umask
pub umask: usize,
/// Status of context
pub status: Status,
/// Context running or not
......@@ -116,6 +121,10 @@ pub struct Context {
pub cpu_id: Option<usize>,
/// Current system call
pub syscall: Option<(usize, usize, usize, usize, usize, usize)>,
/// Head buffer to use when system call buffers are not page aligned
pub syscall_head: Box<[u8]>,
/// Tail buffer to use when system call buffers are not page aligned
pub syscall_tail: Box<[u8]>,
/// Context is halting parent
pub vfork: bool,
/// Context is being waited on
......@@ -150,10 +159,6 @@ pub struct Context {
pub name: Arc<Mutex<Box<[u8]>>>,
/// The current working directory
pub cwd: Arc<Mutex<Vec<u8>>>,
/// Kernel events
pub events: Arc<WaitQueue<Event>>,
/// The process environment
pub env: Arc<Mutex<BTreeMap<Box<[u8]>, Arc<Mutex<Vec<u8>>>>>>,
/// The open files in the scheme
pub files: Arc<Mutex<Vec<Option<FileDescriptor>>>>,
/// Singal actions
......@@ -162,6 +167,9 @@ pub struct Context {
impl Context {
pub fn new(id: ContextId) -> Context {
let syscall_head = unsafe { Box::from_raw(::ALLOCATOR.alloc(Layout::from_size_align_unchecked(PAGE_SIZE, PAGE_SIZE)) as *mut [u8; PAGE_SIZE]) };
let syscall_tail = unsafe { Box::from_raw(::ALLOCATOR.alloc(Layout::from_size_align_unchecked(PAGE_SIZE, PAGE_SIZE)) as *mut [u8; PAGE_SIZE]) };
Context {
id: id,
pgid: id,
......@@ -172,10 +180,13 @@ impl Context {
euid: 0,
egid: 0,
ens: SchemeNamespace::from(0),
umask: 0o022,
status: Status::Blocked,
running: false,
cpu_id: None,
syscall: None,
syscall_head: syscall_head,
syscall_tail: syscall_tail,
vfork: false,
waitpid: Arc::new(WaitMap::new()),
pending: VecDeque::new(),
......@@ -193,8 +204,6 @@ impl Context {
grants: Arc::new(Mutex::new(Vec::new())),
name: Arc::new(Mutex::new(Vec::new().into_boxed_slice())),
cwd: Arc::new(Mutex::new(Vec::new())),
events: Arc::new(WaitQueue::new()),
env: Arc::new(Mutex::new(BTreeMap::new())),
files: Arc::new(Mutex::new(Vec::new())),
actions: Arc::new(Mutex::new(vec![(
SigAction {
......@@ -290,15 +299,14 @@ impl Context {
pub fn unblock(&mut self) -> bool {
if self.status == Status::Blocked {
self.status = Status::Runnable;
if cfg!(feature = "multi_core") {
if let Some(cpu_id) = self.cpu_id {
if cpu_id != ::cpu_id() {
// Send IPI if not on current CPU
// TODO: Make this more architecture independent
unsafe { device::local_apic::LOCAL_APIC.set_icr(3 << 18 | 1 << 14 | 0x40) };
}
}
if let Some(cpu_id) = self.cpu_id {
if cpu_id != ::cpu_id() {
// Send IPI if not on current CPU
ipi(IpiKind::Wakeup, IpiTarget::Other);
}
}
true
} else {
false
......
use alloc::arc::{Arc, Weak};
use alloc::BTreeMap;
use spin::{Once, RwLock, RwLockReadGuard, RwLockWriteGuard};
use context;
use scheme::{FileHandle, SchemeId};
use sync::WaitQueue;
use syscall::data::Event;
type EventList = Weak<WaitQueue<Event>>;
#[derive(PartialEq, Eq, PartialOrd, Ord)]
pub struct RegKey {
scheme_id: SchemeId,
event_id: usize,
}
#[derive(PartialEq, Eq, PartialOrd, Ord)]
pub struct ProcessKey {
context_id: context::context::ContextId,
fd: FileHandle,
}
type Registry = BTreeMap<RegKey, BTreeMap<ProcessKey, EventList>>;
static REGISTRY: Once<RwLock<Registry>> = Once::new();
/// Initialize registry, called if needed
fn init_registry() -> RwLock<Registry> {
RwLock::new(Registry::new())
}
/// Get the global schemes list, const
fn registry() -> RwLockReadGuard<'static, Registry> {
REGISTRY.call_once(init_registry).read()
}
/// Get the global schemes list, mutable
pub fn registry_mut() -> RwLockWriteGuard<'static, Registry> {
REGISTRY.call_once(init_registry).write()
}
pub fn register(fd: FileHandle, scheme_id: SchemeId, event_id: usize) -> bool {
let (context_id, events) = {
let contexts = context::contexts();
let context_lock = contexts.current().expect("event::register: No context");
let context = context_lock.read();
(context.id, Arc::downgrade(&context.events))
};
let mut registry = registry_mut();
let entry = registry.entry(RegKey {
scheme_id: scheme_id,
event_id: event_id
}).or_insert_with(|| {
BTreeMap::new()
});
let process_key = ProcessKey {
context_id: context_id,
fd: fd
};
if entry.contains_key(&process_key) {
false
} else {
entry.insert(process_key, events);
true
}
}
pub fn unregister(fd: FileHandle, scheme_id: SchemeId, event_id: usize) {
let mut registry = registry_mut();
let mut remove = false;
let key = RegKey {
scheme_id: scheme_id,
event_id: event_id
};
if let Some(entry) = registry.get_mut(&key) {
let process_key = ProcessKey {
context_id: context::context_id(),
fd: fd,
};
entry.remove(&process_key);
if entry.is_empty() {
remove = true;
}
}
if remove {
registry.remove(&key);
}
}
pub fn trigger(scheme_id: SchemeId, event_id: usize, flags: usize, data: usize) {
let registry = registry();
let key = RegKey {
scheme_id: scheme_id,
event_id: event_id
};
if let Some(event_lists) = registry.get(&key) {
for entry in event_lists.iter() {
if let Some(event_list) = entry.1.upgrade() {
event_list.send(Event {
id: (entry.0).fd.into(),
flags: flags,
data: data
});
}
}
}
}
//! File structs
use alloc::arc::Arc;
use alloc::sync::Arc;
use event;
use spin::RwLock;
use scheme::{self, SchemeId};
use syscall::error::{Result, Error, EBADF};
use scheme::FileHandle;
use context;
/// A file description
#[derive(Debug)]
......@@ -20,23 +19,21 @@ pub struct FileDescription {
/// A file descriptor
#[derive(Clone, Debug)]
#[must_use = "File descriptors must be closed"]
pub struct FileDescriptor {
/// Corresponding file description
pub description: Arc<RwLock<FileDescription>>,
/// If events are on, this is the event ID
pub event: Option<usize>,
/// Cloexec flag
pub cloexec: bool,
}
impl FileDescriptor {
pub fn close(self, fd: FileHandle) -> Result<usize> {
if let Some(event_id) = self.event {
context::event::unregister(fd, self.description.read().scheme, event_id);
}
pub fn close(self) -> Result<usize> {
if let Ok(file) = Arc::try_unwrap(self.description) {
let file = file.into_inner();
event::unregister_file(file.scheme, file.number);
let scheme = {
let schemes = scheme::schemes();
let scheme = schemes.get(file.scheme).ok_or(Error::new(EBADF))?;
......
use alloc::allocator::{Alloc, Layout};
use alloc::arc::Arc;
use alloc::sync::Arc;
use alloc::boxed::Box;
use alloc::heap::Heap;
use alloc::BTreeMap;
use alloc::collections::BTreeMap;
use core::alloc::{GlobalAlloc, Layout};
use core::mem;
use core::sync::atomic::Ordering;
use paging;
......@@ -36,7 +35,7 @@ impl ContextList {
self.map.get(&super::CONTEXT_ID.load(Ordering::SeqCst))
}
pub fn iter(&self) -> ::alloc::btree_map::Iter<ContextId, Arc<RwLock<Context>>> {
pub fn iter(&self) -> ::alloc::collections::btree_map::Iter<ContextId, Arc<RwLock<Context>>> {
self.map.iter()
}
......@@ -67,7 +66,7 @@ impl ContextList {
let context_lock = self.new_context()?;
{
let mut context = context_lock.write();
let mut fx = unsafe { Box::from_raw(Heap.alloc(Layout::from_size_align_unchecked(512, 16)).unwrap() as *mut [u8; 512]) };