diff --git a/11_virtual_memory/Makefile b/11_virtual_memory/Makefile index 688b1796..39f77ce4 100644 --- a/11_virtual_memory/Makefile +++ b/11_virtual_memory/Makefile @@ -21,7 +21,7 @@ ifeq ($(BSP),rpi3) QEMU_RELEASE_ARGS = -serial stdio -display none OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi3.cfg JTAG_BOOT_IMAGE = jtag_boot_rpi3.img - LINKER_FILE = src/bsp/rpi/link.ld + LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a53 else ifeq ($(BSP),rpi4) TARGET = aarch64-unknown-none-softfloat @@ -31,7 +31,7 @@ else ifeq ($(BSP),rpi4) # QEMU_RELEASE_ARGS = -serial stdio -display none OPENOCD_ARG = -f /openocd/tcl/interface/ftdi/olimex-arm-usb-tiny-h.cfg -f /openocd/rpi4.cfg JTAG_BOOT_IMAGE = jtag_boot_rpi4.img - LINKER_FILE = src/bsp/rpi/link.ld + LINKER_FILE = src/bsp/raspberrypi/link.ld RUSTC_MISC_ARGS = -C target-cpu=cortex-a72 endif @@ -74,8 +74,7 @@ $(OUTPUT): $(CARGO_OUTPUT) $(OBJCOPY_CMD) $< $(OUTPUT) doc: - cargo xdoc --target=$(TARGET) --features bsp_$(BSP) --document-private-items - xdg-open target/$(TARGET)/doc/kernel/index.html + cargo xdoc --target=$(TARGET) --features bsp_$(BSP) --document-private-items --open ifeq ($(QEMU_MACHINE_TYPE),) qemu: diff --git a/11_virtual_memory/README.md b/11_virtual_memory/README.md index 539fba04..c5d39ff5 100644 --- a/11_virtual_memory/README.md +++ b/11_virtual_memory/README.md @@ -10,8 +10,9 @@ purposes, we write to a remapped `UART`. - [Introduction](#introduction) - [MMU and paging theory](#mmu-and-paging-theory) - [Approach](#approach) - * [BSP: `bsp/rpi/virt_mem_layout.rs`](#bsp-bsprpivirt_mem_layoutrs) - * [Arch: `arch/aarch64/mmu.rs`](#arch-archaarch64mmurs) + * [Generic Kernel code: `memory/mmu.rs`](#generic-kernel-code-memorymmurs) + * [BSP: `bsp/raspberrypi/memory/mmu.rs`](#bsp-bspraspberrypimemorymmurs) + * [AArch64: `_arch/aarch64/memory/mmu.rs`](#aarch64-_archaarch64memorymmurs) * [`link.ld`](#linkld) - [Address translation examples](#address-translation-examples) * [Address translation using a 64 KiB page descriptor](#address-translation-using-a-64-kib-page-descriptor) @@ -40,32 +41,42 @@ Back from reading `Chapter 12` already? Good job :+1:! ## Approach -- Everything is mapped using a `64 KiB` granule. -- Attributes of different regions (e.g. R/W, no-execute, cached/uncached, etc...) are set in a - high-level data structure in `BSP` code. -- `Arch` code picks up this high-level description and maps it using its specific MMU HW. +1. The generic `kernel` part: `src/memory/mmu.rs` provides architecture-agnostic descriptor types + for composing a high-level data structure that describes the kernel's virtual memory layout: + `memory::mmu::KernelVirtualLayout`. +2. The `BSP` part: `src/bsp/raspberrypi/memory/mmu.rs` contains a static instance of + `KernelVirtualLayout` and makes it accessible throug the function + `bsp::memory::mmu::virt_mem_layout()`. +3. The `aarch64` part: `src/_arch/aarch64/memory/mmu.rs` contains the actual `MMU` driver. It picks + up the `BSP`'s high-level `KernelVirtualLayout` and maps it using a `64 KiB` granule. -### BSP: `bsp/rpi/virt_mem_layout.rs` +### Generic Kernel code: `memory/mmu.rs` -This file is used to describe our kernel's memory layout in a high-level abstraction using our own -descriptor format. We can define ranges of arbitrary length and set respective attributes, for -example if the bits and bytes in this range should be executable or not. +The descriptor types provided in this file are building blocks which help to describe attributes of +different memory regions. For example, R/W, no-execute, cached/uncached, and so on. -The descriptors we use here are agnostic of the hardware `MMU`'s actual descriptors, and we are also -agnostic of the paging granule the `MMU` will use. Having this distinction is less of a technical -need and more a convenience feature for us in order to easily describe the kernel's memory layout, -and hopefully it makes the whole concept a bit more graspable for the reader. +The descriptors are agnostic of the hardware `MMU`'s actual descriptors. Different `BSP`s can use +these types to produce a high-level description of the kernel's virtual memory layout. The actual +`MMU` driver for the real HW will consume these types as an input. -The file contains an instance of `memory::KernelVirtualLayout`,which stores these descriptors. The -policy is to only store regions that are **not** ordinary, normal chacheable DRAM. However, nothing -prevents you from defining those too if you wish to. Here is an example for the device MMIO region: +This way, we achieve a clean abstraction between `BSP` and `_arch` code, which allows exchanging one +without needing to adapt the other. + +### BSP: `bsp/raspberrypi/memory/mmu.rs` + +This file contains an instance of `KernelVirtualLayout`, which stores the descriptors mentioned +previously. The `BSP` is the correct place to do this, because it has knowledge of the target +board's memory map. + +The policy is to only describe regions that are **not** ordinary, normal chacheable DRAM. However, +nothing prevents you from defining those too if you wish to. Here is an example for the device MMIO +region: ```rust -// Device MMIO. RangeDescriptor { name: "Device MMIO", virtual_range: || { - RangeInclusive::new(memory_map::mmio::BASE, memory_map::mmio::END_INCLUSIVE) + RangeInclusive::new(memory::map::mmio::BASE, memory::map::mmio::END_INCLUSIVE) }, translation: Translation::Identity, attribute_fields: AttributeFields { @@ -76,27 +87,27 @@ RangeDescriptor { }, ``` -`KernelVirtualLayout` also provides the following method: +`KernelVirtualLayout` itself implements the following method: ```rust pub fn get_virt_addr_properties( - &self, - virt_addr: usize, - ) -> Result<(usize, AttributeFields), &'static str> + &self, + virt_addr: usize, +) -> Result<(usize, AttributeFields), &'static str> ``` -It will be used by the `arch`'s MMU code to request attributes for a virtual address and the -translation of the address. The function scans for a descriptor that contains the queried address, -and returns the respective findings for the first entry that is a hit. If no entry is found, it -returns default attributes for normal chacheable DRAM and the input address, hence telling the `MMU` -code that the requested address should be `identity mapped`. +It will be used by the `_arch/aarch64`'s `MMU` code to request attributes for a virtual address and +the translation of the address. The function scans for a descriptor that contains the queried +address, and returns the respective findings for the first entry that is a hit. If no entry is +found, it returns default attributes for normal chacheable DRAM and the input address, hence telling +the `MMU` code that the requested address should be `identity mapped`. Due to this default return, it is technicall not needed to define normal cacheable DRAM regions. -### Arch: `arch/aarch64/mmu.rs` +### AArch64: `_arch/aarch64/memory/mmu.rs` -This file contains the `AArch64` specific code. It is a driver, if you like, and the paging granule -is hardcoded here (`64 KiB` page descriptors). +This file contains the `AArch64` `MMU` driver. The paging granule is hardcoded here (`64 KiB` page +descriptors). The actual page tables are stored in a global instance of the `PageTables` struct: @@ -120,27 +131,30 @@ struct PageDescriptor(u64); #[repr(C)] #[repr(align(65536))] struct PageTables { - // Page descriptors, covering 64 KiB windows per entry. + /// Page descriptors, covering 64 KiB windows per entry. lvl3: [[PageDescriptor; 8192]; N], - // Table descriptors, covering 512 MiB windows. + + /// Table descriptors, covering 512 MiB windows. lvl2: [TableDescriptor; N], } /// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. -const ENTRIES_512_MIB: usize = bsp::addr_space_size() >> FIVETWELVE_MIB_SHIFT; +const ENTRIES_512_MIB: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; /// The page tables. /// -/// Supposed to land in `.bss`. Therefore, ensure that they boil down to all "0" entries. +/// # Safety +/// +/// - Supposed to land in `.bss`. Therefore, ensure that they boil down to all "0" entries. static mut TABLES: PageTables<{ ENTRIES_512_MIB }> = PageTables { lvl3: [[PageDescriptor(0); 8192]; ENTRIES_512_MIB], lvl2: [TableDescriptor(0); ENTRIES_512_MIB], }; ``` -They are populated using `get_virt_addr_properties()` and a bunch of utility functions that convert -our own descriptors to the actual `64 bit` integer entries needed by the MMU hardware for the page -table arrays. +They are populated using `bsp::memory::mmu::virt_mem_layout().get_virt_addr_properties()` and a +bunch of utility functions that convert our own descriptors to the actual `64 bit` integer entries +needed by the `MMU` hardware for the page table arrays. Each page table has an entry (`AttrIndex`) that indexes into the [MAIR_EL1] register, which holds information about the cacheability of the respective page. We currently define normal cacheable @@ -165,9 +179,9 @@ fn set_up_mair() { ``` Afterwards, the [Translation Table Base Register 0 - EL1] is set up with the base address of the -`lvl2` and the [Translation Control Register - EL1] is configured. +`lvl2` tables and the [Translation Control Register - EL1] is configured. -Finally, the MMU is turned on through the [System Control Register - EL1]. The last step also +Finally, the `MMU` is turned on through the [System Control Register - EL1]. The last step also enables caching for data and instructions. [Translation Table Base Register 0 - EL1]: https://docs.rs/crate/cortex-a/2.4.0/source/src/regs/ttbr0_el1.rs @@ -202,7 +216,8 @@ The following block diagram visualizes the underlying translation for the second The MMU init code is again a good example to see the great potential of Rust's zero-cost abstractions[[1]][[2]] for embedded programming. -Take this piece of code for setting up the `MAIR_EL1` register using the [cortex-a] crate: +Let's take a look again at the piece of code for setting up the `MAIR_EL1` register using the +[cortex-a] crate: [1]: https://blog.rust-lang.org/2015/05/11/traits.html [2]: https://ruudvanasseldonk.com/2016/11/30/zero-cost-abstractions @@ -232,17 +247,17 @@ data sheet. Looking at the generated code, we can see that despite all the type- abstractions, it boils down to two assembly instructions: ```text -00000000000818d8 kernel::arch::aarch64::mmu::init::h97006e19222f36e2: +00000000000815e0 ::init::hed32b31a58c93b32: ... - 8191c: 88 e0 9f 52 mov w8, #0xff04 + 8161c: mov w8, #0xff04 ... - 81924: 08 a2 18 d5 msr MAIR_EL1, x8 + 81644: msr MAIR_EL1, x8 ``` ## Test it ```console -» make chainboot +$ make chainboot [...] Minipush 1.0 @@ -259,776 +274,24 @@ Minipush 1.0 [MP] ⏩ Pushing 64 KiB ========================================🦀 100% 32 KiB/s Time: 00:00:02 [ML] Loaded! Executing the payload now -[ 2.932547] Booting on: Raspberry Pi 3 -[ 2.933631] MMU online. Special regions: -[ 2.935543] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data -[ 2.939626] 0x1fff0000 - 0x1fffffff | 64 KiB | Dev RW PXN | Remapped Device MMIO -[ 2.943579] 0x3f000000 - 0x3fffffff | 16 MiB | Dev RW PXN | Device MMIO -[ 2.947141] Current privilege level: EL1 -[ 2.949052] Exception handling state: -[ 2.950833] Debug: Masked -[ 2.952397] SError: Masked -[ 2.953960] IRQ: Masked -[ 2.955524] FIQ: Masked -[ 2.957088] Architectural timer resolution: 52 ns -[ 2.959390] Drivers loaded: -[ 2.960737] 1. GPIO -[ 2.961996] 2. PL011Uart -[ 2.963473] Timer test, spinning for 1 second +[ 3.085343] Booting on: Raspberry Pi 3 +[ 3.086427] MMU online. Special regions: +[ 3.088339] 0x00080000 - 0x0008ffff | 64 KiB | C RO PX | Kernel code and RO data +[ 3.092422] 0x1fff0000 - 0x1fffffff | 64 KiB | Dev RW PXN | Remapped Device MMIO +[ 3.096375] 0x3f000000 - 0x4000ffff | 16 MiB | Dev RW PXN | Device MMIO +[ 3.099937] Current privilege level: EL1 +[ 3.101848] Exception handling state: +[ 3.103629] Debug: Masked +[ 3.105192] SError: Masked +[ 3.106756] IRQ: Masked +[ 3.108320] FIQ: Masked +[ 3.109884] Architectural timer resolution: 52 ns +[ 3.112186] Drivers loaded: +[ 3.113532] 1. BCM GPIO +[ 3.114966] 2. BCM PL011 UART +[ 3.116660] Timer test, spinning for 1 second [ !!! ] Writing through the remapped UART at 0x1FFF_1000 -[ 3.967641] Echoing input now - +[ 4.120828] Echoing input now ``` ## Diff to previous -```diff - -diff -uNr 10_privilege_level/src/arch/aarch64/mmu.rs 11_virtual_memory/src/arch/aarch64/mmu.rs ---- 10_privilege_level/src/arch/aarch64/mmu.rs -+++ 11_virtual_memory/src/arch/aarch64/mmu.rs -@@ -0,0 +1,295 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2020 Andre Richter -+ -+//! Memory Management Unit. -+//! -+//! Static page tables, compiled on boot; Everything 64 KiB granule. -+ -+use crate::{ -+ bsp, interface, -+ memory::{AccessPermissions, AttributeFields, MemAttributes}, -+}; -+use core::convert; -+use cortex_a::{barrier, regs::*}; -+use register::register_bitfields; -+ -+// A table descriptor, as per AArch64 Reference Manual Figure D4-15. -+register_bitfields! {u64, -+ STAGE1_TABLE_DESCRIPTOR [ -+ /// Physical address of the next page table. -+ NEXT_LEVEL_TABLE_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] -+ -+ TYPE OFFSET(1) NUMBITS(1) [ -+ Block = 0, -+ Table = 1 -+ ], -+ -+ VALID OFFSET(0) NUMBITS(1) [ -+ False = 0, -+ True = 1 -+ ] -+ ] -+} -+ -+// A level 3 page descriptor, as per AArch64 Reference Manual Figure D4-17. -+register_bitfields! {u64, -+ STAGE1_PAGE_DESCRIPTOR [ -+ /// Privileged execute-never. -+ PXN OFFSET(53) NUMBITS(1) [ -+ False = 0, -+ True = 1 -+ ], -+ -+ /// Physical address of the next page table (lvl2) or the page descriptor (lvl3). -+ OUTPUT_ADDR_64KiB OFFSET(16) NUMBITS(32) [], // [47:16] -+ -+ /// Access flag. -+ AF OFFSET(10) NUMBITS(1) [ -+ False = 0, -+ True = 1 -+ ], -+ -+ /// Shareability field. -+ SH OFFSET(8) NUMBITS(2) [ -+ OuterShareable = 0b10, -+ InnerShareable = 0b11 -+ ], -+ -+ /// Access Permissions. -+ AP OFFSET(6) NUMBITS(2) [ -+ RW_EL1 = 0b00, -+ RW_EL1_EL0 = 0b01, -+ RO_EL1 = 0b10, -+ RO_EL1_EL0 = 0b11 -+ ], -+ -+ /// Memory attributes index into the MAIR_EL1 register. -+ AttrIndx OFFSET(2) NUMBITS(3) [], -+ -+ TYPE OFFSET(1) NUMBITS(1) [ -+ Block = 0, -+ Table = 1 -+ ], -+ -+ VALID OFFSET(0) NUMBITS(1) [ -+ False = 0, -+ True = 1 -+ ] -+ ] -+} -+ -+const SIXTYFOUR_KIB_SHIFT: usize = 16; // log2(64 * 1024) -+const FIVETWELVE_MIB_SHIFT: usize = 29; // log2(512 * 1024 * 1024) -+ -+/// A table descriptor for 64 KiB aperture. -+/// -+/// The output points to the next table. -+#[derive(Copy, Clone)] -+#[repr(transparent)] -+struct TableDescriptor(u64); -+ -+/// A page descriptor with 64 KiB aperture. -+/// -+/// The output points to physical memory. -+#[derive(Copy, Clone)] -+#[repr(transparent)] -+struct PageDescriptor(u64); -+ -+/// Big monolithic struct for storing the page tables. Individual levels must be 64 KiB aligned, -+/// hence the "reverse" order of appearance. -+#[repr(C)] -+#[repr(align(65536))] -+struct PageTables { -+ // Page descriptors, covering 64 KiB windows per entry. -+ lvl3: [[PageDescriptor; 8192]; N], -+ // Table descriptors, covering 512 MiB windows. -+ lvl2: [TableDescriptor; N], -+} -+ -+/// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. -+const ENTRIES_512_MIB: usize = bsp::addr_space_size() >> FIVETWELVE_MIB_SHIFT; -+ -+/// The page tables. -+/// -+/// Supposed to land in `.bss`. Therefore, ensure that they boil down to all "0" entries. -+static mut TABLES: PageTables<{ ENTRIES_512_MIB }> = PageTables { -+ lvl3: [[PageDescriptor(0); 8192]; ENTRIES_512_MIB], -+ lvl2: [TableDescriptor(0); ENTRIES_512_MIB], -+}; -+ -+trait BaseAddr { -+ fn base_addr_u64(&self) -> u64; -+ fn base_addr_usize(&self) -> usize; -+} -+ -+impl BaseAddr for [T; N] { -+ fn base_addr_u64(&self) -> u64 { -+ self as *const T as u64 -+ } -+ -+ fn base_addr_usize(&self) -> usize { -+ self as *const T as usize -+ } -+} -+ -+impl convert::From for TableDescriptor { -+ fn from(next_lvl_table_addr: usize) -> Self { -+ let shifted = next_lvl_table_addr >> SIXTYFOUR_KIB_SHIFT; -+ let val = (STAGE1_TABLE_DESCRIPTOR::VALID::True -+ + STAGE1_TABLE_DESCRIPTOR::TYPE::Table -+ + STAGE1_TABLE_DESCRIPTOR::NEXT_LEVEL_TABLE_ADDR_64KiB.val(shifted as u64)) -+ .value; -+ -+ TableDescriptor(val) -+ } -+} -+ -+/// Convert the kernel's generic memory range attributes to HW-specific attributes of the MMU. -+impl convert::From -+ for register::FieldValue -+{ -+ fn from(attribute_fields: AttributeFields) -> Self { -+ // Memory attributes. -+ let mut desc = match attribute_fields.mem_attributes { -+ MemAttributes::CacheableDRAM => { -+ STAGE1_PAGE_DESCRIPTOR::SH::InnerShareable -+ + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::NORMAL) -+ } -+ MemAttributes::Device => { -+ STAGE1_PAGE_DESCRIPTOR::SH::OuterShareable -+ + STAGE1_PAGE_DESCRIPTOR::AttrIndx.val(mair::DEVICE) -+ } -+ }; -+ -+ // Access Permissions. -+ desc += match attribute_fields.acc_perms { -+ AccessPermissions::ReadOnly => STAGE1_PAGE_DESCRIPTOR::AP::RO_EL1, -+ AccessPermissions::ReadWrite => STAGE1_PAGE_DESCRIPTOR::AP::RW_EL1, -+ }; -+ -+ // Execute Never. -+ desc += if attribute_fields.execute_never { -+ STAGE1_PAGE_DESCRIPTOR::PXN::True -+ } else { -+ STAGE1_PAGE_DESCRIPTOR::PXN::False -+ }; -+ -+ desc -+ } -+} -+ -+impl PageDescriptor { -+ fn new(output_addr: usize, attribute_fields: AttributeFields) -> PageDescriptor { -+ let shifted = output_addr >> SIXTYFOUR_KIB_SHIFT; -+ let val = (STAGE1_PAGE_DESCRIPTOR::VALID::True -+ + STAGE1_PAGE_DESCRIPTOR::AF::True -+ + attribute_fields.into() -+ + STAGE1_PAGE_DESCRIPTOR::TYPE::Table -+ + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted as u64)) -+ .value; -+ -+ PageDescriptor(val) -+ } -+} -+ -+/// Constants for indexing the MAIR_EL1. -+#[allow(dead_code)] -+mod mair { -+ pub const DEVICE: u64 = 0; -+ pub const NORMAL: u64 = 1; -+} -+ -+/// Setup function for the MAIR_EL1 register. -+fn set_up_mair() { -+ // Define the memory types being mapped. -+ MAIR_EL1.write( -+ // Attribute 1 - Cacheable normal DRAM. -+ MAIR_EL1::Attr1_HIGH::Memory_OuterWriteBack_NonTransient_ReadAlloc_WriteAlloc -+ + MAIR_EL1::Attr1_LOW_MEMORY::InnerWriteBack_NonTransient_ReadAlloc_WriteAlloc -+ -+ // Attribute 0 - Device. -+ + MAIR_EL1::Attr0_HIGH::Device -+ + MAIR_EL1::Attr0_LOW_DEVICE::Device_nGnRE, -+ ); -+} -+ -+/// Iterates over all static page table entries and fills them at once. -+/// -+/// # Safety -+/// -+/// - Modifies a `static mut`. Ensure it only happens from here. -+unsafe fn populate_pt_entries() -> Result<(), &'static str> { -+ for (l2_nr, l2_entry) in TABLES.lvl2.iter_mut().enumerate() { -+ *l2_entry = TABLES.lvl3[l2_nr].base_addr_usize().into(); -+ -+ for (l3_nr, l3_entry) in TABLES.lvl3[l2_nr].iter_mut().enumerate() { -+ let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); -+ -+ let (output_addr, attribute_fields) = -+ bsp::virt_mem_layout().get_virt_addr_properties(virt_addr)?; -+ -+ *l3_entry = PageDescriptor::new(output_addr, attribute_fields); -+ } -+ } -+ -+ Ok(()) -+} -+ -+/// Configure various settings of stage 1 of the EL1 translation regime. -+fn configure_translation_control() { -+ let ips = ID_AA64MMFR0_EL1.read(ID_AA64MMFR0_EL1::PARange); -+ TCR_EL1.write( -+ TCR_EL1::TBI0::Ignored -+ + TCR_EL1::IPS.val(ips) -+ + TCR_EL1::TG0::KiB_64 -+ + TCR_EL1::SH0::Inner -+ + TCR_EL1::ORGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable -+ + TCR_EL1::IRGN0::WriteBack_ReadAlloc_WriteAlloc_Cacheable -+ + TCR_EL1::EPD0::EnableTTBR0Walks -+ + TCR_EL1::T0SZ.val(32), // TTBR0 spans 4 GiB total. -+ ); -+} -+ -+//-------------------------------------------------------------------------------------------------- -+// Arch-public -+//-------------------------------------------------------------------------------------------------- -+ -+pub struct MMU; -+ -+//-------------------------------------------------------------------------------------------------- -+// OS interface implementations -+//-------------------------------------------------------------------------------------------------- -+ -+impl interface::mm::MMU for MMU { -+ unsafe fn init(&self) -> Result<(), &'static str> { -+ // Fail early if translation granule is not supported. Both RPis support it, though. -+ if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { -+ return Err("64 KiB translation granule not supported"); -+ } -+ -+ // Prepare the memory attribute indirection register. -+ set_up_mair(); -+ -+ // Populate page tables. -+ populate_pt_entries()?; -+ -+ // Set the "Translation Table Base Register". -+ TTBR0_EL1.set_baddr(TABLES.lvl2.base_addr_u64()); -+ -+ configure_translation_control(); -+ -+ // Switch the MMU on. -+ // -+ // First, force all previous changes to be seen before the MMU is enabled. -+ barrier::isb(barrier::SY); -+ -+ // Enable the MMU and turn on data and instruction caching. -+ SCTLR_EL1.modify(SCTLR_EL1::M::Enable + SCTLR_EL1::C::Cacheable + SCTLR_EL1::I::Cacheable); -+ -+ // Force MMU init to complete before next instruction. -+ barrier::isb(barrier::SY); -+ -+ Ok(()) -+ } -+} - -diff -uNr 10_privilege_level/src/arch/aarch64.rs 11_virtual_memory/src/arch/aarch64.rs ---- 10_privilege_level/src/arch/aarch64.rs -+++ 11_virtual_memory/src/arch/aarch64.rs -@@ -5,6 +5,7 @@ - //! AArch64. - - mod exception; -+mod mmu; - pub mod sync; - mod time; - -@@ -77,6 +78,7 @@ - //-------------------------------------------------------------------------------------------------- - - static TIMER: time::Timer = time::Timer; -+static MMU: mmu::MMU = mmu::MMU; - - //-------------------------------------------------------------------------------------------------- - // Implementation of the kernel's architecture abstraction code -@@ -104,6 +106,11 @@ - } - } - -+/// Return a reference to an `interface::mm::MMU` implementation. -+pub fn mmu() -> &'static impl interface::mm::MMU { -+ &MMU -+} -+ - /// Information about the HW state. - pub mod state { - use crate::arch::PrivilegeLevel; - -diff -uNr 10_privilege_level/src/bsp/rpi/link.ld 11_virtual_memory/src/bsp/rpi/link.ld ---- 10_privilege_level/src/bsp/rpi/link.ld -+++ 11_virtual_memory/src/bsp/rpi/link.ld -@@ -8,6 +8,7 @@ - /* Set current address to the value from which the RPi starts execution */ - . = 0x80000; - -+ __ro_start = .; - .text : - { - *(.text._start) *(.text*) -@@ -17,6 +18,8 @@ - { - *(.rodata*) - } -+ . = ALIGN(65536); /* Fill up to 64 KiB */ -+ __ro_end = .; - - .data : - { - -diff -uNr 10_privilege_level/src/bsp/rpi/memory_map.rs 11_virtual_memory/src/bsp/rpi/memory_map.rs ---- 10_privilege_level/src/bsp/rpi/memory_map.rs -+++ 11_virtual_memory/src/bsp/rpi/memory_map.rs -@@ -4,6 +4,14 @@ - - //! The board's memory map. - -+#[cfg(feature = "bsp_rpi3")] -+#[rustfmt::skip] -+pub const END_INCLUSIVE: usize = 0x3FFF_FFFF; -+ -+#[cfg(feature = "bsp_rpi4")] -+#[rustfmt::skip] -+pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; -+ - /// Physical devices. - #[rustfmt::skip] - pub mod mmio { -@@ -15,4 +23,5 @@ - - pub const GPIO_BASE: usize = BASE + 0x0020_0000; - pub const PL011_UART_BASE: usize = BASE + 0x0020_1000; -+ pub const END_INCLUSIVE: usize = super::END_INCLUSIVE; - } - -diff -uNr 10_privilege_level/src/bsp/rpi/virt_mem_layout.rs 11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs ---- 10_privilege_level/src/bsp/rpi/virt_mem_layout.rs -+++ 11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs -@@ -0,0 +1,82 @@ -+// SPDX-License-Identifier: MIT OR Apache-2.0 -+// -+// Copyright (c) 2018-2020 Andre Richter -+ -+//! The virtual memory layout. -+//! -+//! The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. -+//! It is agnostic of the paging granularity that the architecture's MMU will use. -+ -+use super::memory_map; -+use crate::memory::*; -+use core::ops::RangeInclusive; -+ -+//-------------------------------------------------------------------------------------------------- -+// BSP-public -+//-------------------------------------------------------------------------------------------------- -+ -+pub const NUM_MEM_RANGES: usize = 3; -+ -+pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( -+ memory_map::END_INCLUSIVE, -+ [ -+ RangeDescriptor { -+ name: "Kernel code and RO data", -+ virtual_range: || { -+ // Using the linker script, we ensure that the RO area is consecutive and 64 KiB -+ // aligned, and we export the boundaries via symbols: -+ // -+ // [__ro_start, __ro_end) -+ extern "C" { -+ // The inclusive start of the read-only area, aka the address of the first -+ // byte of the area. -+ static __ro_start: usize; -+ -+ // The exclusive end of the read-only area, aka the address of the first -+ // byte _after_ the RO area. -+ static __ro_end: usize; -+ } -+ -+ unsafe { -+ // Notice the subtraction to turn the exclusive end into an inclusive end. -+ #[allow(clippy::range_minus_one)] -+ RangeInclusive::new( -+ &__ro_start as *const _ as usize, -+ &__ro_end as *const _ as usize - 1, -+ ) -+ } -+ }, -+ translation: Translation::Identity, -+ attribute_fields: AttributeFields { -+ mem_attributes: MemAttributes::CacheableDRAM, -+ acc_perms: AccessPermissions::ReadOnly, -+ execute_never: false, -+ }, -+ }, -+ RangeDescriptor { -+ name: "Remapped Device MMIO", -+ virtual_range: || { -+ // The last 64 KiB slot in the first 512 MiB -+ RangeInclusive::new(0x1FFF_0000, 0x1FFF_FFFF) -+ }, -+ translation: Translation::Offset(memory_map::mmio::BASE + 0x20_0000), -+ attribute_fields: AttributeFields { -+ mem_attributes: MemAttributes::Device, -+ acc_perms: AccessPermissions::ReadWrite, -+ execute_never: true, -+ }, -+ }, -+ RangeDescriptor { -+ name: "Device MMIO", -+ virtual_range: || { -+ RangeInclusive::new(memory_map::mmio::BASE, memory_map::mmio::END_INCLUSIVE) -+ }, -+ translation: Translation::Identity, -+ attribute_fields: AttributeFields { -+ mem_attributes: MemAttributes::Device, -+ acc_perms: AccessPermissions::ReadWrite, -+ execute_never: true, -+ }, -+ }, -+ ], -+); - -diff -uNr 10_privilege_level/src/bsp/rpi.rs 11_virtual_memory/src/bsp/rpi.rs ---- 10_privilege_level/src/bsp/rpi.rs -+++ 11_virtual_memory/src/bsp/rpi.rs -@@ -5,9 +5,10 @@ - //! Board Support Package for the Raspberry Pi. - - mod memory_map; -+mod virt_mem_layout; - - use super::driver; --use crate::interface; -+use crate::{interface, memory::KernelVirtualLayout}; - use core::fmt; - - /// Used by `arch` code to find the early boot core. -@@ -72,3 +73,13 @@ - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); - } -+ -+/// Return the address space size in bytes. -+pub const fn addr_space_size() -> usize { -+ memory_map::END_INCLUSIVE + 1 -+} -+ -+/// Return a reference to the virtual memory layout. -+pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ virt_mem_layout::NUM_MEM_RANGES }> { -+ &virt_mem_layout::LAYOUT -+} - -diff -uNr 10_privilege_level/src/bsp.rs 11_virtual_memory/src/bsp.rs ---- 10_privilege_level/src/bsp.rs -+++ 11_virtual_memory/src/bsp.rs -@@ -4,7 +4,7 @@ - - //! Conditional exporting of Board Support Packages. - --mod driver; -+pub mod driver; - - #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] - mod rpi; - -diff -uNr 10_privilege_level/src/interface.rs 11_virtual_memory/src/interface.rs ---- 10_privilege_level/src/interface.rs -+++ 11_virtual_memory/src/interface.rs -@@ -131,3 +131,17 @@ - fn spin_for(&self, duration: Duration); - } - } -+ -+/// Memory Management interfaces. -+pub mod mm { -+ /// MMU functions. -+ pub trait MMU { -+ /// Called by the kernel during early init. Supposed to take the page tables from the -+ /// `BSP`-supplied `virt_mem_layout()` and install/activate them for the respective MMU. -+ /// -+ /// # Safety -+ /// -+ /// - Changes the HW's global state. -+ unsafe fn init(&self) -> Result<(), &'static str>; -+ } -+} - -diff -uNr 10_privilege_level/src/main.rs 11_virtual_memory/src/main.rs ---- 10_privilege_level/src/main.rs -+++ 11_virtual_memory/src/main.rs -@@ -19,6 +19,8 @@ - //! [Architecture-specific code]: arch/index.html - //! [`kernel::interface`]: interface/index.html - -+#![allow(incomplete_features)] -+#![feature(const_generics)] - #![feature(format_args_nl)] - #![feature(panic_info_message)] - #![feature(trait_alias)] -@@ -47,8 +49,18 @@ - /// # Safety - /// - /// - Only a single core must be active and running this function. --/// - The init calls in this function must appear in the correct order. -+/// - The init calls in this function must appear in the correct order: -+/// - Virtual memory must be activated before the device drivers. -+/// - Without it, any atomic operations, e.g. the yet-to-be-introduced spinlocks in the device -+/// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on -+/// the RPi SoCs. - unsafe fn kernel_init() -> ! { -+ use interface::mm::MMU; -+ -+ if let Err(string) = arch::mmu().init() { -+ panic!("MMU: {}", string); -+ } -+ - for i in bsp::device_drivers().iter() { - if let Err(()) = i.init() { - panic!("Error loading driver: {}", i.compatible()) -@@ -68,6 +80,9 @@ - - info!("Booting on: {}", bsp::board_name()); - -+ info!("MMU online. Special regions:"); -+ bsp::virt_mem_layout().print_layout(); -+ - let (_, privilege_level) = arch::state::current_privilege_level(); - info!("Current privilege level: {}", privilege_level); - -@@ -87,6 +102,13 @@ - info!("Timer test, spinning for 1 second"); - arch::timer().spin_for(Duration::from_secs(1)); - -+ let remapped_uart = unsafe { bsp::driver::PL011Uart::new(0x1FFF_1000) }; -+ writeln!( -+ remapped_uart, -+ "[ !!! ] Writing through the remapped UART at 0x1FFF_1000" -+ ) -+ .unwrap(); -+ - info!("Echoing input now"); - loop { - let c = bsp::console().read_char(); - -diff -uNr 10_privilege_level/src/memory.rs 11_virtual_memory/src/memory.rs ---- 10_privilege_level/src/memory.rs -+++ 11_virtual_memory/src/memory.rs -@@ -4,7 +4,10 @@ - - //! Memory Management. - --use core::ops::Range; -+use core::{ -+ fmt, -+ ops::{Range, RangeInclusive}, -+}; - - /// Zero out a memory region. - /// -@@ -23,3 +26,144 @@ - ptr = ptr.offset(1); - } - } -+ -+#[allow(dead_code)] -+#[derive(Copy, Clone)] -+pub enum Translation { -+ Identity, -+ Offset(usize), -+} -+ -+#[derive(Copy, Clone)] -+pub enum MemAttributes { -+ CacheableDRAM, -+ Device, -+} -+ -+#[derive(Copy, Clone)] -+pub enum AccessPermissions { -+ ReadOnly, -+ ReadWrite, -+} -+ -+#[derive(Copy, Clone)] -+pub struct AttributeFields { -+ pub mem_attributes: MemAttributes, -+ pub acc_perms: AccessPermissions, -+ pub execute_never: bool, -+} -+ -+impl Default for AttributeFields { -+ fn default() -> AttributeFields { -+ AttributeFields { -+ mem_attributes: MemAttributes::CacheableDRAM, -+ acc_perms: AccessPermissions::ReadWrite, -+ execute_never: true, -+ } -+ } -+} -+ -+/// An architecture agnostic descriptor for a memory range. -+pub struct RangeDescriptor { -+ pub name: &'static str, -+ pub virtual_range: fn() -> RangeInclusive, -+ pub translation: Translation, -+ pub attribute_fields: AttributeFields, -+} -+ -+/// Human-readable output of a RangeDescriptor. -+impl fmt::Display for RangeDescriptor { -+ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { -+ // Call the function to which self.range points, and dereference the result, which causes -+ // Rust to copy the value. -+ let start = *(self.virtual_range)().start(); -+ let end = *(self.virtual_range)().end(); -+ let size = end - start + 1; -+ -+ // log2(1024). -+ const KIB_RSHIFT: u32 = 10; -+ -+ // log2(1024 * 1024). -+ const MIB_RSHIFT: u32 = 20; -+ -+ let (size, unit) = if (size >> MIB_RSHIFT) > 0 { -+ (size >> MIB_RSHIFT, "MiB") -+ } else if (size >> KIB_RSHIFT) > 0 { -+ (size >> KIB_RSHIFT, "KiB") -+ } else { -+ (size, "Byte") -+ }; -+ -+ let attr = match self.attribute_fields.mem_attributes { -+ MemAttributes::CacheableDRAM => "C", -+ MemAttributes::Device => "Dev", -+ }; -+ -+ let acc_p = match self.attribute_fields.acc_perms { -+ AccessPermissions::ReadOnly => "RO", -+ AccessPermissions::ReadWrite => "RW", -+ }; -+ -+ let xn = if self.attribute_fields.execute_never { -+ "PXN" -+ } else { -+ "PX" -+ }; -+ -+ write!( -+ f, -+ " {:#010x} - {:#010x} | {: >3} {} | {: <3} {} {: <3} | {}", -+ start, end, size, unit, attr, acc_p, xn, self.name -+ ) -+ } -+} -+ -+/// Type for expressing the kernel's virtual memory layout. -+pub struct KernelVirtualLayout { -+ max_virt_addr_inclusive: usize, -+ inner: [RangeDescriptor; NUM_SPECIAL_RANGES], -+} -+ -+impl KernelVirtualLayout<{ NUM_SPECIAL_RANGES }> { -+ pub const fn new(max: usize, layout: [RangeDescriptor; NUM_SPECIAL_RANGES]) -> Self { -+ Self { -+ max_virt_addr_inclusive: max, -+ inner: layout, -+ } -+ } -+ -+ /// For a virtual address, find and return the output address and corresponding attributes. -+ /// -+ /// If the address is not found in `inner`, return an identity mapped default with normal -+ /// cacheable DRAM attributes. -+ pub fn get_virt_addr_properties( -+ &self, -+ virt_addr: usize, -+ ) -> Result<(usize, AttributeFields), &'static str> { -+ if virt_addr > self.max_virt_addr_inclusive { -+ return Err("Address out of range"); -+ } -+ -+ for i in self.inner.iter() { -+ if (i.virtual_range)().contains(&virt_addr) { -+ let output_addr = match i.translation { -+ Translation::Identity => virt_addr, -+ Translation::Offset(a) => a + (virt_addr - (i.virtual_range)().start()), -+ }; -+ -+ return Ok((output_addr, i.attribute_fields)); -+ } -+ } -+ -+ Ok((virt_addr, AttributeFields::default())) -+ } -+ -+ /// Print the memory layout. -+ pub fn print_layout(&self) { -+ use crate::info; -+ -+ for i in self.inner.iter() { -+ info!("{}", i); -+ } -+ } -+} - -``` diff --git a/11_virtual_memory/kernel b/11_virtual_memory/kernel index f7f4669d..b4606584 100755 Binary files a/11_virtual_memory/kernel and b/11_virtual_memory/kernel differ diff --git a/11_virtual_memory/kernel8.img b/11_virtual_memory/kernel8.img index d2ab81f3..088f9890 100755 Binary files a/11_virtual_memory/kernel8.img and b/11_virtual_memory/kernel8.img differ diff --git a/11_virtual_memory/src/_arch/aarch64/cpu.rs b/11_virtual_memory/src/_arch/aarch64/cpu.rs new file mode 100644 index 00000000..ac9c4a88 --- /dev/null +++ b/11_virtual_memory/src/_arch/aarch64/cpu.rs @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Architectural processor code. + +use crate::{bsp, cpu}; +use cortex_a::{asm, regs::*}; + +//-------------------------------------------------------------------------------------------------- +// Boot Code +//-------------------------------------------------------------------------------------------------- + +/// The entry of the `kernel` binary. +/// +/// The function must be named `_start`, because the linker is looking for this exact name. +/// +/// # Safety +/// +/// - Linker script must ensure to place this function at `0x80_000`. +#[naked] +#[no_mangle] +pub unsafe extern "C" fn _start() -> ! { + // Expect the boot core to start in EL2. + if (bsp::cpu::BOOT_CORE_ID == cpu::smp::core_id()) + && (CurrentEL.get() == CurrentEL::EL::EL2.value) + { + el2_to_el1_transition() + } else { + // If not core0, infinitely wait for events. + wait_forever() + } +} + +/// Transition from EL2 to EL1. +/// +/// # Safety +/// +/// - The HW state of EL1 must be prepared in a sound way. +/// - Exception return from EL2 must must continue execution in EL1 with +/// `runtime_init::runtime_init()`. +#[inline(always)] +unsafe fn el2_to_el1_transition() -> ! { + use crate::runtime_init; + + // Enable timer counter registers for EL1. + CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); + + // No offset for reading the counters. + CNTVOFF_EL2.set(0); + + // Set EL1 execution state to AArch64. + HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); + + // Set up a simulated exception return. + // + // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a + // stack pointer. + SPSR_EL2.write( + SPSR_EL2::D::Masked + + SPSR_EL2::A::Masked + + SPSR_EL2::I::Masked + + SPSR_EL2::F::Masked + + SPSR_EL2::M::EL1h, + ); + + // Second, let the link register point to runtime_init(). + ELR_EL2.set(runtime_init::runtime_init as *const () as u64); + + // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. + SP_EL1.set(bsp::cpu::BOOT_CORE_STACK_START); + + // Use `eret` to "return" to EL1. This results in execution of runtime_init() in EL1. + asm::eret() +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +pub use asm::nop; + +/// Spin for `n` cycles. +#[inline(always)] +pub fn spin_for_cycles(n: usize) { + for _ in 0..n { + asm::nop(); + } +} + +/// Pause execution on the core. +#[inline(always)] +pub fn wait_forever() -> ! { + loop { + asm::wfe() + } +} diff --git a/11_virtual_memory/src/_arch/aarch64/cpu/smp.rs b/11_virtual_memory/src/_arch/aarch64/cpu/smp.rs new file mode 100644 index 00000000..8429e1d2 --- /dev/null +++ b/11_virtual_memory/src/_arch/aarch64/cpu/smp.rs @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Architectural symmetric multiprocessing. + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return the executing core's id. +#[inline(always)] +pub fn core_id() -> T +where + T: From, +{ + const CORE_MASK: u64 = 0b11; + + T::from((MPIDR_EL1.get() & CORE_MASK) as u8) +} diff --git a/11_virtual_memory/src/_arch/aarch64/exception.rs b/11_virtual_memory/src/_arch/aarch64/exception.rs new file mode 100644 index 00000000..ebfef8c8 --- /dev/null +++ b/11_virtual_memory/src/_arch/aarch64/exception.rs @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Architectural synchronous and asynchronous exception handling. + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- +use crate::exception::PrivilegeLevel; + +/// The processing element's current privilege level. +pub fn current_privilege_level() -> (PrivilegeLevel, &'static str) { + let el = CurrentEL.read_as_enum(CurrentEL::EL); + match el { + Some(CurrentEL::EL::Value::EL2) => (PrivilegeLevel::Hypervisor, "EL2"), + Some(CurrentEL::EL::Value::EL1) => (PrivilegeLevel::Kernel, "EL1"), + Some(CurrentEL::EL::Value::EL0) => (PrivilegeLevel::User, "EL0"), + _ => (PrivilegeLevel::Unknown, "Unknown"), + } +} diff --git a/11_virtual_memory/src/_arch/aarch64/exception/asynchronous.rs b/11_virtual_memory/src/_arch/aarch64/exception/asynchronous.rs new file mode 100644 index 00000000..1cc2fba8 --- /dev/null +++ b/11_virtual_memory/src/_arch/aarch64/exception/asynchronous.rs @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Architectural asynchronous exception handling. + +use cortex_a::regs::*; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +trait DaifField { + fn daif_field() -> register::Field; +} + +struct Debug; +struct SError; +struct IRQ; +struct FIQ; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + +impl DaifField for Debug { + fn daif_field() -> register::Field { + DAIF::D + } +} + +impl DaifField for SError { + fn daif_field() -> register::Field { + DAIF::A + } +} + +impl DaifField for IRQ { + fn daif_field() -> register::Field { + DAIF::I + } +} + +impl DaifField for FIQ { + fn daif_field() -> register::Field { + DAIF::F + } +} + +fn is_masked() -> bool { + DAIF.is_set(T::daif_field()) +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Print the AArch64 exceptions status. +#[rustfmt::skip] +pub fn print_state() { + use crate::info; + + let to_mask_str = |x| -> _ { + if x { "Masked" } else { "Unmasked" } + }; + + info!(" Debug: {}", to_mask_str(is_masked::())); + info!(" SError: {}", to_mask_str(is_masked::())); + info!(" IRQ: {}", to_mask_str(is_masked::())); + info!(" FIQ: {}", to_mask_str(is_masked::())); +} diff --git a/11_virtual_memory/src/arch/aarch64/mmu.rs b/11_virtual_memory/src/_arch/aarch64/memory/mmu.rs similarity index 81% rename from 11_virtual_memory/src/arch/aarch64/mmu.rs rename to 11_virtual_memory/src/_arch/aarch64/memory/mmu.rs index c0ba5b26..e9c9d038 100644 --- a/11_virtual_memory/src/arch/aarch64/mmu.rs +++ b/11_virtual_memory/src/_arch/aarch64/memory/mmu.rs @@ -2,19 +2,21 @@ // // Copyright (c) 2018-2020 Andre Richter -//! Memory Management Unit. +//! Memory Management Unit Driver. //! //! Static page tables, compiled on boot; Everything 64 KiB granule. -use crate::{ - bsp, interface, - memory::{AccessPermissions, AttributeFields, MemAttributes}, -}; +use super::{AccessPermissions, AttributeFields, MemAttributes}; +use crate::{bsp, memory}; use core::convert; use cortex_a::{barrier, regs::*}; use register::register_bitfields; -// A table descriptor, as per AArch64 Reference Manual Figure D4-15. +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + +// A table descriptor, as per ARMv8-A Architecture Reference Manual Figure D4-15. register_bitfields! {u64, STAGE1_TABLE_DESCRIPTOR [ /// Physical address of the next page table. @@ -32,7 +34,7 @@ register_bitfields! {u64, ] } -// A level 3 page descriptor, as per AArch64 Reference Manual Figure D4-17. +// A level 3 page descriptor, as per ARMv8-A Architecture Reference Manual Figure D4-17. register_bitfields! {u64, STAGE1_PAGE_DESCRIPTOR [ /// Privileged execute-never. @@ -101,18 +103,21 @@ struct PageDescriptor(u64); #[repr(C)] #[repr(align(65536))] struct PageTables { - // Page descriptors, covering 64 KiB windows per entry. + /// Page descriptors, covering 64 KiB windows per entry. lvl3: [[PageDescriptor; 8192]; N], - // Table descriptors, covering 512 MiB windows. + + /// Table descriptors, covering 512 MiB windows. lvl2: [TableDescriptor; N], } /// Usually evaluates to 1 GiB for RPi3 and 4 GiB for RPi 4. -const ENTRIES_512_MIB: usize = bsp::addr_space_size() >> FIVETWELVE_MIB_SHIFT; +const ENTRIES_512_MIB: usize = bsp::memory::mmu::addr_space_size() >> FIVETWELVE_MIB_SHIFT; /// The page tables. /// -/// Supposed to land in `.bss`. Therefore, ensure that they boil down to all "0" entries. +/// # Safety +/// +/// - Supposed to land in `.bss`. Therefore, ensure that they boil down to all "0" entries. static mut TABLES: PageTables<{ ENTRIES_512_MIB }> = PageTables { lvl3: [[PageDescriptor(0); 8192]; ENTRIES_512_MIB], lvl2: [TableDescriptor(0); ENTRIES_512_MIB], @@ -123,6 +128,30 @@ trait BaseAddr { fn base_addr_usize(&self) -> usize; } +/// Constants for indexing the MAIR_EL1. +#[allow(dead_code)] +mod mair { + pub const DEVICE: u64 = 0; + pub const NORMAL: u64 = 1; +} + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Memory Management Unit type. +pub struct MemoryManagementUnit; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static MMU: MemoryManagementUnit = MemoryManagementUnit; + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + impl BaseAddr for [T; N] { fn base_addr_u64(&self) -> u64 { self as *const T as u64 @@ -180,7 +209,7 @@ impl convert::From } impl PageDescriptor { - fn new(output_addr: usize, attribute_fields: AttributeFields) -> PageDescriptor { + fn new(output_addr: usize, attribute_fields: AttributeFields) -> Self { let shifted = output_addr >> SIXTYFOUR_KIB_SHIFT; let val = (STAGE1_PAGE_DESCRIPTOR::VALID::True + STAGE1_PAGE_DESCRIPTOR::AF::True @@ -189,17 +218,10 @@ impl PageDescriptor { + STAGE1_PAGE_DESCRIPTOR::OUTPUT_ADDR_64KiB.val(shifted as u64)) .value; - PageDescriptor(val) + Self(val) } } -/// Constants for indexing the MAIR_EL1. -#[allow(dead_code)] -mod mair { - pub const DEVICE: u64 = 0; - pub const NORMAL: u64 = 1; -} - /// Setup function for the MAIR_EL1 register. fn set_up_mair() { // Define the memory types being mapped. @@ -227,7 +249,7 @@ unsafe fn populate_pt_entries() -> Result<(), &'static str> { let virt_addr = (l2_nr << FIVETWELVE_MIB_SHIFT) + (l3_nr << SIXTYFOUR_KIB_SHIFT); let (output_addr, attribute_fields) = - bsp::virt_mem_layout().get_virt_addr_properties(virt_addr)?; + bsp::memory::mmu::virt_mem_layout().get_virt_addr_properties(virt_addr)?; *l3_entry = PageDescriptor::new(output_addr, attribute_fields); } @@ -252,16 +274,19 @@ fn configure_translation_control() { } //-------------------------------------------------------------------------------------------------- -// Arch-public +// Public Code //-------------------------------------------------------------------------------------------------- -pub struct MMU; +/// Return a reference to the MMU. +pub fn mmu() -> &'static impl memory::mmu::interface::MMU { + &MMU +} -//-------------------------------------------------------------------------------------------------- -// OS interface implementations -//-------------------------------------------------------------------------------------------------- +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ -impl interface::mm::MMU for MMU { +impl memory::mmu::interface::MMU for MemoryManagementUnit { unsafe fn init(&self) -> Result<(), &'static str> { // Fail early if translation granule is not supported. Both RPis support it, though. if !ID_AA64MMFR0_EL1.matches_all(ID_AA64MMFR0_EL1::TGran64::Supported) { diff --git a/11_virtual_memory/src/arch/aarch64/time.rs b/11_virtual_memory/src/_arch/aarch64/time.rs similarity index 69% rename from 11_virtual_memory/src/arch/aarch64/time.rs rename to 11_virtual_memory/src/_arch/aarch64/time.rs index 249c498a..fb01ced1 100644 --- a/11_virtual_memory/src/arch/aarch64/time.rs +++ b/11_virtual_memory/src/_arch/aarch64/time.rs @@ -2,25 +2,45 @@ // // Copyright (c) 2018-2020 Andre Richter -//! Timer primitives. +//! Architectural timer primitives. -use crate::{interface, warn}; +use crate::{time, warn}; use core::time::Duration; use cortex_a::regs::*; +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + const NS_PER_S: u64 = 1_000_000_000; //-------------------------------------------------------------------------------------------------- -// Arch-public +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// ARMv8 Generic Timer. +pub struct GenericTimer; + +//-------------------------------------------------------------------------------------------------- +// Global instances //-------------------------------------------------------------------------------------------------- -pub struct Timer; +static TIME_MANAGER: GenericTimer = GenericTimer; //-------------------------------------------------------------------------------------------------- -// OS interface implementations +// Public Code //-------------------------------------------------------------------------------------------------- -impl interface::time::Timer for Timer { +/// Return a reference to the time manager. +pub fn time_manager() -> &'static impl time::interface::TimeManager { + &TIME_MANAGER +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl time::interface::TimeManager for GenericTimer { fn resolution(&self) -> Duration { Duration::from_nanos(NS_PER_S / (CNTFRQ_EL0.get() as u64)) } diff --git a/11_virtual_memory/src/arch.rs b/11_virtual_memory/src/arch.rs deleted file mode 100644 index c3a24d0d..00000000 --- a/11_virtual_memory/src/arch.rs +++ /dev/null @@ -1,21 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! Conditional exporting of processor architecture code. - -#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] -mod aarch64; - -#[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] -pub use aarch64::*; - -/// Architectural privilege level. -#[allow(missing_docs)] -#[derive(PartialEq)] -pub enum PrivilegeLevel { - User, - Kernel, - Hypervisor, - Unknown, -} diff --git a/11_virtual_memory/src/arch/aarch64.rs b/11_virtual_memory/src/arch/aarch64.rs deleted file mode 100644 index dccf866e..00000000 --- a/11_virtual_memory/src/arch/aarch64.rs +++ /dev/null @@ -1,148 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! AArch64. - -mod exception; -mod mmu; -pub mod sync; -mod time; - -use crate::{bsp, interface}; -use cortex_a::{asm, regs::*}; - -/// The entry of the `kernel` binary. -/// -/// The function must be named `_start`, because the linker is looking for this exact name. -/// -/// # Safety -/// -/// - Linker script must ensure to place this function at `0x80_000`. -#[no_mangle] -pub unsafe extern "C" fn _start() -> ! { - const CORE_MASK: u64 = 0x3; - - // Expect the boot core to start in EL2. - if (bsp::BOOT_CORE_ID == MPIDR_EL1.get() & CORE_MASK) - && (CurrentEL.get() == CurrentEL::EL::EL2.value) - { - el2_to_el1_transition() - } else { - // If not core0, infinitely wait for events. - wait_forever() - } -} - -/// Transition from EL2 to EL1. -/// -/// # Safety -/// -/// - The HW state of EL1 must be prepared in a sound way. -/// - Exception return from EL2 must must continue execution in EL1 with ´runtime_init::init()`. -#[inline(always)] -unsafe fn el2_to_el1_transition() -> ! { - // Enable timer counter registers for EL1. - CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET); - - // No offset for reading the counters. - CNTVOFF_EL2.set(0); - - // Set EL1 execution state to AArch64. - HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64); - - // Set up a simulated exception return. - // - // First, fake a saved program status, where all interrupts were masked and SP_EL1 was used as a - // stack pointer. - SPSR_EL2.write( - SPSR_EL2::D::Masked - + SPSR_EL2::A::Masked - + SPSR_EL2::I::Masked - + SPSR_EL2::F::Masked - + SPSR_EL2::M::EL1h, - ); - - // Second, let the link register point to init(). - ELR_EL2.set(crate::runtime_init::runtime_init as *const () as u64); - - // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. - SP_EL1.set(bsp::BOOT_CORE_STACK_START); - - // Use `eret` to "return" to EL1. This will result in execution of `reset()` in EL1. - asm::eret() -} - -//-------------------------------------------------------------------------------------------------- -// Global instances -//-------------------------------------------------------------------------------------------------- - -static TIMER: time::Timer = time::Timer; -static MMU: mmu::MMU = mmu::MMU; - -//-------------------------------------------------------------------------------------------------- -// Implementation of the kernel's architecture abstraction code -//-------------------------------------------------------------------------------------------------- - -pub use asm::nop; - -/// Spin for `n` cycles. -pub fn spin_for_cycles(n: usize) { - for _ in 0..n { - asm::nop(); - } -} - -/// Return a reference to a `interface::time::TimeKeeper` implementation. -pub fn timer() -> &'static impl interface::time::Timer { - &TIMER -} - -/// Pause execution on the calling CPU core. -#[inline(always)] -pub fn wait_forever() -> ! { - loop { - asm::wfe() - } -} - -/// Return a reference to an `interface::mm::MMU` implementation. -pub fn mmu() -> &'static impl interface::mm::MMU { - &MMU -} - -/// Information about the HW state. -pub mod state { - use crate::arch::PrivilegeLevel; - use cortex_a::regs::*; - - /// The processing element's current privilege level. - pub fn current_privilege_level() -> (PrivilegeLevel, &'static str) { - let el = CurrentEL.read_as_enum(CurrentEL::EL); - match el { - Some(CurrentEL::EL::Value::EL2) => (PrivilegeLevel::Hypervisor, "EL2"), - Some(CurrentEL::EL::Value::EL1) => (PrivilegeLevel::Kernel, "EL1"), - Some(CurrentEL::EL::Value::EL0) => (PrivilegeLevel::User, "EL0"), - _ => (PrivilegeLevel::Unknown, "Unknown"), - } - } - - /// Print the AArch64 exceptions status. - #[rustfmt::skip] - pub fn print_exception_state() { - use super::{ - exception, - exception::{Debug, SError, FIQ, IRQ}, - }; - use crate::info; - - let to_mask_str = |x| -> _ { - if x { "Masked" } else { "Unmasked" } - }; - - info!(" Debug: {}", to_mask_str(exception::is_masked::())); - info!(" SError: {}", to_mask_str(exception::is_masked::())); - info!(" IRQ: {}", to_mask_str(exception::is_masked::())); - info!(" FIQ: {}", to_mask_str(exception::is_masked::())); - } -} diff --git a/11_virtual_memory/src/arch/aarch64/exception.rs b/11_virtual_memory/src/arch/aarch64/exception.rs deleted file mode 100644 index acfe13df..00000000 --- a/11_virtual_memory/src/arch/aarch64/exception.rs +++ /dev/null @@ -1,48 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! Exception handling. - -use cortex_a::regs::*; - -//-------------------------------------------------------------------------------------------------- -// Arch-public -//-------------------------------------------------------------------------------------------------- - -pub trait DaifField { - fn daif_field() -> register::Field; -} - -pub struct Debug; -pub struct SError; -pub struct IRQ; -pub struct FIQ; - -impl DaifField for Debug { - fn daif_field() -> register::Field { - DAIF::D - } -} - -impl DaifField for SError { - fn daif_field() -> register::Field { - DAIF::A - } -} - -impl DaifField for IRQ { - fn daif_field() -> register::Field { - DAIF::I - } -} - -impl DaifField for FIQ { - fn daif_field() -> register::Field { - DAIF::F - } -} - -pub fn is_masked() -> bool { - DAIF.is_set(T::daif_field()) -} diff --git a/11_virtual_memory/src/arch/aarch64/sync.rs b/11_virtual_memory/src/arch/aarch64/sync.rs deleted file mode 100644 index 1d1e459f..00000000 --- a/11_virtual_memory/src/arch/aarch64/sync.rs +++ /dev/null @@ -1,53 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! Synchronization primitives. - -use crate::interface; -use core::cell::UnsafeCell; - -//-------------------------------------------------------------------------------------------------- -// Arch-public -//-------------------------------------------------------------------------------------------------- - -/// A pseudo-lock for teaching purposes. -/// -/// Used to introduce [interior mutability]. -/// -/// In contrast to a real Mutex implementation, does not protect against concurrent access to the -/// contained data. This part is preserved for later lessons. -/// -/// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is -/// executing single-threaded, aka only running on a single core with interrupts disabled. -/// -/// [interior mutability]: https://doc.rust-lang.org/std/cell/index.html -pub struct NullLock { - data: UnsafeCell, -} - -unsafe impl Send for NullLock {} -unsafe impl Sync for NullLock {} - -impl NullLock { - /// Wraps `data` into a new `NullLock`. - pub const fn new(data: T) -> NullLock { - NullLock { - data: UnsafeCell::new(data), - } - } -} - -//-------------------------------------------------------------------------------------------------- -// OS interface implementations -//-------------------------------------------------------------------------------------------------- - -impl interface::sync::Mutex for &NullLock { - type Data = T; - - fn lock(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R { - // In a real lock, there would be code encapsulating this line that ensures that this - // mutable reference will ever only be given out once at a time. - f(unsafe { &mut *self.data.get() }) - } -} diff --git a/11_virtual_memory/src/bsp.rs b/11_virtual_memory/src/bsp.rs index 7b246afa..5add2e38 100644 --- a/11_virtual_memory/src/bsp.rs +++ b/11_virtual_memory/src/bsp.rs @@ -2,12 +2,12 @@ // // Copyright (c) 2018-2020 Andre Richter -//! Conditional exporting of Board Support Packages. +//! Conditional re-exporting of Board Support Packages. -pub mod driver; +pub mod device_driver; #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] -mod rpi; +mod raspberrypi; #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] -pub use rpi::*; +pub use raspberrypi::*; diff --git a/11_virtual_memory/src/bsp/driver.rs b/11_virtual_memory/src/bsp/device_driver.rs similarity index 93% rename from 11_virtual_memory/src/bsp/driver.rs rename to 11_virtual_memory/src/bsp/device_driver.rs index f75093a5..4508e953 100644 --- a/11_virtual_memory/src/bsp/driver.rs +++ b/11_virtual_memory/src/bsp/device_driver.rs @@ -2,7 +2,7 @@ // // Copyright (c) 2018-2020 Andre Richter -//! Drivers. +//! Device driver. #[cfg(any(feature = "bsp_rpi3", feature = "bsp_rpi4"))] mod bcm; diff --git a/11_virtual_memory/src/bsp/driver/bcm.rs b/11_virtual_memory/src/bsp/device_driver/bcm.rs similarity index 70% rename from 11_virtual_memory/src/bsp/driver/bcm.rs rename to 11_virtual_memory/src/bsp/device_driver/bcm.rs index 40232f30..59071d5d 100644 --- a/11_virtual_memory/src/bsp/driver/bcm.rs +++ b/11_virtual_memory/src/bsp/device_driver/bcm.rs @@ -7,5 +7,5 @@ mod bcm2xxx_gpio; mod bcm2xxx_pl011_uart; -pub use bcm2xxx_gpio::GPIO; -pub use bcm2xxx_pl011_uart::{PL011Uart, PanicUart}; +pub use bcm2xxx_gpio::*; +pub use bcm2xxx_pl011_uart::*; diff --git a/11_virtual_memory/src/bsp/driver/bcm/bcm2xxx_gpio.rs b/11_virtual_memory/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs similarity index 69% rename from 11_virtual_memory/src/bsp/driver/bcm/bcm2xxx_gpio.rs rename to 11_virtual_memory/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs index 1bcc9a64..0c17f498 100644 --- a/11_virtual_memory/src/bsp/driver/bcm/bcm2xxx_gpio.rs +++ b/11_virtual_memory/src/bsp/device_driver/bcm/bcm2xxx_gpio.rs @@ -2,11 +2,15 @@ // // Copyright (c) 2018-2020 Andre Richter -//! GPIO driver. +//! GPIO Driver. -use crate::{arch, arch::sync::NullLock, interface}; +use crate::{cpu, driver, synchronization, synchronization::NullLock}; use core::ops; -use register::{mmio::ReadWrite, register_bitfields, register_structs}; +use register::{mmio::*, register_bitfields, register_structs}; + +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- // GPIO registers. // @@ -66,12 +70,23 @@ register_structs! { } } -/// The driver's private data. struct GPIOInner { base_addr: usize, } -/// Deref to RegisterBlock. +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Representation of the GPIO HW. +pub struct GPIO { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + impl ops::Deref for GPIOInner { type Target = RegisterBlock; @@ -81,29 +96,28 @@ impl ops::Deref for GPIOInner { } impl GPIOInner { - const fn new(base_addr: usize) -> GPIOInner { - GPIOInner { base_addr } + const fn new(base_addr: usize) -> Self { + Self { base_addr } } - /// Return a pointer to the register block. + /// Return a pointer to the associated MMIO register block. fn ptr(&self) -> *const RegisterBlock { self.base_addr as *const _ } } //-------------------------------------------------------------------------------------------------- -// BSP-public +// Public Code //-------------------------------------------------------------------------------------------------- -use interface::sync::Mutex; - -/// The driver's main struct. -pub struct GPIO { - inner: NullLock, -} impl GPIO { - pub const unsafe fn new(base_addr: usize) -> GPIO { - GPIO { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide the correct `base_addr`. + pub const unsafe fn new(base_addr: usize) -> Self { + Self { inner: NullLock::new(GPIOInner::new(base_addr)), } } @@ -122,24 +136,25 @@ impl GPIO { // Enable pins 14 and 15. inner.GPPUD.set(0); - arch::spin_for_cycles(150); + cpu::spin_for_cycles(150); inner .GPPUDCLK0 .write(GPPUDCLK0::PUDCLK14::AssertClock + GPPUDCLK0::PUDCLK15::AssertClock); - arch::spin_for_cycles(150); + cpu::spin_for_cycles(150); inner.GPPUDCLK0.set(0); }) } } -//-------------------------------------------------------------------------------------------------- -// OS interface implementations -//-------------------------------------------------------------------------------------------------- +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::Mutex; -impl interface::driver::DeviceDriver for GPIO { +impl driver::interface::DeviceDriver for GPIO { fn compatible(&self) -> &str { - "GPIO" + "BCM GPIO" } } diff --git a/11_virtual_memory/src/bsp/driver/bcm/bcm2xxx_pl011_uart.rs b/11_virtual_memory/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs similarity index 88% rename from 11_virtual_memory/src/bsp/driver/bcm/bcm2xxx_pl011_uart.rs rename to 11_virtual_memory/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs index b9dd63b6..714e293e 100644 --- a/11_virtual_memory/src/bsp/driver/bcm/bcm2xxx_pl011_uart.rs +++ b/11_virtual_memory/src/bsp/device_driver/bcm/bcm2xxx_pl011_uart.rs @@ -4,10 +4,14 @@ //! PL011 UART driver. -use crate::{arch, arch::sync::NullLock, interface}; +use crate::{console, cpu, driver, synchronization, synchronization::NullLock}; use core::{fmt, ops}; use register::{mmio::*, register_bitfields, register_structs}; +//-------------------------------------------------------------------------------------------------- +// Private Definitions +//-------------------------------------------------------------------------------------------------- + // PL011 UART registers. // // Descriptions taken from @@ -109,7 +113,12 @@ register_bitfields! { ] } +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + register_structs! { + #[allow(missing_docs)] #[allow(non_snake_case)] pub RegisterBlock { (0x00 => DR: ReadWrite), @@ -126,13 +135,25 @@ register_structs! { } } -/// The driver's mutex protected part. +#[allow(missing_docs)] pub struct PL011UartInner { base_addr: usize, chars_written: usize, chars_read: usize, } +// Export the inner struct so that BSPs can use it for the panic handler. +pub use PL011UartInner as PanicUart; + +/// Representation of the UART. +pub struct PL011Uart { + inner: NullLock, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + /// Deref to RegisterBlock. /// /// Allows writing @@ -152,8 +173,13 @@ impl ops::Deref for PL011UartInner { } impl PL011UartInner { - pub const unsafe fn new(base_addr: usize) -> PL011UartInner { - PL011UartInner { + /// Create an instance. + /// + /// # Safety + /// + /// - The user must ensure to provide the correct `base_addr`. + pub const unsafe fn new(base_addr: usize) -> Self { + Self { base_addr, chars_written: 0, chars_read: 0, @@ -164,7 +190,7 @@ impl PL011UartInner { /// /// Results in 8N1 and 230400 baud (if the clk has been previously set to 48 MHz by the /// firmware). - pub fn init(&self) { + pub fn init(&mut self) { // Turn it off temporarily. self.CR.set(0); @@ -186,7 +212,7 @@ impl PL011UartInner { fn write_char(&mut self, c: char) { // Spin while TX FIFO full is set, waiting for an empty slot. while self.FR.matches_all(FR::TXFF::SET) { - arch::nop(); + cpu::nop(); } // Write the character to the buffer. @@ -215,42 +241,28 @@ impl fmt::Write for PL011UartInner { } } -//-------------------------------------------------------------------------------------------------- -// Export the inner struct so that BSPs can use it for the panic handler -//-------------------------------------------------------------------------------------------------- -pub use PL011UartInner as PanicUart; - -//-------------------------------------------------------------------------------------------------- -// BSP-public -//-------------------------------------------------------------------------------------------------- - -/// The driver's main struct. -pub struct PL011Uart { - inner: NullLock, -} - impl PL011Uart { /// # Safety /// - /// The user must ensure to provide the correct `base_addr`. - pub const unsafe fn new(base_addr: usize) -> PL011Uart { - PL011Uart { + /// - The user must ensure to provide the correct `base_addr`. + pub const unsafe fn new(base_addr: usize) -> Self { + Self { inner: NullLock::new(PL011UartInner::new(base_addr)), } } } -//-------------------------------------------------------------------------------------------------- -// OS interface implementations -//-------------------------------------------------------------------------------------------------- -use interface::sync::Mutex; +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use synchronization::interface::Mutex; -impl interface::driver::DeviceDriver for PL011Uart { +impl driver::interface::DeviceDriver for PL011Uart { fn compatible(&self) -> &str { - "PL011Uart" + "BCM PL011 UART" } - fn init(&self) -> interface::driver::Result { + fn init(&self) -> Result<(), ()> { let mut r = &self.inner; r.lock(|inner| inner.init()); @@ -258,7 +270,7 @@ impl interface::driver::DeviceDriver for PL011Uart { } } -impl interface::console::Write for PL011Uart { +impl console::interface::Write for PL011Uart { /// Passthrough of `args` to the `core::fmt::Write` implementation, but guarded by a Mutex to /// serialize access. fn write_char(&self, c: char) { @@ -274,23 +286,23 @@ impl interface::console::Write for PL011Uart { } fn flush(&self) { - let mut r = &self.inner; // Spin until TX FIFO empty is set. + let mut r = &self.inner; r.lock(|inner| { while !inner.FR.matches_all(FR::TXFE::SET) { - arch::nop(); + cpu::nop(); } }); } } -impl interface::console::Read for PL011Uart { +impl console::interface::Read for PL011Uart { fn read_char(&self) -> char { let mut r = &self.inner; r.lock(|inner| { // Spin while RX FIFO empty is set. while inner.FR.matches_all(FR::RXFE::SET) { - arch::nop(); + cpu::nop(); } // Read one character. @@ -319,7 +331,7 @@ impl interface::console::Read for PL011Uart { } } -impl interface::console::Statistics for PL011Uart { +impl console::interface::Statistics for PL011Uart { fn chars_written(&self) -> usize { let mut r = &self.inner; r.lock(|inner| inner.chars_written) diff --git a/11_virtual_memory/src/bsp/raspberrypi.rs b/11_virtual_memory/src/bsp/raspberrypi.rs new file mode 100644 index 00000000..c976cc29 --- /dev/null +++ b/11_virtual_memory/src/bsp/raspberrypi.rs @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Top-level BSP file for the Raspberry Pi 3 and 4. + +pub mod console; +pub mod cpu; +pub mod driver; +pub mod memory; + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- +use super::device_driver; + +static GPIO: device_driver::GPIO = + unsafe { device_driver::GPIO::new(memory::map::mmio::GPIO_BASE) }; + +static PL011_UART: device_driver::PL011Uart = + unsafe { device_driver::PL011Uart::new(memory::map::mmio::PL011_UART_BASE) }; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Board identification. +pub fn board_name() -> &'static str { + #[cfg(feature = "bsp_rpi3")] + { + "Raspberry Pi 3" + } + + #[cfg(feature = "bsp_rpi4")] + { + "Raspberry Pi 4" + } +} diff --git a/11_virtual_memory/src/bsp/raspberrypi/console.rs b/11_virtual_memory/src/bsp/raspberrypi/console.rs new file mode 100644 index 00000000..061f9c1c --- /dev/null +++ b/11_virtual_memory/src/bsp/raspberrypi/console.rs @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! BSP console facilities. + +use super::{super::device_driver, memory::map}; +use crate::console; +use core::fmt; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// In case of a panic, the panic handler uses this function to take a last shot at printing +/// something before the system is halted. +/// +/// # Safety +/// +/// - Use only for printing during a panic. +pub unsafe fn panic_console_out() -> impl fmt::Write { + let mut uart = device_driver::PanicUart::new(map::mmio::PL011_UART_BASE); + uart.init(); + uart +} + +/// Return a reference to the console. +pub fn console() -> &'static impl console::interface::All { + &super::PL011_UART +} diff --git a/11_virtual_memory/src/bsp/raspberrypi/cpu.rs b/11_virtual_memory/src/bsp/raspberrypi/cpu.rs new file mode 100644 index 00000000..19db276e --- /dev/null +++ b/11_virtual_memory/src/bsp/raspberrypi/cpu.rs @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! BSP Processor code. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Used by `arch` code to find the early boot core. +pub const BOOT_CORE_ID: usize = 0; + +/// The early boot core's stack address. +pub const BOOT_CORE_STACK_START: u64 = 0x80_000; diff --git a/11_virtual_memory/src/bsp/raspberrypi/driver.rs b/11_virtual_memory/src/bsp/raspberrypi/driver.rs new file mode 100644 index 00000000..86526dc0 --- /dev/null +++ b/11_virtual_memory/src/bsp/raspberrypi/driver.rs @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! BSP driver support. + +use crate::driver; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Device Driver Manager type. +pub struct BSPDriverManager { + device_drivers: [&'static (dyn DeviceDriver + Sync); 2], +} + +//-------------------------------------------------------------------------------------------------- +// Global instances +//-------------------------------------------------------------------------------------------------- + +static BSP_DRIVER_MANAGER: BSPDriverManager = BSPDriverManager { + device_drivers: [&super::GPIO, &super::PL011_UART], +}; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return a reference to the driver manager. +pub fn driver_manager() -> &'static impl driver::interface::DriverManager { + &BSP_DRIVER_MANAGER +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ +use driver::interface::DeviceDriver; + +impl driver::interface::DriverManager for BSPDriverManager { + fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)] { + &self.device_drivers[..] + } + + fn post_device_driver_init(&self) { + // Configure PL011Uart's output pins. + super::GPIO.map_pl011_uart(); + } +} diff --git a/11_virtual_memory/src/bsp/rpi/link.ld b/11_virtual_memory/src/bsp/raspberrypi/link.ld similarity index 100% rename from 11_virtual_memory/src/bsp/rpi/link.ld rename to 11_virtual_memory/src/bsp/raspberrypi/link.ld diff --git a/11_virtual_memory/src/bsp/raspberrypi/memory.rs b/11_virtual_memory/src/bsp/raspberrypi/memory.rs new file mode 100644 index 00000000..62f0f74d --- /dev/null +++ b/11_virtual_memory/src/bsp/raspberrypi/memory.rs @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! BSP Memory Management. + +pub mod mmu; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// The board's memory map. +#[rustfmt::skip] +pub(super) mod map { + pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; + + pub const GPIO_OFFSET: usize = 0x0020_0000; + pub const UART_OFFSET: usize = 0x0020_1000; + + /// Physical devices. + #[cfg(feature = "bsp_rpi3")] + pub mod mmio { + use super::*; + + pub const BASE: usize = 0x3F00_0000; + pub const GPIO_BASE: usize = BASE + GPIO_OFFSET; + pub const PL011_UART_BASE: usize = BASE + UART_OFFSET; + pub const END_INCLUSIVE: usize = 0x4000_FFFF; + } + + /// Physical devices. + #[cfg(feature = "bsp_rpi4")] + pub mod mmio { + use super::*; + + pub const BASE: usize = 0xFE00_0000; + pub const GPIO_BASE: usize = BASE + GPIO_OFFSET; + pub const PL011_UART_BASE: usize = BASE + UART_OFFSET; + pub const END_INCLUSIVE: usize = 0xFF84_FFFF; + } +} diff --git a/11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs b/11_virtual_memory/src/bsp/raspberrypi/memory/mmu.rs similarity index 73% rename from 11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs rename to 11_virtual_memory/src/bsp/raspberrypi/memory/mmu.rs index f2fc5388..a6ca52a1 100644 --- a/11_virtual_memory/src/bsp/rpi/virt_mem_layout.rs +++ b/11_virtual_memory/src/bsp/raspberrypi/memory/mmu.rs @@ -2,23 +2,24 @@ // // Copyright (c) 2018-2020 Andre Richter -//! The virtual memory layout. -//! -//! The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. -//! It is agnostic of the paging granularity that the architecture's MMU will use. +//! BSP Memory Management Unit. -use super::memory_map; -use crate::memory::*; +use super::super::memory; +use crate::memory::mmu::*; use core::ops::RangeInclusive; //-------------------------------------------------------------------------------------------------- -// BSP-public +// Public Definitions //-------------------------------------------------------------------------------------------------- -pub const NUM_MEM_RANGES: usize = 3; +const NUM_MEM_RANGES: usize = 3; +/// The virtual memory layout. +/// +/// The layout must contain only special ranges, aka anything that is _not_ normal cacheable DRAM. +/// It is agnostic of the paging granularity that the architecture's MMU will use. pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout::new( - memory_map::END_INCLUSIVE, + memory::map::END_INCLUSIVE, [ RangeDescriptor { name: "Kernel code and RO data", @@ -59,7 +60,7 @@ pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout // The last 64 KiB slot in the first 512 MiB RangeInclusive::new(0x1FFF_0000, 0x1FFF_FFFF) }, - translation: Translation::Offset(memory_map::mmio::BASE + 0x20_0000), + translation: Translation::Offset(memory::map::mmio::BASE + 0x20_0000), attribute_fields: AttributeFields { mem_attributes: MemAttributes::Device, acc_perms: AccessPermissions::ReadWrite, @@ -69,7 +70,7 @@ pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout RangeDescriptor { name: "Device MMIO", virtual_range: || { - RangeInclusive::new(memory_map::mmio::BASE, memory_map::mmio::END_INCLUSIVE) + RangeInclusive::new(memory::map::mmio::BASE, memory::map::mmio::END_INCLUSIVE) }, translation: Translation::Identity, attribute_fields: AttributeFields { @@ -80,3 +81,17 @@ pub static LAYOUT: KernelVirtualLayout<{ NUM_MEM_RANGES }> = KernelVirtualLayout }, ], ); + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +/// Return the address space size in bytes. +pub const fn addr_space_size() -> usize { + memory::map::END_INCLUSIVE + 1 +} + +/// Return a reference to the virtual memory layout. +pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ NUM_MEM_RANGES }> { + &LAYOUT +} diff --git a/11_virtual_memory/src/bsp/rpi.rs b/11_virtual_memory/src/bsp/rpi.rs deleted file mode 100644 index bdb76c81..00000000 --- a/11_virtual_memory/src/bsp/rpi.rs +++ /dev/null @@ -1,85 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! Board Support Package for the Raspberry Pi. - -mod memory_map; -mod virt_mem_layout; - -use super::driver; -use crate::{interface, memory::KernelVirtualLayout}; -use core::fmt; - -/// Used by `arch` code to find the early boot core. -pub const BOOT_CORE_ID: u64 = 0; - -/// The early boot core's stack address. -pub const BOOT_CORE_STACK_START: u64 = 0x80_000; - -//-------------------------------------------------------------------------------------------------- -// Global BSP driver instances -//-------------------------------------------------------------------------------------------------- - -static GPIO: driver::GPIO = unsafe { driver::GPIO::new(memory_map::mmio::GPIO_BASE) }; -static PL011_UART: driver::PL011Uart = - unsafe { driver::PL011Uart::new(memory_map::mmio::PL011_UART_BASE) }; - -//-------------------------------------------------------------------------------------------------- -// Implementation of the kernel's BSP calls -//-------------------------------------------------------------------------------------------------- - -/// Board identification. -pub fn board_name() -> &'static str { - #[cfg(feature = "bsp_rpi3")] - { - "Raspberry Pi 3" - } - - #[cfg(feature = "bsp_rpi4")] - { - "Raspberry Pi 4" - } -} - -/// Return a reference to a `console::All` implementation. -pub fn console() -> &'static impl interface::console::All { - &PL011_UART -} - -/// In case of a panic, the panic handler uses this function to take a last shot at printing -/// something before the system is halted. -/// -/// # Safety -/// -/// - Use only for printing during a panic. -pub unsafe fn panic_console_out() -> impl fmt::Write { - let uart = driver::PanicUart::new(memory_map::mmio::PL011_UART_BASE); - uart.init(); - uart -} - -/// Return an array of references to all `DeviceDriver` compatible `BSP` drivers. -/// -/// # Safety -/// -/// The order of devices is the order in which `DeviceDriver::init()` is called. -pub fn device_drivers() -> [&'static dyn interface::driver::DeviceDriver; 2] { - [&GPIO, &PL011_UART] -} - -/// BSP initialization code that runs after driver init. -pub fn post_driver_init() { - // Configure PL011Uart's output pins. - GPIO.map_pl011_uart(); -} - -/// Return the address space size in bytes. -pub const fn addr_space_size() -> usize { - memory_map::END_INCLUSIVE + 1 -} - -/// Return a reference to the virtual memory layout. -pub fn virt_mem_layout() -> &'static KernelVirtualLayout<{ virt_mem_layout::NUM_MEM_RANGES }> { - &virt_mem_layout::LAYOUT -} diff --git a/11_virtual_memory/src/bsp/rpi/memory_map.rs b/11_virtual_memory/src/bsp/rpi/memory_map.rs deleted file mode 100644 index 18505939..00000000 --- a/11_virtual_memory/src/bsp/rpi/memory_map.rs +++ /dev/null @@ -1,27 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! The board's memory map. - -#[cfg(feature = "bsp_rpi3")] -#[rustfmt::skip] -pub const END_INCLUSIVE: usize = 0x3FFF_FFFF; - -#[cfg(feature = "bsp_rpi4")] -#[rustfmt::skip] -pub const END_INCLUSIVE: usize = 0xFFFF_FFFF; - -/// Physical devices. -#[rustfmt::skip] -pub mod mmio { - #[cfg(feature = "bsp_rpi3")] - pub const BASE: usize = 0x3F00_0000; - - #[cfg(feature = "bsp_rpi4")] - pub const BASE: usize = 0xFE00_0000; - - pub const GPIO_BASE: usize = BASE + 0x0020_0000; - pub const PL011_UART_BASE: usize = BASE + 0x0020_1000; - pub const END_INCLUSIVE: usize = super::END_INCLUSIVE; -} diff --git a/11_virtual_memory/src/console.rs b/11_virtual_memory/src/console.rs new file mode 100644 index 00000000..e6323a20 --- /dev/null +++ b/11_virtual_memory/src/console.rs @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! System console. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Console interfaces. +pub mod interface { + use core::fmt; + + /// Console write functions. + pub trait Write { + /// Write a single character. + fn write_char(&self, c: char); + + /// Write a Rust format string. + fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; + + /// Block execution until the last character has been physically put on the TX wire + /// (draining TX buffers/FIFOs, if any). + fn flush(&self); + } + + /// Console read functions. + pub trait Read { + /// Read a single character. + fn read_char(&self) -> char { + ' ' + } + + /// Clear RX buffers, if any. + fn clear(&self); + } + + /// Console statistics. + pub trait Statistics { + /// Return the number of characters written. + fn chars_written(&self) -> usize { + 0 + } + + /// Return the number of characters read. + fn chars_read(&self) -> usize { + 0 + } + } + + /// Trait alias for a full-fledged console. + pub trait All = Write + Read + Statistics; +} diff --git a/11_virtual_memory/src/cpu.rs b/11_virtual_memory/src/cpu.rs new file mode 100644 index 00000000..9c67c0e7 --- /dev/null +++ b/11_virtual_memory/src/cpu.rs @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020 Andre Richter + +//! Processor code. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/cpu.rs"] +mod arch_cpu; +pub use arch_cpu::*; + +pub mod smp; diff --git a/11_virtual_memory/src/cpu/smp.rs b/11_virtual_memory/src/cpu/smp.rs new file mode 100644 index 00000000..b1428884 --- /dev/null +++ b/11_virtual_memory/src/cpu/smp.rs @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Symmetric multiprocessing. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/cpu/smp.rs"] +mod arch_cpu_smp; +pub use arch_cpu_smp::*; diff --git a/11_virtual_memory/src/driver.rs b/11_virtual_memory/src/driver.rs new file mode 100644 index 00000000..c63b8301 --- /dev/null +++ b/11_virtual_memory/src/driver.rs @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2018-2020 Andre Richter + +//! Driver support. + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Driver interfaces. +pub mod interface { + + /// Device Driver functions. + pub trait DeviceDriver { + /// Return a compatibility string for identifying the driver. + fn compatible(&self) -> &str; + + /// Called by the kernel to bring up the device. + fn init(&self) -> Result<(), ()> { + Ok(()) + } + } + + /// Device driver management functions. + /// + /// The `BSP` is supposed to supply one global instance. + pub trait DriverManager { + /// Return a slice of references to all `BSP`-instantiated drivers. + /// + /// # Safety + /// + /// - The order of devices is the order in which `DeviceDriver::init()` is called. + fn all_device_drivers(&self) -> &[&'static (dyn DeviceDriver + Sync)]; + + /// Initialization code that runs after driver init. + /// + /// For example, device driver code that depends on other drivers already being online. + fn post_device_driver_init(&self); + } +} diff --git a/11_virtual_memory/src/exception.rs b/11_virtual_memory/src/exception.rs new file mode 100644 index 00000000..d2e62179 --- /dev/null +++ b/11_virtual_memory/src/exception.rs @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020 Andre Richter + +//! Synchronous and asynchronous exception handling. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/exception.rs"] +mod arch_exception; +pub use arch_exception::*; + +pub mod asynchronous; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Kernel privilege levels. +#[allow(missing_docs)] +#[derive(PartialEq)] +pub enum PrivilegeLevel { + User, + Kernel, + Hypervisor, + Unknown, +} diff --git a/11_virtual_memory/src/exception/asynchronous.rs b/11_virtual_memory/src/exception/asynchronous.rs new file mode 100644 index 00000000..3c75f90a --- /dev/null +++ b/11_virtual_memory/src/exception/asynchronous.rs @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020 Andre Richter + +//! Asynchronous exception handling. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/exception/asynchronous.rs"] +mod arch_exception_async; +pub use arch_exception_async::*; diff --git a/11_virtual_memory/src/interface.rs b/11_virtual_memory/src/interface.rs deleted file mode 100644 index 3988beef..00000000 --- a/11_virtual_memory/src/interface.rs +++ /dev/null @@ -1,147 +0,0 @@ -// SPDX-License-Identifier: MIT OR Apache-2.0 -// -// Copyright (c) 2018-2020 Andre Richter - -//! Trait definitions for coupling `kernel` and `BSP` code. -//! -//! ``` -//! +-------------------+ -//! | Interface (Trait) | -//! | | -//! +--+-------------+--+ -//! ^ ^ -//! | | -//! | | -//! +----------+--+ +--+----------+ -//! | Kernel code | | BSP Code | -//! | | | | -//! +-------------+ +-------------+ -//! ``` - -/// System console operations. -pub mod console { - use core::fmt; - - /// Console write functions. - pub trait Write { - /// Write a single character. - fn write_char(&self, c: char); - - /// Write a Rust format string. - fn write_fmt(&self, args: fmt::Arguments) -> fmt::Result; - - /// Block execution until the last character has been physically put on the TX wire - /// (draining TX buffers/FIFOs, if any). - fn flush(&self); - } - - /// Console read functions. - pub trait Read { - /// Read a single character. - fn read_char(&self) -> char { - ' ' - } - - /// Clear RX buffers, if any. - fn clear(&self); - } - - /// Console statistics. - pub trait Statistics { - /// Return the number of characters written. - fn chars_written(&self) -> usize { - 0 - } - - /// Return the number of characters read. - fn chars_read(&self) -> usize { - 0 - } - } - - /// Trait alias for a full-fledged console. - pub trait All = Write + Read + Statistics; -} - -/// Synchronization primitives. -pub mod sync { - /// Any object implementing this trait guarantees exclusive access to the data contained within - /// the mutex for the duration of the lock. - /// - /// The trait follows the [Rust embedded WG's - /// proposal](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md) and therefore - /// provides some goodness such as [deadlock - /// prevention](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md#design-decisions-and-compatibility). - /// - /// # Example - /// - /// Since the lock function takes an `&mut self` to enable deadlock-prevention, the trait is - /// best implemented **for a reference to a container struct**, and has a usage pattern that - /// might feel strange at first: - /// - /// ``` - /// static MUT: Mutex> = Mutex::new(RefCell::new(0)); - /// - /// fn foo() { - /// let mut r = &MUT; // Note that r is mutable - /// r.lock(|data| *data += 1); - /// } - /// ``` - pub trait Mutex { - /// Type of data encapsulated by the mutex. - type Data; - - /// Creates a critical section and grants temporary mutable access to the encapsulated data. - fn lock(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R; - } -} - -/// Driver interfaces. -pub mod driver { - /// Driver result type, e.g. for indicating successful driver init. - pub type Result = core::result::Result<(), ()>; - - /// Device Driver functions. - pub trait DeviceDriver { - /// Return a compatibility string for identifying the driver. - fn compatible(&self) -> &str; - - /// Called by the kernel to bring up the device. - fn init(&self) -> Result { - Ok(()) - } - } -} - -/// Timekeeping interfaces. -pub mod time { - use core::time::Duration; - - /// Timer functions. - pub trait Timer { - /// The timer's resolution. - fn resolution(&self) -> Duration; - - /// The uptime since power-on of the device. - /// - /// This includes time consumed by firmware and bootloaders. - fn uptime(&self) -> Duration; - - /// Spin for a given duration. - fn spin_for(&self, duration: Duration); - } -} - -/// Memory Management interfaces. -pub mod mm { - /// MMU functions. - pub trait MMU { - /// Called by the kernel during early init. Supposed to take the page tables from the - /// `BSP`-supplied `virt_mem_layout()` and install/activate them for the respective MMU. - /// - /// # Safety - /// - /// - Changes the HW's global state. - unsafe fn init(&self) -> Result<(), &'static str>; - } -} diff --git a/11_virtual_memory/src/main.rs b/11_virtual_memory/src/main.rs index 0303b25f..6ff3292f 100644 --- a/11_virtual_memory/src/main.rs +++ b/11_virtual_memory/src/main.rs @@ -5,47 +5,131 @@ // Rust embedded logo for `make doc`. #![doc(html_logo_url = "https://git.io/JeGIp")] -//! The `kernel` +//! The `kernel` binary. //! -//! The `kernel` is composed by glueing together code from +//! # TL;DR - Overview of important Kernel entities //! -//! - [Hardware-specific Board Support Packages] (`BSPs`). -//! - [Architecture-specific code]. -//! - HW- and architecture-agnostic `kernel` code. +//! - [`bsp::console::console()`] - Returns a reference to the kernel's [console interface]. +//! - [`bsp::driver::driver_manager()`] - Returns a reference to the kernel's [driver interface]. +//! - [`memory::mmu::mmu()`] - Returns a reference to the kernel's [MMU interface]. +//! - [`time::time_manager()`] - Returns a reference to the kernel's [timer interface]. //! -//! using the [`kernel::interface`] traits. +//! [console interface]: ../libkernel/console/interface/index.html +//! [driver interface]: ../libkernel/driver/interface/trait.DriverManager.html +//! [MMU interface]: ../libkernel/memory/mmu/interface/trait.MMU.html +//! [timer interface]: ../libkernel/time/interface/trait.TimeManager.html //! -//! [Hardware-specific Board Support Packages]: bsp/index.html -//! [Architecture-specific code]: arch/index.html -//! [`kernel::interface`]: interface/index.html +//! # Code organization and architecture +//! +//! The code is divided into different *modules*, each representing a typical **subsystem** of the +//! `kernel`. Top-level module files of subsystems reside directly in the `src` folder. For example, +//! `src/memory.rs` contains code that is concerned with all things memory management. +//! +//! ## Visibility of processor architecture code +//! +//! Some of the `kernel`'s subsystems depend on low-level code that is specific to the target +//! processor architecture. For each supported processor architecture, there exists a subfolder in +//! `src/_arch`, for example, `src/_arch/aarch64`. +//! +//! The architecture folders mirror the subsystem modules laid out in `src`. For example, +//! architectural code that belongs to the `kernel`'s memory subsystem (`src/memory.rs`) would go +//! into `src/_arch/aarch64/memory.rs`. The latter file is directly included and re-exported in +//! `src/memory.rs`, so that the architectural code parts are transparent with respect to the code's +//! module organization. That means a public function `foo()` defined in +//! `src/_arch/aarch64/memory.rs` would be reachable as `crate::memory::foo()` only. +//! +//! The `_` in `_arch` denotes that this folder is not part of the standard module hierarchy. +//! Rather, it's contents are conditionally pulled into respective files using the `#[path = +//! "_arch/xxx/yyy.rs"]` attribute. +//! +//! ## BSP code +//! +//! `BSP` stands for Board Support Package. `BSP` code is organized under `src/bsp.rs` and contains +//! target board specific definitions and functions. These are things such as the board's memory map +//! or instances of drivers for devices that are featured on the respective board. +//! +//! Just like processor architecture code, the `BSP` code's module structure tries to mirror the +//! `kernel`'s subsystem modules, but there is no transparent re-exporting this time. That means +//! whatever is provided must be called starting from the `bsp` namespace, e.g. +//! `bsp::driver::driver_manager()`. +//! +//! ## Kernel interfaces +//! +//! Both `arch` and `bsp` contain code that is conditionally compiled depending on the actual target +//! and board for which the kernel is compiled. For example, the `interrupt controller` hardware of +//! the `Raspberry Pi 3` and the `Raspberry Pi 4` is different, but we want the rest of the `kernel` +//! code to play nicely with any of the two without much hassle. +//! +//! In order to provide a clean abstraction between `arch`, `bsp` and `generic kernel code`, +//! `interface` traits are provided *whenever possible* and *where it makes sense*. They are defined +//! in the respective subsystem module and help to enforce the idiom of *program to an interface, +//! not an implementation*. For example, there will be a common IRQ handling interface which the two +//! different interrupt controller `drivers` of both Raspberrys will implement, and only export the +//! interface to the rest of the `kernel`. +//! +//! ``` +//! +-------------------+ +//! | Interface (Trait) | +//! | | +//! +--+-------------+--+ +//! ^ ^ +//! | | +//! | | +//! +----------+--+ +--+----------+ +//! | kernel code | | bsp code | +//! | | | arch code | +//! +-------------+ +-------------+ +//! ``` +//! +//! # Summary +//! +//! For a logical `kernel` subsystem, corresponding code can be distributed over several physical +//! locations. Here is an example for the **memory** subsystem: +//! +//! - `src/memory.rs` and `src/memory/**/*` +//! - Common code that is agnostic of target processor architecture and `BSP` characteristics. +//! - Example: A function to zero a chunk of memory. +//! - Interfaces for the memory subsystem that are implemented by `arch` or `BSP` code. +//! - Example: An `MMU` interface that defines `MMU` function prototypes. +//! - `src/bsp/__board_name__/memory.rs` and `src/bsp/__board_name__/memory/**/*` +//! - `BSP` specific code. +//! - Example: The board's memory map (physical addresses of DRAM and MMIO devices). +//! - `src/_arch/__arch_name__/memory.rs` and `src/_arch/__arch_name__/memory/**/*` +//! - Processor architecture specific code. +//! - Example: Implementation of the `MMU` interface for the `__arch_name__` processor +//! architecture. +//! +//! From a namespace perspective, **memory** subsystem code lives in: +//! +//! - `crate::memory::*` +//! - `crate::bsp::memory::*` #![allow(incomplete_features)] #![feature(const_generics)] #![feature(format_args_nl)] +#![feature(naked_functions)] #![feature(panic_info_message)] #![feature(trait_alias)] #![no_main] #![no_std] -// Conditionally includes the selected `architecture` code, which provides the `_start()` function, -// the first function to run. -mod arch; +// `mod cpu` provides the `_start()` function, the first function to run. `_start()` then calls +// `runtime_init()`, which jumps to `kernel_init()`. -// `_start()` then calls `runtime_init()`, which on completion, jumps to `kernel_init()`. -mod runtime_init; - -// Conditionally includes the selected `BSP` code. mod bsp; - -mod interface; +mod console; +mod cpu; +mod driver; +mod exception; mod memory; mod panic_wait; mod print; +mod runtime_init; +mod synchronization; +mod time; /// Early init code. /// -/// Concerned with with initializing `BSP` and `arch` parts. -/// /// # Safety /// /// - Only a single core must be active and running this function. @@ -55,18 +139,19 @@ mod print; /// drivers (which currently employ NullLocks instead of spinlocks), will fail to work on /// the RPi SoCs. unsafe fn kernel_init() -> ! { - use interface::mm::MMU; + use driver::interface::DriverManager; + use memory::mmu::interface::MMU; - if let Err(string) = arch::mmu().init() { + if let Err(string) = memory::mmu::mmu().init() { panic!("MMU: {}", string); } - for i in bsp::device_drivers().iter() { - if let Err(()) = i.init() { + for i in bsp::driver::driver_manager().all_device_drivers().iter() { + if i.init().is_err() { panic!("Error loading driver: {}", i.compatible()) } } - bsp::post_driver_init(); + bsp::driver::driver_manager().post_device_driver_init(); // println! is usable from here on. // Transition from unsafe to safe. @@ -75,34 +160,40 @@ unsafe fn kernel_init() -> ! { /// The main function running after the early init. fn kernel_main() -> ! { + use console::interface::All; use core::time::Duration; - use interface::{console::All, time::Timer}; + use driver::interface::DriverManager; + use time::interface::TimeManager; info!("Booting on: {}", bsp::board_name()); info!("MMU online. Special regions:"); - bsp::virt_mem_layout().print_layout(); + bsp::memory::mmu::virt_mem_layout().print_layout(); - let (_, privilege_level) = arch::state::current_privilege_level(); + let (_, privilege_level) = exception::current_privilege_level(); info!("Current privilege level: {}", privilege_level); info!("Exception handling state:"); - arch::state::print_exception_state(); + exception::asynchronous::print_state(); info!( "Architectural timer resolution: {} ns", - arch::timer().resolution().as_nanos() + time::time_manager().resolution().as_nanos() ); info!("Drivers loaded:"); - for (i, driver) in bsp::device_drivers().iter().enumerate() { + for (i, driver) in bsp::driver::driver_manager() + .all_device_drivers() + .iter() + .enumerate() + { info!(" {}. {}", i + 1, driver.compatible()); } info!("Timer test, spinning for 1 second"); - arch::timer().spin_for(Duration::from_secs(1)); + time::time_manager().spin_for(Duration::from_secs(1)); - let remapped_uart = unsafe { bsp::driver::PL011Uart::new(0x1FFF_1000) }; + let remapped_uart = unsafe { bsp::device_driver::PL011Uart::new(0x1FFF_1000) }; writeln!( remapped_uart, "[ !!! ] Writing through the remapped UART at 0x1FFF_1000" @@ -111,7 +202,7 @@ fn kernel_main() -> ! { info!("Echoing input now"); loop { - let c = bsp::console().read_char(); - bsp::console().write_char(c); + let c = bsp::console::console().read_char(); + bsp::console::console().write_char(c); } } diff --git a/11_virtual_memory/src/memory.rs b/11_virtual_memory/src/memory.rs index 49639a8d..989fa21f 100644 --- a/11_virtual_memory/src/memory.rs +++ b/11_virtual_memory/src/memory.rs @@ -4,10 +4,13 @@ //! Memory Management. -use core::{ - fmt, - ops::{Range, RangeInclusive}, -}; +pub mod mmu; + +use core::ops::Range; + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- /// Zero out a memory region. /// @@ -26,144 +29,3 @@ where ptr = ptr.offset(1); } } - -#[allow(dead_code)] -#[derive(Copy, Clone)] -pub enum Translation { - Identity, - Offset(usize), -} - -#[derive(Copy, Clone)] -pub enum MemAttributes { - CacheableDRAM, - Device, -} - -#[derive(Copy, Clone)] -pub enum AccessPermissions { - ReadOnly, - ReadWrite, -} - -#[derive(Copy, Clone)] -pub struct AttributeFields { - pub mem_attributes: MemAttributes, - pub acc_perms: AccessPermissions, - pub execute_never: bool, -} - -impl Default for AttributeFields { - fn default() -> AttributeFields { - AttributeFields { - mem_attributes: MemAttributes::CacheableDRAM, - acc_perms: AccessPermissions::ReadWrite, - execute_never: true, - } - } -} - -/// An architecture agnostic descriptor for a memory range. -pub struct RangeDescriptor { - pub name: &'static str, - pub virtual_range: fn() -> RangeInclusive, - pub translation: Translation, - pub attribute_fields: AttributeFields, -} - -/// Human-readable output of a RangeDescriptor. -impl fmt::Display for RangeDescriptor { - fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { - // Call the function to which self.range points, and dereference the result, which causes - // Rust to copy the value. - let start = *(self.virtual_range)().start(); - let end = *(self.virtual_range)().end(); - let size = end - start + 1; - - // log2(1024). - const KIB_RSHIFT: u32 = 10; - - // log2(1024 * 1024). - const MIB_RSHIFT: u32 = 20; - - let (size, unit) = if (size >> MIB_RSHIFT) > 0 { - (size >> MIB_RSHIFT, "MiB") - } else if (size >> KIB_RSHIFT) > 0 { - (size >> KIB_RSHIFT, "KiB") - } else { - (size, "Byte") - }; - - let attr = match self.attribute_fields.mem_attributes { - MemAttributes::CacheableDRAM => "C", - MemAttributes::Device => "Dev", - }; - - let acc_p = match self.attribute_fields.acc_perms { - AccessPermissions::ReadOnly => "RO", - AccessPermissions::ReadWrite => "RW", - }; - - let xn = if self.attribute_fields.execute_never { - "PXN" - } else { - "PX" - }; - - write!( - f, - " {:#010x} - {:#010x} | {: >3} {} | {: <3} {} {: <3} | {}", - start, end, size, unit, attr, acc_p, xn, self.name - ) - } -} - -/// Type for expressing the kernel's virtual memory layout. -pub struct KernelVirtualLayout { - max_virt_addr_inclusive: usize, - inner: [RangeDescriptor; NUM_SPECIAL_RANGES], -} - -impl KernelVirtualLayout<{ NUM_SPECIAL_RANGES }> { - pub const fn new(max: usize, layout: [RangeDescriptor; NUM_SPECIAL_RANGES]) -> Self { - Self { - max_virt_addr_inclusive: max, - inner: layout, - } - } - - /// For a virtual address, find and return the output address and corresponding attributes. - /// - /// If the address is not found in `inner`, return an identity mapped default with normal - /// cacheable DRAM attributes. - pub fn get_virt_addr_properties( - &self, - virt_addr: usize, - ) -> Result<(usize, AttributeFields), &'static str> { - if virt_addr > self.max_virt_addr_inclusive { - return Err("Address out of range"); - } - - for i in self.inner.iter() { - if (i.virtual_range)().contains(&virt_addr) { - let output_addr = match i.translation { - Translation::Identity => virt_addr, - Translation::Offset(a) => a + (virt_addr - (i.virtual_range)().start()), - }; - - return Ok((output_addr, i.attribute_fields)); - } - } - - Ok((virt_addr, AttributeFields::default())) - } - - /// Print the memory layout. - pub fn print_layout(&self) { - use crate::info; - - for i in self.inner.iter() { - info!("{}", i); - } - } -} diff --git a/11_virtual_memory/src/memory/mmu.rs b/11_virtual_memory/src/memory/mmu.rs new file mode 100644 index 00000000..17184b08 --- /dev/null +++ b/11_virtual_memory/src/memory/mmu.rs @@ -0,0 +1,198 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020 Andre Richter + +//! Memory Management Unit. +//! +//! In order to decouple `BSP` and `arch` parts of the MMU code (to keep them pluggable), this file +//! provides types for composing an architecture-agnostic description of the kernel 's virtual +//! memory layout. +//! +//! The `BSP` provides such a description through the `bsp::memory::mmu::virt_mem_layout()` +//! function. +//! +//! The `MMU` driver of the `arch` code uses `bsp::memory::mmu::virt_mem_layout()` to compile and +//! install respective page tables. + +#[cfg(target_arch = "aarch64")] +#[path = "../_arch/aarch64/memory/mmu.rs"] +mod arch_mmu; +pub use arch_mmu::*; + +use core::{fmt, ops::RangeInclusive}; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Memory Management interfaces. +pub mod interface { + + /// MMU functions. + pub trait MMU { + /// Called by the kernel during early init. Supposed to take the page tables from the + /// `BSP`-supplied `virt_mem_layout()` and install/activate them for the respective MMU. + /// + /// # Safety + /// + /// - Changes the HW's global state. + unsafe fn init(&self) -> Result<(), &'static str>; + } +} + +/// Architecture agnostic translation types. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub enum Translation { + Identity, + Offset(usize), +} + +/// Architecture agnostic memory attributes. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub enum MemAttributes { + CacheableDRAM, + Device, +} + +/// Architecture agnostic access permissions. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub enum AccessPermissions { + ReadOnly, + ReadWrite, +} + +/// Collection of memory attributes. +#[allow(missing_docs)] +#[derive(Copy, Clone)] +pub struct AttributeFields { + pub mem_attributes: MemAttributes, + pub acc_perms: AccessPermissions, + pub execute_never: bool, +} + +/// Architecture agnostic descriptor for a memory range. +#[allow(missing_docs)] +pub struct RangeDescriptor { + pub name: &'static str, + pub virtual_range: fn() -> RangeInclusive, + pub translation: Translation, + pub attribute_fields: AttributeFields, +} + +/// Type for expressing the kernel's virtual memory layout. +pub struct KernelVirtualLayout { + /// The last (inclusive) address of the address space. + max_virt_addr_inclusive: usize, + + /// Array of descriptors for non-standard (normal cacheable DRAM) memory regions. + inner: [RangeDescriptor; NUM_SPECIAL_RANGES], +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +impl Default for AttributeFields { + fn default() -> AttributeFields { + AttributeFields { + mem_attributes: MemAttributes::CacheableDRAM, + acc_perms: AccessPermissions::ReadWrite, + execute_never: true, + } + } +} + +/// Human-readable output of a RangeDescriptor. +impl fmt::Display for RangeDescriptor { + fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { + // Call the function to which self.range points, and dereference the result, which causes + // Rust to copy the value. + let start = *(self.virtual_range)().start(); + let end = *(self.virtual_range)().end(); + let size = end - start + 1; + + // log2(1024). + const KIB_RSHIFT: u32 = 10; + + // log2(1024 * 1024). + const MIB_RSHIFT: u32 = 20; + + let (size, unit) = if (size >> MIB_RSHIFT) > 0 { + (size >> MIB_RSHIFT, "MiB") + } else if (size >> KIB_RSHIFT) > 0 { + (size >> KIB_RSHIFT, "KiB") + } else { + (size, "Byte") + }; + + let attr = match self.attribute_fields.mem_attributes { + MemAttributes::CacheableDRAM => "C", + MemAttributes::Device => "Dev", + }; + + let acc_p = match self.attribute_fields.acc_perms { + AccessPermissions::ReadOnly => "RO", + AccessPermissions::ReadWrite => "RW", + }; + + let xn = if self.attribute_fields.execute_never { + "PXN" + } else { + "PX" + }; + + write!( + f, + " {:#010x} - {:#010x} | {: >3} {} | {: <3} {} {: <3} | {}", + start, end, size, unit, attr, acc_p, xn, self.name + ) + } +} + +impl KernelVirtualLayout<{ NUM_SPECIAL_RANGES }> { + /// Create a new instance. + pub const fn new(max: usize, layout: [RangeDescriptor; NUM_SPECIAL_RANGES]) -> Self { + Self { + max_virt_addr_inclusive: max, + inner: layout, + } + } + + /// For a virtual address, find and return the output address and corresponding attributes. + /// + /// If the address is not found in `inner`, return an identity mapped default with normal + /// cacheable DRAM attributes. + pub fn get_virt_addr_properties( + &self, + virt_addr: usize, + ) -> Result<(usize, AttributeFields), &'static str> { + if virt_addr > self.max_virt_addr_inclusive { + return Err("Address out of range"); + } + + for i in self.inner.iter() { + if (i.virtual_range)().contains(&virt_addr) { + let output_addr = match i.translation { + Translation::Identity => virt_addr, + Translation::Offset(a) => a + (virt_addr - (i.virtual_range)().start()), + }; + + return Ok((output_addr, i.attribute_fields)); + } + } + + Ok((virt_addr, AttributeFields::default())) + } + + /// Print the memory layout. + pub fn print_layout(&self) { + use crate::info; + + for i in self.inner.iter() { + info!("{}", i); + } + } +} diff --git a/11_virtual_memory/src/panic_wait.rs b/11_virtual_memory/src/panic_wait.rs index 67cf1ee6..1386e1e2 100644 --- a/11_virtual_memory/src/panic_wait.rs +++ b/11_virtual_memory/src/panic_wait.rs @@ -4,13 +4,17 @@ //! A panic handler that infinitely waits. -use crate::{arch, bsp}; +use crate::{bsp, cpu}; use core::{fmt, panic::PanicInfo}; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + fn _panic_print(args: fmt::Arguments) { use fmt::Write; - unsafe { bsp::panic_console_out().write_fmt(args).unwrap() }; + unsafe { bsp::console::panic_console_out().write_fmt(args).unwrap() }; } /// Prints with a newline - only use from the panic handler. @@ -31,5 +35,5 @@ fn panic(info: &PanicInfo) -> ! { panic_println!("\nKernel panic!"); } - arch::wait_forever() + cpu::wait_forever() } diff --git a/11_virtual_memory/src/print.rs b/11_virtual_memory/src/print.rs index 1d0736af..cc303bfc 100644 --- a/11_virtual_memory/src/print.rs +++ b/11_virtual_memory/src/print.rs @@ -4,16 +4,24 @@ //! Printing facilities. -use crate::{bsp, interface}; +use crate::{bsp, console}; use core::fmt; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + #[doc(hidden)] pub fn _print(args: fmt::Arguments) { - use interface::console::Write; + use console::interface::Write; - bsp::console().write_fmt(args).unwrap(); + bsp::console::console().write_fmt(args).unwrap(); } +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + /// Prints without a newline. /// /// Carbon copy from https://doc.rust-lang.org/src/std/macros.rs.html @@ -33,14 +41,14 @@ macro_rules! println { }) } -/// Prints an info, with newline. +/// Prints an info, with a newline. #[macro_export] macro_rules! info { ($string:expr) => ({ #[allow(unused_imports)] - use crate::interface::time::Timer; + use crate::time::interface::TimeManager; - let timestamp = $crate::arch::timer().uptime(); + let timestamp = $crate::time::time_manager().uptime(); let timestamp_subsec_us = timestamp.subsec_micros(); $crate::print::_print(format_args_nl!( @@ -52,9 +60,9 @@ macro_rules! info { }); ($format_string:expr, $($arg:tt)*) => ({ #[allow(unused_imports)] - use crate::interface::time::Timer; + use crate::time::interface::TimeManager; - let timestamp = $crate::arch::timer().uptime(); + let timestamp = $crate::time::time_manager().uptime(); let timestamp_subsec_us = timestamp.subsec_micros(); $crate::print::_print(format_args_nl!( @@ -67,14 +75,14 @@ macro_rules! info { }) } -/// Prints a warning, with newline. +/// Prints a warning, with a newline. #[macro_export] macro_rules! warn { ($string:expr) => ({ #[allow(unused_imports)] - use crate::interface::time::Timer; + use crate::time::interface::TimeManager; - let timestamp = $crate::arch::timer().uptime(); + let timestamp = $crate::time::time_manager().uptime(); let timestamp_subsec_us = timestamp.subsec_micros(); $crate::print::_print(format_args_nl!( @@ -86,9 +94,9 @@ macro_rules! warn { }); ($format_string:expr, $($arg:tt)*) => ({ #[allow(unused_imports)] - use crate::interface::time::Timer; + use crate::time::interface::TimeManager; - let timestamp = $crate::arch::timer().uptime(); + let timestamp = $crate::time::time_manager().uptime(); let timestamp_subsec_us = timestamp.subsec_micros(); $crate::print::_print(format_args_nl!( diff --git a/11_virtual_memory/src/runtime_init.rs b/11_virtual_memory/src/runtime_init.rs index 8132a20b..d517bd64 100644 --- a/11_virtual_memory/src/runtime_init.rs +++ b/11_virtual_memory/src/runtime_init.rs @@ -7,6 +7,10 @@ use crate::memory; use core::ops::Range; +//-------------------------------------------------------------------------------------------------- +// Private Code +//-------------------------------------------------------------------------------------------------- + /// Return the range spanning the .bss section. /// /// # Safety @@ -36,6 +40,10 @@ unsafe fn zero_bss() { memory::zero_volatile(bss_range()); } +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + /// Equivalent to `crt0` or `c0` code in C/C++ world. Clears the `bss` section, then jumps to kernel /// init code. /// diff --git a/11_virtual_memory/src/synchronization.rs b/11_virtual_memory/src/synchronization.rs new file mode 100644 index 00000000..caa2794a --- /dev/null +++ b/11_virtual_memory/src/synchronization.rs @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020 Andre Richter + +//! Synchronization primitives. + +use core::cell::UnsafeCell; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Synchronization interfaces. +pub mod interface { + + /// Any object implementing this trait guarantees exclusive access to the data contained within + /// the Mutex for the duration of the provided closure. + /// + /// The trait follows the [Rust embedded WG's + /// proposal](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md) and therefore + /// provides some goodness such as [deadlock + /// prevention](https://github.com/korken89/wg/blob/master/rfcs/0377-mutex-trait.md#design-decisions-and-compatibility). + /// + /// # Example + /// + /// Since the lock function takes an `&mut self` to enable deadlock-prevention, the trait is + /// best implemented **for a reference to a container struct**, and has a usage pattern that + /// might feel strange at first: + /// + /// ``` + /// static MUT: Mutex> = Mutex::new(RefCell::new(0)); + /// + /// fn foo() { + /// let mut r = &MUT; // Note that r is mutable + /// r.lock(|data| *data += 1); + /// } + /// ``` + pub trait Mutex { + /// The type of encapsulated data. + type Data; + + /// Creates a critical section and grants temporary mutable access to the encapsulated data. + fn lock(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R; + } +} + +/// A pseudo-lock for teaching purposes. +/// +/// Used to introduce [interior mutability]. +/// +/// In contrast to a real Mutex implementation, does not protect against concurrent access from +/// other cores to the contained data. This part is preserved for later lessons. +/// +/// The lock will only be used as long as it is safe to do so, i.e. as long as the kernel is +/// executing single-threaded, aka only running on a single core with interrupts disabled. +/// +/// [interior mutability]: https://doc.rust-lang.org/std/cell/index.html +pub struct NullLock { + data: UnsafeCell, +} + +//-------------------------------------------------------------------------------------------------- +// Public Code +//-------------------------------------------------------------------------------------------------- + +unsafe impl Sync for NullLock {} + +impl NullLock { + /// Wraps `data` into a new `NullLock`. + pub const fn new(data: T) -> Self { + Self { + data: UnsafeCell::new(data), + } + } +} + +//------------------------------------------------------------------------------ +// OS Interface Code +//------------------------------------------------------------------------------ + +impl interface::Mutex for &NullLock { + type Data = T; + + fn lock(&mut self, f: impl FnOnce(&mut Self::Data) -> R) -> R { + // In a real lock, there would be code encapsulating this line that ensures that this + // mutable reference will ever only be given out once at a time. + let data = unsafe { &mut *self.data.get() }; + + f(data) + } +} diff --git a/11_virtual_memory/src/time.rs b/11_virtual_memory/src/time.rs new file mode 100644 index 00000000..cd3ceec3 --- /dev/null +++ b/11_virtual_memory/src/time.rs @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// +// Copyright (c) 2020 Andre Richter + +//! Timer primitives. + +#[cfg(target_arch = "aarch64")] +#[path = "_arch/aarch64/time.rs"] +mod arch_time; +pub use arch_time::*; + +//-------------------------------------------------------------------------------------------------- +// Public Definitions +//-------------------------------------------------------------------------------------------------- + +/// Timekeeping interfaces. +pub mod interface { + use core::time::Duration; + + /// Time management functions. + /// + /// The `BSP` is supposed to supply one global instance. + pub trait TimeManager { + /// The timer's resolution. + fn resolution(&self) -> Duration; + + /// The uptime since power-on of the device. + /// + /// This includes time consumed by firmware and bootloaders. + fn uptime(&self) -> Duration; + + /// Spin for a given duration. + fn spin_for(&self, duration: Duration); + } +}