Using the Memory Subsystem Simulator (MSS)
Introduction
COMP3361 – Operating Systems I Mike Goss (Michael.Goss@du.edu)
Winter 2019
Last Modified: 2/1/19 5:34:00 PM
The Memory Subsystem Simulator (MSS) is a software simulation of a simplified version of a modern computer memory controller. It provides virtual memory functionality for a simulation of a 26-bit address space using 32-bit values.
The interface allows you to read data from and write data to a simulated Memory Management Unit (MMU). You can think of your program as simulating some functions of the operating system and some functions of the CPU and/or Direct Memory Access (DMA) unit transferring data using the MMU.
Organization
MSS is implemented in C++. MSS consists of a main MMU class and a number of auxiliary classes.
MMU, the main class for user access, provides a simulation of a complete Memory Management Unit for mapped access to simulated physical memory. The memory model uses a single level page table with 16KB (0x4000) page size. Page table entries are 32 bits, and a page table contains 4K (0x1000) entries. This allows a maximum of 64MB (0x4000000 bytes) of memory. The top 6 bits of any 32-bit address or page table entry are unused and must be zero.
The MMU starts initially in physical memory mode, entering protected (virtual) mode when requested by the user. Once in virtual mode, the MMU cannot be switched back to physical memory mode.
Obtaining the MSS Software
The MSS software has been developed on a 32-bit Linux system using NetBeans 8.2 and g++ 7.3.0. It is configured as a NetBeans project, including unit tests. The software should also work properly on Windows, Mac, and Linux 64-bit computers using NetBeans (however, the final version of any submitted assignments must run on the Linux virtual machine environment used in the course). The project is stored in the DU git server.* To download the project into NetBeans:
1. Click on the “Team” menu
2. Select “Git”, and then “Clone…”
3. In the dialog box, enter this Repository URL:
https://git.cs.du.edu/mikegoss/memorysubsystemw2019.git
4. Enter your Computer Science Dept. Linux system user name and password (if you don’t have a CS Linux account, or have forgotten your user name or password, see https://support.cs.du.edu/ for assistance).
5. Enter the destination folder for the project. This should be the same parent folder where your program
is located. For example, if the NetBeans project for your Lab99 code is located in the path /home/yourname/OS/Lab99, then the destination folder should be /home/yourname/OS.
* Note: git.cs.du.edu is not reliably accessible from DU WiFi. Please use the “eduroam” WiFi network, the VPN, or an external network. See
https://www.du.edu/it/services/networks-internet/wireless-wired/connect for details on connecting to “eduroam”. See https://www.du.edu/it/services/software/vpn for VPN info (you may use the VPN on the DU WiFi network, or externally).
6. Click “Finish”. This will create a subdirectory under the destination folder called MemorySubsystemW2019 (in the example above, the path to this subdirectory would be /home/yourname/OS/MemorySubsystemW2019).
7. You’ll see a dialog box asking if you want to open the project. Click “Open Project”. This will open a project in NetBeans called MemorySubsystem.
Updating the MSS Software
If updates are issued to the MSS software, update your local copy as follows:
• Right click on the MemorySubsystem project, and select Git -> Remote -> Pull from Upstream from
the menu. Click “Revert” if you are asked to revert local changes (this may occur due to changes to the NetBeans files when building the project).
MSS Download
Using the version of MSS provided through Git is highly recommended in case updates to the software are necessary. If you are unable to fetch the software through Git, a zip file of the MSS code is also available on Canvas in the “MSS” folder (https://canvas.du.edu/courses/75772/files/folder/MSS).
Configuring MSS For Your Project
To use MSS in your project:
• Open or create your project as a separate project, if you haven’t already (don’t use the
MemorySubsystem project for your code, it is configured as a library, not an application).
• Right click on your project name in NetBeans.
• Select “Properties” from the menu.
• Select “C++ Compiler” in the dialog box “Categories” list under “Build”.
o In the “Configuration” selector at the top of the dialog, select “All Configurations”. o Click on “Include Directories”, click the “…” box, and click “Add”. Select the
MemorySubsystem project directory.
o Select “Relative” for the “Store Path As:” option. o Click on “C++ Standard” and select “C++14”.
• Select “Linker” in the dialog box Categories list under “Build”.
o Select “Libraries” under the “Libraries” section. Click the “…” box, select “Add Project”, and
then select the MemorySubsystem project.
• Close both dialog boxes using the “OK” buttons in each.
In Your Source Code
In source files using the MSS, use the following line to define MSS functionality:
#include
This requires that you have successfully completed the configuration above.
Note that all MSS constructs are defined in C++ namespace “mem”.
Using MSS
This section describes how to use MSS.
2
Creating a Simulated Memory Subsystem
Use the MMU constructor to create an instance of a memory subsystem. The (required) first argument to the constructor is the number of page frames to configure (total physical memory size is the number of page frames × 16384 bytes). The maximum number of page frames is 4096 (0x1000), which would result in a total of 64MB of memory.
The optional second argument specifies the number of Translation Lookaside Buffer (TLB) entries. Don’t use this second argument unless it’s specified in a Lab or Programming assignment (your program will need to flush the TLB at appropriate times in order to operate properly). If this argument is omitted, no TLB is configured.
Transferring Data
The MMU class provides the movb operation to transfer bytes into, out of, and within the MMU. See comments in MMU.h for details on the versions of this operation.
MSS Addressing
The MMU has two addressing modes: physical and virtual. On initialization, the MMU is in physical address mode; all virtual addresses are mapped directly to physical addresses.
Here’s a sample program which just writes a few bytes to memory in physical mode, and reads them back:
#include
#include
int main(int argc, char** argv) {
mem::MMU mem(16);
mem::Addr vaddress = 0x300;
uint8_t data[] = {‘a’, ‘b’, ‘c’, ‘\0’ }; mem.movb(vaddress, data, sizeof (data)); uint8_t buffer[sizeof (data)]; mem.movb(buffer, vaddress, sizeof (buffer)); std::cout << buffer;
}
Virtual Mode
The MMU is controlled by the PMCB (Processor Memory Control Block). The format of the PMCB is defined in PMCB.h (which is included with MMU.h). PMCB values are only used in virtual mode; the PMCB is ignored in physical memory mode.
These fields are defined in the PMCB:
• page_table_base:thephysicaladdressofthepagetable.Itmustbeonapageaddressboundary
(multiple of 0x4000).
The following fields are used internally to preserve the state of a partially executed operation which has been interrupted by a page fault and may be resumed after the required page has been mapped.
• operation_state:thestateofanoperationwhichhasbeeninterruptedbyapagefaultorother error. The constructor will initialize this value to PMCB::NONE for a new PMCB. The MMU may set
3
this value to PMCB::READ_OP or PMCB::WRITE_OP as necessary. When setting the PMCB, set this
field to PMCB::NONE.
• next_vaddress:thenextvirtualaddresstobeaccessedbytheinterruptedoperation.TheMMU
user may read this address to determine the address of a page fault or other error.
• remaining_count:numberofbytesremainingtotransferintheinterruptedinstruction.
• user_buffer:pointertothenextbyteintheMMUuserbuffersupplyingthesourceforawrite,or
the destination for a read.
Page Table
The page table consists of 4K (0x1000) entries, each entry 4 bytes long. The table occupies exactly 1 page frame (16K bytes), and must start on a page frame boundary. The entry format is defined in PageTable.h. Each page table entry has this format:
Bit field descriptions (bit positions and masks defined in PageTable.h and MemoryDefs.h):
• P: “Present” bit (bit 1). If set, other fields are valid. If not set, then the corresponding page is not
mapped, and the other fields in the entry are not used. (kPTE_Present, kPTE_PresentMask)
• W: “Writable” bit (bit 0). If set, writing to the page is allowed. If not set, page is read-only.
(kPTE_Writable, kPTE_WritableMask)
• A: “Accessed” bit (bit 2). Set to 1 by the MMU whenever the page is accessed (read or write) using this
page table entry. You can initialize this bit to 0, and then later examine it to see if the page was used.
(kPTE_Accessed, kPTE_AccessedMask)
• M: “Modified” or “dirty” bit (bit 3). Set to 1 by the MMU whenever a page is modified using this page
table entry. You can initialize this bit to 0, and then later examine it to see if the page was modified.
(kPTE_Modified, kPTE_ModifiedMask)
• Page Frame Number (bits 14-25): the number of the page frame mapped by this page table entry.
Notice that if all other bits are masked to 0, the result is the physical address of the page frame, with
no arithmetic or shifting required. (kPTE_FrameMask)
• 0: fields shown as “0” must be set to 0. Results are undefined if these fields are non-zero.
Virtual Address Format
Virtual addresses have the following format:
• Page Table Index (bits 14-25): selects the page table entry to be used to map the virtual address to a physical address. (kPageNumberMask)
• Offset (bits 0-13): offset within page. (kPageOffsetMask)
• 0: the upper 6 bits of the virtual address must be 0. Non-zero values will result in an MMU error.
4
PMCB Control
The MMU maintains two PMCBs, one for kernel mode, one for user mode. One of these will be active at any given time. Several MMU functions are provided to work with the PMCB:
• enter_virtual_modeinitializesthekernelmodePMCBwithacopyofthespecifiedPMCB,makes it the current PMCB, and puts the MMU into virtual (protected) mode. The page table at the base address specified in the PMCB will be used whenever kernel mode is active.
• set_user_PMCBloadsanewusermodePMCBintotheMMU,andmakesitthecurrentPMCB, including using the page table specified in the PMCB. The argument is copied into the MMU. The page table at the base address specified in the PMCB becomes the current page table.
• get_user_PMCBcopiesthecurrentuserPMCBintoaspecifiedvariableinthecallingprogram.
• set_kernel_PMCBmakescurrentthekernelPMCBandassociatedpagetablespecifiedby
enter_virtual_mode. Returns a copy of the previous PMCB.
• get_kernel_PMCBcopiesthekernelmodePMCBintoaspecifiedvariableinthecallingprogram.
When switching contexts due to a task switch, get the current PMCB, then set it back when the interrupted task is ready to resume.
Translation Lookaside Buffer (TLB)
The MMU class constructor specifies whether the TLB is enabled, and the number of entries in the TLB. The TLB is never flushed automatically; your code must explicitly flush the TLB whenever the page table is changed or a new page table is put into use by set_user_PMCB or set_kernel_PMCB. Three functions are provided by the MMU class to control the TLB:
• isTLBEnabledreturnstrueiftheTLBwasenabledbytheMMUconstructor,falseotherwise.
• FlushTLBclearsallentriesfromtheTLB.
• get_TLBStatsreturnsstatisticsaboutthenumberofTLBhitsandmisses,andthemaximum
number of entries used. Statistics are provided for recent history (since the last TLB flush) and complete history since the MMU object was created.
Memory Subsystem Faults
Faults which occur during MMU calls due to a page not present or a write permission violation will result in a call to a user-supplied fault handler. This emulates the CPU interrupt process when such a fault occurs on actual hardware. If a fault occurs and no handler has been specified, a default hander will terminate the process.
Fault types are:
• Page Fault: generated when a get or put operation attempts to access a page which is not mapped
(Present bit not set in page table).
• Write Permission Fault: generated when a put operation attempts to access a mapped page which
does not have the Writable bit set in the page table entry.
For either of these exceptions, the PMCB next_vaddress will contain the virtual address at which the fault occurred.
Fault Handlers
In virtual memory mode, fault handlers should be set for page faults and write permission faults. The default handlers simply throw a fatal exception.
5
A fault handler is a class object which is a user-defined subclass of base class mem::MMU::FaultHandler. This subclass may store additional data or define additional methods beyond the base class. On detection of a fault, the MMU will switch to kernel mode and call the Run method of the specified fault handler. A single argument will be supplied to the Run method, a read-only copy of the user mode PMCB. The fault handler may attempt to remedy the fault (for example, by allocating a new page at the location of a page fault), in which case it should return true to retry to operation which caused the fault. If the fault handler returns false, the operation will not be retried, and the original MOVB operation will return false.
A fault handler is set by creating a shared pointer (type std::shared_ptr) to a fault handler object allocated using std::make_shared, and then passing that shared pointer as the argument to either of the MMU
methods SetPageFaultHandler or SetWritePermissionFaultHandler. You can see examples of fault handlers in the Memory Subsystem project test code.
Exceptions
The MMU will throw an exception if a user request would cause a fatal malfunction. This will usually indicate an error in the user program, possibly in the page table contents or other errors. Your program should not attempt to catch these exceptions; they do not represent recoverable conditions.
Examples
See the file MMUTests.cpp under “Test Files” in the MemorySubsystem project.
Bug Reports
If you think you’ve found a bug in the MSS or a documentation error, please email the instructor. Include a description of the bug or error with as much detail as possible, together with (in case of a bug) a zip file containing an export of your NetBeans project. Finding an actual bug or documentation error may result in up to 5 points extra credit on the assignment where the bug or error was found (only for the first person to report each bug or error).
Before reporting a bug, examine your code carefully to make sure that the bug is in MSS and not in your code.
Appendix – Running the tests in the MemorySubsystem Project
If you’re interested in how to run the tests, the details are below. This is completely optional.
The unit tests included with the project are configured to use the Google Test testing framework. To install the Google Test framework on your Linux VM for use with NetBeans:
1. Install the NBCndUnit plugin for NetBeans (see http://plugins.netbeans.org/plugin/57174/nbcndunit for instructions).
2. Install the Google Test framework for Linux by entering the following commands in a terminal on your Linux VM:
sudo apt-get install libgtest-dev cmake
cd /usr/src/gtest
sudo cmake CMakeLists.txt
sudo make
3. You can now build and run the unit tests by right-clicking on the MemorySubsystem project in NetBeans, then selecting “Test” from the menu.
6