COMP3301 Assignment 3

OpenBSD VHD Kernel Driver – Filing the System

Due: 3pm Monday in Week 13 (21st of October) Submission: Git

Code submission is marked in your prac session in week 13 Last Updated: October 12, 2024

1. Academic Integrity

   All assessments are individual. You should feel free to discuss aspects of C programming and assessment specifications with fellow students and discuss the related APIs in general terms. You should not actively help (or seek help from) other students with the actual coding of your assessment. It is cheating to look at another student’s code, and it is cheating to allow your code to be seen or shared in printed or electronic form. You should note that all submitted code will be subject to automated checks for plagiarism and collusion. If we detect plagiarism or collusion (outside of the base code given to everyone), formal misconduct proceedings will be initiated against you. If you’re having trouble, seek help from a teaching staff member. Do not be tempted to copy another student’s code. You should read and understand the statements on student misconduct in the course profile and on the school website: https://eecs.uq.edu. au/current-students/guidelines-and-policies-students/student-conduct

   1. Use of AI Tools

      All assessment tasks evaluate students’ abilities, skills and knowledge without the aid of gen- erative Artificial Intelligence (AI) or Machine Translation (MT). Students are advised that the use of AI technologies to develop responses (e.g. code generation) is strictly prohibited and may constitute student misconduct under the Student Code of Conduct.

2. Background

   This assignment extends the OpenBSD kernel to add support for using VHD disk images as block devices. This is similar to the existing functionality provided by the vnd(4) driver, which supports using files containing a disk image as a block device. The purpose of this assignment is to exercise concepts of low-level disk operations and caching in an operating system kernel environment.

   From a high-level point of view, a physical disk device presents a sequence of bytes that can be written to or read from, with the ability to quickly seek an arbitrary position and read and write at that point. Note that this is a simplification that ignores that disks address and provide access to blocks of bytes, not individual bytes.

   A file on most operating systems offers similar features, i.e., a sequence of bytes that can be accessed by address. Because of these similarities, it is possible for an operating system to provide a common set of operations on both files and disks (e.g., open, close, read, write, seek, etc.) and allow them to be used interchangeably. For example, you could use tar to write an archive to a file in a filesystem, or directly to a disk device. dd, cp, cat, etc can read the bytes

   from a raw disk into a file or visa versa. However, operating systems generally provide extra functionality on top of disk devices such as the ability to partition disks and mount filesystems from them.

   1. vnd(4)

      The vnd(4) driver in OpenBSD provides a ”disk-like interface to a file”. This means the OpenBSD kernel can open a file and present it as a block device to the rest of the system, which in turn allows for the creation and use of filesystems on these disk images.

      The vnd(4) driver currently only supports using raw disk images as backing files. There’s a one-to-one mapping of data offsets for data in the end disk device and the byte offset of that data in the underlying file. This makes the implementation very simple, with the downside that the backing file has to be the same size as the disk vnd is presenting. If you have a 32G disk image, the file will be 32G regardless of how much data is actually stored inside a filesystem mounted on top of it. Similar functionality exists in the loop driver in Linux, and the lofi driver in Solaris and Illumos.

   2. Virtual Hard Disk (VHD)

      Virtual Hard Disk (VHD) is a file format for disk images used by Connectix’s Virtual PC hypervisor, which was later acquired by Microsoft and renamed to Microsoft Virtual PC. A defining feature of VHD files is that they allocate space and extend the size of the file backing a disk image on demand. This is different to raw disk images where the backing file has to pre-allocate space for the entire disk image up front. With VHD, you can also have fixed-size images, works in a very similar way to the raw images used by vnd.

      VHD also supports using a read-only file as a base image and making writing changes to a separate file. When used by a hypervisor, this allows for thin provisioning of virtual machines where only changes made by each virtual machine are stored rather than repeated copies of disks. Due to the popularity of Connectix/Microsoft Virtual PC, it is common for disk images to be distributed as VHD files.

      The VHD format used by this assignment is documented at: https://stluc.manta.uqcloud.net/comp3301/public/2024/comp3301-vhd-spec.pdf

      The same file can be found on Blackboard. This specification differs from the official Microsoft specification in that it includes annotations for ease of understanding and contains corrections to errors in the official specification. The official VHD format is documented by Microsoft in Virtual Hard Disk Format Spec_10_18_06.doc.

3. Instructions

   To complete the assignment, you will need to do the following:

   1. Download the base code patch

      cd ~

      ftp https: // stluc. manta. uqcloud . net/ comp 3301 / public /2024 / comp3301 -2024 – a3 . patch

1. Create the a3 branch

   cd / usr/ src

   git checkout -b a3 openbsd -7.5

1. Apply the base code patch

   git am < ~/ comp3301 -2024 – a3 . patch

1. Install the includes

   cd / usr/ src/ include doas make includes

1. Build the kernel

   cd / usr/ src/ sys/ arch / amd64 / compile / GENERIC . MP make obj

   make config make – j4

   doas make install

1. Reboot

   doas reboot

1. Build and install vhdctl

   cd / usr/ src/ usr. sbin / vhdctl make obj

   make

   doas make install

1. Create vhd(4) device node 0

1. Specifications

   You will be extending the OpenBSD kernel to add support for using VHD files as a backend for a vhd(4) virtual block device. vhd(4) is roughly based on vnd(4). Boilerplate code for the device entry points and command line utility will be provided, but you will be implementing the handling of the file and the VHD file format within the provided kernel driver.

   Only a subset of the VHD functionality listed in Microsoft’s specification of the file format is required. The following functionality is required:

   • Read support

   • Write support

   • Fixed-size images

   • Dynamic images

     Differencing images do not need to be supported. In addition to supporting the VHD file format, the kernel should implement the following:

   • Deny attaching VHD files which are invalid, corrupted or contain features not supported by your kernel driver.

   • Deny detaching VHD files when the disk is open unless the force flag is passed to the

     VHDIOCDETACH ioctl.

   • Return the name of the VHD file the device was attached to for the VHDIOCFNAME ioctl.

   • Populate a struct stat for the currently attached VHD file for the VHDIOCSTAT ioctl.

   1. VHD Disk I/O

      Reads and writes against the vhd block device should be mapped to vn_rdwr() against the VHD backing file. Writing to the vhd device must not corrupt the backing VHD disk image. The read and write functionality contributes to the majority of the marks for this assignment.

      Caching of VHD structures (e.g., footer, dynamic disk header and block allocation table) and data blocks should be implemented for dynamic images, for improved performance. For ex- ample, if you read a sector from block 0x3301 and immediately read another sector from the same block, you should not need to read the backing VHD file twice. Same goes for mixed reads and writes, for example, read on block 0x3010 followed by a write on the same block should result in one read and one write (not 2 reads and 1 write). The number of data block cached, the cache replacement algorithm (if you intend on caching 2 or more blocks) and the implementation details of the caching is up to you to decide, as long as some form of caching is implemented.

   2. ioctl interface

      The following ioctls should only work on the raw partition of the character device associated with each vhd disk. Except for VHDIOCATTACH, they should only work when a vhd disk is attached to a backing file.

      VHDIOCATTACH

      Specify the VHD file to attach as a block device, and parameters for using the disk in the kernel. The vhd_attach struct contains the following fields:

      • vhd_file – The name of the VHD file to attach a vhd disk to.

      • vhd_readonly – The vhd disk can be written to when set to 0, and if the VHD file is read-only, the vhd disk should fail to attach. The vhd disk should be read-only when set to a non-zero value, and the VHD file should be opened read-only.

        VHDIOCDETACH

        This ioctl requests the block device be detached, and the backing file closed. If the disk is in use, the request should fail with EBUSY unless the unsigned int ioctl argument is set to a non-zero value. A non zero value requests the forced removal of the block device and close of the backing VHD file.

        VHDIOCFNAME

        This ioctl requests the name of the VHD file used for the currently attached block device. The name should be the same as what was passed as the filename in the VHDIOCATTACH ioctl.

        VHDIOCSTAT

        This ioctl is equivalent to an fstat(2) system call against the backing file.

2. Provided Tools/Files

   1. vhdctl

      The patch includes source for a vhdctl utility that uses the ioctls defined above to control vhd devices. It is similar to vnconfig. The source can be found after the patch is applied in src/usr.sbin/vhdctl.

   2. vhdtool

      vhdtool allows you to create VHD files and convert disk images between raw and VHD. It also allows you to perform basic consistency checks on VHD files, as well as to dump their footers and dynamic disk headers in a human-readable format onto the console. It may be installed by running the following commands:

      cd ~

      ftp https: // stluc. manta. uqcloud . net/ comp 3301 / public /2024 / vhdtool. tar doas tar – xpf vhdtool. tar -C / usr/ local/ bin

1. rawtest.img

This is a raw disk of size 10 MiB which contains a filesystem with the following files:

Download: https://stluc.manta.uqcloud.net/comp3301/public/2024/rawtest.img.