Symmetry

After a long time of not working on Symmetry due to University and other commitments, I put in a bit of work today and made a lot of progress in getting Symmetry multitasking. Today it actually did show some evidence of running two things at once, even if it does crash after about four seconds…

Still, it’s a big improvement from before, when either it would just run one of the threads, or immediately restart the virtual machine.

The updated code is in Git, but is disabled.

I made a small breakthrough today with the first Symmetry release, 0.1 (Codename Interceptor), with the two most critical defects in the development branch being fixed. These were both problems that stemmed from a problem in Symmetry’s memory management, and have been blocking progress in the kernel for around three months now. With these fixed, as well as some good progress on vector support, Symmetry is well on the way to gaining multitasking and Elf executable support!

According to Trac. we are now 58% of the way to the 0.1 release. I expect that this will be done mid-year.

Symmetry’s build system has recently been completely revamped, making builds faster and more efficient, and making the source tree much cleaner in general. The old makefile used to build every source file every time it was run, which makes it far slower than it needs to be having only changed one or two files. It would also dump every object file in the top of the kernel source directory, which would make it rather messy.

The new build system solves all these problems, and will be far more portable than the old version. Whereas the old version was hard-wired to only compile for the x86 architecture, it would be very difficult to port it without writing large parts of the kernel target for every additional arch. The new one should be able to just work when you change one of the variables.

The new makefile now exists out of the source tree, in the build directory (instead of being in the kernel directory before). I also moved the image and ramdisk folders out so now just the source is in the kernel’s source directory, as it should be.

I haven’t really been able to do any work on Symmetry recently, given that I have been quite busy with the end of school recently, and that my development computer is not currently set up. So at the moment, it is actually quite badly broken - the latest version I compiled would just quit with a triple fault. This is because I have been getting ready to add multitasking support recently, and some of the infrastructure that I have implemented is fairly broken.

I am going to try and get this fixed as soon as possible.

I have spent most of the time that I have been working on Symmetry trying to find a bug that was causing a General Page Fault (or GPF) at startup. The fault would be raised when I tried to create a heap, so I thought it would therefore be something wrong with the paging or heap management code. After a week or two, it occured to me that there might be a problem with the GDT entries being wrong, and sure enough, it turned out that there was an error setting up the gates, making the whole memory not work as expected. It turned out that this was also causing another GPF in the ACPI initialisation.

So now I am able to test in Bochs and on real machines, instead of just in VirtualBox and QEMU like before. Now I can finish off the heap management (I just have to finish deallocation) and then I’ll move on to multitasking and user mode. I’ve already started to write an ELF executable parser, and once I have those things finished, hopefully I can get Newlib ported to my OS, and then compile apps for it!

I finished Symmetry’s basic virtual file system today. Right now, the only thing that gets mounted on it is the dev directory, and a few files off the ramdisk. The ramdisk (or initrd) is just a tar archive that GRUB loads, and Symmetry parses and mounts on the filesystem. I chose a tar archive over a custom tutorial (as is used in JamesM’s tutorials, which my VFS was somewhat based on), because this way I don’t have to make a program to write the initrd, parsing it is almost as simple as a basic custom FS, and most people would have tar installed, so I can use it in the makefile.

The code, as always, is in our repository at Gitorious.

I’m working on heap management at the moment with Symmetry - now I can dynamically allocate memory, and I have the necessary infrastructure to free it as well - I just have to finish off that function.

I also added a PIT driver (as I will need that for multitasking) and I also did a quick keyboard manager (the callback just prints out a word), but that isn’t working too well - as soon as a key is pressed, I get a general protection fault…

Speaking of general protection faults, my ACPI code triggers one on my computer (when running it as the main OS) but not in Virtualbox, QEMU or Bochs… So, I have to fix that, although it could be quite hard to track down, not being able to test it in an emulator… I also get a GPF when enabling memory management in Bochs, so I will also have to see if that happens on real hardware…

Today I wrote an ostream implementation for Symmetry that uses the existing textmode functions. Now, instead of using textmode::write(text), you can just use cout << “text” like you do in normal C++. It has some non-standard additions (like the ability to set the colour of the text) and doesn’t implement many of the standards, but that doesn’t matter - this implementation is just for the kernel (not general userspace programs).

I’m still working on the ramdisk and vfs, but I’m having a problem with my memory allocator… Right now, if I try to use allocated memory, I just get a page fault. It’s probably because I haven’t made my heap management able to deal with paging. So I suppose I’ll work on that next - I do (as I’ve been saying for a while) need a better heap manager anyway!

I just pushed some code into master that can parse a tar archive. I decided to use tar as the format for Symmetry’s ramdisks because it means that I can have the ramdisk generated by just putting one line in a makefile (instead of having to make a custom utility to make it), and because it makes it far easier to edit the ramdisk again without needing a custom tool.

Right now, the names and contents of the files on the ramdisk are just printed out at startup, but eventually the files on the ramdisk will be added to the VFS (which I am yet to write). After I’ve got the VFS going, (and written some better heap management), I’m probably going to have a crack at putting elf executables on the ramdisk and have Symmetry parse and execute them! I’ll also do multitasking and user mode then, so I can run more than one executable at once, and run them at lower privileges than the kernel.

Well, I finished paging at last… I’m not sure what I’m going to do now - I do need proper heap management, but it would probably be more fun to get multitasking done.

I should probably get my paging code to use the amount of memory reported by Grub rather than assuming a number like 16 MB…

I’d also like to start porting Newlib to my OS. It is kind of useless as long as I don’t have a VFS and ramdisk to load some executables from, and I will need to work out how to load ELF executables as well. Maybe I’ll get the C library and ELF parser working, and then just load a statically linked executable (like a shell) as a module with Grub just for fun at first…

But once I do have a VFS and multitasking working, Symmetry might actually be usable… I’ll port a shell like Bash or Dash, and then maybe an editor like nano. These will have to be run from a ramdisk until I get a CD-ROM driver and support for some filesystems.