Lexos will use buddy system algorithm for page allocator. I am still learning buddy allocator, since, I don't find much free time nowadays. Flaws in memory management algorithms will make an operating system weaker. So, I am taking much time to learn and design buddy allocator in my OS.
Lexos will have VMM on top of PMM. PMM (Physical Memory Manager) will service the applications and kernel by providing and freeing page frames on demand. The allocated pages will be mapped into virtual address by VMM (Virtual Memory Mapper).
Edit: As for the time being, I have planned to use stack based approach to manage physical page frames. Later, I would use buddy allocator. Though, I am thinking of using a modified version of stack-based approach, which would have individual stack that holds free pages in the order of 2^n.
Thursday, December 15, 2005
Thursday, October 20, 2005
Detecting physical memory
Hi all,
Now, I have come up to explain how to determine the physical memory in the system. You should detect the physical memory map and pass this info to the kernel, which will be used by the physical memory manager. BIOS interrupt 15h, functions E820h, E801h and 88h are there to help you out in this. You should try using these functions in the order mentioned, if one of the function failed. It would be better, if your bootloader supports all these functions. If you use GRUB, instead of your own bootloader, the multiboot structure will be passed to your kernel, which will hold the memory map.
Memory map is a table that holds the information about your physical memory. Each entry in the table holds base address (8 bytes), length (8 bytes) and the type of the memory (8 bytes). You can look into more details about all these BIOS interrupts at http://www.uruk.org/orig-grub/mem64mb.html
When you enable paging in your kernel, READ_ONLY permission can be given to the memory blocks, which are not available for Operating System use.
Now, I have come up to explain how to determine the physical memory in the system. You should detect the physical memory map and pass this info to the kernel, which will be used by the physical memory manager. BIOS interrupt 15h, functions E820h, E801h and 88h are there to help you out in this. You should try using these functions in the order mentioned, if one of the function failed. It would be better, if your bootloader supports all these functions. If you use GRUB, instead of your own bootloader, the multiboot structure will be passed to your kernel, which will hold the memory map.
Memory map is a table that holds the information about your physical memory. Each entry in the table holds base address (8 bytes), length (8 bytes) and the type of the memory (8 bytes). You can look into more details about all these BIOS interrupts at http://www.uruk.org/orig-grub/mem64mb.html
When you enable paging in your kernel, READ_ONLY permission can be given to the memory blocks, which are not available for Operating System use.
Tuesday, October 11, 2005
Memory Management
Hi,
My keyboard driver is complete now and I have started learning about Memory Management. Normally, any 32-bit protected mode operating systems will have atleast two memory managers. One running in the lower level that allocates the actual physical memory and the next that works in the virtual memory layer that interacts with the physical memory manager to actually allocate physical memory. An operating system should efficiently manage memory. There are various techniques available to manage memory efficiently, stack-based approach, bitmap approach and linux's buddy system algorithm. You are not restricted to used only these algorithms. You can very well implement your own idea or the combination of any of those mentioned. Now, let us see the stack-based approach and the bitmap approach to manage the memory. I will let you all know the buddy system algorithm sooner or later :).
* Stack-based approach:
In the stack-based approach, the starting address of the physical pages are pushed into the stack initially. Then, when one or more pages are requested, they will be popped out from the stack as appropriate and the address will be passed to the virtuall mm. The advantage of using stack-based approach is that the speed of alloting pages. But, the main downfall of the stack-based approach is the fragmentation. Fragmentation arises when the free pages in the memory are not contiguous. Although you can run a daemon to reduce fragmentation when the system is free and the memory is considered to be more fragmented. Also, the space this approach takes for storing the free page addresses is also huge. Because, the size of a pointer in a 32-bit environment will be 32 bits. So, to keep track of 128 MB of memory (32768 pages), you need 128 KB of memory. You need not allocate memory pages to maintain these free page information.
You are going to return the actual physical memory to the virtual memory manager. How will the DMA controller manage when you send the physical address > 16 MB? Because, the DMA controller is not going to convert the address into physical address. Rather, it will just send the address through its address line. Well, now, you need to maintain two different stacks, one that will contain page addresses < 16 MB (for DMA) and the other will contain page addresses > 16 MB. Freeing page is as easy as allocating page. Just push the address of the page into the stack, which will used by the physical memory manager.
* Bitmap approach:
The bitmap approach is the alternative approach available to you, if you think that the Stack-based approach eats up more memory. Here, you use only one bit to store the status of the free page. 1 for available, 0 for occupied. So, in a single byte, you can store the status of 8 pages, which uses very less amount when compared with the previous stack-based approach. You can also use the bitmap to effectively allocate the best-fit and first-fit approach, which is not possible in the stack based approach. But, you need to start scanning the bitmap from the beginning, which may consume more time to allocate in best-fit approach. Some macros can be written to substitute the status of page by giving the page address.
I will explain the buddy system that is used in Linux later. I have not yet decided which algorithm to use in my physical memory manager. Recently, I have read some articles that use binary trees to track the free pages, instead of using linked list.
-NOTNULL
My keyboard driver is complete now and I have started learning about Memory Management. Normally, any 32-bit protected mode operating systems will have atleast two memory managers. One running in the lower level that allocates the actual physical memory and the next that works in the virtual memory layer that interacts with the physical memory manager to actually allocate physical memory. An operating system should efficiently manage memory. There are various techniques available to manage memory efficiently, stack-based approach, bitmap approach and linux's buddy system algorithm. You are not restricted to used only these algorithms. You can very well implement your own idea or the combination of any of those mentioned. Now, let us see the stack-based approach and the bitmap approach to manage the memory. I will let you all know the buddy system algorithm sooner or later :).
* Stack-based approach:
In the stack-based approach, the starting address of the physical pages are pushed into the stack initially. Then, when one or more pages are requested, they will be popped out from the stack as appropriate and the address will be passed to the virtuall mm. The advantage of using stack-based approach is that the speed of alloting pages. But, the main downfall of the stack-based approach is the fragmentation. Fragmentation arises when the free pages in the memory are not contiguous. Although you can run a daemon to reduce fragmentation when the system is free and the memory is considered to be more fragmented. Also, the space this approach takes for storing the free page addresses is also huge. Because, the size of a pointer in a 32-bit environment will be 32 bits. So, to keep track of 128 MB of memory (32768 pages), you need 128 KB of memory. You need not allocate memory pages to maintain these free page information.
You are going to return the actual physical memory to the virtual memory manager. How will the DMA controller manage when you send the physical address > 16 MB? Because, the DMA controller is not going to convert the address into physical address. Rather, it will just send the address through its address line. Well, now, you need to maintain two different stacks, one that will contain page addresses < 16 MB (for DMA) and the other will contain page addresses > 16 MB. Freeing page is as easy as allocating page. Just push the address of the page into the stack, which will used by the physical memory manager.
* Bitmap approach:
The bitmap approach is the alternative approach available to you, if you think that the Stack-based approach eats up more memory. Here, you use only one bit to store the status of the free page. 1 for available, 0 for occupied. So, in a single byte, you can store the status of 8 pages, which uses very less amount when compared with the previous stack-based approach. You can also use the bitmap to effectively allocate the best-fit and first-fit approach, which is not possible in the stack based approach. But, you need to start scanning the bitmap from the beginning, which may consume more time to allocate in best-fit approach. Some macros can be written to substitute the status of page by giving the page address.
I will explain the buddy system that is used in Linux later. I have not yet decided which algorithm to use in my physical memory manager. Recently, I have read some articles that use binary trees to track the free pages, instead of using linked list.
-NOTNULL
Friday, September 09, 2005
Back again
Hi all,
Bloggin' you after a long time. :-) I would like to add my current OS status. I have started writing the keyboard driver and it is nearing completion. Finally, I learnt that developing an Operating System on your own needs a plenty of theoritical knowledge. So, I started Learning various documents. I would suggest Intel x86 manuals (if you are coding for the same), GCC manual, NASM manual, Operating Systems theory books (to know the various techniques used for mm, pm, dm, etc). Also, learn the book "Understanding the Linux Kernel" by DANIEL P. BOVET and MARCO CESATI. It explains all sorts of things Linux does, device management, IO Interuppts and exception handling, memory management, virtual memory, etc. It is a must read book for OS developers.
You should also have good design skills. To be able to design a system, you should have a good theoritical knowledge. In order to gain theoritical knowledge, you should rad more books. So, I started reading various operating system books in my free time. I find it quite helpful in designing a system. Recently, I learnt about the system calls. I will explain you about what is a system call, how does it get activated? Are open, creat, read, write actually system calls? Or there exists something below these functions above which these functions act as system calls?
-NOTNULL
Bloggin' you after a long time. :-) I would like to add my current OS status. I have started writing the keyboard driver and it is nearing completion. Finally, I learnt that developing an Operating System on your own needs a plenty of theoritical knowledge. So, I started Learning various documents. I would suggest Intel x86 manuals (if you are coding for the same), GCC manual, NASM manual, Operating Systems theory books (to know the various techniques used for mm, pm, dm, etc). Also, learn the book "Understanding the Linux Kernel" by DANIEL P. BOVET and MARCO CESATI. It explains all sorts of things Linux does, device management, IO Interuppts and exception handling, memory management, virtual memory, etc. It is a must read book for OS developers.
You should also have good design skills. To be able to design a system, you should have a good theoritical knowledge. In order to gain theoritical knowledge, you should rad more books. So, I started reading various operating system books in my free time. I find it quite helpful in designing a system. Recently, I learnt about the system calls. I will explain you about what is a system call, how does it get activated? Are open, creat, read, write actually system calls? Or there exists something below these functions above which these functions act as system calls?
-NOTNULL
Friday, July 08, 2005
Loading the kernel
After POST [http://www.pcguide.com/ref/mbsys/bios/bootPOST-c.html], the BIOS looks at the first sector of the primary boot device. If boot code is written in the first sector, it is loaded into the memory at address 0x7C00 by the BIOS and the system jumps to that code. Now, how does the BIOS identify the boot loader code? Since, this boot code is the first to get loaded into memory, we call it as Primary Bootloader, which can only be a maximum of 512 bytes. The last 2 bytes of a boot loader has, 0xAA55. This is the standard signature for any primary boot loader. Since, the primary loader needs to be less than 512 bytes, it should be written in x86 assembly language.
The primary loader loads either secondary bootloader or kernel into memory. The choice is left to you. The secondary loader does not have the constraint of being less than 512 bytes. It can be of any size and it can be written in a high-level language, such as C, C++, etc. The primary job of a secondary loader needs to set up the environment for kernel, i.e., GDT, IDT, etc, load the kernel and jump to it.
-NOTNULL
The primary loader loads either secondary bootloader or kernel into memory. The choice is left to you. The secondary loader does not have the constraint of being less than 512 bytes. It can be of any size and it can be written in a high-level language, such as C, C++, etc. The primary job of a secondary loader needs to set up the environment for kernel, i.e., GDT, IDT, etc, load the kernel and jump to it.
-NOTNULL
Types of kernel
Since, I am running out of time today, I will explain you the types of kernel shortly. Basically, there are three types of kernel:
- Monolithic Kernel
- Micro Kernel
- ExoKernel
Monolithic:
As the name itself suggests, the kernel has every services like, FS Management, MM, Process Management, etc. in the kernel space. It does not run as a seperate process. So, as you guess, there is no context switching, when you ask for a service. But, the probability of a monolithic kernel getting struck is more. Because, if there is a bug in the kernel itself, nothing can rescue it. Linux and Windows are good examples of Monolithic kernel. Linux, being a monolithic kernel, you can insert modules into the kernel dynamically using insmod command.
Micro Kernel:
Micro kernel runs all the services as a daemon in the user space. So, if a problem occurs in any of the service, the kernel will be able to decide what to do next. But, you pay-off the time to switch to a service in this type of kernel. Micro kernels are some what difficult to design and build than the monolithic kernel. There are always a discussion over the internet, talking about the advantage and disadvantages of monolithic and micro kernel.
Exo Kernel:
Exo kernel is not yet stabilized. It's under design and research. The user mode processes running in this type of kernel has the ability to access kernel resources like process tables, etc directly.
Requirements of OS Deving
Requirements for writing an operating system:
Ok. Let me get into the topic straight away. Today, we shall see, what are the software requirements for developing an OS.
First, decide which platform are you gonna use for your os deving. You can either choose Windows or Linux. Wide variety of tools are available in both platforms.
* Text editor - (I would recommend ConTEXT for windows users, good syntax highlighting and easy to use)
* Assembler NASM (Netwide ASeMbler), etc.If you are very well used to any other assemblers, you can use it. MASM, TASM and FASM are some other assemblers to mention. You will be required to write some part of your OS in assembly language, such as primary bootloader, which when written in other langages will take more than 512 bytes to fit in.
* C Compiler - GCC, etc.You can choose to write OS as a whole in x86 assembly language or by mixing some high level languages such as C and C++ with ASM. How to mix asm and C? That, I will explain you latter.
* Linker - ld, jloc, etc.If you are a good designer, you will be creating multiple files for your OS and compile it individually. At last, you will need a single file by linking all the compiled files. There's where a linker is used. A linker takes multiple input object files and finally gives you a single linked file. Linkers like ld, gives you the full freedom to specify code, data, etc. segments in the linked file through a special file called linker.ld.
* Cygwin or DJGPP [If you are deving in Windows]If you choose to use Linux command and other stuffs like gcc, ld in Windows, you will surely need Cygwin or DJGPP installed.
* partcopyFor windows users only. Linux users, don't worry. We have dd :-). Your primary loader needs to be written to the first 512 bytes of the floppy or harddisk image.
* Virtual floppy drive - VFD for windows users.Again linux users, dont' bother. I will explain you later how to mount your floppy or hard disk image using linux loop devices. When you coding hard, you will be required to write boot loader or your kernel file to your physical floppy drive to test it. This method has two disadvantages. First thing is, the number of floppies you are gonna spend. The second is, the time. Since disk drive accesses take more time, you have to wait some till it gets writed into the drive.
* And at last, an x86 machine simulator, if you choose x86 processor -BOCHS, etc.
Operating systems cannot be run like other application programs. You have to reboot your system everytime to see the results. Don't worry. Your fellow devers has made it easy. Simply, get the image of the floppy drive or hard disk (in which your files are residing) and give it to the bochs. Bochs will do the rest. It will show you the results that you will see in the real hardware. But, we cannot expect 100% accuracy. My suggestion would be to test your OS in the real system, at some regular check points, i.e., when you see a new functionality working in the BOCHS.
Note: Please, see the links for the above mentioned tools in the LINKS section of the LEXOS - main blog page.
Though, I have just now started my way in OS deving, I have given you the basic softwares that you will require. Finally, apart from software requirements, you will need lots of time for debugging itself and persistance, when you see your code doesn't work for a long time :-) Hope this will be helpful for beginners. Comments welcome.
- NOTNULL
Ok. Let me get into the topic straight away. Today, we shall see, what are the software requirements for developing an OS.
First, decide which platform are you gonna use for your os deving. You can either choose Windows or Linux. Wide variety of tools are available in both platforms.
* Text editor - (I would recommend ConTEXT for windows users, good syntax highlighting and easy to use)
* Assembler NASM (Netwide ASeMbler), etc.If you are very well used to any other assemblers, you can use it. MASM, TASM and FASM are some other assemblers to mention. You will be required to write some part of your OS in assembly language, such as primary bootloader, which when written in other langages will take more than 512 bytes to fit in.
* C Compiler - GCC, etc.You can choose to write OS as a whole in x86 assembly language or by mixing some high level languages such as C and C++ with ASM. How to mix asm and C? That, I will explain you latter.
* Linker - ld, jloc, etc.If you are a good designer, you will be creating multiple files for your OS and compile it individually. At last, you will need a single file by linking all the compiled files. There's where a linker is used. A linker takes multiple input object files and finally gives you a single linked file. Linkers like ld, gives you the full freedom to specify code, data, etc. segments in the linked file through a special file called linker.ld.
* Cygwin or DJGPP [If you are deving in Windows]If you choose to use Linux command and other stuffs like gcc, ld in Windows, you will surely need Cygwin or DJGPP installed.
* partcopyFor windows users only. Linux users, don't worry. We have dd :-). Your primary loader needs to be written to the first 512 bytes of the floppy or harddisk image.
* Virtual floppy drive - VFD for windows users.Again linux users, dont' bother. I will explain you later how to mount your floppy or hard disk image using linux loop devices. When you coding hard, you will be required to write boot loader or your kernel file to your physical floppy drive to test it. This method has two disadvantages. First thing is, the number of floppies you are gonna spend. The second is, the time. Since disk drive accesses take more time, you have to wait some till it gets writed into the drive.
* And at last, an x86 machine simulator, if you choose x86 processor -BOCHS, etc.
Operating systems cannot be run like other application programs. You have to reboot your system everytime to see the results. Don't worry. Your fellow devers has made it easy. Simply, get the image of the floppy drive or hard disk (in which your files are residing) and give it to the bochs. Bochs will do the rest. It will show you the results that you will see in the real hardware. But, we cannot expect 100% accuracy. My suggestion would be to test your OS in the real system, at some regular check points, i.e., when you see a new functionality working in the BOCHS.
Note: Please, see the links for the above mentioned tools in the LINKS section of the LEXOS - main blog page.
Though, I have just now started my way in OS deving, I have given you the basic softwares that you will require. Finally, apart from software requirements, you will need lots of time for debugging itself and persistance, when you see your code doesn't work for a long time :-) Hope this will be helpful for beginners. Comments welcome.
- NOTNULL
Subscribe to:
Posts (Atom)