Setting up a debugging environment for the Linux Kernel

Whether you are developing a kernel module or doing some kernel hacking, it’s essential that you set up a development and debugging environment which will help you in complete the project faster and with ease. The method of using printk to display debug info is a quite limiting one when it comes to developing more complex modules/drivers. That being said, the reason I’m writing this is to share with you everything I learned from setting up the tools for testing and debugging a LKM rootkit.

When it comes to debugging the Linux kernel there are a few options to choose. If you have available hardware laying around you could use two machines; a development machine connected to the KGDB infrastructure of the kernel running on the target machine via a serial connection as described in here. An other option, more practical, is the use of KDB which does not require additional hardware although has other limitations because it’s not a source level debugger like kgdb but rather a kernel-level debugger. The final and most compelling option in my opinion is visualization, you can run the kernel on a virtual machine with VMware that will run as a userspace application which can be manipulated easily and even if the kernel crashes, the development machine will not be affected. More fitting even than the the VMware or some other visualization application, is a machine emulator and visualizer called QEMU, which gives us the option to test our modules for a range of different architectures because QEMU can emulate different processor architectures. In this article we will describe the process of debugging the kernel with the use if QEMU/KVM.

Next we have to examine our options regarding running the kernel inside QEMU. I see no point running a complete Linux based operating system since we are more interested in the kernel and the modules of that kernel, so a better option is to use tools like Buildroot that is an easy to use tool in order create simple embedded systems. With buildroot we can obtain an image of a kernel version we want to test and a small root filesystem which will contain our Linux Kernel Module we want to test/debug. This is convenient because QEMU can run directly from a kernel image and a rootfs.

Continue reading “Setting up a debugging environment for the Linux Kernel”