Debugging Tips

1. Unoptimization for debugging

Patches below makes a program slow, but you can debug the program line by line and inspect variables on gdb. This can also be used on Fedora Core 4. This is only for target i386-softmmu.

Patch for Windows host.

Patch for Linux host.

After AIO is introduced, SIGALRM and SIGUSR2 are used on Linux host. They have effect on gdb. This patch disables SIGUSR2 and AIO uses SIGALRM. It also disable timer interrupt because it is not good for debugging.

On Windows host, these signal handling is not used so that debugging is easier.

-O0 option of gcc is for unoptimization. Removing -fomit-frame-pointer makes easy to debug because normal function call is used. But these can't be used for OP_CFLAGS because Object file op.o is analyzed by dyngen. Only -O2 option is too optimized for other source codes. -O2 -fno-gcse or -O1 can be used on your environment( version of gcc etc.).

To use -O0 option, it needs to unregister variables of env and from T0 to T3 in exec.h. This enables inspecting struct CPUX86State.

ASM_SOFTMMU in target-i386/op.c can't be used. The program can't be compiled.

This is my debugging method on Linux host.

(1)Apply patch to current CVS and make.

$ cvs -z3 co qemu

$ cd qemu
qemu$ patch -p0 <../qemu-20061108-debug-on-linux.patch
qemu$ ./configure --target-list=i386-softmmu --cc=gcc32
qemu$ make
(2) Change directory to i386-softmmu and edit .gdbinit file.
qemu$ cd i386-softmmu
i386-softmmu$ vi .gdbinit
This is .gdbinit file.
file qemu
set args -L ../pc-bios -hda ../../linux.img
b main

define hook-stop
handle SIGALRM nopass

define hook-run
handle SIGALRM pass

defilen hook-continue
handle SIGALRM pass


AIO used SIGALRM in the patch and set gdb ingore it.

(3) Start gdb.
i386-softmmu$ gdb
Then the program stops at main routine.

If you see an error that "Couldn't get registers: No such process" on Fedora Core 4, it is a bug of gdb 6.0. A workaround is :
[Fedora Core 4] LD_ASSUME_KERNEL=2.2.5
[Fedora Core 4] gdb

2. How to debug the program on gdb

After applying the patch and making the program, start gdb as follows.

i386-softmmu$ gdb qemu.exe
GNU gdb 5.2.1
Copyright 2002 Free Software Foundaton, Inc.

To set starting options by set args and set breakpoint by b(break).

(gdb) set args -L ../pc-bios -hda ../linux.img
(gdb) b main

To start the program, use r(run). To execute a next line in source code, use n(next) and to inspect variables, use p(print).

Starting prgram: C:\qemu\i386-softmmu\qemu.exe -L ../pc-bios -hda ../linux.img

Breakpoint 1, main (argc=5, argv=0x1584a08)
	at C:/qemu/vl.c:2758
2758         DisplayState *ds = &display_state;   (This is source code to be executed next.)
(gdb)p display_state           (This shows contents of display_state.)
$1 = {data = 0x0, linesize =0, depth = 0, width = 0, height = 0,
   dpy_update = 0, dpy_size = 0, dpy_refresh = 0}
(gdb)p &display_state          (This shows memory address of display_state.)
$2 = (DisplayState *) 0x4b6018      (This value depends on your environment.)
(gdb)p ds                      (This shows value of ds.)
$3 = (DisplayState *) 0x7800bd6     (This variable is not initialized now.)
(gdb)n                         (This executes 2758 line in vl.c.)
(gdb)p ds
$4 = (DisplayState *) 0x4b6018      (This value became equal to memory address of display_state.)
(gdb)p *ds
$5 = {data = 0x0, linesize =0, depth = 0, width = 0, height = 0,
   dpy_update = 0, dpy_size = 0, dpy_refresh = 0}
                                    (These values are equal to contents of display_state.)

Then you can see that memory address of display_state is entered to ds.

To set break point at main_loop and execute till it, c(continue). To show source code, l(list).

(gdb)b main_loop
Breakpoint 2 at 0xxxxxx: file C:/qemu/vl.c, line 2459.
(gdb) c

Breakpoint 2, main_loop () at C:/qemu/vl.c, line 2459
2459        CPUState *env = global_env;
(gdb) list
2454    }
2456    int main_loop(void)
2457    {
2458         int ret, timeout;
2459         CPUState *env = global_env;
(gdb) n
2462              if (vm_running) {
(gdb) p env
$7 = (CPUX86State *) 0xa520048         (CPUState is changed to CPUX86State by macro.)
(gdb) p *env
$8 = {regs = {0, 0, 1536, etc.....

To set break point at 2463 line in vl.c, use b vl.c:2463. To enter a function, use s(step). To exit the function, use finish.

(gdb) b vl.c:2463
Breakpoint 3 at 0x403f5e: file c:/qemu/vl.c, line 2463.
(gdb) c

Breakpoint 3, main_loop () at C:/qemu/vl.c:2463
2463                   ret = cpu_exec(env);
(gdb) s
cpu_x86_exec (env1=0xa520048) at C:/qemu/cpu-exec.c:113
113           saved_T0 = T0;
(gdb) finish
Run till exit from #0 cpu_x86_exec (env1=0xa520048)
      at C:/qemu/cpu-exec.c:113
0x00403f69 in main_loop () at C:/qemu/vl.c:2463
2463                    ret = cpu_exec(env);
Value returned is $10 = 256

To show break points, use i(info) b(break). To delete that, del(delete) number. To enable and disable, enab(enable) number, dis(disable) number.

(gdb) i b
(gdb) del 1
(gdb) enab 1
(gdb) dis 1

To exit the program, use q(quit).

(gdb) q
The program is running. Exit anyway? (y or n) y

Please see other commands by h(help).

(gdb) h
(gdb) h running

You can set initial state of gdb by a file .gdbinit.

file qemu.exe
set args -L ../pc-bios -hda ../linux.img
b main

3. Debug print

A program noconsole.exe add/remove a console to/from an existing executable.

Here is souce code.

4. Remote debugging using windbg

I will show you how to debug Windows 2000 guest from a Windows XP host using windbg.

4.1 Installing windbg

Download Debugging Tools for Windows from Microsoft's site and intall it to your host OS.

4.2 Setting windbg

Start windbg and select menu [File]->[Symbol File Path]. It sets the guest OS'es symbol path.
Put srv*c:\tmp* in the dialog. It means that symbol files are read from Microsoft's symbol file server and their temporary files are saved to c:\tmp. Create c:\tmp folder if it doesn't exist.

4.3 Creating a shortcut of windbg

Create a shortcut of windbg on Desktop. Select start->all programs->Debugging Tools for Windows->windbg, and right click on it, and select Send->Desktop(creating shortcut).

Right click on the shortcut and select Property.

You will see link to field. Then add a start option -k com:pipe,port=\\.\pipe\com_1,resets=0,reconnect . com_1 is a name of named pipe.

"C:\Program Files\Debugging Tools for Windows\windbg.exe" -k com:pipe,port=\\.\pipe\com_1,resets=0,reconnect

4.4 Setting Guest OS

If the guest OS is Windows 2000, start it and go to command prompt. Change attribute of boot.ini and open it by notepad.exe .
C:\> attrib -r -s -h boot.ini
C:\> notepad boot.ini

Then copy a entry and add /debug /debugport=com1 /baudrate=115200.

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows 2000 Professional" /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows 2000 Professional" /fastdetect /debug /debugport=com1 /baudrate=115200
An each entry is a one line.
After adding the entry, shutdown the guest OS.

4.5 Starting the guest OS and Connecting from the host OS

Start the guest OS with -serial pipe:com_1 option. com_1 is the name of named pipe above.

qemu.exe -L . -hda win2k.img -serial pipe:com_1

Qemu will stop to wait connection from windbg.

Then click the shortcut of windbg.

You will see selecting menu for debugger enabled in Windows 2000.
Select debugger enabled.

Microsoft Windows 2000 Professional
Microsoft Windows 2000 Professional [debugger enabled]

Then you will see connected messages in windbg. If you can't see this message, close the guest OS and windbg and restart.

Connected to Windows 2000 2195 x86 compatible target, ptr64 FALSE
Kernel Debugger connection established.
Symbol search path is: srv*c:\tmp*
Executable search path is:

After a while, ntoskernl.exe is loaded and get messages.

Windows 2000 Kernel Version 2195 UP Free x86 compatible
Kernel base = 0x80400000 PsLoadedModuleList = 0x8046a4c0
System Uptime: not available

Selecting menu [Debug]->[break] will break an executiong of the guest OS.
It takes some time to see break.

4.6 Some tips

4.6.1 Breaking at loading ntoskrnl.exe

At first, start windbg. Then press Ctrl-Alt-k two times. You will see these messages.

Will request initial breakpoint at next boot.
Will breakin on first symbol load at next boot.
After that, start the guest OS. Then the guest OS will break at loading ntoskrnl.exe

4.6.2 Commands

List drivers loaded.

kd> lm t n
List modules loaded.
kd> x *!
List symbols for a module (for example ntoskrnl.exe)
kd> x nt!*
kd> u
kd> g

5. Debugging DLL with gdb

There are some tips to set breakpoints when you debug DLL with gdb.

For example, when you debug SDL.dll with QEMU, you have to add option -g to make SDL library.

SDL-1.2.9>$ CFLAGS="-O0 -g" configure
Then copy SDL.dll to the place where QEMU is. Then start debugging qemu.

At this point, there are two ways to set breakpoints in SDL.dll. One is to set them after qemu runs and the other is to set them before that.

[1] After qemu runs.

Set argments for qemu and stop at main function.

$ gdb qemu
(gdb) set args -L ../pc-bios -hda ../../linux.img
(gdb) b main
(gdb) run
Then set functions you want to see.

(gdb) b SDL_VideoInit
(gdb) continue
Then you will find the program stops at the function you want to stop.

[2] Before qemu runs.

Set argments for qemu and use dll-symbols to set SDL.dll.
$ gdb qemu
(gdb) set args -L ../pc-bios -hda ../../linux.img
(gdb) dll-symbols SDL.dll
List a function you want to see.
(gdb) list SDL_VideoInit
142     /*
143      * Initialize the video and event subsystems -- determine native pixel format
144      */
145     int SDL_VideoInit (const char *driver_name, Uint32 flags)
146     {
147             SDL_VideoDevice *video;
148             int index;
149             int i;
150             SDL_PixelFormat vformat;
Set a breakpoint at the line you want to stop.
(gdb)@break 145

(gdb) run
Then you will see the program stops at the line you set. If a source code doesn't appear, you may have to use directory to set the place where souce codes are.
(gdb) directory ~/sdl-1.2.9/src/video/
I think address of DLL is not set before the program is loaded to memory so that you cannot set breakpoints by function name.