Debugging the x86 Kernel with kdebug and VMWare#

This page will eventually be a technote on how to use the vmware virtual machine to debug the QNX microkernel. For now it's a placeholder documenting how I did it, and the problems I encountered.

VMWare Workstation#

VMWare Workstation (the latest version is 6.0) is a useful tool in that it allows you to run one or more virtual x86 cpus on your development system. As far as the OS is concerned, it's an independant x86 machine, complete with BIOS, serial, disk and network devices.

Our standard qnxbasedma.build and bios.build will work with vmware.

VMWare supports PXE netbooting, but probably the simplest method to load a fresh image is to generate a boot floppy image. You can do this with the mkifs filter, mkifsf_vmware

NB: The current 6.3.0 version has a bug in it under windows, since it assumes that PATH_MAX is greater than BLOCKSIZE(512). This is not the case under cygwin. PR 25358 indicates that this is now fixed.

To generate a floppy boot image with this filter, simple replace

virtual=x86,bios +compress

with virtual=x86,vmware +compress

Using the GNU linker to build a relocateable procnto-instr#

gdb requires a relocated symbol file with debugging info. The current way we do things is to produce a fully relocated procnto-instr (x86 only) and then use ldrel to relocate it via mkifs.

In order to produce a useable symbol file for gdb, use the GNU linker to build procnto.

Step 1: fixup qconf-win32.mk

Make the following change to your installed copy of qconf-win32.mk

Index: qconf-win32.mk
===================================================================
RCS file: /product/tools/mkfiles/qconf-win32.mk,v
retrieving revision 1.62
diff -c -r1.62 qconf-win32.mk
*** qconf-win32.mk      20 Apr 2005 16:31:22 -0000      1.62
--- qconf-win32.mk      24 May 2005 02:31:23 -0000
***************
*** 123,130 ****
  LD_nto_x86_gcc_qcc = $(QNX_HOST)/usr/bin/qcc -V$(gcc_ver_string)gcc_ntox86
  AR_nto_x86_gcc_qcc = $(AR_nto_x86_gcc)
  UM_nto_x86_gcc_qcc = $(UM_nto_x86_gcc)
! LDBOOTSTRAP_nto_x86_gcc_qcc = $(LDBOOTSTRAP_nto_x86_gcc)
! LDBOOTSTRAPPOST_nto_x86_gcc_qcc = $(LDBOOTSTRAPPOST_nto_x86_gcc)

  CC_nto_x86_icc = icc -c
  AS_nto_x86_icc = icc -c
--- 123,130 ----
  LD_nto_x86_gcc_qcc = $(QNX_HOST)/usr/bin/qcc -V$(gcc_ver_string)gcc_ntox86
  AR_nto_x86_gcc_qcc = $(AR_nto_x86_gcc)
  UM_nto_x86_gcc_qcc = $(UM_nto_x86_gcc)
! LDBOOTSTRAP_nto_x86_gcc_qcc = $(LR_nto_x86_gcc_qcc) -Bstatic -Wl,--no-keep-memory -nostartup -Wl,-u_start
! LDBOOTSTRAPPOST_nto_x86_gcc_qcc = $(LRPOST_nto_x86_gcc_qcc)

  CC_nto_x86_icc = icc -c
  AS_nto_x86_icc = icc -c

:_NB:_ You will probably have to copy this over to $QNX_TARGET/usr/include/mk/qconfig.mk since links don't work right under windows. :_NBB:_ You will also probably want to compile with CCOPTS += -fno-omit-frame-pointer -g3 for building procnto

Step 2: fix your build file to use the right linker

Now mkifs will complain because it will try to use gcc_ntox86le as the target to qcc to relocate your procnto (sheesh!), so add in the following line to your build script

[virtual=x86,vmware +compress ] boot = {
[linker=\"qcc -v -bootstrap -M -nostdlib -Wl,--no-keep-memory -Vgcc_ntox86 %(h!=0, -Wl,-Ttext -Wl,0x%t%)%(d!=0, -Wl,-Tdata -Wl,0x%d%)  -o%o %i\"]
...
}

Also add

 [+keeplinked]

before your procnto-instr so that you get a procnto-instr.sym symbol file to use for debugging

:_NB_ The current qcc conf file lacks a ld_script attribute and a nto.link file. This needs to be fixed (or at least mkifs does to pad out the bss section of procnto-instr. I just copied the shle nto.link and moved the bss section into the data group, ie:

    .data ALIGN(0x1000) : {
        *(.data)
        *(.bss)
        *(COMMON)
        _end = .;
        __end = .;
        end = .;
    }:data

And then you will need to add to the gcc_ntox86+static portion of your $QNX_HOST/etc/qcc/gcc/2.95.3/gcc_ntox86.conf

ld_script=-T"$(QNX_TARGET)/x86/lib/nto.link"

This is in GNATS now... PR 26915 PR 26916

:_NB_ If you see it complaining about not being able to find a watcom ld then you need to update your bios.boot or vmware.boot to the latest version

Step 3: Download and install the vmwaregateway application

VMWare can redirect it's serial port to a named pipe. gdb doesn't support connected to windows named pipes (why not?!!) but there is a useful util called vmwaregateway.exe that is over on the L4Ka site. You can get it here

 [http://l4ka.org/tools/vmwaregateway.php] 

Install it, register it as a services and start the service.

 vmwaregateway.exe /r 

followed by a

 net start vmwaregateway

Step 4: Add a serial port to your vmware configuration

Go to the add hardware wizard, select a serial port and then redirect the output to the named pipe
.\pipe\vmwaredebug

In the settings you should specify "This end is the client" "The other end is an application" and "Yield CPU on poll"

Step 5: Add a floppy image to your vmware configuration

Go the add hardware wizard, select floppy drive and specify the boot image you created in your build file

Step 6: build and install services/kdebug

You will need to hack services/kdebug/gdb/gdb.c to remove support for the qOffsets command. This doesn't work properly, isn't needed anymore and will screw up gdb - just take it out (Colin will do this soon)

Step 7: add kdebug support to your build file

add the following to your build file

startup-bios -D 8250.2f8.38400 -K 8250.3f8.38400 -v
gdb_kdebug -K -D1

Note that you can omit the -K from the gdb_kdebug line if you don't want procnto to stop on it's entry point.

Also note that the debug port is now going out the serial port, even though you probably don't have one setup. You need this though since otherwise when you 'ungrab' the input from vmware with ctrl-alt, you will abort the kernel debugger!

Step 8: connect with gdb

Start the vmware session. If you specified -K to gdb_kdebug then it should hang waiting for gdb to connect.

kdebug uses the gdb remote protocol. So load the procnto-instr.sym into gdb, and then connect to localhost port 567

 (gdb) file procnto-instr.sym
 (gdb) target remote localhost:567

You should now be live! Set a breakpoint and hit the slopes, dude!

Step 9: use the IDE to debug

Ooh!! Ahh!!

Step 10: go back to using command line gdb ... cause that is what kernel guys do

Frankly, gdb 5.2.1 isn't up to the MI demands that the IDE places on it. Maybe gdb 6 will be better! ;v) (with apologies to the tools team)

== Lastly ==

A couple of gotchas

• Our mix of cygwin and non cygwin tools don't play well with each other. If you have cygwin installed, don't BUILD from a mountpoint, and make sure you pass windows pathnames (but strangely, with unix forward-slashes) for command line arguments. Yay.

• Build speed under windows seems really slow. It could be that I'm running too much at once (think disk paging!) but it's most likely cygwin being slow. We really could do with a anti-cygwin liberation army... maybe windows services for unix would be worth it?