KGDB remote debugging connection issue via USB and Serial connection

I am having issues with serial and usb connection between host and target. Below is my setup. Both host and target do NOT have any serial (DB9) ports.

Host : Running windows + VMshare + Ubuntu

Target: Running linux kernel 3.19 . Has a MINI usb port which acts as a serial port, i think its ( CP210x uart to usb )

Connection 1 : Host ( USB to DB9 male-PL2303) + DB9 female to female + (DB9 male to USB) target.

Connection 2 : Host ( USB ) --cable-- (USB mini) Target

Host ( ubuntu VM ), can recognize the USB device (both connections types ) as /dev/ttyUSB0. The device does not show up on the windows device manager since VM takes over the device control.

Target boots into UEFI shell. I modify the syslinux.cfg file to append "kgdbwait kgdboc =ttyS0, 115200" to APPEND flag. Save the change ( press F2) then exit ( press F3 ). Boot into the image. Target now enters the kdb prompt with the following message

kgdb: Waiting for connection from remote gdb...
Entering kdb ( current= <64bit address>, pid 1) on processor 0 due to Keyboard Entry
Kgdb > _

on the host side, i do the following commands and below is error

root@ubuntu: cd /images
root@ubuntu: sudo gdb ./vmlinux
Reading symbols from ./vmlinux done.
(gdb)
(gdb) target remote /dev/ttyUSB0
Remote debugging using /dev/ttyUSB0
Ignoring packet error, continuing...
warning: unrecognized item "timeout" in "qSupported" response
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Bogus trace status reply from target: timeout

Experiments i tried

on Host i used " target remote /dev/ttyS0 " , still same issue
Tried different cables in each connection ( 1 and 2 ) mentioned above
on Target removed the edit in the syslinux.cfg file in UEFI shell, booted the image and entered kgdb using "echo g > /proc/sysrq-trigger"
All kernel configuration related to KGDB* , KGDB_SERIAL*, KGDB_USB* are enabled
all available baud rates

Questions

If i use "kgdbwait kgdboc=ttyUSB0, 115200" instead of " kgdbwait kgdboc=ttyS0, 115200" the target does NOT halt. When the target boots up fully to login prompt, i can see the device is recognized as ttyUSB0 when using connection 1. However as it does not stop does that mean, KGDB using USB does not work ? or for USB debug , i need to use direct USB--USB wire ( connection 3 ) ?
does syslinux.cfg support USB debug? becuase there is a SERIAL flag which has value" 0, 115200 " where 0 refers to ttyS0. syslinux documentation does not have any values for USB type device.
using connection 2, why am i seeing the timeout and packet error issues
occasionally with connection 2, when i execute " target remote /dev/ttyUSB0 " on the host, i notice junk characters on the target. So there is some communication happening, so tried different baud rates still same issue. Does this indicate anything inherently wrong with my setup?
In many online forums/ documents i do not see the "entering kdb due to keyboard entry" when the kernel enters kdb prompt. is this unusual?

Solution

The setup of remote kgdb debug is a bit tedious. There are several prerequisites/limitations for the kgdb to work. I will try to break it down.

You have to prepare two machines in this setup.
HOST: Where the agent-proxy & GDB installed.
TARGET: The Linux system being debugged.

Connection Setup

[Host /dev/ttyUSB0] USB to Serial --------- COM port [Target /dev/ttyS0]

On the TARGET side, it's not possible to use the USB interface with kgdb. This is because all the USB-Serial driver (CP210x, PL2303, ...etc) did not implement the polling hook. You have to connect the COM port with a serial cable directly. It is ok to use the USB interface on the HOST side. Since it is a serial connection, you have to use a USB-to-Serial converter and install the proper driver on HOST.

Set the proper baud rate on both sides:

[Target] stty -F /dev/ttyS0 115200
[Host] stty -F /dev/ttyUSB0 115200

Make sure the serial connection works on both direction. You can use:

[Host] cat /dev/ttyUSB0
[Target] echo 'from TARGET to HOST' > /dev/ttyS0

[Target] cat /dev/ttyS0
[Host] echo 'from HOST to TARGET' > /dev/ttyUSB0

You should see the messages on both side of machine. If not, there might be some problems on the cable or driver.

Compile Kernel

Enable KGDB* , KGDB_SERIAL*, KGDB_USB*, DEBUG_INFO, DEBUG_INFO_DWARF4, MAGIC_SYSRQ in the kernel config. Compile and install on the TARGET.

The main purpose here is to enable KGDB feature & preserve debug information in vmlinux.

agent-proxy Setup

agent-proxy acts as a proxy for the TARGET's serial port. It splits up the serial port for multiplexing. One for primary console I/O, the other for GDB session. Thus, we can work on both simultaneously. You should run the agent-proxy on HOST machine.

git clone http://git.kernel.org/pub/scm/utils/kernel/kgdb/agent-proxy.git
cd agent-proxy ; make
./agent-proxy 5550^5551 0 /dev/ttyUSB0,115200

This will redirect:

TARGET's console to HOST:5550
TARGET's kgdb listening port to HOST:5551

Start To Debug

First, open the primary console:

[Host] telnet localhost 5550

Entering the kdb mode, either by:

[Target] echo ttyS0,115200 > /sys/module/kgdboc/parameters/kgdboc
[Target] dmesg | tail
(you should see KGDB: Registered I/O driver kgdboc, otherwise it failed)

[Target] echo g >/proc/sysrq-trigger

Or, by adding the following kernel parameters in TARGET's bootloader (for early kernel debug):

console=tty0 console=ttyS0,115200 kgdbwait kgdboc=ttyS0,115200

The TARGET machine will halt immediately once it breaks into the kdb.
At the same time, you will see a kdb prompt on the primary console:

....
Entering kdb (current=0xcb846c80, pid 2301) on processor 3 due to Keyboard Entry
[3]kdb>

Type kgdb then enter. The TARGET is now pending for remote GDB's connection. We will connect it from the HOST.

Host> gdb vmlinux
(gdb) target remote localhost:5551
Remote debugging using localhost:5551
kgdb_breakpoint () at kernel/debug/debug_core.c:1072
1072             wmb(); /* Sync point after breakpoint */
(gdb)

Enjoy your kernel debugging!