I'm following linux bootloading using u-boot (using SPL falcon mode where u-boot-spl launches linux directly) on a qemu virtual machine. Now the code jumped to linux kernel and because I have done add-symbol-file vmlinux 0x80081000 I can follow the kernel code step by step using gdb connected to the virtual machine. Actually I loaded the kernel image to 0x80080000 but I had to set the address to 0x80081000 to make the source code appear on the gdb correctly according to the PC value(I don't know why this difference of 0x1000 is needed).
Later I found the kernel sets up the page table (identity mapping and swap table) and jumps to __primary_switched and this is where pure kernel virtual address is used first time for the PC. This is where the call is made at the end of the head.S file.
ldr x8, =__primary_switched
adrp x0, __PHYS_OFFSET
br x8
In the symbol file (vmlinux, an elf file), the symbols before __primary_switched are all mapped at virtual addresses (starting with 0xffffffc0..... high addresses) but the gdb could follow the source even when the PC value was using physical address. (The PC was initially loaded with physical address of the kernel start and PC relative jumps were being used until it jumps to __primary_switched, mmu disabled or using identity mapping) So does this mean, in doing add-symbol-file only the offset of the symbols from the start of text matters?
Another quetion : I can follow the kernel source with gdb but after __primary_switched, I cannot see the source. The debugger doesn't show the correct source location according to the now kernel virtual PC value. Should I tell the debugger to use correct offset using add-symbol-file again? if so how?
ADD (8:32 AM Wednesday, January 12, 2022, UTC)
I found from gdb manual,
"add-symbol-file filename [ -readnow | -readnever ] [ -o offset ] [ textaddress ] [ -s section address ... ] The add-symbol-file command reads additional symbol table information from the file filename. You would use this command when filename has been dynamically loaded (by some other means) into the program that is running. The textaddress parameter gives the memory address at which the file's text section has been loaded. You can additionally specify the base address of other sections using an arbitrary number of '-s section address' pairs. If a section is omitted, gdb will use its default addresses as found in filename. Any address or textaddress can be given as an expression. ..."
I changed my program a little bit to fix a problem. The readelf shows the .text section starting at ffffffc010080800. So I adjusted the command to "add-symbol-file vmlinux 0x80000800" and gdb shows the kernel source correct after jump to linux. Still it doesn't show me the source code after __primary_switched.
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .head.text PROGBITS ffffffc010080000 00010000
0000000000000040 0000000000000000 AX 0 0 4
[ 2] .text PROGBITS ffffffc010080800 00010800
0000000000304370 0000000000000000 AX 0 0 2048
[ 3] .rodata PROGBITS ffffffc010390000 00320000
.... (skip) ...
[12] .notes NOTE ffffffc01045be18 003ebe18
000000000000003c 0000000000000000 A 0 0 4
[13] .init.text PROGBITS ffffffc010470000 003f0000
0000000000027ec8 0000000000000000 AX 0 0 4
[14] .exit.text PROGBITS ffffffc010497ec8 00417ec8
000000000000046c 0000000000000000 AX 0 0 4
Since '__primary_switched' resides in section .init.text, I tried adding "-s .init.text 0xffffffc010470000" or "-s .init_text 0x803ef800"(physcial address) to the add-symbol-file command to no avail. Is my command wrong? Or could this be from page table (virtual -> Physical) problem because I see synchronous exception right after I enter __primary_switched (I see PC value has become 0x200. If the exception vector is located in 0x0, this is the vector entry for synch exception like undefined instruction. I should also check the vector base address has not been set correctly.)
I found my kernel load address was wrong (__PHYS_OFFSET was below physical ddr address start).
After fixing it, the PC increments normally with kernel virtual address and I should just apply the add-symbol-file command using the virtual address.
This was the new section addresses.
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .head.text PROGBITS ffffffc010080000 00010000
0000000000000040 0000000000000000 AX 0 0 4
[ 2] .text PROGBITS ffffffc010080800 00010800
0000000000304370 0000000000000000 AX 0 0 2048
[ 3] .rodata PROGBITS ffffffc010390000 00320000
00000000000a6385 0000000000000000 WA 0 0 4096
[ 4] .modinfo PROGBITS ffffffc010436385 003c6385
00000000000018ff 0000000000000000 A 0 0 1
[ 5] .pci_fixup PROGBITS ffffffc010437c90 003c7c90
00000000000020f0 0000000000000000 A 0 0 16
[ 6] __ksymtab PROGBITS ffffffc010439d80 003c9d80
0000000000006d20 0000000000000000 A 0 0 4
[ 7] __ksymtab_gpl PROGBITS ffffffc010440aa0 003d0aa0
0000000000005808 0000000000000000 A 0 0 4
[ 8] __ksymtab_strings PROGBITS ffffffc0104462a8 003d62a8
00000000000134f2 0000000000000000 A 0 0 1
[ 9] __param PROGBITS ffffffc0104597a0 003e97a0
0000000000000b68 0000000000000000 A 0 0 8
[10] __modver PROGBITS ffffffc01045a308 003ea308
0000000000000cf8 0000000000000000 A 0 0 8
[11] __ex_table PROGBITS ffffffc01045b000 003eb000
0000000000000e18 0000000000000000 A 0 0 8
[12] .notes NOTE ffffffc01045be18 003ebe18
000000000000003c 0000000000000000 A 0 0 4
[13] .init.text PROGBITS ffffffc010470000 003f0000
0000000000027ec8 0000000000000000 AX 0 0 4
[14] .exit.text PROGBITS ffffffc010497ec8 00417ec8
The final kernel image is loaded at 0x80080000. Then __PHYS_OFFSET becomes 0x80000000. (TEXT_OFFSET is 0x80000 by default). Now I can debug the kernel source before __primary_switch using this command.
add-symbol-file images/vmlinux 0x80080800 -s .head.text 0x80080000 -s .init.text 0x803f7800
And after the kernel entered __primary_switched (now kernel virtual address is used), I added this command to see the source and I can follow code using qemu and gdb step-by-step.
add-symbol-file images/vmlinux 0xffffffc010080800 -s .head.text 0xffffffc010080000 -s .init.text 0xffffffc010470000 Hope this helps someone later.
But after some days, I think I could just use add-symbol-file images/vmlinux 0xffffffc010080800 (applying all the section info).