How to enable MMU in QEMU (virt machine a57 cpu)

I am writing a bare-matel ARMv8 program on QEMU, but when i enable the MMU, it can't continue execute any instructions.

The QEMU option is "-machine virt -cpu cortex-a57 -smp 1 -m 1G -nographic -serial mon:stdio -kernel a.bin"

Here is my code https://github.com/zhulangpi/NBOS/blob/mmu/arch/start.S

I try to map 0x4000 0000~0x7fbf ffff(DRAM) to 0xffff 0000 0000 0000~0xffff 0000 3fbf ffff(total 1020MB).

I use GDB to debug the binary image by connect QEMU, when i have enabled MMU, if i execute the next instruction, it shows:

(gdb) x/x 0x400800c8
0x400800c8:     0xd28014b4

(gdb) si

0x00000000400800c8 in ?? ()
=> 0x00000000400800c8:  Cannot access memory at address 0x400800c8

The 0x400800c8 is PA and the corresponding VA(address in linker script) is 0xffff 0000 0008 00c8.

I can access the memory correctly by virtual address in GDB as follows,

(gdb) x/x 0xffff0000000800c8
0xffff0000000800c8 <_start+200>:        0xd28014b4

I config the MMU as follows,

    adrp    x0,  pg_tbl_start    //defined in linker script
    msr ttbr1_el1, x0

    ldr x0, =(TCR_VALUE)        //(TCR_T0SZ | TCR_T1SZ | TCR_TG0_4K | TCR_TG1_4K)
    msr tcr_el1, x0

    ldr x0, =(MAIR_VALUE)       //(0<<(8*1))|(0x44<<(8*0))
    msr mair_el1, x0

    //Initialize VBAR_EL1
    ldr x0, =vector_table_el1
    msr vbar_el1,   x0  

    /* configure init kernel task stack */
    ldr x0, =__init_stack_top   //defined in linker script
    mov sp, x0                  //sp_el1

    mrs x0, s3_1_c15_c2_1
    orr x0, x0, #(0x1<<6)       //cpuectlr.smpen = 1
    msr s3_1_c15_c2_1, x0

    mrs x0, sctlr_el1
    orr x0, x0, #1              // M bit, mmu
    msr sctlr_el1, x0           //enable the MMU

I expect to access the memory and device correctly. Or someone can show me the code how to enable the mmu in qemu virt machine.

Solution

What is happening here is that as soon as you turn on the MMU, the next instruction fetch is performed by doing a virtual-to-physical translation of the program counter value (in your case 0x400800c8). If this translation fails, then the CPU will take an exception, and the gdbstub will not be able to read memory from that address either (since it also operates on virtual addresses).

More specifically, the address 0x400800c8 is in the low half of the physical address range (its bit 55 is 0), so an EL1 translation is going to use the page tables pointed to by TTBR0_EL1. But you haven't initialized TTBR0_EL1, so it is zero still. All the page table entries at that address will read as zero, which is the "invalid" descriptor, and so the MMU translation will fail.

You can fix this by setting up your page tables correctly, so that there is a 1:1 mapping for the addresses your startup/initialization code is running at.