I'm writing an ARMv7 disassembler. The ways to switch between ARM and Thumb mode are clearly described in the ARM reference manual, but how do you know what mode a program starts in?
I am using Xcode which compiles to thumb by default, so I know that all of my own programs will start in Thumb unless I force compilation to ARM mode. But, I would like to be able to take an arbitrary mach-o executable and find out the instruction set mode at the beginning of the code.
Is there somewhere in the mach-o header that specifies the instruction set upon entry point?
The processor knows that it's in Thumb mode by turning on the least-significant bit of the program counter, causing the program counter to have an odd value. This bit is ignored for the purpose of instruction fetching and you can switch between ARM and Thumb mode by toggling this bit.
When you create an ARM binary, the linker will set the least significant bit of the address of a symbol depending on whether this symbol points to ARM or Thumb code so the processor automatically picks the right mode on program start. You don't need to care about this.