Windriver VxWorks Simulator Self modifying code

Question

Good morning.

I have a program that is Self-Modifying-Code.

Really, it build the binaries, which then are changed by ELFPatch and changes some function's prologues.

I am working with Windriver WorkBench 3.3 & VxWorks 6.9 Update3.

I created a standard simulator (PENTIUM), when i run my code on the simulator:

void replace_prolog(void* func_ptr) {
    char* p = (char*)func_ptr;
    for (int i=0; i < PROLOGUE_SIZE; ++i)
        p[i]=m_prologue[i]; // << prologue is a member array. 
    ...
}

Let's call the Real Prologue : Original Prologue;
The Changed Prologue : Changed Prologue;
The One that is placed at Run-Time : Replacement Prologue;

I get an Exception (signal 11 - Segmentation Fault).
!! I realized it is VxWorks's .text Segment Protection.

So, I created a SimPC based VIP to be my simulator BSP, and excluded INCLUDE_PROTECT_TEXT (and all it's relevant kernel components)
and run the simulator:
Now, there is no exception!

Facts

1. Looking at Memory Browser I see the Changed Prologue Bytes (memory didn't change)!
2. Printing the buffer to console, prints the Replacement Prologue Bytes values! (Weird)
3. looking at assembly view (Mega Weird): shows the Changed Prologue Hex values but the Original Prologue asm commands (push bp;...) even though the byte value does not match them.

My Questions

• Anyone had any experience with modifying .text segment?
• Anyone encountered memory that would not change (without an exception/signal) on simulator, which is not a memory mapped port/volatile ?

Long Shot Assumption

I have an assumption it is about caching, hinting that vxWorks know this region shouldn't change, so it doesn't write_through, but don't know how i can check it...

EDIT 2: tried setting my pointers to be volatile => same behavior!

Please Help.

Answer 1

Forgot about the question: but still have an answer.

1. there is an issue with the Host_Tools which does not show the changes to .text section.
   while on the target, the bytes actually changed.

2. the function didn't work because my transformation was ruining dynamic linking.

   • my function code, had a call to function with a constant string "Whatever"
   • when i transformed the function code, i unintentionally, changed the reference of a relocation pointer which at loading time got a bad absolute PTR.
     Lucky me, it pointed to a 0x00 buffer, and therefore printed an empty string without crashing.

Suggested Solutions:

1. Do not touch the relocated Pointers both when altering the Executable and altering at Run-time.
2. Create a static self-contained executable with absolute footprint => no dynamic relocation occurs that way.
3. alter dl() to transform the altered reloacted pointers back to their expected relocated.
4. alter dl() to infer from the altered relocated pointers the expected altered absolute pointer, so the transformation will create absolute pointer.

Note: I Choose #2 because it is the simplest, and because in my system, I do not need shared objects anyway.