Removing function call from application using a debugger

Question

I have an application which calls a function for every iteration of the main program loop. I want to remove this function call using a debugger like OllyDBG.

I think I approximately located where the function should be based on some strings that are in it.

How would I go about doing this without breaking the program?

Answer 1

It really depends on how the program uses the function. And you'll need some basic assembly language to successfully do that, as well as perhaps understanding the different calling conventions.

I'll give a few examples, with increasing difficulty.

A simple example

Lets assume the function you want to avoid has the following prototype:

void bad_function();

We can clearly see it has no arguments and does not return any value. Lets also say it does not change any global variables or makes any other change to the program. In that case, we can safely just "remove" the call to the function and everything will work great.

However, we cannot just remove machine code bytes from the stream, as there are relative and hardcoded offsets we will then need to adjust by the number of bytes removed. Luckily (though, luck has nothing to do with it) there are specific instructions called nop instructions that are just empty placeholders - instructing the CPU to execute nothing but move on to the next instruction.

A side note: although most instruction sets define a nop instruction, they are usually just conventions of instructions that are already exist but have no side-effects. For example, the common x86 nop instruction is 0x90, which is technically xchg eax, eax. Obviously exchanging a register with itself does nothing.

A simple return value

Now, lets say the function returns a boolean determining whether execution should continue or terminate abruptly (say, because something the program considers bad happened).

The function's prototype may then look like:

bool bad_function();

and the code using it will be:

bool bad;
bad = bad_function();
if (bad)
{
    ExitProcess();
}

The resulting assembly may be similar to:

    call bad_function
    test eax, eax
    jz not_bad_function
    call ExitProcess

not_bad_function:
    ... some more assembly

You'll notice here we'll want to make sure the code under not_bad_function (which is a label I named to ease reading. In reality it'll have an offset) execute, as well as avoid the call to bad_function.

We'll wanna replace the unneeded instructions with nops. Sometimes instructions may be larger than one byte and we'll replace a single instruction with a few nop instructions and not just one. There are also multiple-byte long nop instructions, but we won't cover that here.

In the following case, we'll want to nop-out all four instructions up until the not_bad_function (those are the two calls, the test and jz instructions). We could also just make the jump unconditional, and leave the test and calls intact. That may be handy if we do want the bad_function to execute because it does make certain side-effects we're interesed in.

"cleaning" the stack

Now, lets assume bad_function accepts parameters. Depending on calling conventions, the set of arguments passed and the compiler used and target architecture we can end up with several push instructions before bad_function is called. Those are used to push parameters to the stack, for the called function to use. Again depending on calling conventions it may be the caller's responsibility to clean up the stack afterwards. In that case, we'll need to remove both the push instructions and the cleanup after the function is called.

As the code is becoming longer, I'll leave creating an example as an exercise for the reader.