Can't find how to use ptrace() properly

Question

Currently, for a project, I need to code some kind of debugger using ptrace(). At the end, it should show every function/syscall entered/exited in the program to trace.

Right now, I'm pretty stuck. I made a small program that should try to trace a given program, and print if it finds a call or a syscall based on the opcode (retreived with the registers). Here it is:

#include <sys/ptrace.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <sys/reg.h>
#include <sys/syscall.h>
#include <sys/user.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main()
{
        pid_t child;
        const int long_size = sizeof(long);

        child = fork();

        if(child == 0) {
                ptrace(PTRACE_TRACEME, 0, NULL, NULL);
                execl("./bin", "bin", NULL);
        } else {
                int status;
                unsigned ins;
                struct user_regs_struct regs;
                unsigned char prim, sec;

                while (1) {
                        wait(&status);
                        if (WIFEXITED(status))
                                break;
                        ptrace(PTRACE_GETREGS, child, NULL, &regs);
                        ins = ptrace(PTRACE_PEEKTEXT, child, regs.rip, NULL);
                        prim = (unsigned)0xFF & ins;
                        sec = ((unsigned)0xFF00 & ins) >> 8;
                        if (prim == 0xE8 && sec == 0xCD)
                                printf("call found!\n");
                        if (prim == 0x80 && sec == 0xCD)
                                printf("syscall found!\n");
                        ptrace(PTRACE_SINGLESTEP, child, NULL, NULL);
                }
        }
        return 0;
}

And here's the code for the "bin" binary :

#include <unistd.h>

void toto()
{
        write(1, "hello\n", 6);
}

int main()
{
        toto();
        toto();
        return (1);
}

When I'm looking the output of my mini debugger, it seems to find only one syscall and one call... I tried messing with the registers and the offset, but every tutorial I found on internet seems to be for a 32bits machine, which won't work in my case :/

Can someone give me a small hint to help me continue?

Thanks and have a nice day !

Answer 1

You were almost there but your masking (as first suspected) wasn't catching the callq opcode. There is also going to be a whole lot of extra callq codes caught by using PTRACE_SINGLESTEP too, I'm not sure if you realize this or not.

I compiled your bin program statically so you can get a consistent address for main and toto

gcc bin.c -o bin -g -Wall -static on a 64-bit machine.

And then in the main-script, I changed the masking opertion on ins variable:

#include <sys/ptrace.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <sys/reg.h>
#include <sys/syscall.h>
#include <sys/user.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main()
{
        pid_t child;

        child = fork();

        if(child == 0) {
                ptrace(PTRACE_TRACEME, 0, NULL, NULL);
                execl("./bin", "bin", NULL);
        } else {
                int status;
                unsigned ins;
                struct user_regs_struct regs;
                unsigned char prim;

                while (1) {
                        ptrace(PTRACE_SINGLESTEP, child, NULL, NULL);
                        wait(&status);
                        if (WIFEXITED(status))
                                break;
                        ptrace(PTRACE_GETREGS, child, NULL, &regs);
                        ins = ptrace(PTRACE_PEEKTEXT, child, regs.rip, NULL);
                        prim = (unsigned)0xFF & ins;
                        // Here in prim just mask for the first byte
                        if (prim == 0xe8) {
                        // Print the addresses to check out too
                                printf("RIP: %#x --> %#x\n", regs.rip, ins);
                                printf("call found!\n");
                        }
                }
        }
        return 0;
}

You only need to mask to check the first byte to see if it matches the call opcode. I added some extra print statements with the instruction pointer address so you can check the static source code just to make sure you are catching the right calls.

You can redirect your output from the main-program (I called it stepper):

./stepper > calls.txt

If you then do objdump -S bin > dump.txt you can see the addresses from toto and main where the callq instruction is made will also be in calls.txt file

You end up with all the extra calls being made from the crt-functions, the linker, and library calls.