I am looking to log all accesses to addresses in a block allocated in heap, pointed by a given pointer. In the following example, it would be monitoring all accesses to 1024 bytes from p.
#define BLK 1024
#include <stdlib.h>
int monfun(int *p){
int t;
for(int i=0; i<BLK; i++){
t=p[i];
}
return t;
}
int main(void){
int *p=(int*)malloc(BLK);
int *p = a;
monfun(p);
return 0;
}
And obtain something like:
w 4b 0x12345
r 4b 0x12346
r 8b 0x12348
...
Performance is not a problem. This is likely slow down execution by a good deal and requires a lot of extra memory. Instead, I am looking for precise measurements.
I have been exploring valgrind, its massif tool gives me a tree with allocation/free pairs. I keep reading online that valgrind can do what I need, but I can only get access counters rather than the actual sequence of touched addresses.
I have read other tools, such as radare2, qemu-mtrace, a tracegrind plugin for valgrind (for v3.15. I have v3.21), intel's pin, but at this point I am not sure which tool/manual to invest my learning on in order to get this done.
If this can be done with valgrind, could you please guide me on how to achieve it?
Using a debugger could be a solution, but definitely a clunky one. What you really need is either some kind of compile-time instrumentation (not sure how tho, maybe a LLVM plugin, pretty hardcore solution) or emulation (QEMU, Intel PIN, Dynamorio, etc).
You can achieve this by using the tracing events memory_region_ops_{read,write} or by implementing your own. See QEMU doc about tracing for more info. You probably need a recent version of QEMU for this, for example, the QEMU user on my Debian 11 system does not support these events. Compiling QEMU from source is pretty easy anyway, you could try with the latest. The downside of this approach is that it could be quite slow and you will need to filter out the output since this will log all reads and writes.
Assuming you are using an x86 CPU, with PIN you can do this and much more, after you get familiar enough with the tool. Both the documentation and the PIN kit obtainable from the download page have examples that do almost exactly what you want. Downloading the latest version of the PIN kit and looking at these files:
source/tools/ManualExamples/pinatrace.cpp for memory R/W tracingsource/tools/ManualExamples/malloctrace.cpp for tracing malloc() callsIt should be pretty easy to combine the two into a single tracing tool. The only functionality to add is dumping values from memory, which can be done with PIN_SafeCopy().
I wrote a tracing tool combining code from the two examples above that should do what you want. It should contain enough comments to understand what is going on, and the examples contain even more comments, so you can read those if something is unclear.
It expects a call to a dummy function trace_next_allocation() to enable tracing of the chunk allocated by the next malloc() call. The chunk is then automatically un-tracked when freed through free().
Here's the code (see below for a usage example):
/**
* tracechunk.cpp
*
* Copyright (C) 2004-2021 Intel Corporation.
* Copyright (C) 2023 Marco Bonelli.
* SPDX-License-Identifier: MIT
*/
#include "pin.H"
#include <iostream>
#include <fstream>
struct tracked_malloc_chunk {
ADDRINT start;
ADDRINT end;
ADDRINT size;
};
struct last_write_info {
ADDRINT addr;
UINT32 size;
};
bool tracking = false;
bool track_next = false;
struct tracked_malloc_chunk tracked_chunk;
struct last_write_info last_write;
static PIN_LOCK pin_lock;
std::ofstream trace_file;
KNOB<std::string> trace_output_fname(KNOB_MODE_WRITEONCE, "pintool", "o", "tracechunk.out", "specify trace file name");
/**
* Dump `size` bytes of tracee memory starting at `addr`.
*/
void hexdump(ADDRINT addr, UINT32 size) {
char old_fill = trace_file.fill();
static UINT8 data[512];
size_t actual;
actual = PIN_SafeCopy(data, (void *)addr, size);
trace_file << std::noshowbase << std::setfill('0');
for (UINT32 i = 0; i < actual; i++)
trace_file << std::setw(2) << (unsigned)data[i];
if (actual != (size_t)size)
trace_file << " (err: could only read " << actual << " bytes)";
trace_file << std::endl << std::showbase << std::setfill(old_fill);
}
/**
* Executed *before* a malloc() call: save the allocation size for later.
*/
VOID malloc_before(ADDRINT size) {
if (!track_next)
return;
tracked_chunk.size = size;
}
/**
* Executed *after* a malloc() call: save the chunk address and start tracing.
*/
VOID malloc_after(ADDRINT retval) {
if (retval == 0) {
trace_file << "ERROR: malloc() call to track failed!" << std::endl;
return;
}
if (!track_next || !tracked_chunk.size)
return;
tracked_chunk.start = retval;
tracked_chunk.end = retval + tracked_chunk.size;
trace_file << "START tracking memory R/W for chunk ["
<< tracked_chunk.start << ","
<< tracked_chunk.end << ") of size "
<< tracked_chunk.size << std::endl;
tracking = true;
track_next = false;
}
/**
* Executed *before* a free() call: stop tracing if the chunk we are currently
* tracking is freed.
*/
VOID free_before(ADDRINT addr) {
if (!tracking || addr != tracked_chunk.start)
return;
trace_file << "STOP tracking memory R/W for chunk ["
<< tracked_chunk.start << ","
<< tracked_chunk.end << ") of size "
<< tracked_chunk.size << std::endl;
tracking = false;
tracked_chunk = {0};
}
/**
* Enable tracking of R/W operations for the chunk returned by the next malloc()
* invocation.
*/
VOID enable_track_next(void) {
track_next = true;
}
/**
* Executed *before* a read operation: dump instruction pointer, address, size
* and memory content.
*/
VOID trace_read(ADDRINT ip, ADDRINT addr, UINT32 size) {
ADDRINT offset;
if (!tracked_chunk.size || !size)
return;
offset = addr - tracked_chunk.start;
if (offset < 0 || offset >= tracked_chunk.size)
return;
trace_file << ip << ": READ of size " << size << " at " << addr
<< " (offset " << offset << "): ";
PIN_GetLock(&pin_lock, ip);
hexdump(addr, size);
PIN_ReleaseLock(&pin_lock);
}
/**
* Executed *before* a write operation: dump instruction pointer, address and
* size, then save write address and size for later.
*/
VOID trace_write_before(ADDRINT ip, ADDRINT addr, UINT32 size) {
ADDRINT offset = addr - tracked_chunk.start;
if (!tracked_chunk.size || !size || offset < 0 || offset >= tracked_chunk.size)
return;
last_write.addr = addr;
last_write.size = size;
trace_file << ip << ": WRITE of size " << size << " at " << addr
<< " (offset " << offset << "): ";
}
/**
* Executed *after* a write operation: dump memory content (in big endian order)
* from previously saved address and size.
*/
VOID trace_write_after(ADDRINT ip) {
if (!last_write.size)
return;
PIN_GetLock(&pin_lock, ip);
hexdump(last_write.addr, last_write.size);
PIN_ReleaseLock(&pin_lock);
last_write.size = 0;
}
VOID Image(IMG img, VOID* v) {
// Instrument malloc() to save the allocation size and the chunk address
// when we want to trace the next allocation
RTN mallocRtn = RTN_FindByName(img, "malloc");
if (RTN_Valid(mallocRtn)) {
RTN_Open(mallocRtn);
RTN_InsertCall(mallocRtn, IPOINT_BEFORE, (AFUNPTR)malloc_before,
IARG_FUNCARG_ENTRYPOINT_VALUE, 0, IARG_END);
RTN_InsertCall(mallocRtn, IPOINT_AFTER, (AFUNPTR)malloc_after,
IARG_FUNCRET_EXITPOINT_VALUE, IARG_END);
RTN_Close(mallocRtn);
}
// Instrument free() to stop tracing the chunk
RTN freeRtn = RTN_FindByName(img, "free");
if (RTN_Valid(freeRtn)) {
RTN_Open(freeRtn);
RTN_InsertCall(freeRtn, IPOINT_BEFORE, (AFUNPTR)free_before,
IARG_FUNCARG_ENTRYPOINT_VALUE, 0, IARG_END);
RTN_Close(freeRtn);
}
// Instrument the dummy trace_next_allocation() function to enable tracing
// the next malloc() allocation
RTN triggerRtn = RTN_FindByName(img, "trace_next_allocation");
if (RTN_Valid(triggerRtn)) {
RTN_Open(triggerRtn);
RTN_InsertCall(triggerRtn, IPOINT_BEFORE, (AFUNPTR)enable_track_next,
IARG_END);
RTN_Close(triggerRtn);
}
}
VOID Instruction(INS ins, VOID* v) {
UINT32 n = INS_MemoryOperandCount(ins);
for (UINT32 i = 0; i < n; i++) {
// Instrument read operations to dump address, size and memory content
if (INS_MemoryOperandIsRead(ins, i)) {
INS_InsertPredicatedCall(ins, IPOINT_BEFORE, (AFUNPTR)trace_read,
IARG_INST_PTR, IARG_MEMORYOP_EA, i, IARG_MEMORYOP_SIZE, i,
IARG_END);
}
// Instrument write operations to dump address, size and memory content.
// This needs to be done in two steps as we can't get both the effective
// address and the memory content at the same time (the effective
// address and the size are not provided at IPOINT_AFTER).
if (INS_MemoryOperandIsWritten(ins, i)) {
INS_InsertPredicatedCall(ins, IPOINT_BEFORE,
(AFUNPTR)trace_write_before, IARG_INST_PTR, IARG_MEMORYOP_EA, i,
IARG_MEMORYOP_SIZE, i, IARG_END);
if (INS_IsValidForIpointAfter(ins)) {
INS_InsertPredicatedCall(ins, IPOINT_AFTER,
(AFUNPTR)trace_write_after, IARG_END);
}
}
}
}
VOID Fini(INT32 code, VOID* v) {
trace_file.close();
}
INT32 Usage() {
std::cerr << "This tool produces a trace of memory read/write operations"
<< " on specific malloc() chunks" << std::endl;
std::cerr << std::endl << KNOB_BASE::StringKnobSummary() << std::endl;
return 1;
}
int main(int argc, char **argv) {
PIN_InitSymbols();
if (PIN_Init(argc, argv))
return Usage();
// Write to a file since stdout and stderr may be closed by the application
trace_file.open(trace_output_fname.Value().c_str());
trace_file << std::hex << std::showbase;
IMG_AddInstrumentFunction(Image, 0);
INS_AddInstrumentFunction(Instruction, 0);
PIN_AddFiniFunction(Fini, 0);
PIN_StartProgram();
return 0;
}
Here's an example program to trace:
// example.c
#include <stdlib.h>
#define N 4
// Do not optimize away, we need this function to be called to enable tracing
void __attribute__((optimize("O0"))) trace_next_allocation(void) {}
int main(void){
volatile int *chunk;
trace_next_allocation();
chunk = malloc(N * sizeof(int));
for(unsigned i = 0; i < N; i++) {
chunk[i] += 123;
}
free(chunk);
return 0;
}
The tool can then be used like this (where /path/to/pin-xxx is the path to the
extracted PIN kit downloaded from here):
cd /path/to/pin-xxx/source/tools/ManualExamples
# Write the tool code in a file named tracechunk.cpp inside this directory...
# Compile the tool
make obj-intel64/tracechunk.so
# Compile example program
gcc -o example example.c
# Trace with PIN
../../../pin -t obj-intel64/tracechunk.so -- ./example
# Show trace output
cat tracechunk.out
I tested it with PIN 3.28 and the output looks like this:
START tracking memory R/W for chunk [0x55ec376bb2a0,0x55ec376bb2b0) of size 0x10
0x55ec3591d07a: READ of size 0x4 at 0x55ec376bb2a0 (offset 0): 00000000
0x55ec3591d083: WRITE of size 0x4 at 0x55ec376bb2a0 (offset 0): 7b000000
0x55ec3591d07a: READ of size 0x4 at 0x55ec376bb2a4 (offset 0x4): 00000000
0x55ec3591d083: WRITE of size 0x4 at 0x55ec376bb2a4 (offset 0x4): 7b000000
0x55ec3591d07a: READ of size 0x4 at 0x55ec376bb2a8 (offset 0x8): 00000000
0x55ec3591d083: WRITE of size 0x4 at 0x55ec376bb2a8 (offset 0x8): 7b000000
0x55ec3591d07a: READ of size 0x4 at 0x55ec376bb2ac (offset 0xc): 00000000
0x55ec3591d083: WRITE of size 0x4 at 0x55ec376bb2ac (offset 0xc): 7b000000
STOP tracking memory R/W for chunk [0x55ec376bb2a0,0x55ec376bb2b0) of size 0x10