Unexpected variable values reading from file (ESP32)

Question

I am still learning Cpp, so please advise if I am misunderstanding here.

Using an ESP32, I am trying to read / write files to Flash / FFat. This is the method I have created which should read a file from flash and load it into PSRAM:

unsigned char* storage_read(char* path) {

    File file = FFat.open(path);
    if(!file) {
        Serial.println("no file");
        return 0x00;
    }

    int count = file.size();
    unsigned char* buffer = (unsigned char*)ps_malloc(count);

    Serial.printf("Bytes: %d\n", count);
    Serial.printf("Count: %d\n", sizeof(buffer));

    for (int i = 0; i < count; i++) {
        buffer[i] = (unsigned char)file.read();
    }

    file.close();
    return buffer;
}

The problem is that I get the contents of my b64 data file, with the addition of several extra bytes of data globbed on the end.

Calling the method with:

Serial.printf("Got: %s", storage_read("/frame/testframe-000.b64"));

I get the output:

Bytes: 684
Count: 4
Got: <myb64string> + <68B of garbage>

Why would sizeof not be returning the proper size?

What would be the proper way of loading this string into a buffer?

Answer 1

Why would sizeof not be returning the proper size?

That's because sizeof() has a very specific function (not very intuitive). It is used - compile time - to query the size of the data type passed to it. Calling sizeof(buffer) returns the size, in bytes, of the type of variable buffer. It's an unsigned char*, so a 4-byte memory address. So that's what you get.

What would be the proper way of loading this string into a buffer?

What I noticed is that you're expecting to load string data from your file, but you don't explicitly terminate it with a zero byte. As you probably know, all C strings must be terminated with a zero byte. Data that you load from the file most likely doesn't have one (unless you took extra care to add it while saving). So when you read a string from a file sized N bytes, allocate a buffer of N+1 bytes, load the file into it and terminate it with a zero. Something like this:

unsigned char* storage_read(char* path) {
    File file = FFat.open(path);
    if(!file) {
        Serial.println("no file");
        return 0x00;
    }
    int count = file.size();
    unsigned char* buffer = (unsigned char*)ps_malloc(count + 1); //< Updated

    Serial.printf("Bytes: %d\n", count);
    Serial.printf("Count: %d\n", sizeof(buffer));

    for (int i = 0; i < count; i++) {
        buffer[i] = (unsigned char)file.read();
    }
    buffer[count] = 0; //< Added

    file.close();
    return buffer;
}

And since you're returning a heap-allocated buffer from your function, take extra care to remember to delete it in caller when finished. This line in your code will leak the memory:

Serial.printf("Got: %s", storage_read("/frame/testframe-000.b64"));