invalid pointer when using strtok_r

Question

When running my code (shown in the first code block), I get this error: *** Error in `./a.out': free(): invalid pointer: 0x0000000001e4c016 *** I found a fix (which is shown in the second code block), but I don't understand why the error is happening in the first place.

I read the documentation regarding strtok_r, but I don't understand why assigning "str" to a new char* fixes the problem.

Doesn't "rest = str" mean that rest and str point to the same block of memory. How does this fix the problem???

Broken code:

#include <stdio.h> 
#include <string.h> 
#include <stdlib.h>

int main() 
{ 
    char* str = (char*) malloc(sizeof(char) * 128);
    char* token;
    
    printf("Enter something: ");  
    fgets(str, 128, stdin);
  
    while ((token = strtok_r(str, " ", &str))) { 
        printf("%s\n", token); 
    }
    
    free(str);
    return (0); 
}

Fixed code:

#include <stdio.h> 
#include <string.h> 
#include <stdlib.h>

int main() 
{ 
    char* str = (char*) malloc(sizeof(char) * 128);
    char* token; 
    char* rest = str; 
    
    printf("Enter something: ");  
    fgets(str, 128, stdin);
  
    while ((token = strtok_r(rest, " ", &rest))) { 
        printf("%s\n", token); 
    }
    
    free(str);
    return (0); 
}

Answer 1

It looks evidently that a call of strtok_r changes the pointer str that is passed to the call by reference as the third parameter.

while ((token = strtok_r(str, " ", &str))) { 
                                   ^^^^
    printf("%s\n", token); 
}

So after a call of the function the pointer str can point inside the original string. So it will not store the value that it had after a call of malloc.

Thus using the auxiliary variable rest allows to keep the initial value in the pointer str.

Pay attention to that you are calling the function incorrectly. Here is its description

On the first call to strtok_r(), str should point to the string to be parsed, and the value of saveptr is ignored. In subsequent calls, str should be NULL, and saveptr should be unchanged since the previous call.

So for the second and subsequent calls of the function the first argument shall be NULL.