Copying content of strtok token in C

Need to separate a string and then do another separation.

char *token = strtok(str, ",");
while(token){
    char *current_string = malloc(sizeof(char) * strlen(token));
    strcpy(current_string, token);

    char *tk = strtok(current_string, ":"); // KEY
    printf("key: %s ", tk);
    tk = strtok(0, ":");                    // VALUE
    printf("value: %s\r\n", tk);
    printf("%s\n", token);
    token = strtok(0, ",");
}

printf("Done\n");

Trying to copy the content of token, but doing so messes with what remains in the token variable. It only processes one line instead of the three it should. I suspect the issue is with the strcpy(current_string, token) but unsure how I should go about it.

Solution

The strtok function uses an internal static buffer to keep track of where it left off. This means you can't use it to go back and forth parsing two different strings.

In your specific case, on this call:

token = strtok(0, ",");

The internal buffer is still pointing to a location inside of current_string, so attempting to go back to token won't work.

What you need to strtok_r. This version takes an additional parameter to keep track of the current state. That way, you can interchangeably parse two or more string by using a different state pointer for each one:

char *state1, *state2;
char *token = strtok_r(str, ",", &state1);
while(token){
    char *current_string = strdup(token);

    char *tk = strtok_r(current_string, ":", &state2); // KEY
    printf("key: %s ", tk);
    tk = strtok_r(NULL, ":", &state2);                    // VALUE
    printf("value: %s\r\n", tk);
    printf("%s\n", token);
    free(current_string);
    token = strtok_r(NULL, ",", &state1);
}

printf("Done\n");

Note that NULL was passed to the later strtok_r calls instead of 0, since NULL may not necessarily be 0. Also, the calls to malloc/strcpy were replaced with a call to strdup which does the same, and also a call to free was added to prevent a memory leak.

The strtok_r is available on UNIX/Linux system. On Windows, use strtok_s which works the same way.