Search code examples
cstringtokenizec-stringsstrtok

strtok weird behavior on tokenization

The below code gives unexpected behaviour.

#include <stdio.h>
#include <string.h>

int main()
{
    
    char s[] = "hello $";
    char *t;
    t = strtok(s, "$$");
    printf("%s", t);

    return 0;
}

Why does this output hello and not hello $?

Solution

  • In a comment you wrote:

    so how should I split by a string?

    There is no standard way (no predefined library function) for doing this, that I am aware of.

    You could write your own split-on-string loop, using strstr. Here is one way:

    #include <stdio.h>
#include <string.h>

int main()
{
    char s[] = "hello$$this$$is$$a$$test";
    char *sep = "$$";
    char *p = s;
    char *p2;
    int i = 1;
    do {
        p2 = strstr(p, sep);
        if(p2 != NULL) *p2 = '\0';
        printf("%d: %s\n", i++, p);
        if(p2 != NULL) p = p2 + strlen(sep);
    } while(p2 != NULL);
}

    This could be improved, but it works, and it should get you started.