strtok() giving unexpected results

Question

this is a question that has puzzled me for a little bit would love to get some input. I am inputing an ascii file with many complex numbers which I am tokenizing using strtok with the delimiters "+" and " ". the file is the following format: [[[0+0i 0+0i 0+0i 0+0i.......]]]. I expect the first token to be 0 and the second to be 0i. I am actually getting "0,0i" as my first token. the following the code being used:

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

char data[]={0};
int line = 0;
char *datap[231000];
int i,n;

void main()
{
    FILE * filei = fopen("signal_subband_16.ascii","r");
    FILE * fileo = fopen("Output_file_simple.txt","a");
    if(filei==NULL){
        printf("There was an error opening the input file");
        exit(1);
    }
    else if(fileo==NULL){
        printf("There was an error opening the output file");
        exit(1);
    }
    else{
        while(fgets(data,999999,filei)){
            line ++;
            //      printf("Line: %d -> %s", line, data);
        }
    }

    datap[0]=strtok(data,"[+");
    n = 1;
    while(datap[n-1]!=NULL){
        datap[n] = strtok(NULL,"i+");
        n++;
    }

    //for(i=0;i<n;i+2){
    //  printf("%s %s\n", datap[i], datap[i+1]);
    //}

    printf("%s\n, %s\n",datap[0], datap[1]);

    fclose(filei);
    fclose(fileo);
}

Any help or push to the right answer would be greatly appreciated. Thanks

NEW CODE FUNCTIONAL

#include <stdlib.h>
#include <stdio.h>
#include <string.h>




char data[999999];
int line = 0;
char *datap[231000];
int i,n;


void main()
{
FILE * filei = fopen("signal_subband_16.ascii","r");
FILE * fileo = fopen("Output_file_simple.txt","a");
if(filei==NULL){
    printf("There was an error opening the input file");
    exit(1);
}
else if(fileo==NULL){
    printf("There was an error opening the output file");
    exit(1);
}
else{
    while(fgets(data,999999,filei)){
        datap[0]=strtok(data,"[ +");
        n = 1;
        while(datap[n-1]!=NULL){
            datap[n] = strtok(NULL,"i +]");
            n++;
            }

    for(i=0;i<n;i+2){
    printf("%s %s\n", datap[i], datap[i+1]);
}
    memset(data,0,999999);
    line ++;

    }

}






fclose(filei);
fclose(fileo);

}

Answer 1

The biggest problem with your code is that you're writing outside the bounds of the data array. When you declare an array with empty size, it makes it just big enough to hold the values in the initializer list. So

char data[] = {0};

declares an array with just 1 element, equivalent to

char data[1] = {0};

You need to make this big enough to hold the lines you're reading with fgets(), so it should be:

char data[999999];

There's no need to initialize it, since you don't use it until after you fill it in with fgets().

The problem with your use of strtok() is that you didn't include space in the delimiter list. Also, if you want the i character to be part of the token, it should not be in the delimiter list. And you need to include ] in the delimiters so you don't include ]]] in the last token. So the loop should be:

datap[0]=strtok(data,"[+");
n = 1;
while(datap[n-1]!=NULL){
    datap[n] = strtok(NULL," +]");
    n++;
}

Note also that at the point where this loop appears, data contains only the last line of the file. You're not parsing any other lines in the file. It's not clear whether this is intended or not. If you want to parse all lines of the file, this loop should be inside the loop that reads each line. And you'll need to make a copy of data for each line, because you're overwriting it each time you read a line, and strtok() simply returns pointers into the array.