Trying to tokenize a string

Question

Possible Duplicate:
How do I tokenize a string in C++?

I have this text file

q1 t q2

q2 e q3

q3 x q4

q4 t q5 q6 q11

q5 | q6 q11 

I want to extract each element that is separated by a space. For example, in line one I want to be able to extract "q1" "t" and "q2" as separate tokens.

I'm thinking there's two ways to go about this

1. read from the file, token by token, using ifstream>>. The problem I'm having with this approach is that I don't know how to tell when the end of a line has been reached so that I can move onto the next line.

2. The other approach, is to get the whole line at once with getline(); The problem with this approach is that I have to tokenize the string myself, and each line is different so I'm not sure that's the best idea. I'm pretty blown away there's no built in way to do this, besides strtok() which looks to be not at all what I want. Thanks guys any help is appreciated.

Answer 1

Use getline, and to tokenize the resulting string, put it into a std::stringstream and extract the tokens from that.

std::string line_string;
while ( getline( file, line_string ) ) {
    std::istringstream line( line_string );
    std::string token;
    while ( line >> token ) {
        do something with token
    }
}