Tokenizing a string inside a Switch case

Question

I am tokenizing a string based on a delimiter (using Boost Tokenizer) and then do stuff with the tokens based on the value of a state variable.

The problem I'm having: One of the cases requires the string to be tokenized further. This causes errors because the tok1 and token iterator variables are declared in the case. I've tried declaring them outside the switch and just assigning them inside the case, but this doesn't work.

Does anyone know of a way how I can make this work or if there a better way to go about further tokenizing the string inside the case? Thanks!

Example code below:

 boost::char_separator<char> sep(TOKEN_DELIMETER_NEWLINE);                  
 boost::char_separator<char> sep2(TOKEN_DELIMETER_SPACE);                  
 tokenizer tok(_text, sep);

while(lineToken!=tok.end())
{
    switch(state)
    {
        case FIRST_STATE:

            lineToken++;
            tokenizer tok1(*lineToken, sep2);
            tokenizer::iterator token=tok1.begin();  

        break;
    // Other Cases follow...
    }

}

Answer 1

After filling in the gaps I compiled the following:

std::string _text1 = "The rain,In Spain,Lies Mainly,On the plain";

boost::char_separator<char> sep(",");                  
boost::char_separator<char> sep2(" ");                  
boost::tokenizer<boost::char_separator<char>> tok(_text1,sep);
boost::tokenizer<boost::char_separator<char>>::iterator lineToken = tok.begin();
unsigned int state = 0;

while(lineToken!=tok.end())
{
    switch(state)
    {
    case 0:
     lineToken++;
     boost::tokenizer<boost::char_separator<char>> tok1(*lineToken, sep2);
     boost::tokenizer<boost::char_separator<char>>::iterator token=tok1.begin();  

    break;
    // Other Cases follow...
}
}

Which works for me - well it compiles and tokenizes.... Note that my example is not like yours in that the increment of the iterator will cause a crash before the end because I don't do any checking.....

Maybe you didn't use the template or missed it out?