Search code examples
c++c++11tokenize

How to tokenize a string with percentages?


My purpose is to tokenize the following input string %08%04root%08%03KEY%08%05site2%08%11ksk-1483008950879%08%07ID-CERT in this way:

4root
3KEY
5site2
11ksk-1483008950879
7ID-CERT

Then I tried the following approach:

char_separator<char> sepSig("%08");    
boost::tokenizer< char_separator<char> > tokenSig(inputString, sepSig);   
BOOST_FOREACH (const string& t, tokenSig)
    log << "Token: " << t << std::endl; 

Unfortunately, all the items '8' are catched, with the following division:

Token: 4root //OK
Token: 3KEY  //OK
Token: 5site2 //OK
Token: 11ksk-14 
Token: 3
Token: 95
Token: 79
Token: 7ID-CERT //OK

Then I need a way to differentiate the item %08 from 8. I tried also the separators \08 and \x08, but in this way the token is the same of the whole string.


Solution

  • boost::char_separator treats any of the characters as separators. I can't see a string_separator - you could write your own, or just replace every occurance of "%08" with "\t", and then separate on \t.