Search code examples
javatokenize

Tokenize a string with a space in java


I want to tokenize a string like this

String line = "a=b c='123 456' d=777 e='uij yyy'";

I cannot split based like this

String [] words = line.split(" ");

Any idea how can I split so that I get tokens like

a=b
c='123 456'
d=777
e='uij yyy';  

Solution

  • The simplest way to do this is by hand implementing a simple finite state machine. In other words, process the string a character at a time:

    • When you hit a space, break off a token;
    • When you hit a quote keep getting characters until you hit another quote.