I have been looking at the differences between Lucene 2.9 particular the redone tokenstream API and it just occurs to me its particularly ugly compared to the old just return a new or repopulate the given with values if your reusing said Token.
I have not done any profiling but it seems using a MAP to store attributes is not that efficient and it would be easier to just create a new value type holding values etc. The TokenStream and Attribute stuff looks like object pooling which is pretty much never necessary these days for simple value types like a Token of text.
creat()
When Ken Thompson and Dennis Ritchie received the 1983 Turing Award, after their respective acceptance speeches, someone in the audience asked Ken what he would do differently with Unix if he were to do it all over again. He said, "I'd spell 'creat' with an 'e'."