I'm trying to implement few algorithms for measuring similarity between two sentences. I think it would be helpful if I had some kind of expanded representation of a sentence.
We could get this expanded representation with google/bing search, but putting whole sentence as a keyword doesn't give good results, especially if sentence is long. So I was thinking that finding keywords in a sentence and using them as search terms would improve our results.
Anybody aware of an algorithm for finding keywords in a sentence?
Thank you!
Lucene can handle that extraction as well as the comparison.