Search code examples
nlpazure-cognitive-servicesazure-language-understanding

Searching for known phrases in text using Azure Cognitive Services


I'm trying to ascertain the "right tool for the job" here, and I believe Cognitive Services can do this but without disappearing down an R&D rabbit-hole I thought I'd make sure I was tunnelling in the right direction first.

So, here is the brief:

I have a collection of known existing phrases which I want to look for, but these might be written in slightly different ways, be that grammar or language.

I want to be able to parse a (potentially large) volume of text to scan and look for those phrases so that I can identify them.

For example, my phrase could be "the event will be in person" but that also needs to identify different uses of language; for example "in-person event", "face to face event", or "on-site event" - as well as the various synonyms and variations you can get with such things.

LUIS initially appeared to be the go-to tool for this kind of thing, and includes the ability to write your own Features (aka Phrase Lists) to augment the model, but it isn't clear whether that would hit the brief - LUIS appears to be much more about "intent" and user interaction (for example building a chat Bot, or understanding intent from emails).

Text Analytics also seems a likely candidate, but again seems more focused about identifying "entities" (such as people / places / organisations) rather than a natural language "phrase" - would this tool work if I was defining my own "Topics" or is that really just barking up the wrong tree?

.. or ... is there actually something else I should be looking at completely different?

At this point - I'm really looking for a "which tool should I spend lots of time learning about".

Thanks all in advance - I appreciate this is a fairly open-ended requirement.


Solution

  • It seems your scenario aligns more with our text analytics service. I was going to recommend Key Phrase Extraction API which evaluates unstructured text and returns a list of key phrases. However, since you require to use known (custom) phrase list, it may not be the solution you're looking for. We currently don't support custom key phrase extraction today, however it's on our roadmap. If interested, we can connect you with the product team to learn more about your scenario.

    Updated: Please try custom NER capability.