java.lang.IllegalStateException: should be impossible for sortable string fields to be present in embedded documents

Question

I have created an index in Atlas Search like that:

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "_id": {
        "type": "objectId"
      },
      ...
      "properties": {
        "type": "embeddedDocuments",
        "dynamic": false,
        "fields": {
          ...
          "stringValueAsToken": {
            "type": "token"
          },
          ...
        }
      },
      ...
    }
  }
}

but i get this error on the mongot server -

java.lang.IllegalStateException: should be impossible for sortable string fields to be present in embedded documents
         at com.xgen.mongot.util.Check.stateAssertion(Check.java:168)
         at com.xgen.mongot.index.lucene.document.single.LuceneSearchIndexFieldValueHandler.handleSortableString(LuceneSearchIndexFieldValueHandler.java:493)
         at com.xgen.mongot.index.lucene.document.single.LuceneSearchIndexFieldValueHandler.handleString(LuceneSearchIndexFieldValueHandler.java:425)
         at com.xgen.mongot.index.lucene.document.block.EmbeddedFieldValueHandler.handleString(EmbeddedFieldValueHandler.java:138)
         at com.xgen.mongot.index.ingestion.BsonDocumentProcessor.handleField(BsonDocumentProcessor.java:152)
         at com.xgen.mongot.index.ingestion.BsonDocumentProcessor.handleDocumentField(BsonDocumentProcessor.java:52)
         at com.xgen.mongot.index.ingestion.BsonDocumentProcessor.handleField(BsonDocumentProcessor.java:93)
         at com.xgen.mongot.index.ingestion.BsonDocumentProcessor.handleField(BsonDocumentProcessor.java:76)
         at com.xgen.mongot.index.ingestion.BsonDocumentProcessor.handleDocumentField(BsonDocumentProcessor.java:52)
         at com.xgen.mongot.index.ingestion.BsonDocumentProcessor.process(BsonDocumentProcessor.java:38)
         at com.xgen.mongot.index.lucene.SingleLuceneIndexWriter.handleInsertOrUpdate(SingleLuceneIndexWriter.java:407)
         at com.xgen.mongot.index.lucene.SingleLuceneIndexWriter.updateIndex(SingleLuceneIndexWriter.java:242)
         at com.xgen.mongot.index.lucene.SingleLuceneIndexWriter.updateIndex(SingleLuceneIndexWriter.java:227)
         at com.xgen.mongot.index.MeteredIndexWriter.updateIndex(MeteredIndexWriter.java:20)
         at com.xgen.mongot.replication.mongodb.common.DefaultDocumentIndexer.indexDocumentEvent(DefaultDocumentIndexer.java:49)
         at com.xgen.mongot.replication.mongodb.common.IndexingWorkScheduler$IndexingTask.run(IndexingWorkScheduler.java:239)
         at com.xgen.mongot.util.FutureUtils.lambda$checkedRunAsync$2(FutureUtils.java:108)
         at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
         at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:49)
         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base/java.lang.Thread.run(Thread.java:829)

The issue is with the type token. why is that exception is thrown and how can i create the stringValueAsToken field to be a token?

Tried to create a field in the index that not exists on the mongoDB document and it was succeeded, but once i entered data to that field the searched index failed.

Answer 1

From the - documentation

You can't index children of fields indexed as the embeddedDocuments type as the token type.

A kind of work around, you can use keyword analyzer

"stringValueAsToken": {
    "type": "string",
    "analyzer": "lucene.keyword"
}

Keyword analyzer make the string indexed as a single token even if it is contains spaces.

By the way, if you wanted to make the field as token in order to use the equals operator, you can achieve the same using string and lucene.keyword with the text operator.

In order to keep the full text search capabilities of the field, you can use text operator with path of multi. Define the field in the index like that:

"stringValue": {
  "type": "string",
  "multi": {
    "keywordAnalyzer": {
      "type": "string",
      "analyzer": "lucene.keyword"
    }
  }
}

and do the equality in the query like that:

"text": {
  "query": "item1",
  "path": {
    "value": "properties.stringValue",
    "multi": "keywordAnalyzer"
  }
}