Search code examples
solrsunspotsunspot-railssunspot-solr

View raw Solr tokens for a single field on a single document

I'm debugging my Solr schema and I'd like to see the results of tokenizing a specific field.

For a simplified example, if I have:

<fieldType name="text" class="solr.TextField" omitNorms="false">
    <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.StandardFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.PorterStemFilterFactory"/>
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front"/>
    </analyzer>
</fieldType>

and I indexed a field with the value "Hello, worlds!", I want to see something along the lines of:

hello world he el ll lo hel ell llo hell ello hello wo or rl ld wor orl rld worl orld

to ensure that everything is being tokenized as I envisage it is.

Is this in any way possible?

Solution

  • Yes, Admin > Analysis is exactly what you want.

    But there's another great tool that allows you to read index and see how exactly a field or document was indexed.

    It's called Luke and it's invaluable when troubleshooting and tweaking your schema.