Search code examples
javascriptjquerysolrlukebigdata

Dynamically retrieve all fields present in Solr documents

Is it possible to dynamically retrieve all fields present in a set of Solr documents and still maintain reasonable performance? The end goal here is to dynamically populate a list of numeric fields for users to sort their current query upon.

In a perfect world, I'd like to be able to have this list contain all of the numeric fields present in the docs returned by the user's query.

If this isn't possible to achieve, though, I'm going to populate the list with numeric fields via the luke handler. Unfortunately it seems that the luke handler returns fields for the entire collection, but can't be restricted to only the current query.

I'm fairly new to Solr, so any help/discussion would be greatly appreciated!

Solution

  • This is how I solved my problem:

    1. Add new fields to schema: SORT_DOC_FIELDS and NO_SORT_DOC_FIELDS.
    2. Wrote some JavaScript using the ScriptUpdateProcessor that determined whether a field could be sorted upon based off of several criteria (e.g. not multi-valued, not type tdate or string). If it could be sorted upon, I added it to the SORT_DOC_FIELDS field for that document; if not, I added it to the NO_SORT_DOC_FIELDS field.
    3. Whenever I want to retrieve the fields that are present in a query, I add &facet=true&facet.field=SORT_DOC_FIELDS&facet.field=NO_SORT_DOC_FIELDS&facet.limit=-1 to the query string.

    The performance is OK. I have an index of 30 million documents, and making the query and getting a response, round-trip, takes about 2s.