Search code examples
elasticsearchstormcrawler

Is there a way to get the `metadata.depth` value also be added to a field in the doc index?


Stormcrawler, through the process of crawling, is adding a field to the status index called metadata.depth. I am not sure where that is generated from, but would it be possible to somehow send that depth value to the, er, index index in elasticsearch as well as the status index? That value would help a bit with ranking for searching against the index index.


Solution

  • If it's in the metadata, it can be indexed using indexer.md.mapping, like anything else.

    It is generated by MetadataTransfer when an outlink is discovered.