Search code examples
meta-tagsnutch

Issue in retrieving Metatags - Nutch 2.3 version

I'm using Nutch2.3-src version. Am able to crawl the webpages, but it is only taking description, and not other metatags like LastModified, Author.

I have updated Index.metadata and metatags.names property. But still no luck. Getting only null as value.

<property>
<name>metatags.names</name>
<value>*</value>
<description>Names of the metatags to extract, separated by ','.
  Use '*' to extract all metatags. Prefixes the names with 'meta_' in
  the parse-metadata. For instance, to index description and keywords,
  you need to activate the plugins parse-metadata and index-metadata
  and set the value of the properties 'metatags.names' and
  'index.metadata' to 'description,keywords'.
  </description>
</property>

<property>
  <name>index.metadata</name>
  <value>description,LastModified,Created,WCMCategories,WCMKeywords,Authors,SiteName,title,lastmodified,created,wcmcategories,wcmkeywords,authors,sitename,meta_description,meta_LastModified,meta_Created,meta_WCMCategories,meta_WCMKeywords,meta_Authors,meta_SiteName,meta_title,meta_lastmodified,meta_created,meta_wcmcategories,meta_wcmkeywords,meta_authors,meta_sitename</value>
  <description>
  Comma-separated list of keys to be taken from the metadata to generate fields.
  Can be used e.g. for 'description' or 'keywords' provided that these values are generated
  by a parser (see parse-metatags plugin), and property 'metatags.names'.
  </description>
</property>
Solution

  • Resolved this issue. Metatags were case-sensitive. The attribute name should match in both webpage and nutch-site.xml.