Search code examples
javadbpediaspotlight-dbpedia

dbpedia spotlight dataset


I installed the DBpedia Spotlight from http://spotlight.dbpedia.org/download/release-0.5/dbpedia-spotlight-quickstart.zip and wanted to improve its dataset by downloading from https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki/Downloads.

Can someone tell me how to use the data from spotter lexicon and disambiguation index with the jar files.


Solution

  • Assuming you have already downloaded and decompressed the files below:

    wget http://spotlight.dbpedia.org/download/release-0.5/context-index-compact.tgz
    tar zxvf context-index-compact.tgz
    wget http://spotlight.dbpedia.org/download/release-0.4/surface_forms-Wikipedia-TitRedDis.uriThresh75.tsv.spotterDictionary.gz
    gunzip surface_forms-Wikipedia-TitRedDis.uriThresh75.tsv.spotterDictionary.gz
    

    Now you just need to change the server.properties file to point to your newly extracted files:

    org.dbpedia.spotlight.index.dir = index-withSF-withTypes-compressed
    org.dbpedia.spotlight.spot.dictionary = surface_forms-Wikipedia-TitRedDis.uriThresh75.tsv.spotterDictionary
    

    If you are using the largest spotter dict, you may need to increase the java heap space -- e.g. -Xmx10G in your command line.