I am trying to use the Stanford NLP toolkit to evaluate dependency grammars.
Is it possible to make the LexicalizedParser class use only the rules from the DEPENDENCY_GRAMMAR section of the model file?
Leaving the other sections of the model file blank causes exceptions, and changing the 'doPCFG' flag to 'false' in the model file causes a null pointer exception in ExhaustiveDependencyParser.getBestScore.
It appears that the LexicalizeParser class falls back on the (constituency) PCFG when it cannot parse the input using the dependency rules.
Is this correct? Can this behaviour be switched off so that dependency grammars can be evaluated in isolation?
What you're looking for is the Stanford Dependency Parser, which only produces a dependency parser and not a constituency parse. The lexicalized parser, by contrast, only produces a constituency parse, which is then converted to a dependency parse with a converter. Therefore, there is no option to not produce this constituency parse and still produce a dependency parse, as there's nothing for the converter to convert.
If you're only doing evaluation of grammatical formalisms, however, you may consider using the dependency converter on gold trees, or using the new Universal Dependencies annotated data.