Search code examples
tensorflowsyntaxnet

How to interpret the output of a synaxnet when annotating a corpus

I annotated a corpus using pre-trained syntaxnet model (i.e. using Parse McParseface). I am having a problem understanding the output. There are two metrics reproted in the output. Are those for POS tagging and dependency parsing? If yes, which one is POS tagging performance and which one is for dependency parsing performance?

Here is the output:

INFO:tensorflow:Total processed documents: 21710 INFO:tensorflow:num correct tokens: 454150 INFO:tensorflow:total tokens: 560993 INFO:tensorflow:Seconds elapsed in evaluation: 1184.63, eval metric: 80.95% INFO:tensorflow:Processed 206 documents INFO:tensorflow:Total processed documents: 21710 INFO:tensorflow:num correct tokens: 291851 INFO:tensorflow:total tokens: 504496 INFO:tensorflow:Seconds elapsed in evaluation: 1193.17, eval metric: 57.85%

Solution

  • If you're using https://github.com/tensorflow/models/blob/master/syntaxnet/syntaxnet/demo.sh then the first metric is POS tag accuracy, the second UAS. They are only meaningful if the conll data you input contains gold POS tags and gold dependencies.