AllenNLP) Is there a way to set config for evaluation, epeicially for reader?

Question

I am new to allenNLP library. In order to set args for dataset_reader, I want to set config for evaluation, like train (https://github.com/allenai/allennlp-template-config-files/blob/master/training_config/my_model_trained_on_my_dataset.jsonnet)

But I am not sure if there is a config file template for evaluation, like train, and the config file below works (where train_data_path and trainer parts are deleted.)

{
    "dataset_reader" : {
        // This name needs to match the name that you used to register your dataset reader, with
        // the call to `@DatasetReader.register()`.
        "type": "classification-tsv",
        // These other parameters exactly match the constructor parameters of your dataset reader class.
        "token_indexers": {
            "tokens": {
                "type": "single_id"
            }
        }
    },
    "validation_data_path": "/path/to/your/validation/data/here.tsv",
    "model": {
        // This name needs to match the name that you used to register your model, with
        // the call to `@Model.register()`.
        "type": "simple_classifier",
        // These other parameters exactly match the constructor parameters of your model class.
        "embedder": {
            "token_embedders": {
                "tokens": {
                    "type": "embedding",
                    "embedding_dim": 10
                }
            }
        },
        "encoder": {
            "type": "bag_of_embeddings",
            "embedding_dim": 10
        }
    },
    "data_loader": {
        // See http://docs.allennlp.org/master/api/data/dataloader/ for more info on acceptable
        // parameters here.
        "batch_size": 8,
        "shuffle": true
    },
}

Thanks in advance.

Answer 1

@petew's answer is correct. allennlp evaluate does not read a configuration file. It uses the configuration file that is stored with the model. Run allennlp evaluate -h to get more info.

If you need the dataset reader to behave differently at evaluation time, use the validation_dataset_reader field in the configuration file. This will take effect both during training (for evaluations at the end of the epoch), and later, when you run allennlp evaluate.