Search code examples
quanteda

Quanteda, textmodel_wordfish, "the algorithm did not converge"

I am trying to estimate a wordfish model with the latest version of quanteda (1.3.0) - yet, keep getting the message:

Warning message:
In textmodel_wordfish.dfm(speaker_dfm, dir = c(28, 2)) :
Warning: The algorithm did not converge.

The dfm is grouped by docvars and currently relatively small at around 700000 elements (I have tried several sizes and kinds of dfms, always the same warning message). A couple of days ago (before the most recent quanteda update) it went smoothly with a much bigger dfm. Any idea how to fix this? Thank you in advance!

Solution

  • This depends a lot on the sparseness of the dfm, which is unknown from your question. This unsupervised model will have a lot of trouble if your dfm is nearly all zeroes.

    Steps you can take:

    1. Reduce the number of low-frequency features using dfm_trim().

    2. Try running the model with sparse = TRUE.