Search code examples
pythonmachine-learningpipnlpfasttext

CompressFastText pqkmeans does not install

I would like to use compressfasttext library: https://github.com/avidale/compress-fasttext

I use this command to install:

pip install compress-fasttext[full]

I get this pqkmeans error:

running install_egg_info   Copying lshash3.egg-info to
build/bdist.linux-x86_64/wheel/lshash3-0.0.8-py3.8.egg-info   
running
install_scripts   
error: invalid command 'bdist_wininst'   [end of
output]
note: This error originates from a subprocess, and is likely
not a problem with pip. ERROR: Failed building wheel for lshash3

I tried python2.7, python3.7, python3.8 with Ubunto 18.0.

For python 2.7, I can pip install the pqkmeans without this error. But compressFast Text library does not work.

For python3.7, 3.8, I get the same error message.

Any clue about this please?

UPDATE: Just want to add that I made a related issue here in case anybody wants to check: https://github.com/avidale/compress-fasttext/issues/19

Solution

  • As per my comment, from the error message, it looks like the real problem is in the install of lshash3 (not updated since 2017), which is only brought in via what appears to be an undocumented dataset package texmex-python used only for optional eval/demo purposes in pqkmeans.

    So, if you can make pip simply skip that dependency, you may get over this error with no other ill effects.

    I haven't tested this, but you may want to try, in a fresh Python 3.X virtual environment:

    • manually installing all the requirements of pqkmeans except texmex-python - see that project's requirements.txt for the list
    • then, install pqkmeans with the pip -no-deps option: pip install --no-deps pqkmeans
    • if that succeeds, perhaps pip install compress-fasttext[full] will then consider pqkmeans already present, and not try the problem texmex-python & lshash3 installs, and you're in business
    • but if it does still try, you could try manually installing compress-fastttext[full]'s unique non-pqkmeans requirements – really just gensim – then pip install --no-deps compress-fasttext[full]

    The potential end-result would be you'd have everything but the problem packages texmex-python and lshash3, which you probably don't really need for any of the code you'll be calling.

    Good luck, & let me know if any form of this works!