Search code examples
huggingface-transformers

Why does huggingface hang on list input for pipeline sentiment-analysis?


With python 3.10 and latest version of huggingface.

for simple code likes this

from transformers import pipeline

input_list = ['How do I test my connection? (Windows)', 'how do I change my payment method?', 'How do I contact customer support?']

classifier = pipeline('sentiment-analysis')
results = classifier(input_list)

the program hangs and returns error messages:

File ".......env/lib/python3.10/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

but replace the list input with a string, it works

from transformers import pipeline
classifier = pipeline('sentiment-analysis')
result = classifier('How do I test my connection? (Windows)')

Solution

  • It needs to define a main function to run multitask that the list input depends on. Following update works

    from transformers import pipeline
    
    def main():
        input_list = ['How do I test my connection? (Windows)', 
        'how do I change my payment method?',
        'How do I contact customer support?']
    
        classifier = pipeline('sentiment-analysis')
        results = classifier(input_list)
    
    if __name__ == '__main__':
        main()
    

    The question is reduced to where to put freeze_support() in a Python script?