Search code examples
huggingface-transformershuggingface

How to ask multiple choice questions using pipeline

I want to ask multiple choice questions to the Hugging Face transformer pipeline; however, there does not seem to be a good task choice for this.

I am referencing this:

qa_pipeline = pipeline("question-answering", model="distilbert-base-uncased")
result = qa_pipeline(question = question, context = context)

I know question-answering exists; however, this task requires context where then the pipeline would extract the answer from the context. For specifically multiple choice answering, this method does not make much sense because I am trying to recieve a specific letter, not a word associated with the letter. Additionally, I could possiblely try text-generation, but I was hoping there was some better solution than this. Let me know your thoughts!

Solution

  • You could hand multiple sentence pairs to the model and then infer which one gives you the highest probability:

    https://huggingface.co/docs/transformers/tasks/multiple_choice#inference

    In your case prompt would be the question, and each answer option one of the candidates.