Search code examples
pythonlangchainlarge-language-model

RetrievalQAWithSourcesChain - NotImplementedError: Saving not supported for this chain type

I built a RAG pipeline and now want to save the model/pipeline locally. However when I try to save it I get an error message. Here is the code and the error output:

prompt_template = """ You are a chatbot having a conversation with a human. You can only answer in the German language.
Do not put English language or English translations into your answer.
{summaries}
Human: {question}
Chatbot:
"""

from langchain.chains import RetrievalQA
from langchain.chains import RetrievalQAWithSourcesChain

prompt = PromptTemplate(input_variables=["summaries","question"],
            template=prompt_template)

chain_type_kwargs = {"prompt": prompt}
#search_kwargs={'k': 7} -> The more the better, aber irgendwann ist Context Limit errreicht
rag_pipeline = RetrievalQAWithSourcesChain.from_chain_type(
    llm=model, chain_type='stuff',
    retriever=vectordb.as_retriever(),
    chain_type_kwargs=chain_type_kwargs,
)

rag_pipeline.save("llama_rag_modell.json")

Results in:

NotImplementedError: Saving not supported for this chain type.

So how can I save my pipeline?

Solution

  • As OP mentioned in the comments above, the problem was an outdated version of LangChain specifically version 0.0.240.

    The issue was fixed in more recent versions, here: https://github.com/langchain-ai/langchain/pull/10132 and here https://github.com/langchain-ai/langchain/issues/3983.

    The solution in this case was to upgrade LangChain:

    pip install langchain==0.0.327

    Since LangChain and similar LLM frameworks are fairly new and getting major updates really fast, I advise anyone facing similar issues to check the GitHub repo of the framework, or the official Python package site to determine why some methods may not work as provided in the official docs.

    Here's another answer that is related to an outdated version of LangChain: Langchain's AttributeError: 'RunnableSequence' object has no attribute 'input_schema'

    In this one week old post, the version went up from 0.0.321 to 0.0.327, so you know what I mean when I say things are moving fast!