Search code examples
pythonllamallama-cpp-python

TypeError in Python 3.11 when Using BasicModelRunner from llama-cpp-python

I'm currently taking the DeepAI's Finetuning Coursera course and encountered a bug while trying to run one of their demonstrations locally in a Jupyter notebook.

Environment:

  • Python version: 3.11
  • Required packages:
    • notebook
    • lamini
    • llama-cpp-python (> 0.1.53)

Issue:

When attempting to run the following code:

import os
import lamini

lamini.api_url = os.getenv("POWERML__PRODUCTION__URL")
lamini.api_key = os.getenv("POWERML__PRODUCTION__KEY")

from llama import BasicModelRunner

non_finetuned = BasicModelRunner("meta-llama/Llama-2-7b-hf")

non_finetuned_output = non_finetuned("Tell me how to train my dog to sit")

I receive a TypeError at the last line:

TypeError: can only concatenate str (not "NoneType") to str

This error occurs when I try to pass a string to the non_finetuned BasicModelRunner object.

Solution

  • I had the same problem. You have to create an account in lamini then you can create a key copy/past to POWERML__PRODUCTION__KEY for POWERML__PRODUCTION__URL let it as empty string ""