I have a basic chain that classifies some text based on the Common European Framework of Reference for Languages. I'm timing the difference between normal chain.apply
and chain.aapply
but can't get it to work.
What am I doing wrong?
import os
from time import time
import openai
from dotenv import load_dotenv, find_dotenv
from langchain.chains import LLMChain
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
_ = load_dotenv(find_dotenv())
openai.api_key = os.getenv('OPENAI_API_KEY')
llm = ChatOpenAI(temperature=0)
prompt = ChatPromptTemplate.from_template(
'Classify the text based on the Common European Framework of Reference '
'for Languages (CEFR). Give a single value: {text}',
)
chain = LLMChain(llm=llm, prompt=prompt)
texts = [
{'text': 'Hallo, ich bin 25 Jahre alt.'},
{'text': 'Wie geht es dir?'},
{'text': 'In meiner Freizeit, spiele ich gerne Fussball.'}
]
start = time()
res_a = chain.apply(texts)
print(res_a)
print(f"apply time taken: {time() - start:.2f} seconds")
print()
start = time()
res_aa = chain.aapply(texts)
print(res_aa)
print(f"aapply time taken: {time() - start:.2f} seconds")
Output
[{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}]
apply time taken: 2.24 seconds
<coroutine object LLMChain.aapply at 0x0000025EA95BE3B0>
aapply time taken: 0.00 seconds
C:\Users\User\AppData\Local\Temp\ipykernel_13620\1566967258.py:34: RuntimeWarning: coroutine 'LLMChain.aapply' was never awaited
res_aa = chain.aapply(texts)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Changing
res_aa = chain.aapply(texts)
to
res_aa = await chain.aapply(texts)
did the job!
Now it works (damn these methods are much faster than doing it sequentially)
[{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}]
apply time taken: 2.87 seconds
[{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}]
aapply time taken: 1.34 seconds