Search code examples
pythonwindows-subsystem-for-linuxlarge-language-modelllama-indexollama

Client error '404 Not Found' for url 'http://localhost:11434/api/chat' while using ReActAgent of llama_index.core.agent

I am following this tutorial, https://youtu.be/JLmI0GJuGlY?si=eeffNvHjaRHVV6r7&t=1915, and trying to build a simple LLM agent.

I am on WSL2, Windows 11, and I am coding in VSC. I use Ollama to download and store my LLMs. My python is 3.9.

My script my_main3.py is very simple:

from llama_index.llms.ollama import Ollama
from llama_parse import LlamaParse
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, PromptTemplate
from llama_index.core.embeddings import resolve_embed_model
from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.core.agent import ReActAgent
from prompts import context

from dotenv import load_dotenv
load_dotenv()

llm = Ollama(model="mistral", request_timeout=30.0)

parser = LlamaParse(result_type="markdown") 

file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader("./data", file_extractor=file_extractor).load_data()

embed_model = resolve_embed_model("local:BAAI/bge-m3")

vector_index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)
query_engine = vector_index.as_query_engine(llm=llm)

tools = [
    QueryEngineTool(
        query_engine=query_engine,
        metadata=ToolMetadata(
            name="api_documentation",
            description="this gives documentation about code for an API. Use this for reading docs for the API",
        ),
    )
]

code_llm = Ollama(model="codellama")
agent = ReActAgent.from_tools(tools, llm=code_llm, verbose=True, context=context) # context is from prompts.py

while (prompt := input("Enter a prompt (q to quit): ")) != "q":
    result = agent.query(prompt)
    print(result)

Then I run Python main.py in my Terminal. The script runs well until the while loop.

It prompts me to input, then in input:

Enter a prompt (q to quit): send a post request to make a new item using the api in Python.

It then throws me this error.

Traceback (most recent call last):
  File "/home/ubuntu2022/MyUbunDev/210_AI_agent_basic/my_main3.py", line 38, in <module>
    result = agent.query(prompt)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 102, in wrapper
    self.span_drop(*args, id=id, err=e, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 77, in span_drop
    h.span_drop(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/base.py", line 48, in span_drop
    span = self.prepare_to_drop_span(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/null.py", line 35, in prepare_to_drop_span
    raise err
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 100, in wrapper
    result = func(*args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/base/base_query_engine.py", line 51, in query
    query_result = self._query(str_or_query_bundle)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
    return func(self, *args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/agent/types.py", line 40, in _query
    agent_response = self.chat(
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 102, in wrapper
    self.span_drop(*args, id=id, err=e, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 77, in span_drop
    h.span_drop(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/base.py", line 48, in span_drop
    span = self.prepare_to_drop_span(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/null.py", line 35, in prepare_to_drop_span
    raise err
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 100, in wrapper
    result = func(*args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
    return func(self, *args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/agent/runner/base.py", line 604, in chat
    chat_response = self._chat(
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 102, in wrapper
    self.span_drop(*args, id=id, err=e, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 77, in span_drop
    h.span_drop(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/base.py", line 48, in span_drop
    span = self.prepare_to_drop_span(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/null.py", line 35, in prepare_to_drop_span
    raise err
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 100, in wrapper
    result = func(*args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/agent/runner/base.py", line 539, in _chat
    cur_step_output = self._run_step(
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 102, in wrapper
    self.span_drop(*args, id=id, err=e, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 77, in span_drop
    h.span_drop(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/base.py", line 48, in span_drop
    span = self.prepare_to_drop_span(*args, id=id, err=err, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/span_handlers/null.py", line 35, in prepare_to_drop_span
    raise err
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/instrumentation/dispatcher.py", line 100, in wrapper
    result = func(*args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/agent/runner/base.py", line 382, in _run_step
    cur_step_output = self.agent_worker.run_step(step, task, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
    return func(self, *args, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/agent/react/step.py", line 653, in run_step
    return self._run_step(step, task)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/agent/react/step.py", line 463, in _run_step
    chat_response = self._llm.chat(input_chat)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/core/llms/callbacks.py", line 130, in wrapped_llm_chat
    f_return_val = f(_self, messages, **kwargs)
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/llama_index/llms/ollama/base.py", line 105, in chat
    response.raise_for_status()
  File "/home/ubuntu2022/miniconda/envs/llm/lib/python3.9/site-packages/httpx/_models.py", line 761, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://localhost:11434/api/chat'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404

I checked my Edge browser, http://localhost:11434/ is running Ollama. Is this causing the clash? And Noticed I have never set up that http://localhost:11434/api/chat endpoint in my script.

Solution

  • Someone posted further down in the comments on the video. I had this same issue.

    @HodBuri 1 month ago Error 404 not found - local host - api - chat [FIX] If anyone else gets an error like that when trying to run the llamacode agent, just run the llamacode llm in terminal to download it, as it did not download it automatically for me at least as he said around 29:11

    So similar to what he showed at the start with Mistral: ollama run mistral.

    You can run this in a new terminal to download codellama: ollama run codellama

    After running the line above in a new terminal I kept it up and reran the main.py in the terminal I was previously working in and it worked