I am using Google's gemini-1.5-pro-002 model to build a chat bot. The model is deployed on VertexAI. I need the bot to do auto function calling. It works if the function parameters are simple types such as string or integer. I am having issues getting it to work with complex types.
The following request does not work:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:generateContent \
-d '{
"contents": [
{
"role": "user",
"parts": [
{
"text": "Search for German people with the last name of Greene"
}
]
}
],
"tools": [
{
"functionDeclarations": [
{
"name": "SearchPlugin_SearchPerson",
"description": "Find a person given a search request",
"parameters": {
"type": "object",
"properties": {
"search_request": {
"type": "object",
"properties": {
"first_name": {
"type": "string",
"description": "The person first_name"
},
"last_name": {
"type": "string",
"description": "The person last_name"
},
"nationality": {
"type": "string",
"description": "The person nationality"
}
}
}
},
"required": [
"search_request"
]
}
}
]
}
]
}'
Gemini replies with
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"functionCall": {
"name": "SearchPlugin_SearchPerson",
"args": {
"search_request": "unknown"
}
}
}
]
},
"finishReason": "STOP"
}
],
"usageMetadata": {
"promptTokenCount": 54,
"candidatesTokenCount": 9,
"totalTokenCount": 63
},
"modelVersion": "gemini-1.5-pro-002"
}
Gemini can use the tool correctly if I define the tool like this:
{
"functionDeclarations": [
{
"name": "SearchPlugin_SearchPerson",
"description": "Find a person given a search request.",
"parameters": {
"type": "object",
"properties": {
"first_name": {
"type": "string",
"description": "The person first_name."
},
"last_name": {
"type": "string",
"description": "The person last_name."
},
"nationality": {
"type": "string",
"description": "The person nationality."
}
},
"required": [
"last_name"
]
}
}
]
}
Can Gemini call functions with complex parameters?
About your question Can Gemini call functions with complex parameters?, it's yes. But, in the current stage, the result depends on the model.
When I tested your showing request body with several models, I noticed that the result depends on the model. It seems that in the current stage (September 30, 2024), the following result is obtained. So, as the current workaround, if you want to directly use your showing request body, how about changing the model from gemini-1.5-pro-002 to another?
| model | Result |
|---|---|
| gemini-1.5-pro | failure |
| gemini-1.5-pro-001 | failure |
| gemini-1.5-pro-latest | failure |
| gemini-1.5-pro-exp-0827 | failure |
| gemini-1.5-pro-002 | failure |
| gemini-1.5-flash | success |
| gemini-1.5-flash-001 | success |
| gemini-1.5-flash-latest | success |
| gemini-1.5-flash-8b-exp-0827 | failure |
| gemini-1.5-flash-002 | failure |
| gemini-1.5-flash-8b-exp-0924 | success |
failure: "search_request": "unknown" was obtained from your request body.
success: "search_request": {"last_name": "Greene","nationality": "German"} was obtained from your request body.
success might also be changed in the update. So, please think of this answer as of September 30, 2024.