Functional Calling in LLMS

What is functional Calling...?

This refers to the ability to reliably connect LLM`s to external tools to enable effective tool usage and interaction with external APIs.

This capability allows the models to detect when an API call is needed and generate a JSON file with the necessary parameters. These functions, acting as tools, can be defined and utilised within a single request to enhance the AI’s interaction with external systems.

Merits

Functional calling enables AI developers to create:

• Conversational agents that can efficiently use external tools to answer questions. I.e., the query "What is the weather like in Wajir ?" will be converted to a function call such as get_current_weather(location: string, unit: 'celsius' | 'fahrenheit')

• LLM-powered solutions for extracting and tagging data (e.g., extracting people names from a Wikipedia article)

• Applications that can convert natural language to API calls or valid database queries

• Conversational knowledge retrieval engines that interact with a knowledge base

Functional Calling with GPT 3.5-Turbo 1106

say we want to get a weather conditions of a place like Wajir

The AI Model alone isn't equipped to handle this task due to its data limitations. To address this, you should integrate the model with an additional tool. By using the function-calling features, you can identify which external tool to use and provide the necessary parameters, ultimately generating a complete solution.

Let's say a user is asking the following question to the model:

What is the weather like in Moyale ?

To handle this request using function calling, the first step is to define a weather function or set of functions that you will be passing as part of the OpenAI API request:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Retrieve the current weather for a specified location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "Specify the city and state, for example, Moyale,Kenya",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit to use; either Celsius or Fahrenheit.",
                    },
                },
                "required": ["location"],
            },
        }
    }
]

The function get_current_weather returns the current weather in a given location. When you pass this function definition as part of the request, it doesn't actually executes a function, it just returns a JSON object containing the arguments needed to call the function. Here are some code snippets of how to achieve this.

def get_completion(messages, model="gpt-3.5-turbo-1106", temperature=0, max_tokens=300, tools=None):
    response = openai.chat.completions.create(
        model=model,
        messages=messages,
        temperature=temperature,
        max_tokens=max_tokens,
        tools=tools
    )
    return response.choices[0].message

the prompt design:

messages = [
    {
        "role": "user",
        "content": "What is the weather like in Moyale?"
    }
]

now call the get_completion function with the defined arguments

response = get_completion(messages, tools=tools)

The response object contains the following:

ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='...', function=Function(arguments='{"location":"Moyale","unit":"celsius"}', name='get_current_weather'), type='function')])

In particular, the arguments object contains the important arguments extracted by the model and that will be needed to complete the request.

You can then choose to call an external weather API for the actual weather. Once you have the weather information available you can pass it back to the model to summarise a final response given the original user question.

Conclusion

Recent research highlights that GPT-4 Turbo excels in functional calling, with an impressive 95% usage rate for this capability. In contrast, GPT-3.5 Turbo (version 1106) shows notable deficiencies, often encountering errors with functional calling APIs. Given these insights, GPT-4 Turbo is strongly recommended for applications that rely heavily on functional calling.