Granite 20b Functioncalling

🚀 Granite-20B-FunctionCalling

Granite-20B-FunctionCalling 是一個基於 IBM 的 granite-20b-code-instruct 模型微調得到的模型，旨在為 Granite 模型家族引入函數調用能力。該模型採用多任務訓練方法，針對函數調用中包含的七項基本任務進行訓練，包括嵌套函數調用、函數鏈、並行函數、函數名稱檢測、參數值對檢測、最優下一個函數和響應生成。

🚀 快速開始

模型概述

• 開發者：IBM Research
• 論文：Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks
• 發佈日期：2024 年 7 月 9 日
• 許可證：Apache 2.0

預期用途

該模型旨在響應與函數調用相關的指令。

💻 使用示例

基礎用法

以下是一個如何使用 Granite-20B-Code-FunctionCalling 模型的簡單示例：

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda" # or "cpu"
model_path = "ibm-granite/granite-20b-functioncalling"
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()

# define the user query and list of available functions
query = "What's the current weather in New York?"
functions = [
    {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                }
            },
            "required": ["location"]
        }
    },
    {
        "name": "get_stock_price",
        "description": "Retrieves the current stock price for a given ticker symbol. The ticker symbol must be a valid symbol for a publicly traded company on a major US stock exchange like NYSE or NASDAQ. The tool will return the latest trade price in USD. It should be used when the user asks about the current or most recent price of a specific stock. It will not provide any other information about the stock or company.",
        "parameters": {
            "type": "object",
            "properties": {
                "ticker": {
                    "type": "string",
                    "description": "The stock ticker symbol, e.g. AAPL for Apple Inc."
                }
            },
            "required": ["ticker"]
        }
    }
]


# serialize functions and define a payload to generate the input template
payload = {
    "functions_str": [json.dumps(x) for x in functions],
    "query": query,
}

instruction = tokenizer.apply_chat_template(payload, tokenize=False, add_generation_prompt=True)

# tokenize the text
input_tokens = tokenizer(instruction, return_tensors="pt").to(device)

# generate output tokens
outputs = model.generate(**input_tokens, max_new_tokens=100)

# decode output tokens into text
outputs = tokenizer.batch_decode(outputs)

# loop over the batch to print, in this example the batch size is 1
for output in outputs:
    # Each function call in the output will be preceded by the token "<function_call>" followed by a 
    # json serialized function call of the format {"name": $function_name$, "arguments" {$arg_name$: $arg_val$}}
    # In this specific case, the output will be: <function_call> {"name": "get_current_weather", "arguments": {"location": "New York"}}
    print(output)

📄 許可證

本項目採用 Apache 2.0 許可證。