On-Device: Functional Tokens (Octopus v2)
New LLMs w/ functional token fine-tuning for edge devices (including iPhone, Pixel).
New tech (functional tokens, Octopus v2 fine-tuning) increases energy efficiency of LLM function calls significantly.
Octopus v2: Stanford Univ test new on-device LLMs w/ function calling, or tool use, for AI agent accuracy and inference speed. They applied a special fine-tuning with FUNCTIONAL TOKENS to GEMMA 2B from Google.
Plus function calling code for OpenAI, Anthropic /Claude 3 and Cohere Command R PLUS.
00:00 On Device LLM
00:20 Octopus v2 Function Calling (Apple, Google)
15:45 CODE (Anthropic, Cohere) Function Calling
24:24 Implications for NVIDIA, Microsoft?
#airesearch
#ai
#pythonprogramming
