LLM Tool use in vLLM
- Track: Low-level AI Engineering and Hacking
- Room: UB2.252A (Lameere)
- Day: Sunday
- Start: 14:25
- End: 14:40
- Video only: ub2252a
- Chat: Join the conversation!
Tool use or function calling is one of the key features required to realize the promises of agentic applications. While it is still a novelty with a lot of experimentation and variation among model providers, protocols such as the OpenAI function calling extensions of the chat completions API have become a standard for a specific case of tool calling. In this talk we're going to explain how function calling works all the way from the API endpoint down to the actual model prompt. We're going to explore the different kinds of interaction flows that are possible with this protocol and what the current limitations are.
Speakers
Max de Bayser |