Local AI requires a full-stack problem-solving approach to succeed. At Homebrew, we’ve addressed key issues like hardware compatibility, multimodal model optimization, and enabling efficient inference on edge devices.

In this talk, we’ll draw from our experience deploying local AI to over 1M+ devices with Jan, a small multi-modal AI. We’ll share insights into optimization hurdles, architectural trade-offs, hardware requirements, and their impact on developers and users alike.

Plus, we’ll share how a full-stack approach enables us to integrate user feedback loops and collect RLHF data, continuously improving model performance and delivering more effective solutions.