Loading…
← Back to schedule
KEYNOTE (AI DevWorld): SambaNova Systems -- Composition of Experts: A Modular Approach to Scaling AI Models
Thursday February 13, 2025 2:30pm - 2:55pm PST
AI DevWorld Main Stage
Anton McGonnell, SambaNova Systems, VP of Product

This session, led by Anton McGonnell, Head of Software Products at SambaNova Systems, presents an in-depth exploration of the Composition of Experts approach in AI - a model made up of many models that work in conjunction with each other - and in particular the benefits to individual developers. Instead of building one monolithic model, this strategy composes workflows using many specialised, independent "expert" models. This proves to be more efficient, modular, and extensible.

The talk will provide a deep dive into the advantages of this technique, which promotes reusability, saves on alignment tax, and enables granular access control. By detailing real-world implementations, attendees will learn best practices for breaking down problems, assigning appropriate experts, and orchestrating data flow between them. McGonnell will discuss how an architecture with a large memory capacity, enables this approach by hosting and serving thousands of models in parallel on a single node without performance degradation. This makes the Composition of Experts approach highly scalable and reduces the total cost of ownership for inference.

Additionally, the session will offer crucial hardware considerations for executing Composition of Experts at scale. It will examine how specialised architectures can allow hosting thousands of models in parallel on a single node. With companies increasingly relying on AI, reliability and cost-efficiency are paramount concerns. Composition of Experts tackles these challenges through modularity and inference optimisation. By attending, developers and engineers can expect to gain highly applicable skills for improving the scalability, performance, and governance of AI in production environments.
Speakers
avatar for Anton McGonnell

Anton McGonnell

VP of Product, SambaNova Systems
Anton McGonnell is SambaNova’s VP of Product and leads the team defining their cutting-edge full-stack chips-to-model platform.Prior to SambaNova, Anton played a pivotal role in shaping the development and implementation of advanced machine learning technologies at UIPath.Anton... Read More →
Thursday February 13, 2025 2:30pm - 2:55pm PST
AI DevWorld Main Stage
  5. KEYNOTES & FEATURED

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Share Modal

Share this link via

Or copy link