MTIA v1: Meta’s first-generation AI inference accelerator

AI workloads are ubiquitous at Meta — forming the basis for a wide range of use cases, including content understanding, Feeds, generative AI, and ads ranking. These workloads run on PyTorch with first-class Python integration, eager-mode development, and the simplicity of APIs. Deep learning recommendation models (DLRMs), in particular, are important for improving experiences across Meta’s services and applications. But as these models increase in size and complexity, the underlying hardware systems need to provide exponentially more memory and compute while remaining efficient.

Read the full article.