TLDR: We are hiring! Specifically, we are hiring PhDs graduating in 2025, professionals with industry experience, and managers. We are NOT hiring new graduates with a bachelor's or master's degree. Our summer internship positions have been filled. Candidates please email us at "codesign AT meta DOT com".

The AI and Systems Co-Design team at Meta (formerly known as Facebook), led by Chunqiang Tang (a.k.a. CQ Tang), consists of over 100 employees, mostly PhDs, including many world-class research scientists and engineers. As reflected in our team name "co-design", we conduct interdisciplinary research and development across AI, hardware, and software, with a focus on performance, efficiency, and scalability.

• We own the company's overall strategy for exploring innovative hardware technologies for CPUs, GPUs, memory, storage, and Meta's custom AI chips, and we productionize them in Meta's hyperscale fleet of O(1,000,000) servers and O(100,000) GPUs, powering all Meta products such as Facebook, Instagram, and meta.ai.
• We apply novel software optimizations across the whole stack---from ML models and applications to the Linux kernel---to achieve optimal performance on the hardware.
• We develop innovative AI technologies for large language models (Llama), ranking systems, and more.

Overall, our work largely corresponds to the research communities of systems in general and especially systems for ML (MLSys, SOSP, OSDI, SIGCOMM, NSDI), hardware architecture (ISCA, ASPLOS), ML (NeurIPS, ICML, ICLR) and supercomputing (SC, ICS). Here are selected publications that showcase our work in diverse areas:

• Systems for ML
  • The Llama 3 Herd of Models. Our contributions include much of the work described in the paper's Section 3.3 "Infrastructure, Scaling, and Efficiency", Section 6 "Inference", and Section 7.3 "Model Scaling".
  • Llama 2: Open foundation and fine-tuned chat models. Our contributions include re-architecting Llama's training infrastructure and transitioning it from a research environment to Meta's hyperscale production infrastructure, enabling future Llama training to scale to tens of thousands of GPUs and beyond.
  • Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
  • PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
• AI chip and server design
  • MTIA: First Generation Silicon Targeting Meta's Recommendation Systems
  • The Grand Teton AI Server
• ML models and kernels
  • Deep Learning Recommendation Model for Personalization and Recommendation Systems
  • Wukong: Towards a Scaling Law for Large-Scale Recommendation
  • FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
• ML numerics, pruning, distillation, and optimizer
  • Microscaling Data Formats for Deep Learning
  INT4 Decoding GQA CUDA Optimizations for LLM Inference
  • Pruning and Distillation to Enable Llama 3.2 1B and 3B Models Suitable for Mobile Devices
  • PyTorch Distributed Shampoo Winning the MLCommon Training Algorithms Competition
• HPC and collective communications library (MPI, NCCL, RCCL)
  • Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression
  • Training Deep Learning Recommendation Model with Quantized Collective Communications
• Performance benchmarking and projection for both AI and non-AI workloads
  • DCPerf: An open source benchmark suite for hyperscale compute applications
  • DLRM: An advanced, open source deep learning recommendation model
• Hardware and software co-design
  • TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory
  • SoftSKU: Optimizing Server Architectures for Microservice Diversity @Scale

Like research labs, our team consists primarily of PhDs, and we strongly encourage and excel in research publications. However, we differ from traditional research labs in several key ways:

• Direct ownership: Like traditional research labs, we build strong partnerships with numerous teams across diverse areas for broad influence. However, what sets us apart is our direct ownership of the hardware strategy for Meta's hyperscale fleet. This enables us to lead in many areas while fostering seamless partnerships in others.
• Production systems: Our primary goal is to develop forward-looking innovations in AI, hardware, and software, and directly implement them in production systems that serve billions of people. The billions of users of Meta products and Meta's hyperscale fleet of O(1,000,000) servers and O(100,000) GPUs are, in effect, our lab. In contrast, traditional research labs often rely on technology transfer for a less direct impact.
• Impact: Our impact is widely acknowledged within the company and throughout the industry. We drive Meta's hardware strategy to save billions of dollars, and directly develop innovative technologies in Meta's flagship products like Llama and Ads ranking models.

Open Source Projects

• DCPerf: An open source benchmark suite for hyperscale compute applications
• DLRM: An advanced, open source deep learning recommendation model
• FBGEMM: ML kernels
• Pytorch distributed Shampoo optimizer. This work won the competition of the external tuning track of the inaugural AlgoPerf training algorithms benchmark.

2024

• The Llama 3 Herd of Models
• (IEEE Micro Top Picks'24) End-to-End Cloud Application Cloning With Ditto
• Movie Gen: A Cast of Media Foundation Models
• Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression
• Wukong: Towards a Scaling Law for Large-Scale Recommendation
• Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large Scale Recommendation
• PyTorch Distributed Shampoo Winning the MLCommon Training Algorithms Competition
• Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
• CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization
• MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale
• QuickUpdate: a Real-Time Personalization System for Large-Scale Recommendation Model
• Toward 100TB Recommendation Models With Embedding Offloading
• DCPerf: An open source benchmark suite for hyperscale compute applications
• Expanding Datacenter Capacity with DVFS Boosting: A safe and scalable deployment experience
• Task Me Anything
• Context Parallelism for Scalable Million-Token Inference

2023

• Llama 2: Open Foundation and Fine-Tuned Chat Models
• MTIA: First Generation Silicon Targeting Meta's Recommendation Systems
• Microscaling Data Formats for Deep Learning
• Shared Microexponents: A Little Shifting Goes a Long Way
• TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory
• Contiguitas: The Pursuit of Physical Memory Contiguity in Datacenters
• Mystique: Enabling Accurate and Scalable Generation of Production AI Benchmarks
• RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure
• AdaEmbed: Adaptive Embedding for Large-Scale Recommendation Models
• Characterization of Data Compression in Datacenters,
• Ditto: End-to-End Application Cloning for Networked Cloud Services,
• FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models

2022

• Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
• DHEN: A deep and hierarchical ensemble network for large-scale click-through rate prediction
• TMO: Transparent Memory Offloading in Datacenters
• Unity: A Unified Graph Representation and Runtime for Distributed DNN Training
• Check-N-Run: a Checkpointing System for Training Deep Learning Recommendation Models
• Supporting Massive DLRM Inference through Software Defined Memory
• RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation

2021

• FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
• Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
• Efficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models
• First-Generation Inference Accelerator Deployment at Facebook
• Understanding Acceleration Opportunities at Hyperscale
• Improving Performance of Flash Based Key-Value Stores Using Storage Class Memory as a Volatile Memory Extension

2020

• Training Deep Learning Recommendation Model with Quantized Collective Communications
• Mixed-Precision Embedding Using a Cache
• FioSynth: A representative I/O benchmark and data visualizer for data center workloads,
• Accelerometer and SoftSKU: Improving hardware platform performance for diverse microservices
Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale

2019

• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• SoftSKU: Optimizing Server Architectures for Microservice Diversity @Scale
• A Study of BFLOAT16 for Deep Learning Training
• Post-Training 4-bit Quantization of Embedding Tables
• Bandana: Using Non-Volatile Memory for Storing Deep Learning Models
• Who's Afraid of Uncorrectable Bit Errors? Online Recovery of Flash Errors with Distributed Redundancy