Welcome to the graduate course on Systems and Machine Learning! This is a research-oriented course covering topics on Systems for Machine Learning and Machine Learning for Systems.

Machine learning is transforming several domains ranging from natural language processing to drug discovery today. One of the key factors that enabled rapid progress in ML/AI in recent years has been fast-evolving underlying hardware and software platforms. In this course, we will cover recent advancements in research and industry on machine learning systems that enabled the AI/ML revolution. Specific topics include domain-specific architectures, deep learning frameworks and compilers, networking and scheduling in deep learning clusters, etc. We will also discuss practical challenges in deploying such systems. In the second half of the course, we will explore how machine learning has been employed to tackle various networking and systems challenges such as Internet congestion control, adaptive bitrate selection in video streaming, flow prediction, etc.

Instructor: Sangeetha Abdu Jyothi

Class Hours: MW 5:00-6:20 PM

Location: PCB 1200

Office Hours: MW 6:20-6:50 pm (in person after class)
and Mon 1:30-2:30 pm on Zoom (https://uci.zoom.us/j/97881946410?pwd=ZjiVhFb7MxYCPZo7jgtCdDtbMLnWT7.1) or by appointment

TA: Kapil Agrawal

TA Office Hours: Wed 3.30 - 4.30 pm, ICS 458B

Course Policies: Course Policies

Prerequisites: Understanding of basic concepts in machine learning and systems (taken at least one undergrad course in ML and (networking or operating systems or distributed systems))

Schedule: (Papers may be updated as the quarter progresses. Please check back periodically. The paper for each class will be final one week prior to the class. )

Date	Category	Lecture	Required Reading	Optional
09/30/24		Lec 1: Introduction
10/02/24	Systems For ML	Lec 2: Domain Specific Architectures	In-Datacenter Performance Analysis of a Tensor Processing Unit (ISCA'17)	A Configurable Cloud-Scale DNN Processor for Real-Time AI (ISCA'18)
10/07/24	Systems For ML	Lec 3: DL Frameworks	PyTorch (NeurIPS'19)	TensorFlow (OSDI'16) MXNet CNTK (KDD'16)
10/09/24	Systems For ML	Lec 4: DL compilers	TVM (OSDI'18)	Glow nGraph XLA TensorComprehensions
10/14/24	Systems For ML	Lec 5: Networking Challenges in DNN training	Parameter Server (OSDI'14)	Horovod (arXiv'18) BytePS (OSDI'20) TicTac (MLSys'19) P3 (MLSys'19)
10/16/24	Systems For ML	Lec 6: Cluster scheduling for DL workloads	Gavel (OSDI'20)	AntMan (OSDI'20) Themis (NSDI'20) Gandiva (OSDI'18)
10/21/24	Systems For ML	Lec 7: Automated ML: Hyperparameter Tuning and Neural Architectural Search (NAS)	Hyperband (JMLR'18)	Cerebro (VLDB'20) Vizier (KDD'17) ASHA (MLSys'20) Designing Neural Architectures using RL (ICLR'17)
10/23/24	Systems For ML	Lec 8: ML at the Edge	Towards Federated Learning at Scale: System Design (MLSys'19)
10/28/24	Systems For ML	Lec 9: LLM training	TeraPipe (ICML'21)	LightSeq2 (SC'22) PipeFisher (MLSys'23)
10/30/24	Systems For ML	Lec 10: LLM serving	Orca (OSDI'22)	Scaling Transformer Inference (MLSys'23)
11/04/24	Systems For ML	Lec 11: Other SoTA Models (Diffusion models, DLRMs, etc.)	AdaEmbed (OSDI '23)	Accelerating Diffusion Models through Block Caching (CVPR '24)
11/06/24	Systems For ML	Lec 12: Challenges in Operational ML Systems	Hidden Technical Debt in Machine Learning Systems (NeurIPS'15)	TFX (KDD'17) Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective (HPCA'18)
11/11/24		Veteran's Day
11/13/24	ML for Systems	Lec 13: Classic control + Learning (congestion control & ABR)	Classic Meets Modern (SIGCOMM'20)	Aurora (ICML'19) Pensieve (SIGCOMM'17) Learning in-situ (ATC'20)
11/18/24	ML for Systems	Lec 14: ML for Index Structures	Learned Index Structures (SIGMOD'18)
11/20/24	ML for Systems	Lec 15: ML for Flow Prediction	Flux (NSDI'19)
11/25/24	ML for Systems	Lec 16: Interpretability of RL-based controllers	Metis (SIGCOMM'20)
11/27/24		Project Presentations
12/02/24		Project Presentations
12/04/24		Project Presentations

Grading

Paper Summaries: 40%

Project: 60%

Title and plan: 5%

Checkpoint 1 (Report + Recorded presentation): 15%

Checkpoint 2 (Report): 10%

Presentation: 15%

Final report: 15%