CSCI1390

Systems for Machine Learning

Spring 2025

Many applications, across industries varying from ecommerce to education, rely on data processing and machine learning systems for data analytics tasks. Deep learning techniques are now being applied to problems such as search, coding assistants, and chip placement. Due to how widely used these applications are, performance, specifically latency, throughput, and hardware efficiency, is very important. However, achieving high performance in these systems can be challenging. ML systems are run on different types of hardware accelerators (GPUs, TPUs) that have unique performance characteristics. Models are becoming larger and larger, and even with access to the most powerful hardware, systems must manage memory bandwidth and network bandwidth carefully when doing training and inference. Additionally, organizations deploying these large-scale ML systems tend to worry about more than just performance: they must worry about other factors such as energy usage, debuggability, and easy