Machine Learning System Design Interview

ML system design interviews are a close cousin of regular system design interviews. You are given a business problem, need to determine requirements, and present a high level overview of your solution. It’s no surprise, then, that the author of the most popular book on system design interviews System Design Interview would write a book on ML system design interviews—in this case, coauthoring with an ML expert.

In the first chapter, the book provides a framework for how to answer ML system design interviews before going through 10 examples. The framework is simple and intuitive. Unlike System Design Interview, Machine Learning System Design Interview doesn’t give a quick primer before diving into the example questions. While ML nowadays is a massive field, too much for any 1 book or person to cover, the algorithms and models are essential to any solution. Changing the model can change the rest of the system. For example, since 2022ish, it has become possible to use prompts and LLMs to do fraud detection as opposed to training xgboost/random forest models. Taking the LLM route totally changes the data you need (text vs. numerical features), data engineering (data lake vs structured tables), inference (guardrails vs no guardrails), infrastructure (GPU instances vs. CPU instances), etc. The book partially makes up for this by explaining some ML concepts in the chapters.

My other gripe with the book is the overrepresentation of search/recommendation style problems. Only 3 of the chapters—”Google Street View Blurring System”, “Harmful Content Detection”, and “People You May Know” (arguably “Ad Click Prediction on Social Platforms” as well)—are about other types of problems. Problems such as content generation, robotics, and chatbots are missing. Otherwise, Machine Learning System Design Interview is a great book that anyone preparing for ML system design interviews should read¹.