Edo Liberty
Edo Liberty
Ex Director of Research, Head of Amazon AI Labs and Founder, Hypercube
Traditional machine learning mostly focuses on prediction and regression. In the last several years however, machine learning and specifically deep learning, show great promise for ranking, information retrieval, recommendation, and deduplication. These new techniques allow companies to improve real time retrieval and ranking of images, videos, customers, jobs, shopping catalog items, friends, places, and many more. Retrieval and ranking, however, pose new technological, algorithmic, and scientific challenges which are not addressed by open source software dedicated to prediction and regression. These include management of large collections of items, real time retrieval, fast batch scoring, metering and testing, distributed systems management, and many more. As a result, deploying and serving such machine learning models at scale is still a herculean effort even today for highly tech-heavy companies. We will introduce a unified framework for all such problems and argue that it simplifies the workflow significantly while improving the end result.