Squeezing the last DRiP: AutoML for Cost-constrained Product Classification
Date:
Resources: [Slides]
Abstract: Current AutoML research aims to minimize the Discovery time of high-performing models, e.g. "find best model within 30 mins". However, ideally most models are trained, and used in production for months before refresh, meaning the costs operational cost of running an AutoML model in production far exceeds the one-time discovery cost. Instead, can AutoML systems discover high-performing models which operate within an explicit budget? We propose a new AutoML paradigm, DRiP (Discover-Refine-Productionize) which not only allows cost-backwards optimization, but produces a cost-performance tradeoff curve for users to choose an appropriate point. We compare to AutoGluon v0.2 and find that DRiP AutoML can be tuned to achieve: (i) On-par performance at low cost (ii) Minimum overall cost (iii) Maximum overall performance.