The Automated Profitability Management Team owns the systems Amazon uses to optimize vendor terms. Our team develops systems that discover negotiation opportunities and automatically engage in them. As new systems are being developed we need a scientific way of evaluating them so that we can focus only on the most promising ones.
To understand the incremental value of any new developed system we typically need to run online experiments that can take several months. In this project, I designed and implemented an evaluation framework that finds the incremental value of any new developed system offline without the need of running time consuming experiments.
The framework leverages ideas from economics and reinforcement learning, and enables fast, scalable and reliable offline comparison of decison systems.
As shown in the figure above, the findings from this project empowered our team in directing our resources to system i.
- Posted on:
- December 30, 2022
- 1 minute read, 145 words