Product for Evaluating Decision Rules
By Murat Unal in Profit Management Causal Inference Causal Machine Learning
December 30, 2022
Summary
The Automated Profitability Management Team owns the systems Amazon uses to optimize vendor terms. Our team develops systems that discover negotiation opportunities and automatically engage in them. As new systems are being developed we need a scientific way of evaluating them so that we can focus only on the most promising ones.
To understand the incremental value of any new developed system we typically need to run online experiments that can take several months. In this project, I designed and implemented an evaluation framework that finds the incremental value of any new developed system offline without the need of running time consuming experiments.
The framework leverages ideas from economics and reinforcement learning, and enables fast, scalable and reliable offline comparison of decison systems.
As shown in the figure above, the findings from this project empowered our team in directing our resources to system i.
- Posted on:
- December 30, 2022
- Length:
- 1 minute read, 145 words
- Categories:
- Profit Management Causal Inference Causal Machine Learning