Five lessons from Amazon’s supply chain optimization technologies organization

Every time you browse the Amazon Store, optimization models developed by Amazon’s Supply Chain Optimization Technologies (SCOT) organization work behind the scenes to make sure that products from Earth’s widest selection are in stock, and that you are able to receive your orders in the promised time windows at the lowest possible cost.

This is no easy feat.

There’s the sheer scale involved—at Amazon re:MARS 2019, the company revealed that it offered hundreds of millions of products in 185 countries across the world. To compound matters further, Amazon offers a variety of delivery speeds for customers. For example, US Prime customers typically receive orders within a one-day delivery window, while Prime Now customers expect their orders to be fulfilled in as little as a few hours.

The same pool of inventory is used to serve demand for orders with different shipping speeds. Consider a box of formula. It could be used to fulfill a Prime order. It could also be sent to parents who have placed an urgent order via Prime Now.

Given that most fulfillment centers in urban areas have extremely limited space for inventory, SCOT’s inventory system must also make predictive tradeoffs on where to stock every item. Should a unit of a particular product be stocked in a fulfillment center in New York City? Or should it be placed at a more “upstream” exurb location that is farther out?

The system must make these decisions in advance of customer demand, even in the face of extreme unpredictability—after all, who could have predicted the increase in demand for loungewear post-COVID-19? Furthermore, many of these decisions related to inventory routing must be made in real time, even as products make their journey from vendors to customer doorsteps.

Since 2011, SCOT had been on a journey to completely automate inventory buying and placement decisions. And yet, midway through the last decade, the organization made a remarkable decision. To help deliver on growing customer expectations for faster deliveries, and to manage an increasingly complex and layered fulfillment network, Amazon would completely redesign its automated inventory management system.

In short: they would replace a system that was still industry-leading in supply chain automation with a system that would work great for the future. They called the new solution multi-echelon inventory planning version 2 (or MEPv2 for short).

“As far back as 2016, we were able to see that the automated systems we had at the time wouldn’t help us meet the ever-growing expectations of our customers and the complexity of our growing network,” Deepak Bhatia, vice president of SCOT, recalled. “As a result, we made the decision to redesign our supply chain decision systems from the ground up.”

Today, MEPv2 is helping buy and place hundreds of billions of dollars in inventory around the world. The new inventory planning and control system is a testament to how optimization and machine learning models can be deployed by businesses to automate and transform their operations, while continuously driving innovation in their operations.

Here are five lessons from Amazon’s SCOT organization on how you can continue to drive innovation while running a highly automated business.

1. Consciously cultivate the appetite for taking risk

“Being involved in the design of the new inventory planning and control system was one of the most rewarding experiences of my career,” says Yan Xia, principal applied scientist at Amazon.

When the system was deployed in 2020, Xia received a promotion as recognition for his work in the project.

“To be completely honest, I would probably have been promoted even if I had failed,” Xia says. “We encourage risk-taking at SCOT, and don’t look at failures as setbacks, as long as we learn something from them.”

Xia gives the example of one of the early in-production systems (aka labs) that the team developed to demonstrate the efficacy of the new model. This lab was conducted two years after the commencement of the project.

“It was a disaster,” Xia says. “Systems didn’t work as expected. Things began to break.”

Xia remembers presenting the results to leadership.

“I remember Deepak’s reaction to this day,” he says. “Deepak pointed out that SCOT had taken three years to deploy the first automated system. He said that failure was expected, and that he would have been surprised if we hadn’t failed. He asked us to keep calm and carry on.”

2. To drive disruptive change, push from both the top and the bottom

Salal Humair, Vice President and Distinguished Scientist, SCOT, compares the move to the new inventory planning system to changing the engines of a plane midflight.

“We already had this complex system responsible for making billions of dollars of inventory decisions,” Humair says. “And yet, it was impossible to ignore the headwinds: Amazon was expanding globally, and customers around the world wanted faster deliveries.”

Humair says that it would have been only too easy to ignore these headwinds, and to get trapped in the “tech burden,” which occurs when organizations invest so much time and resources in developing a technological ecosystem that they become resistant to change.

Humair stresses the importance of driving change from both the top down and bottom up. He says that it is essential that an organization consciously develops the right leaders, scientists and engineers to accomplish this difficult task.

“If you rely on a top-down strategy, then the strategic vision can get diluted as it makes its way down the organization. The result is that people on the frontlines lose the zeal and motivation, as they have no idea why they are doing what they are doing,” says Humair.

“On the other hand, you don’t want a situation that can arise by pursuing an exclusively bottom-up approach, where everyone on the team is working on something new, and by the time it gets to the organization’s leadership, the efforts of the team are not aligned with the broader strategic priorities.”

That is why it is important to make sure that you create a mission-focused group of engineers and scientists who are motivated and vested in driving change,” says Humair. “You also need to keep your leadership in sync, so that the fear of failure does not hold back your front line from forging ahead with the new mandate.”

3. Build trust in change through understandability and explainability

Developing trust in the new system is paramount. Explainability—or understanding why optimization and machine learning models make the decisions they make—is still a largely unsolved problem in the world of artificial intelligence. Explainability allows business decision makers to trust the decisions made by algorithms.

“This is especially important when you own multiple decentralized systems producing a wide array of outputs,” says Humair.

SCOT’s inventory optimization system is made up of multiple models working together to help fulfill customer delivery promises. To give just a few examples, forecasting models developed by SCOT help predict product demand, buying systems tell Amazon how much product to buy from suppliers, while placement systems help inform the best location to place those products.

Humair says that it’s important to audit the aggregate outputs produced by optimization models as well as conduct deep dives on anecdotes to convince stakeholders that the new system can trusted to run the business.

“At Amazon, we define a set of core aggregate business metrics related to customer satisfaction,” says Humair. “When developing MEPv2, not only did we have to ensure that the aggregate metrics improved, but we also did multiple and detailed deep dives at single product levels, to convince stakeholders that the decisions being made by the new systems made sense. It was an essential element of building trust in the new system,” says Humair.

4. Iterate, iterate, iterate, and rely on judgment in addition to data

Disruptive change rarely takes place in one fell swoop. During a profound change, data often fails to provide a clear and unambiguous answer that can help inform the best path forward. As SCOT’s scientists debated the best path to pursue, they realized that they didn’t have the mechanisms to clearly evaluate the models that were on the table.

“Our supply chain is incredibly complex, and there wasn’t a great way to decide between available options,” says Jeff Maurer, vice president of inventory planning and control. “So, what we did was to pick the best options we could identify, built them out, learnt from observations and repeated the process. But the learning from never unambiguous. At every stage, we had to use judgment to decide whether to stay the course or roll back.”

SCOT iterated on the new solution in a sequence of four labs that spanned 2018 to 2020. The first lab incorporated components of the new system coupled with the old platform, while later iterations tested the new system independent of the old one. Many of the labs in the early years failed. Every lab led to a detailed retrospective. This resulting analysis was coupled with judgment to determine if the team was on the right course. It was only when the team had tested the new system for bugs, enabled key functionalities, and proven it in a lab, that the new system went live in 2020.

5. Always take the long-term view

“Chance favors the prepared mind,” says Bhatia. “We had a system that was working. And yet, we were keenly aware that given how fast Amazon’s business was changing, we had to change.”

Bhatia has been involved in the design and deployment of both the initial automated inventory planning and control systems and MEPv2. In each case, the move to a new way of doing things was a multi-year process.

Bhatia quotes a Jeff Bezos shareholder letter, where Amazon’s founder stated the importance of taking a long-term view.

“To drive change, you have to always be able to see the future you are driving for,” says Bhatia. “If you are part of a large organization, driving change is like steering a large ship. You have to be patient and keep pushing at the flywheel. You have to remain patient. Change will happen if you persist.”