Why an ideal ML model is not enough to build a business around ML
In 2021, Zillow — one of the largest real estate marketplaces in the United States — announced a 25% reduction in staff and wrote off $304 million in losses. Following the news, Zillow's stock plummeted.
What went wrong?
A significant part of Zillow's business at that time was built around machine learning technologies that very accurately predicted the current value of real estate. And even though the ML models themselves were good and of high quality, their integration into business processes turned out to be almost catastrophic.
Let's discuss how this came to be.
Zillow has a well-known service called Zestimate, which allows homeowners to track the value of their property in real time. To provide accurate forecasts it uses ML models.
Initially, Zestimate was developed as a mechanism to increase marketplace retention. After all, people usually visit Zillow when they want to buy or sell a home. However, tracking the value of one's home on a regular basis is a perfectly valid monthly use case that can easily become a habit.
In 2018, Zillow decided to launch a new direction of work based on the developments for Zestimate. The marketplace began purchasing homes to resell them at higher prices at a later date.
The value of the new product for users was the ability to close a home sale deal very quickly. After that, Zillow planned to renovate the home and sell it at a markup.
The idea was not original, but Zillow had advantages over competitors: access to capital and highly accurate ML models for predicting home values.
But the business model built around ML did not work out. In 2021, the company announced the closure of its home-buying program, laid off a quarter of its employees, and wrote off colossal losses.
The problem was not so much with the ML models themselves as with how they were integrated into the business. The models were good at evaluating the current value of homes. However, the transactions for buying and subsequently selling homes take time, during which the value of the homes can change significantly. This is exactly what happened in 2021, when the real estate market cooled down due to a series of global economic processes, and the relationships between home characteristics and their values changed. All of this led to approximately two-thirds of the purchased homes being bought at higher prices than their potential selling prices.
What conclusions can be drawn from this story?
The success of a business built around ML depends not only on the machine learning technologies and the quality of the models. It is crucial to properly implement these models into the business. The risk of failure arises from erroneous assumptions that are not related to machine learning.
So, what can you do to avoid such mistakes?
Ask yourself these questions:
If you want to learn how to identify opportunities for using AI and ML in your projects and be able to turn them into working business solutions, join
Yours,
Oleg
