Virtual Closet

Every apparel recommendation engine has the same blind spot, which is that it knows what you bought but it does not know what you have. Virtual Closet was the attempt to close that gap, and it was a digital version of a customer's actual wardrobe, created automatically from that customer's purchase history, which they could then add to by searching Amazon's catalog or by uploading photos of their own.

The premise was that owning that data unlocks a different class of personalization, and by a different class I do not mean "customers who bought this also bought." I mean outfit ideas made of the things already hanging in your closet, and search results that complement what you already own, and the ability to hide the items that are too similar to something you own already.

The prototype

Rather than argue about it we built it, and so I built a React prototype, using Redux with sagas for the data flow, and that prototype stood up the full loop against real customer data:

• Ingest order history and filter it down to the softlines items that plausibly belong in a wardrobe
• Generate outfits from the resulting closet using the outfit builder service
• Collect structured feedback on both the generated outfits and the curated recommendations, item by item, from real reviewers

The prototype ran over cohorts of real customers with anonymized identifiers, and the whole purpose of the prototype was to produce judgments we could learn from, so there was a closet view with the items in it, and a sidebar for building outfits and rating them, and dedicated feedback surfaces for recommendation quality.

There was also an anonymous mode, which was the same closet interface and the same recommendation-building interface, decoupled from a signed-in identity, so that people could be shown a closet and asked to reason about that closet without accounts getting in the way.

What the research found

The interesting findings were nearly all of them about import logic, which is the unglamorous problem of deciding which purchases are actually yours, and which of them are actually still in your closet.

Purchase history is a noisy proxy for a wardrobe, because purchase history contains items bought for children and items bought for other family members, and items that were returned, and items that were never kept at all. Purchase history also misses things, and a meaningful minority of participants reported items they had genuinely bought on Amazon that never showed up in their closet, and for most of those participants it was a handful of items, and for a few of them it was 10 or more.

The clear conclusion was that automatic import should be conservative, so that it excludes what obviously belongs to somebody else, and then lets people pull those items in by hand from their purchase history if they want them. That conclusion carries a design consequence with it, which is that you need a separate store for the softlines purchases that are deliberately not in the closet, so that the system remembers what it chose to leave out and doesn't keep on re-offering it.

We also learned how wide the range of closet sizes is. A meaningful share of participants estimated 500 or more items in their physical closet, and essentially all of those participants scored high on fashion engagement, and nearly all of them were women. So whatever got built had to hold up at that scale, and not merely at the median.

The system behind it

Getting it ready to ship

The path from prototype to a feature inside the Amazon Shopping app ran through the parts of the work nobody writes about: a full architecture review documenting the upstream and downstream services, the classes of data moving between them, authentication and access control, credentials management, database security, input validation, and log management — plus a security review and a manual code review of the client.

Virtual Closet shipped into the Amazon Shopping app as Amazon Closet.