How I Used AI to Fix Inconsistent Attribute Values at Scale in E‑commerce
When people talk about scaling e‑commerce, they focus on big-ticket engineering challenges: distributed search, real‑time inventory, recommendation engines, and checkout optimisation. But beneath all that sits a quieter, more persistent issue almost every retailer struggles with: attribute values.
Attributes are the backbone of product discovery. They power filters, comparisons, search ranking, and recommendation logic. But in real catalogues, attribute values are rarely clean. They’re inconsistent, duplicated, misformatted, or semantically ambiguous.
Take something as simple as Size. You might see:
Code
["XL", "Small", "12cm", "Large", "M", "S"]
Or Colour:
Code
["RAL 3020", "Crimson", "Red", "Dark Red"]
Individually, these inconsistencies look harmless. But multiply them across over 3 million SKUs, each with dozens of attributes, and the problem becomes systemic. Filters behave unpredictably, search engines lose relevance, merchandisers drown in manual cleanup, and product discovery becomes slower and more frustrating for customers.
This was the challenge I faced as a full-stack software engineer at Zoro, a problem that was easy to overlook but affected every product page.
My Approach: Hybrid AI Meets Determinism
I didn’t want a mysterious black box AI that simply sorts things. Systems like that are hard to trust, debug, or scale. Instead, I aimed for a pipeline that was:
- explainable
- predictable
- scalable
- controllable by humans
The result was a hybrid AI pipeline that combines contextual reasoning from LLMs with clear rules and merchandiser controls. It acts smartly when needed, but always stays predictable. This is AI with guardrails, not AI out of control.
Background Jobs: Built for Throughput
All attribute processing happens in offline background jobs, not in real time. This was not a compromise; it was a strategic architectural choice.
Real‑time pipelines sound appealing, but at e‑commerce scale, they introduce:
- unpredictable latency
- brittle dependencies
- expensive compute spikes
- operational fragility
Offline jobs, on the other hand, gave us:
- High throughput: huge batches processed without affecting live systems
- Resilience: failures never affected customer traffic
- Cost control: compute could be scheduled during low-traffic times
- Isolation: LLM latency never affected product pages
- Consistency: updates were atomic and predictable
Keeping customer-facing systems separate from data-processing pipelines is essential when working with millions of SKUs.
Cleaning & Normalization
Before using AI on the data, I ran a clear preprocessing step to remove noise and confusion. This step may sound simple, but it greatly improved the LLM’s reasoning.
The cleaning pipeline included:
- trimming whitespace
- removing empty values
- deduplicating values
- flattening category breadcrumbs into a contextual string
This ensured the LLM received clean, clear input, which is key to consistent results. Garbage in, garbage out. At this scale, even small errors can lead to bigger problems later.
LLM Service with Context
The LLM wasn’t just sorting values alphabetically. It was reasoning about them.
The service received:
- cleaned attribute values
- category breadcrumbs
- attribute metadata
With this context, the model could understand:
- That “Voltage” in Power Tools is numeric
- that “Size” in Clothing follows a known progression
- that “Colour” in Paints might follow RAL standards
- that “Material” in Hardware has semantic relationships
The model returned:
- ordered values
- refined attribute names
- a decision: deterministic or contextual ordering
This lets the pipeline handle different attribute types without hardcoding rules for every category.
Deterministic Fallbacks
Not every attribute needs AI.
In fact, many attributes are better handled by deterministic logic.
Numeric ranges, unit‑based values, and simple sets often benefit from:
- faster processing
- predictable ordering
- lower cost
- zero ambiguity
The pipeline automatically detected these cases and used deterministic logic for them. This kept the system efficient and avoided unnecessary LLM calls.
Manual vs LLM Tagging
Merchandisers still needed control, especially for business‑sensitive attributes.
So each category could be tagged as:
- LLM_SORT — let the model decide
- MANUAL_SORT — merchandisers define the order
This dual-tag system lets people make the final decisions while AI did most of the work. It also built trust, since merchandisers could override the model when needed without breaking the pipeline.
Persistence & Control
All results were stored directly in a Product MongoDB database, keeping the architecture simple and centralised.
MongoDB became the single operational store for:
- sorted attribute values
- refined attribute names
- category‑level sort tags
- product‑level sortOrder fields
This made it easy to review changes, override values, reprocess categories, and sync with other systems.
Search Integration
Once sorted, values flowed into:
- Elasticsearch for keyword‑driven search
- Vespa for semantic and vector‑based search
This ensured that:
- filters appeared in logical order
- Product pages displayed consistent attributes
- search engines ranked products more accurately
- Customers could browse categories more easily
Search is where attribute sorting is most visible, and where consistency matters most.
Architecture Overview
To make this work across millions of SKUs, I designed a modular pipeline built around background jobs, AI reasoning, and search integration. The architecture diagram below captures the full flow:
- Product data enters from the Product Information System
- The Attribute Extraction Job pulls attribute values and category context
- These are passed to the AI Sorting Service
- Updated product documents are written into the Product MongoDB
- The Outbound Sync Job updates the Product Information System with the sort order
- Elasticsearch and Vespa Sync Jobs push sorted data into their respective search systems
- API Services connect Elasticsearch and Vespa to the Client Application
This flow makes sure that every attribute value, whether sorted by AI or set manually, is reflected in search, merchandising, and the customer experience.
The Solution in Action
Here’s how messy values were transformed:
| Attribute | Raw Values | Ordered Output | |----|----|----| | Size | XL, Small, 12cm, Large, M, S | Small, M, Large, XL, 12cm | | Color | RAL 3020, Crimson, Red, Dark Red | Red, Dark Red, Crimson, Red (RAL 3020) | | Material | Steel, Carbon Steel, Stainless, Stainless Steel | Steel, Stainless Steel, Carbon Steel | | Numeric | 5cm, 12cm, 2cm, 20cm | 2cm, 5cm, 12cm, 20cm |
These examples show how the pipeline combines contextual reasoning with clear rules to create clean, easy-to-understand sequences.
Why Offline Jobs Instead of Real‑Time Processing?
Real‑time processing would have introduced:
- unpredictable latency
- Higher computing costs
- brittle dependencies
- operational complexity
Offline jobs gave us:
- batch efficiency
- asynchronous LLM calls
- retry logic and error queues
- human review windows
- predictable compute spend
The trade-off was a small delay between data ingestion and display, but the benefit was consistency at scale, which customers value much more.
Impact
The results were significant:
- Consistent attribute ordering across 3M+ SKUs
- Predictable numeric sorting via deterministic fallbacks
- Merchandiser control through manual tagging
- Cleaner product pages and more intuitive filters
- Improved search relevance
- Higher customer confidence and conversion
This was not just a technical win; it was also a win for user experience and revenue.
Lessons Learned
- Hybrid pipelines outperform pure AI at scale. Guardrails are important.
- Context dramatically improves LLM accuracy
- Offline jobs are essential for throughput and resilience
- Human override mechanisms build trust and adoption
- Clean input is the foundation of reliable AI output
Final Thought
Sorting attribute values sounds simple, but it becomes a real challenge when you have to do it for millions of products.
By combining LLM intelligence with clear rules and merchandiser control, I transformed a complex, hidden issue into a clean, scalable system.
It’s a reminder that some of the biggest wins come from solving the boring problems, the ones that are easy to miss but show up on every product page.
\n \n \n
You May Also Like
Uniswap stemt in met token burn van 100 miljoen UNI
Which Crypto to Buy Today for Long-Term Growth? One DeFi Crypto Stands Out
