From Raw HTML to AI-Ready Data: The Invisible Work of Normalization

How is an AI-ready dataset built? How Senkrondata normalizes scattered product pages into one consistent, structured dataset.

A product might be listed as "500 ML Shampoo" on site A, "0.5 L Shampoo" on site B, and "500ml — Shampoo (Family Size)" on site C. One site shows its price as "149.90 TRY," another as "₺149.90." A human reading these sees the same thing; but to a database, a BI tool, or a language model, these three strings are three different things — until they're normalized.

An AI-ready dataset isn't a direct output of raw HTML. The difference is an invisible but critical layer: normalization.

Why raw data isn't "ready"

Every site has its own design and writing conventions: different unit notations (ml/L, g/kg), different currency formats, different decimal separators, different category naming, stray whitespace/characters baked into the HTML. On top of that, the same piece of information sometimes arrives in a structured field (a "brand" box) and sometimes buried in free text.

Feed this raw form directly into an analytics tool or a language model, and it spends most of its time wrestling with "do these two strings say the same thing?" instead of doing the actual job.

Normalization: manufacturing consistency

The normalization layer reduces the different representations from each source into a common schema:

• Unit and currency standardization: "0.5 L" and "500 ML" are converted to the same numeric value, the same unit.
• Field extraction: information buried in free text — brand, color, size — is pulled out into separate fields (as far as possible; in some categories this stays partial).
• Category mapping: each site's own category tree is mapped to a shared taxonomy.
• Cleanup: stray whitespace, leftover HTML, encoding errors are removed.

The result is a set of records with the same shape, regardless of which source they came from — which makes it ready for the next step: being consumed by a machine.

What "AI-ready" actually means

For a dataset to be "ready" for AI/BI tools requires three things:

1. A consistent schema: every record has the same fields, of the same types — a language model or a query engine knows what to expect, field by field.
2. No loss of context: normalization simplifies without deleting the original information; "500 ML" gets converted to a standard unit while the original source phrasing is preserved.
3. Traceability: which source each normalized field came from, and when, is known — needed for both auditing and debugging.

With all three in place, the dataset isn't just a table a human can read — it's a foundation a model or a reporting engine can build on with confidence.

The principles that keep it trustworthy

• Normalization standardizes form, it doesn't lose information. The original text always stays accessible; a consistent layer is just added on top.
• Missing data is marked "missing," not guessed. If a field couldn't be extracted, it's left blank rather than silently filled with a default — filling it would create a false sense of certainty.
• The schema evolves with sources but doesn't break backward. A new field can be added; fields existing consumers (reports, models) expect don't suddenly disappear.

The bottom line

The difference between raw HTML and an AI-ready dataset is mostly an invisible normalization layer — the work of translating units, currencies, categories, and attributes into a common language, producing consistency without losing information. The stronger that layer is, the more trustworthy every analysis, every report, every model built on top of it becomes.

If you want raw data turned into a trustworthy, AI-ready dataset, talk to the Senkrondata team.