Why your AI can't understand your data

Everyone's talking about AI-powered analytics. The promise is compelling: point an AI at your data and get instant insights. No more waiting for reports. No more relying on overworked analysts. Just ask a question and get an answer.

There's just one problem. It doesn't work.

When large language models query raw enterprise data, they get it wrong roughly 80% of the time. That's not a typo. Four out of five queries return incorrect results. For finance teams making decisions worth millions, those odds aren't just disappointing — they're dangerous.

In this article, we'll explain why AI struggles with raw data, what it actually needs to deliver reliable insights, and how forward-thinking organisations are solving this problem.

The 80% failure rate nobody talks about

Recent research from AtScale tested a leading AI model against a standard analytics benchmark. Without business context, the model achieved just 20% accuracy. Questions that seem simple to a human — like 'What was the sum of web sales for each product brand in 2022?' — consistently produced wrong answers.

The AI wasn't broken. It was missing something fundamental: an understanding of what the data actually means.

Think about your own data. You have tables with columns like 'cat_id2' or 'cust_status_flag'. Your team knows that cat_id2 = 'FW' means Footwear and that status_flag = 'A' means Active. But an AI model? It sees meaningless codes. It can't infer that your 'revenue' column excludes VAT, or that 'active customer' means someone who purchased in the last 90 days.

This isn't a data quality problem. Your data might be perfectly clean and well-structured. The problem is that clean data isn't the same as understood data.

Why cleaning your data isn't enough

Most organisations preparing for AI focus on data quality: removing duplicates, fixing inconsistencies, consolidating sources. This work is essential. But it only solves half the problem.

AI models need two types of knowledge to query your data accurately. The first is explicit knowledge — schema structure, column names, table relationships. Modern AI handles this reasonably well. The second is implicit knowledge — the business meaning behind the data. This is where AI falls apart.

Consider a simple question: 'Show me revenue by region for Q3.' To answer this correctly, the AI needs to know which column contains revenue, whether it's gross or net, which date field defines the reporting period, how quarters are calculated in your fiscal year, and what constitutes a 'region' in your business. Without this context, the AI guesses. And guessing wrong 80% of the time is worse than having no AI at all.

The missing layer between data and AI

The solution isn't more powerful AI models. It's giving AI the business context it needs to understand your data. This is where semantic models come in.

A semantic model sits between your raw data and your AI tools. It defines your business logic in a way machines can understand: what 'revenue' means, how 'active customer' is calculated, which tables join together and why. Think of it as a translation layer that converts your organisation's tribal knowledge into something AI can actually use.

When AtScale repeated their benchmark test with a semantic layer in place, accuracy jumped from 20% to near-perfect. The same AI model. The same data. The only difference was giving the AI the context it needed.

This isn't surprising when you think about it. You wouldn't expect a new analyst to produce accurate reports on day one without any onboarding. They need to learn your definitions, your calculations, your business rules. AI is no different — it just learns from a semantic model instead of sitting in meetings.

What this means for your AI strategy

If you're planning to deploy AI-powered analytics, the order of operations matters. Many organisations rush to implement AI tools before their data foundations are ready. They end up with expensive technology that delivers unreliable results.

The smarter approach is to build in layers:

• Consolidate and cleanse: Bring your data together into a single, governed source of truth.
• Define your business logic: Create a semantic layer that captures your metrics, calculations, and definitions.
• Layer AI on top: Now your AI tools have the context they need to deliver accurate, trustworthy insights.

This approach doesn't just improve accuracy. It also makes AI more accessible. When business definitions are captured in a semantic model, users can ask questions in plain English without needing to understand database structures. The semantic layer handles the translation.

How 5Y approaches AI readiness

At 5Y, we've always believed that transformation starts with data foundations, not technology. That's why our Business Transformation Platform focuses on consolidating fragmented data sources into a single, governed model before any AI is applied.

We embed business logic directly into the platform — your KPIs, your calculations, your definitions. When our AI assistant, Myles, answers a question, he's not guessing at what 'revenue' means. He knows, because the semantic context is built into the foundation.

This approach delivers something most AI tools can't: consistency. Ask the same question twice, get the same answer. Ask it in different ways, get equivalent results. That's the difference between AI as a novelty and AI as a business tool.

Getting started

AI-powered analytics is genuinely transformative — when it works. The gap between promise and reality comes down to preparation. Organisations that invest in data foundations and semantic context will unlock AI's potential. Those that skip these steps will struggle with unreliable results and eroding trust.

If you're evaluating your AI readiness, start by asking a simple question: does your organisation have a single, documented definition for your core metrics? If the answer is no — or if different teams use different calculations — that's your starting point.

Book a demo to see how 5Y can help you build the data foundations that make AI actually work.