There's a persistent belief in enterprise technology that the solution to fragmented data is centralization. Build a single index. Create one source of truth. Make everything searchable from one place.
It's a compelling vision. It just doesn't work. Not at the scale and pace of change of a real enterprise. Not across systems that evolve independently, have different owners, and enforce different access controls. Not without enormous ongoing investment to keep the index current.
Why the single index always fails
The single index approach fails for three fundamental reasons:
Data moves faster than indexes. Enterprise data changes continuously. Emails arrive. Documents are updated. Policies change. An index built yesterday doesn't reflect today's reality. The more dynamic your data, the worse the gap.
Permissions don't travel with the data. When you copy data into a central index, the access controls that govern who can see what don't automatically follow. Re-implementing them in the index is complex, expensive, and almost always incomplete. The result is either over-permissioning (people can see things they shouldn't) or under-permissioning (people can't find things they're authorized to access).
The diversity is a feature, not a bug. Different systems exist because different systems are better at different things. Salesforce is the system of record for CRM. ServiceNow owns IT workflows. SharePoint governs document management. Trying to extract their data into a unified index strips away the context, metadata, and structure that makes that data meaningful.
Why LLMs vindicate federation
Federated search - querying multiple sources simultaneously at query time, rather than pre-indexing everything - has always been architecturally sound. The rap against it was that the results were too heterogeneous to be useful. Different schemas. Different relevance models. No way to compare apples to oranges across systems.
LLMs solve this. They can contextualize, compare, and synthesize across formats and domains in ways traditional systems never could. A good LLM can take results from a SQL database, a SharePoint collection, and a legal publisher and produce a coherent, ranked, synthesized answer - without requiring those sources to share a schema.
Federation was always the right answer. LLMs are what make it work at enterprise scale.
The shift is already happening
We're watching enterprise search vendors and data platform companies quietly shift their messaging. "Federated" is showing up in more roadmaps. "No data migration" is becoming a selling point rather than a limitation to explain away.
The single index was never the goal. Getting to the right answer was. Federation - with modern LLMs doing the synthesis - is how you get there.