To the old saying that three can keep a secret provided two of them are dead, we can add that three can keep data organized provided three of them are dead. Our ability to create and store data far outstrips our ability to organize and find data.  

Search, or the fine art of finding what we’re looking for, is only partially solved. No matter how clever our storage, organization, and indexing, the limits of human cognition mean that methods for organizing information so it can be easily found are always overwhelmed: email categories seemed like a good idea when email first became popular, but our brains can only hold onto as many as nine categories before they blur together. Keywords work for a while, but it doesn’t take long before the sheer volume of information means too many useless results (and that’s not even getting into the problem of “guess the right keywords”). Clever algorithms and heuristics all help, but still put a large burden on human cognition.   

Enterprise Search Is a Nightmare  

Information resides in a great many places: databases, email, calendars, Slack, Webex, Confluence, and other information black holes. Finding the right email or a conversation from a few weeks or months ago is time-consuming and exhausting.   

Some companies might have moderately well-organized wikis, but many do not. And even the most well-organized wikis quickly balloon to a size that makes it difficult to find relevant information.   

It’s no better for those trying to make information available. Letting other people know that you have useful or important information devolves into the online equivalent of yelling loudly. It’s like being at a party: it might start relatively quiet, but as more people start talking, each one raises their voice just a little in order to be heard, which triggers another round of volume increases, and so on. Eventually no one can hear anything.   

The rate at which new information is being created is only accelerating. A knowledge worker can spend easily 8-10 hours per week just hunting for the right data (Bono still hasn’t found what he’s looking for).  

Changing the Paradigm  

You don’t have to climb the highest mountain or run through the fields to realize that we need a different paradigm if we’re going to quickly, easily, and reliably find what we’re looking for. In the realm of enterprise search that new paradigm is where AI Large Language Models (LLMs) come in.   

If you know what you’re looking for, for example the lyrics to a specific song, that’s easy. It’s clear what you want. But as the search becomes more abstract and complex, it becomes harder for a search engine to figure out what you really want. The benefit of LLMs is that figuring out what you want is what they do.   

However, AI search is not a magic fix. AI search can produce errors, hallucinations, and bizarre results (such as Google famously telling people to put glue in pizza to keep the cheese from sliding off). Moreover, enterprise data is increasingly distributed and in a variety of formats (which can include pretty much any data format and technology developed in the past 40 years). Centralizing that information so an AI can process it is not feasible from a technology, cost, or security perspective.  

AI-enhanced search is an important step in changing the enterprise search paradigm, but it doesn’t get us all the way there.  

What’s The Missing Piece?  

So how do you find the perfect pizza recipe or anything else that you’re looking for? The big problem with existing enterprise search solutions has always been that ultimately sorting through results and determining meaning has fallen to human cognition. That was fine when the amount of information was small and manageable, but “small and manageable” is long past. The secret sauce to effective search is combining the best aspects of traditional search—and its decades of work on ranking and sorting results—with the ability to query all available data sources and the ability of LLMs to search by meaning. AI infrastructure software such as SWIRL provides that combination of search techniques (aka metasearch).  

AI Infrastructure Software in Action  

Beyond metasearch, SWIRL provides a framework for managing multiple AI models. SWIRL supports universal connectivity to databases, email, productivity apps (e.g. Slack, WebEx, Teams), and more. Data stays in place and the infrastructure software brings the AI to the data.   

In addition, AI infrastructure software provides: 

• Ability to connect to your choice of AI models 
• AI-enhanced natural language search capabilities 
• Enhanced prompt engineering 
• Retrieval Augmented Generation (RAG) without a vector database 
• Context aware relevancy ranking 
• Personalized results 
• Scalability 

What AI infrastructure software does not do is invent some new—and immediately obsolete—way of organizing your data as a precursor to search. Rather, the software harnesses AI to find relevant information and organize it on the fly before presenting the results to you.   

With SWIRL, our ability to find information is rapidly overtaking our ability to create and store information. We can finally find what we’re looking for.  Learn more about AI infrastructure software-based metasearch. Contact SWIRL today.