The Non-Persistence of Memory—Using AI to Manage Information Overload  

Memory is the first thing to go. I can’t recall what comes next (source forgotten). 

As our information diet accelerates from 10 to 50 to 150 pages per day and counting (Three Numbers Spell Burnout and Information Overload Wrecks Performance), it will come as no surprise that information overload also interferes with learning and memory—if you’ve ever read an article, a paper, or an office memo and a few minutes later can’t remember what you just read, that’s a sign of information overload. There’s just too much coming at us too quickly for us to absorb it and hold on to it. 

For fields that are already information intensive, including medicine, law, and I forget the third field (see what I did there?) the additional weight of information overload comes at a high cost in increased human error and inability to keep up with the flood of information. 

In medicine, healthcare providers need to access patient records, stay abreast of current research, and be knowledgeable about different treatment options. While this task is usually fairly routine, information overload makes it more difficult to spot subtle problems, recognize obscure symptoms, or keep track of proliferating treatment options. 

Lawyers, meanwhile, need to track case law, statutes, precedents, internal documents, memos, and letters. Lawyers in a large firm will need to make sure that legal letters being sent out are consistent with previous letters, possibly going back several years. The amount of information that needs to be reviewed and summarized only grows with each passing day. Information overload only makes this process more and more difficult, increasing the possibility of missing something important. 

Bringing AI to Knowledge Management 

The good news is that AI can be used to implement effective enterprise knowledge management, combatting information overload and preventing many normal human errors. However, using AI requires more than just dumping all your data into a vector database or training an AI model on your data—these “standard” solutions are time-consuming, expensive, and risky.  

Information on the internet is created to be found, whereas enterprise data is created to be used. Some of it is in databases, but relevant data is also in email, Slack, WebEx, and Teams. It’s scattered across document files and PDFs. It’s on internal and external websites. While many search tools are good at finding information in many of those locations, none of them can find data in all of those locations.

Moreover, most of the existing AI tools require you to upload your private data to their web portal if you want the AI to search that data (making it your problem to consolidate everything). 

Overall, not a good solution unless you have lots of money to burn and plenty of technical expertise in-house that isn’t doing anything important. And, of course, the iron nerves to not worry about the risks of a vendor data breach.  

Now, if you’ve been reading this series of articles on information overload, you know that this is the point where I tout SWIRL and list all the ways SWIRL leverages AI to help you deal with information overload. But what if you don’t want SWIRL? Well, as much as that would disappoint me, I do acknowledge that could happen. So instead, what follows are the questions to ask about any AI tool or framework that promises to manage information overload.  

Questions to Ask 

The good news is that there are really only five questions we need to ask to find out if a product will help us safely leverage AI and successfully reduce information overload. 

• Does this software leave my data in place? There is simply no need to upload your data to someone else’s website or copy it all into a vector database. A modern AI framework should be able to bring the AI to the data—no matter the location, format, or technology the data is in—and take advantage of RAG to return results you can trust.
• How does this software keep my data safe? Any AI system should integrate with your existing security protocols—such as Enterprise SSO—so that users can only access the data they’re allowed to see.
• How does the software figure out my intent? Intent matters when asking questions—Apple can be a fruit or a company. A good AI system should be able to use contextual cues—such as role, position, search history—as well as interactive dialog to determine intent so that you get results that make sense.
• How effectively are results ranked by relevancy to my query? This is the whole point of using AI to reduce information overload: we want to reduce the amount of irrelevant information we have to sort through and focus on what matters. Any AI system that can’t do this is a waste of time.
• Which AI LLMs can I use? Freedom of AI is vital: different LLMs have different strengths and weaknesses, so being able to use which models fit your needs is crucial to reducing information overload.

And that’s pretty much it. Yes, there are more details we could worry about, but if we can’t get past the five questions, those details are a waste of time at best, a distraction at worst.  

Ask Questions, Get Answers—AI Knowledge Management is Vital 

In the space of a relatively few years we’ve gone from too little information to information overload, and that’s making it ever harder just to remember what we’ve got. Managing overload requires moving from merely having information to having the right information. That’s where AI comes in. Whatever AI system you end up using, make sure that it gives you freedom to choose the right AI for the job, intelligent relevancy ranking, effective intent detection, security, and, above all, leaves you in control of your data.  

To find out more about how SWIRL can help you overcome information overload, don’t forget to contact us.