This series of blogs will give you a security overview of Swirl.
What is Swirl Search?
Swirl is an open-source search platform software that simultaneously searches multiple content sources and returns AI-ranked results. You can also use Generative AI Models to get answers based on your data. It’s written in Python.
Swirl can connect to various data sources like:
- Databases (SQL & NoSQL, Google BigQuery)
- Public data services (Google Programmable Search, Arxiv.org, etc.)
- Enterprise sources (Microsoft 365, Jira, Miro, etc.)
And generate insights with AI and LLMs like ChatGPT. Start discovering and developing the answers you need based on your data.
Swirl is a powerful search tool comprising two main parts: a metasearch engine and a content transformation process.
Metasearch Engine: This is like a super-search tool. Instead of searching just one place, it simultaneously sends user search requests to multiple sources. These sources can be search engines, databases, or cloud services. It communicates with these sources using their built-in methods (APIs) and follows standard security practices, such as OAUTH2. Once Swirl gets the search results from all these places, it uses advanced language models to organize and prioritize them, ensuring the most relevant results are shown first.
Content Transformation Process: This is an enhancement to the metasearch engine. This process can fetch the actual documents or content from the search results when a search is done. It can also add extra information or modify the content to make it more useful for specific purposes. Some of the things it can be used for include:
- Sending notifications or pushing information to users.
- Assisting in machine learning projects.
- Improving the performance of AI that generates content (known as Retrieval Augmented Generation or RAG).
- Refining the way generative AI models work.
In essence, Swirl helps you quickly find information from multiple sources and can fetch and enhance that information for various advanced applications.
What does Swirl do?
As shown below, Swirl:
- Accepts a query via Adapts the query for each pre-configured SearchProvider
- Asynchronously sends the adapted query to each, waits for the response, and normalizes the responses.
- Stores the results in the configured database
- Uses the included LLM (Large Language Model) to re-rank the results
- Returns the best results from all sources
Not shown: Any search can be put into subscribe mode, causing Swirl to re-run the search, looking for new results on a configured schedule.
The following is an overview of Swirl's working.
