Towards Secure AI and Enterprise Search: What Swirl Version 3.2.0 Brings to Your Business
![Towards Secure AI and Enterprise Search: What Swirl Version 3.2.0 Brings to Your Business](/_next/image?url=https%3A%2F%2F8ph795.p3cdn1.secureserver.net%2Fwp-content%2Fuploads%2F2024%2F02%2FSEARCH-BOX-TO-FIND-IT-ALL.jpg%3Ftime%3D1725282736&w=2048&q=75)
It’s Friday 1st February, 2023. And Swirl releases version 3.2.0. With this, we continue our march towards a secure, tight, air-gapped enterprise-grade unified search and data access platform, which operates within your enterprise firewall, and with your private enterprise large language model.
Swirl is available in Azure with a 30-day free Trial. Check out this page to know more.
This release includes the first version of the AutomaticPayloadMapperResultProcessor
, RequireQueryStringInTitleResultProcessor
, DropIrrelevantPostResultProcessor
, and NoModQueryProcessor
, along with new SearchProviders for OpenSanctions.org and LittleSis.org. We’ve also added support for running OpenAI in Azure and storing query templates in JSON, plus updated Python, Django, and OpenAI API versions and resolved issues found in previous releases. Finally, the Galaxy UI has been updated to star/favorite results above a particular relevancy score.
We’re seeking anyone interested in testing new Connectors to Collibra, Thoughtspot, and Pinecone DB. Please contact Swirl via email or Slack for access.
New Features
🔹 New AutomaticPayloadMapperResultProcessor
profiles response data to find good strings for Swirl’s title, body, and date_published fields. It is intended for SearchProviders with few (or no) good result_mappings options. It should be placed after the MappingResultProcessor, and the result_mappings field should be blank.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/screenshot-1-1024x314.png)
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/4-1024x576.jpg)
🔹 New NoModQueryProcessor
that only removes leading SearchProvider Tags and does not modify the query terms in any way. It is intended for repositories that allow non-search characters (such as brackets).
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/screenshot-2-1024x198.png)
🔹 New RequireQueryStringInTitleResultProcessor
that drops results that do not contain the query_string_to_provider
in the result title
field. It should be added after the MappingResultProcessor
and is now included by default in the “LinkedIn – Google PSE” SearchProvider.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/screenshot-3-1024x285.png)
🔹 New DropIrrelevantPostResultProcessor
drops results with swirl_score < settings.MIN_SWIRL_SCORE
(set to 500
by default) and results with no swirl_score
.
⚠️ The Galaxy UI will not display the correct number of results if this ResultProcessor is deployed. This will be addressed in a future release.
🔹 Swirl now supports using Azure OpenAI models for RAG processing in addition to OpenAI’s ChatGPT models.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/screenshot-4-1024x227.png)
🔹 The Galaxy UI now highlights results with a swirl_score
above a configurable threshold with a star in the results list. The swirl_score
configuration is available in theminimumSwirlScore
entry of static/api/config/default
, and the default is 100
.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/3-1-1024x576.jpg)
🔹 New SearchProvider.query_template_json
field which stores the query_template
as JSON. The MongoDB SearchProvider has been updated to use this feature.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/screenshot-6-1024x256.png)
🔹 Support for lowercasing of URLs is now available via a new LC_URL
option in SearchProvider result_mappings
.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/screenshot-7-1024x169.png)
🔹 Added support for list-of-list responses from source APIs, where the first list element is the column names.
New Search Providers
🔹 Search the LittleSis free database of “who-knows-who at the heights of business and government” with the new LittleSis.org SearchProvider.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/2-1024x576.jpg)
🔹 Find sanctions targets and persons of interest with the new OpenSanctions.org SearchProvider.
![](https://8ph.795.myftpupload.com/wp-content/uploads/2024/02/1-1024x576.jpg)
🔑 Note: Additional SearchProviders contributed by the Swirl Community may be found in the SearchProviders/untested
directory.
Improvements
🔹 Lists of SearchProviders can be added to the endpoint with copy/paste, and Swirl will load them all.
🔹 When signing in to Swirl via Microsoft OIDC, users are now automatically authorized to the M365 SearchProviders.
🔹 Swirl was validated on Python 3.12.1.
🔹 Swirl now runs on the new Django 5.0.x series Python packages.
🔹 Swirl now uses the latest 1.6.x OpenAI Python API.
🔹 The QueryProcessors
base classes were updated.
🔹 The default AI Summary timeout value can now be overridden with a URL parameter in the Galaxy UI. For example http://localhost:8000/galaxy/?q=gig%20economics&rag=true&rag_timeout=90000
🔹 The Snowflake SearchProvider query_template
was revised.
🔹 The BigQuery SearchProvider no longer dedupes results.
🔹 Improved zero results feedback and logfile messages.
Resolved Issues
🔹 Removed duplicate static/api/config/default
configuration file.
🔹 Removed unused timeout settings (SWIRL_Q_WAIT
and SWIRL_RERUN_WAIT
).
🔹 Fixed Elasticsearch authorization and addressed a certificate error.
🔹 Updated the index_email_elastic.py
script and related documentation.
🔹 Stopwords are no longer highlighted in search results or AI Summaries.
🔹 Various small fixes to Galaxy UI behavior.
For More Information
Please check out this release from our GitHub Page.
Don’t forget to join our Swirl Enterprise Search Community Slack.