Documentation

Search Engines

The RheinInsights Retrieval Suite offers to index content sources into any of the following search engines. It also offers secure search for all of these search engines.

On the other hand, you can use the RheinInsights Retrieval Suite’s query processing pipelines to integrate this tailored secure search directly into your bot application or your enterprise search. Also it offers a built-in enterprise search interface with integrated secure search based on the indexed data.

Secure Search

All search engines which are supported by the Suite can be used to build a secure search experience. Here, we use the concept of early-binding security trimming. This works as follows:

  1. Almost all of our connectors (besides Git) enrich the indexed documents with allow and deny ACL fields.

  2. Furthermore the connectors come with a separate crawl mode to index user-group relationships into the search engine’s security store.

  3. At search time, this information can be combined. A user authenticates against the bot or against the search interface and the query pipelines enrich the query with a filter which acts on the allow and deny ACL fields from Step 1.
    In more detail, the query pipeline first retrieves all groups, the user belongs to, and constructs this filter. In turn, the search engine only returns the results, which match the ACL tokens of the user.

The respective search engine pages below, describe in more detail, how you can also leverage early binding security trimming even without using the RheinInsights Suite’s query pipelines.

Vector Search

Using vector search needs the following ingredients:

  1. A large language model which offers so-called embeddings, i.e., a transformation algorithm which converts texts into vectors.
    Here, multiple large language models come in question and the RheinInsights Retrieval Suite’s content transformation allows for calling embedding algorithms before sending the documents to the search engine for indexing.

  2. A search engine, which offers a vector index. This means that after vectorization of the document contents, the vector must be also indexed in search.