Using embeddings is one of the key ingredients for vector search and Retrieval Augmented Generation (see Retrieval Augmented Generation ). The Vectorizer stage uses an embedding provided by a large language model to generate a vector representations for the given document.

In particular, two vectors will be generated. One for the title field and one for the document contents.

The resulting vector(s) will later-on be indexed along with the “normal” document metadata and contents. This means, the vectors will be added as additional fields to the document before indexing.

Configuration Parameters

1. See the parameter description at LLM Specific Configurations - Content Transformers