Our Enterprise Search Connectors
Web Pages
Our Web Pages Connector allows for indexing http pages, as well as their metadata. In turn, you can build great enterprise search experiences, knowledge searches or bot applications powered by retrieval augmented generation.
Features
Our Web Pages Connector comes with the following features.
- Indexing of web pages
- Via http and https
- Full set of mail metadata as part of the indexed documents (via meta-flags)
- Optional support for authorization schemes, based on URLs.
- Efficient incremental crawling
- Optional authentication with basic authentication
- Filters based on URLs and file types
- Easy configuration
Search Engines
Our connectors support the following search engines:
- Microsoft Copilot
(Web Pages Connector for Microsoft Copilot)
- Microsoft Search
(Web Pages Connector for Microsoft Search)
- Microsoft Graph Connectors
(Web Pages Connector for Microsoft Graph)
- Azure AI Search
(Web Pages Connector for Azure AI Search)
- Elasticsearch
(Web Pages Connector for Elasticsearch)
- Apache Solr
(Web Pages Connector for Apache Solr)
- Microsoft Cosmos DB H2/2024
(Web Pages Connector for Microsoft Cosmos DB)
Framework Features
- Scalable connector framework for millions of documents
- Easy to setup and configure
- One interface to manage all connectors
- Robust crawl strategies for content and security principals
- Flexible crawl scheduling built-in
- Document and principal processing pipelines
- Debugging and development made easy: State views which show
- crawled documents
- document metadata
- how the metadata became transformed in the processing pipeline
- indexed security principals and their relationships
- timings
- and more
