Storage
LanceDB provides flexible storage backends that support both cloud object storage and local high-performance storage for different deployment scenarios.| Feature | Description | OSS | Cloud | Enterprise |
|---|---|---|---|---|
| Object, File, Block Storage | Support for AWS, GCS, Azure and S3-compatible vendors. | ✅ | ✅ | ✅ |
| Local SSD/NVMe Storage | Support for storage on customer’s custom servers. | ✅ | ✅ |
Tables
LanceDB’s table abstraction provides ACID-compliant data management with schema evolution, versioning, and consistency guarantees for vector and scalar data.| Feature | Description | OSS | Cloud | Enterprise |
|---|---|---|---|---|
| Tables - CRUD Operations | Basic API to create, read, update, drop tables. | ✅ | ✅ | ✅ |
| Tables - Data Evolution | Alter column schema, datatype, backfill + merge data | ✅ | ✅ | ✅ |
| Tables - Versioning | Append, overwrite, check versions + tag them. | ✅ | ✅ | ✅ |
| Tables - Consistency | Synchronize database with underlying storage. | ✅ | ✅ | ✅ |
Ingestion
LanceDB’s ingestion pipeline handles both vector embedding generation and data loading with support for multiple formats and efficient batch operations.| Feature | Description | OSS | Cloud | Enterprise |
|---|---|---|---|---|
| Embedding - Text Data | Generate vector embeddings from text data using various embedding models. | ✅ | ✅ | ✅ |
| Embedding - Multimodal Data | Generate embeddings from images, audio, and other multimodal content. | ✅ | ✅ | ✅ |
| Embedding - CPU & GPU Device Configuration | Configure CPU or GPU acceleration for embedding generation performance. | ✅ | ✅ | ✅ |
| Embedding - Environment Variables | Manage API keys and configuration for embedding model access. | ✅ | ✅ | ✅ |
| Data Ingestion - Default | Formerly called Adding Data to a Table. | ✅ | ✅ | ✅ |
| Data Ingestion - Formats | Pandas, Polars, Pyarrow, Pydantic | ✅ | ✅ | ✅ |
| Data Ingestion - Upsert | Update existing records or insert new ones based on key. | ✅ | ✅ | ✅ |
| Data Ingestion - Merge Insert | Combine data from multiple sources into a single table. | ✅ | ✅ | ✅ |
Indexing
LanceDB’s indexing system provides multiple vector and scalar index types with automated optimization for fast similarity search and retrieval operations.| Feature | Description | OSS | Cloud | Enterprise |
|---|---|---|---|---|
| Vector Index - IVF_FLAT | Minimal index that looks at IVF partitions, instead of brute forcing. | ✅ | ✅ | ✅ |
| Vector Index - IVF_PQ | Default vector index using Euclidean distance. | ✅ | ✅ | ✅ |
| Vector Index - IVF_SQ | IVF index built using scalar quantized vectors. | ✅ | ✅ | ✅ |
| Vector Index - IVF_HNSW_SQ | HNSW built on IVF’s partitions + vectors that are scalar quantized. | ✅ | ✅ | ✅ |
| Vector Index - Binary | IVF_FLAT with Hamming distance for binary vectors. | ✅ | ✅ | ✅ |
| Scalar Index | BTREE, BITMAP, LABEL_LIST | ✅ | ✅ | ✅ |
| Automated Indexing | Indexing happens in the background no config. | ✅ | ✅ | |
| Bypass Automated Indexing | When you want to search over all available vectors. | ✅ | ✅ | ✅ |
| Reindexing - Manual | User needs to specify that they want to reindex. | ✅ | ✅ | ✅ |
| Reindexing - Automated | Reindexing happens in the background no config | ✅ | ✅ | |
| GPU Indexing - Manual | User needs to specify which indexing device to use. | ✅ | ✅ | ✅ |
| GPU Indexing - Automated | Indexing device is automatically set for user. | ✅ | ✅ | |
| Full Text Search Index | Inverted index | ✅ | ✅ | ✅ |
Search
LanceDB’s search capabilities combine vector similarity search, full-text search, and hybrid approaches to provide comprehensive retrieval functionality across different data types.| Feature | Description | OSS | Cloud | Enterprise |
|---|---|---|---|---|
| Vector Search - No Index | Goes through all the available vectors. | ✅ | ✅ | ✅ |
| Vector Search - ANN Index | Retrieves top K similar vectors. | ✅ | ✅ | ✅ |
| Vector Search - Multivectors | Late interaction vector search. | ✅ | ✅ | ✅ |
| Vector Search - Distance Range | Search for vectors within a specific distance threshold. | ✅ | ✅ | ✅ |
| Vector Search - Binary Vectors | Search using binary vector representations for efficiency. | ✅ | ✅ | ✅ |
| Vector Search - Filtering | Apply scalar filters during vector search operations. | ✅ | ✅ | ✅ |
| Vector Search - Batch API | Process multiple search queries in a single request. | ✅ | ✅ | ✅ |
| Vector Search - Async Indexing | Fallback brute force for fast performance. | ✅ | ✅ | |
| Full Text Search - FTS Index | Inverted Index | ✅ | ✅ | ✅ |
| Full Text Search - Tokenizer | Ngram and other common methods of splitting text data. | ✅ | ✅ | ✅ |
| Full Text Search - Scalar Index | BTREE, BITMAP, LABEL_LIST for non-vector data. | ✅ | ✅ | ✅ |
| Full Text Search - Fuzzy Search | Searching when there is a typo on the query. | ✅ | ✅ | ✅ |
| Full Text Search - Prefix Matching | Search for text that starts with specific characters. | ✅ | ✅ | ✅ |
| Full Text Search - Score Boosting | Increase relevance scores for specific terms or fields. | ✅ | ✅ | ✅ |
| Full Text Search - Boolean Logic | Use AND, OR, NOT operators in text search queries. | ✅ | ✅ | ✅ |
| Full Text Search - Array Fields | Search within array or list data types. | ✅ | ✅ | ✅ |
| Hybrid Search - FTS Index | Combine vector and full-text search in single query. | ✅ | ✅ | ✅ |
| Hybrid Search - Reranking | Reorder search results using additional ranking models. | ✅ | ✅ | ✅ |
| SQL Queries | Execute standard SQL queries on LanceDB tables. | ✅ | ✅ | ✅ |
| Query Optimization | Explain query plan, analyze query plan, optimization config settings. | ✅ | ✅ | ✅ |
Filtering
LanceDB’s filtering system provides flexible query capabilities that can be applied independently or in combination with vector and full-text search operations.| Feature | Description | OSS | Cloud | Enterprise |
|---|---|---|---|---|
| Filtering - no Vector Search | Apply filters without vector search operations. | ✅ | ✅ | ✅ |
| Filtering - Vector Search | Apply filters during vector search operations. | ✅ | ✅ | ✅ |
| Filtering - Full Text Search | Apply filters during full-text search operations. | ✅ | ✅ | ✅ |