Scalar Indexes in LanceDB

Use scalar indexes for efficient metadata filtering and hybrid searches.

Scalar indexes organize scalar attributes (numbers, categories, timestamps) to accelerate filtering alongside vector data.

Supported scalar index types:

• BTREE: sorted values for binary search—best for high-cardinality columns.
• BITMAP: tracks value presence—ideal for low-cardinality columns.
• LABEL_LIST: optimized for array_contains_any / array_contains_all on List<T> columns.

Choosing the Right Index Type

Data Type	Filter	Index Type
Numeric, string, temporal	<, =, >, IN, BETWEEN, IS NULL	BTREE
Boolean or low-cardinality strings/numbers	<, =, >, IN, BETWEEN, IS NULL	BITMAP
Low-cardinality lists	array_has_any, array_has_all	LABEL_LIST

Scalar Index Operations

1. Build the Index

create_scalar_index returns immediately. Call wait_for_index() to block until indexing completes.

2. Check Index Status

3. Update the Index

Update scalar indexes after inserts/deletes by calling optimize().

Cloud automates optimize, so new data remains queryable without extra steps.

4. Run Indexed Searches

Scalar indexes speed up pure scalar filters:

They also accelerate hybrid searches that combine vector similarity with scalar prefilters:

Advanced: Index UUID Columns

LanceDB supports scalar indexes on UUID columns stored as FixedSizeBinary(16), enabling efficient primary-key lookups.

• Python SDK ≥ 0.22.0-beta.4
• TypeScript SDK ≥ 0.19.0-beta.4

1. Define UUID Type

2. Generate UUID Data

3. Create Table with UUID Column

4. Create and Wait for the Index

5. Perform Operations with the UUID Index