Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ opencontext/

5. **Storage Layer** (`storage/`)

- Multi-backend support (SQLite, ChromaDB)
- Multi-backend support (SQLite, ChromaDB, Qdrant, and Milvus)
- Vector storage for similarity search
- Unified storage interface

Expand Down Expand Up @@ -379,6 +379,33 @@ capture:
- `prompts_en.yaml`: English prompt templates
- `prompts_zh.yaml`: Chinese prompt templates

#### Milvus vector backend

Milvus is available as an optional vector backend. Install the extra before selecting it:

```bash
uv sync --extra milvus
```

Then replace the default ChromaDB vector backend in `config/config.yaml`:

```yaml
- name: "default_vector"
storage_type: "vector_db"
backend: "milvus"
config:
vector_size: 2048 # Must match the embedding model output dimension
uri: "${MILVUS_URI:./persist/milvus.db}"
token: "${MILVUS_TOKEN:}"
collection_prefix: "opencontext"
consistency_level: "Session"
```

The same `uri` setting supports Milvus Lite database files, remote Milvus servers such as
`http://localhost:19530`, and Zilliz Cloud endpoints. Set `MILVUS_TOKEN` for authenticated
servers. Milvus Lite is not supported on Windows; Windows users must configure an `http://`
or `https://` remote URI. ChromaDB remains the default backend.

### Running the Server

```bash
Expand Down
21 changes: 20 additions & 1 deletion config/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ processing:
storage:
enabled: true
backends:
# Vector database configuration - Choose either ChromaDB or Qdrant
# Vector database configuration - Choose ChromaDB, Qdrant, or Milvus
# ChromaDB (default)
- name: "default_vector"
storage_type: "vector_db"
Expand Down Expand Up @@ -174,6 +174,25 @@ storage:
# # https: false
# # api_key: "${QDRANT_API_KEY}" # Optional: API key for authentication

# Milvus (alternative vector database)
# Install first with: uv sync --extra milvus
# Uncomment to use Milvus instead of ChromaDB:
# - name: "default_vector"
# storage_type: "vector_db"
# backend: "milvus"
# config:
# # Vector size MUST match your embedding model's output dimension
# vector_size: 2048
# collection_prefix: "opencontext"
# consistency_level: "Session"
#
# # Milvus Lite (local file; not supported on Windows):
# uri: "${MILVUS_URI:./persist/milvus.db}"
#
# # Remote Milvus or Zilliz Cloud (also supported on Windows):
# # uri: "${MILVUS_URI:http://localhost:19530}"
# # token: "${MILVUS_TOKEN:}" # Required for authenticated endpoints

- name: "document_store"
storage_type: "document_db"
backend: "sqlite"
Expand Down
27 changes: 18 additions & 9 deletions opencontext/storage/backends/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,23 @@
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd.
# SPDX-License-Identifier: Apache-2.0

"""
Storage backend package initialization file
"""
"""Storage backend package with lazy provider imports."""

from .chromadb_backend import ChromaDBBackend
from .sqlite_backend import SQLiteBackend
from importlib import import_module

try:
__all__ = ["SQLiteBackend", "ChromaDBBackend"]
except ImportError:
__all__ = ["SQLiteBackend", "ChromaDBBackend"]
__all__ = ["SQLiteBackend", "ChromaDBBackend", "QdrantBackend", "MilvusBackend"]

_BACKEND_MODULES = {
"SQLiteBackend": ".sqlite_backend",
"ChromaDBBackend": ".chromadb_backend",
"QdrantBackend": ".qdrant_backend",
"MilvusBackend": ".milvus_backend",
}


def __getattr__(name: str):
if name not in _BACKEND_MODULES:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
backend = getattr(import_module(_BACKEND_MODULES[name], __name__), name)
globals()[name] = backend
return backend
Loading