Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,10 +205,17 @@ vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000

#### Configuration Template

Create a configuration file `~/.openviking/ov.conf`:
Create a configuration file `~/.openviking/ov.conf`, remove the comments before copy:

```json
{
"storage": {
"workspace": "/home/your-name/openviking_workspace"
},
"log": {
"level": "INFO",
"output": "stdout" // Log output: "stdout" or "file"
},
"embedding": {
"dense": {
"api_base" : "<api-endpoint>", // API endpoint address
Expand Down Expand Up @@ -238,6 +245,13 @@ Create a configuration file `~/.openviking/ov.conf`:

```json
{
"storage": {
"workspace": "/home/your-name/openviking_workspace"
},
"log": {
"level": "INFO",
"output": "stdout" // Log output: "stdout" or "file"
},
"embedding": {
"dense": {
"api_base" : "https://ark.cn-beijing.volces.com/api/v3",
Expand All @@ -263,6 +277,13 @@ Create a configuration file `~/.openviking/ov.conf`:

```json
{
"storage": {
"workspace": "/home/your-name/openviking_workspace"
},
"log": {
"level": "INFO",
"output": "stdout" // Log output: "stdout" or "file"
},
"embedding": {
"dense": {
"api_base" : "https://api.openai.com/v1",
Expand Down
21 changes: 21 additions & 0 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,13 @@ OpenViking 支持多种模型服务:

```json
{
"storage": {
"workspace": "/home/your-name/openviking_workspace"
},
"log": {
"level": "INFO",
"output": "stdout" // 可配置为 "stdout" or "file"
},
"embedding": {
"dense": {
"api_base" : "<api-endpoint>", // API 服务端点地址
Expand Down Expand Up @@ -114,6 +121,13 @@ OpenViking 支持多种模型服务:

```json
{
"storage": {
"workspace": "/home/your-name/openviking_workspace"
},
"log": {
"level": "INFO",
"output": "stdout" // 可配置为 "stdout" or "file"
},
"embedding": {
"dense": {
"api_base" : "https://ark.cn-beijing.volces.com/api/v3",
Expand All @@ -139,6 +153,13 @@ OpenViking 支持多种模型服务:

```json
{
"storage": {
"workspace": "/home/your-name/openviking_workspace"
},
"log": {
"level": "INFO",
"output": "stdout" // 可配置为 "stdout" or "file"
},
"embedding": {
"dense": {
"api_base" : "https://api.openai.com/v1",
Expand Down
53 changes: 53 additions & 0 deletions docs/en/guides/03-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,59 @@ Server connects to remote AGFS and VectorDB services. Configure remote URLs in `
python -m openviking serve
```

## Deploying with Systemd (Recommended)

For Linux systems, you can use Systemd to manage OpenViking as a service, enabling automatic restart and startup on boot. Firstly, you should tried to install and configure openviking on your own.

### Create Systemd Service File

Create `/etc/systemd/system/openviking.service` file:

```ini
[Unit]
Description=OpenViking HTTP Server
After=network.target

[Service]
Type=simple
# Replace with the user running OpenViking
User=your-username
# Replace with the user group
Group=your-group
# Replace with your working directory
WorkingDirectory=/home/your-username/openviking_workspace
# Choose one of the following start methods
ExecStart=/path/to/your/python/bin/openviking-server
Restart=always
RestartSec=5
# Path to config file
Environment="OPENVIKING_CONFIG_FILE=/home/your-username/.openviking/ov.conf"

[Install]
WantedBy=multi-user.target
```

### Manage the Service

After creating the service file, use the following commands to manage the OpenViking service:

```bash
# Reload systemd configuration
sudo systemctl daemon-reload

# Start the service
sudo systemctl start openviking.service

# Enable service on boot
sudo systemctl enable openviking.service

# Check service status
sudo systemctl status openviking.service

# View service logs
sudo journalctl -u openviking.service -f
```

## Connecting Clients

### Python SDK
Expand Down
53 changes: 53 additions & 0 deletions docs/zh/guides/03-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,59 @@ python -m openviking serve
python -m openviking serve
```

## 使用 Systemd 部署服务(推荐)

对于 Linux 系统,可以使用 Systemd 服务来管理 OpenViking,实现自动重启、开机自启等功能。首先,你应该已经成功安装并配置了 OpenViking 服务器,确保它可以正常运行,再进行服务化部署。

### 创建 Systemd 服务文件

创建 `/etc/systemd/system/openviking.service` 文件:

```ini
[Unit]
Description=OpenViking HTTP Server
After=network.target

[Service]
Type=simple
# 替换为运行 OpenViking 的用户
User=your-username
# 替换为用户组
Group=your-group
# 替换为工作目录
WorkingDirectory=/home/your-username/openviking_workspace
# 以下两种启动方式二选一
ExecStart=/path/to/your/python/bin/openviking-server
Restart=always
RestartSec=5
# 配置文件路径
Environment="OPENVIKING_CONFIG_FILE=/home/your-username/.openviking/ov.conf"

[Install]
WantedBy=multi-user.target
```

### 管理服务

创建好服务文件后,使用以下命令管理 OpenViking 服务:

```bash
# 重载 systemd 配置
sudo systemctl daemon-reload

# 启动服务
sudo systemctl start openviking.service

# 设置开机自启
sudo systemctl enable openviking.service

# 查看服务状态
sudo systemctl status openviking.service

# 查看服务日志
sudo journalctl -u openviking.service -f
```

## 连接客户端

### Python SDK
Expand Down
8 changes: 8 additions & 0 deletions openviking/core/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,14 @@ class ContextType(str, Enum):
RESOURCE = "resource"


class ContextLevel(int, Enum):
"""Context level (L0/L1/L2) for vector indexing"""

ABSTRACT = 0 # L0: abstract
OVERVIEW = 1 # L1: overview
DETAIL = 2 # L2: detail/content


class Vectorize:
text: str = ""
# image: str = ""
Expand Down
6 changes: 3 additions & 3 deletions openviking/retrieve/hierarchical_retriever.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ async def _global_vector_search(

global_filter = {
"op": "and",
"conds": [filter, {"op": "must", "field": "is_leaf", "conds": [False]}],
"conds": [filter, {"op": "must", "field": "level", "conds": [0, 1]}],
}
results = await self.storage.search(
collection=collection,
Expand Down Expand Up @@ -351,7 +351,7 @@ def merge_filter(base_filter: Dict, extra_filter: Optional[Dict]) -> Dict:
)

if uri not in visited:
if r.get("is_leaf"):
if r.get("level") == 2:
visited.add(uri)
else:
heapq.heappush(dir_queue, (-final_score, uri))
Expand Down Expand Up @@ -400,7 +400,7 @@ async def _convert_to_matched_contexts(
MatchedContext(
uri=c.get("uri", ""),
context_type=context_type,
is_leaf=c.get("is_leaf", False),
level=c.get("level", 2),
abstract=c.get("abstract", ""),
category=c.get("category", ""),
score=c.get("_final_score", c.get("_score", 0.0)),
Expand Down
1 change: 1 addition & 0 deletions openviking/server/bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ def main():

# Create and run app
app = create_app(config)
print(f"OpenViking HTTP Server is running on {config.host}:{config.port}")
uvicorn.run(app, host=config.host, port=config.port, log_config=None)


Expand Down
21 changes: 19 additions & 2 deletions openviking/storage/collection_schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,32 @@ def context_collection(name: str, vector_dim: int) -> Dict[str, Any]:
"Fields": [
{"FieldName": "id", "FieldType": "string", "IsPrimaryKey": True},
{"FieldName": "uri", "FieldType": "path"},
# type 字段:当前版本未使用,保留用于未来扩展
# 预留用于表示资源的具体类型,如 "file", "directory", "image", "video", "repository" 等
{"FieldName": "type", "FieldType": "string"},
# context_type 字段:区分上下文的大类
# 枚举值:"resource"(资源,默认), "memory"(记忆), "skill"(技能)
# 推导规则:
# - URI 以 viking://agent/skills 开头 → "skill"
# - URI 包含 "memories" → "memory"
# - 其他情况 → "resource"
{"FieldName": "context_type", "FieldType": "string"},
{"FieldName": "vector", "FieldType": "vector", "Dim": vector_dim},
{"FieldName": "sparse_vector", "FieldType": "sparse_vector"},
{"FieldName": "created_at", "FieldType": "date_time"},
{"FieldName": "updated_at", "FieldType": "date_time"},
{"FieldName": "active_count", "FieldType": "int64"},
{"FieldName": "parent_uri", "FieldType": "path"},
{"FieldName": "is_leaf", "FieldType": "bool"},
# level 字段:区分 L0/L1/L2 层级
# 枚举值:
# - 0 = L0(abstract,摘要)
# - 1 = L1(overview,概览)
# - 2 = L2(detail/content,详情/内容,默认)
# URI 命名规则:
# - level=0: {目录}/.abstract.md
# - level=1: {目录}/.overview.md
# - level=2: {文件路径}
{"FieldName": "level", "FieldType": "int64"},
{"FieldName": "name", "FieldType": "string"},
{"FieldName": "description", "FieldType": "string"},
{"FieldName": "tags", "FieldType": "string"},
Expand All @@ -67,7 +84,7 @@ def context_collection(name: str, vector_dim: int) -> Dict[str, Any]:
"updated_at",
"active_count",
"parent_uri",
"is_leaf",
"level",
"name",
"tags",
],
Expand Down
15 changes: 13 additions & 2 deletions openviking/storage/queuefs/embedding_msg_converter.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
to EmbeddingMsg objects for asynchronous vector processing.
"""

from openviking.core.context import Context
from openviking.core.context import Context, ContextLevel
from openviking.storage.queuefs.embedding_msg import EmbeddingMsg
from openviking_cli.utils import get_logger

Expand All @@ -26,9 +26,20 @@ def from_context(context: Context, **kwargs) -> EmbeddingMsg:
if not vectorization_text:
return None

context_dict = context.to_dict()

# 根据 URI 判断 level 字段(用于向量索引)
uri = context_dict.get("uri", "")
if uri.endswith("/.abstract.md"):
context_dict["level"] = ContextLevel.ABSTRACT
elif uri.endswith("/.overview.md"):
context_dict["level"] = ContextLevel.OVERVIEW
else:
context_dict["level"] = ContextLevel.DETAIL

embedding_msg = EmbeddingMsg(
message=vectorization_text,
context_data=context.to_dict(),
context_data=context_dict,
)

# Set any additional fields from kwargs
Expand Down
37 changes: 26 additions & 11 deletions openviking/storage/queuefs/semantic_processor.py
Original file line number Diff line number Diff line change
Expand Up @@ -441,21 +441,36 @@ async def _vectorize_directory_simple(
from openviking.storage.queuefs import get_queue_manager
from openviking.storage.queuefs.embedding_msg_converter import EmbeddingMsgConverter

parent_uri = VikingURI(uri).parent.uri
context = Context(
uri=uri,
parent_uri=parent_uri,
queue_manager = get_queue_manager()
embedding_queue = queue_manager.get_queue(queue_manager.EMBEDDING)

# Vectorize L0: .abstract.md (abstract), level=0
abstract_uri = f"{uri}/.abstract.md"
context_abstract = Context(
uri=abstract_uri,
parent_uri=uri,
is_leaf=False,
abstract=abstract,
context_type=context_type,
)
context.set_vectorize(Vectorize(text=overview))

embedding_msg = EmbeddingMsgConverter.from_context(context)
queue_manager = get_queue_manager()
embedding_queue = queue_manager.get_queue(queue_manager.EMBEDDING)
await embedding_queue.enqueue(embedding_msg) # type: ignore
logger.debug(f"Enqueued directory for vectorization: {uri}")
context_abstract.set_vectorize(Vectorize(text=abstract))
embedding_msg_abstract = EmbeddingMsgConverter.from_context(context_abstract)
await embedding_queue.enqueue(embedding_msg_abstract) # type: ignore
logger.debug(f"Enqueued directory L0 (abstract) for vectorization: {abstract_uri}")

# Vectorize L1: .overview.md (overview), level=1
overview_uri = f"{uri}/.overview.md"
context_overview = Context(
uri=overview_uri,
parent_uri=uri,
is_leaf=False,
abstract=abstract,
context_type=context_type,
)
context_overview.set_vectorize(Vectorize(text=overview))
embedding_msg_overview = EmbeddingMsgConverter.from_context(context_overview)
await embedding_queue.enqueue(embedding_msg_overview) # type: ignore
logger.debug(f"Enqueued directory L1 (overview) for vectorization: {overview_uri}")

async def _vectorize_files(
self,
Expand Down
Loading
Loading