Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 72 additions & 132 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ For complete model support, see [LiteLLM Providers Documentation](https://docs.l

### 3. Environment Configuration

#### Configuration Template
#### Server Configuration Template

Create a configuration file `~/.openviking/ov.conf`, remove the comments before copy:

Expand Down Expand Up @@ -253,7 +253,7 @@ Create a configuration file `~/.openviking/ov.conf`, remove the comments before

> **Note**: For embedding models, currently `volcengine` (Doubao), `openai`, and `jina` providers are supported. For VLM models, we support three providers: `volcengine`, `openai`, and `litellm`. The `litellm` provider supports various models including Anthropic (Claude), DeepSeek, Gemini, Moonshot, Zhipu, DashScope, MiniMax, vLLM, Ollama, and more.

#### Configuration Examples
#### Server Configuration Examples

👇 Expand to see the configuration example for your model service:

Expand Down Expand Up @@ -325,12 +325,12 @@ Create a configuration file `~/.openviking/ov.conf`, remove the comments before

</details>

#### Set Environment Variable
#### Set Server Configuration Environment Variable

After creating the configuration file, set the environment variable to point to it (Linux/macOS):

```bash
export OPENVIKING_CONFIG_FILE=~/.openviking/ov.conf
export OPENVIKING_CONFIG_FILE=~/.openviking/ov.conf # by default
```

On Windows, use one of the following:
Expand All @@ -349,95 +349,75 @@ set "OPENVIKING_CONFIG_FILE=%USERPROFILE%\.openviking\ov.conf"

> 💡 **Tip**: You can also place the configuration file in other locations, just specify the correct path in the environment variable.

### 4. Run Your First Example

> 📝 **Prerequisite**: Ensure you have completed the environment configuration in the previous step.

Now let's run a complete example to experience the core features of OpenViking.
#### CLI/Client Configuration Examples

#### Create Python Script
👇 Expand to see the configuration example for your CLI/Client:

Create `example.py`:
Example: ovcli.conf for visiting localhost server

```python
import openviking as ov
```json
{
"url": "http://localhost:1933",
"timeout": 60.0,
"output": "table"
}
```

# Initialize OpenViking client with data directory
client = ov.SyncOpenViking(path="./data")
After creating the configuration file, set the environment variable to point to it (Linux/macOS):

try:
# Initialize the client
client.initialize()
```bash
export OPENVIKING_CLI_CONFIG_FILE=~/.openviking/ovcli.conf # by default
```

# Add resource (supports URL, file, or directory)
add_result = client.add_resource(
path="https://raw.githubusercontent.com/volcengine/OpenViking/refs/heads/main/README.md"
)
root_uri = add_result['root_uri']
On Windows, use one of the following:

# Explore the resource tree structure
ls_result = client.ls(root_uri)
print(f"Directory structure:\n{ls_result}\n")
PowerShell:

# Use glob to find markdown files
glob_result = client.glob(pattern="**/*.md", uri=root_uri)
if glob_result['matches']:
content = client.read(glob_result['matches'][0])
print(f"Content preview: {content[:200]}...\n")
```powershell
$env:OPENVIKING_CLI_CONFIG_FILE = "$HOME/.openviking/ovcli.conf"
```

# Wait for semantic processing to complete
print("Wait for semantic processing...")
client.wait_processed()
Command Prompt (cmd.exe):

# Get abstract and overview of the resource
abstract = client.abstract(root_uri)
overview = client.overview(root_uri)
print(f"Abstract:\n{abstract}\n\nOverview:\n{overview}\n")
```bat
set "OPENVIKING_CLI_CONFIG_FILE=%USERPROFILE%\.openviking\ovcli.conf"
```

# Perform semantic search
results = client.find("what is openviking", target_uri=root_uri)
print("Search results:")
for r in results.resources:
print(f" {r.uri} (score: {r.score:.4f})")
### 4. Run Your First Example

# Close the client
client.close()
> 📝 **Prerequisite**: Ensure you have completed the configuration (ov.conf and ovcli.conf) in the previous step.

except Exception as e:
print(f"Error: {e}")
```
Now let's run a complete example to experience the core features of OpenViking.

#### Run the Script
#### Launch Server

```bash
python example.py
openviking-server
```

#### Expected Output
or you can run in background

```bash
nohup openviking-server > /data/log/openviking.log 2>&1 &
```
Directory structure:
...

Content preview: ...

Wait for semantic processing...
Abstract:
...

Overview:
...
#### Run the CLI

Search results:
viking://resources/... (score: 0.8523)
...
```bash
ov status
ov add-resource https://github.com/volcengine/OpenViking # --wait
ov ls viking://resources/
ov tree viking://resources/volcengine -L 2
# wait some time for semantic processing if not --wait
ov find "what is openviking"
ov grep "openviking" --uri viking://resources/volcengine/OpenViking/docs/zh
```

Congratulations! You have successfully run OpenViking 🎉

---

## Server Deployment
## Server Deployment Details

For production environments, we recommend running OpenViking as a standalone HTTP service to provide persistent, high-performance context support for your AI Agents.

Expand All @@ -446,7 +426,30 @@ To ensure optimal storage performance and data security, we recommend deploying

👉 **[View: Server Deployment & ECS Setup Guide](./docs/en/getting-started/03-quickstart-server.md)**

---

## OpenClaw Memory Plugin Details

* Test Dataset: Effect testing based on LoCoMo10 (https://github.com/snap-research/locomo) long-range dialogues (1,540 cases in total after removing category5 without ground truth)
* Experimental Groups: Since users may not disable OpenClaw's native memory when using OpenViking, we added experimental groups with native memory enabled or disabled
* OpenViking Version: 0.1.18
* Model: seed-2.0-code
* Evaluation Script: https://github.com/ZaynJarvis/openclaw-eval/tree/main

| Experimental Group | Task Completion Rate | Cost: Input Tokens (Total) |
|----------|------------------|------------------|
| OpenClaw(memory-core) | 35.65% | 24,611,530 |
| OpenClaw + LanceDB (-memory-core) | 44.55% | 51,574,530 |
| OpenClaw + OpenViking Plugin (-memory-core) | 52.08% | 4,264,396 |
| OpenClaw + OpenViking Plugin (+memory-core) | 51.23% | 2,099,622 |

* Experimental Conclusions:
After integrating OpenViking:
- With native memory enabled: 43% improvement over original OpenClaw with 91% reduction in input token cost; 15% improvement over LanceDB with 92% reduction in input token cost.
- With native memory disabled: 49% improvement over original OpenClaw with 83% reduction in input token cost; 17% improvement over LanceDB with 96% reduction in input token cost.

👉 **[View: OpenClaw Memory Plugin](examples/openclaw-memory-plugin/README.md)**

--

## Core Concepts

Expand Down Expand Up @@ -535,77 +538,18 @@ OpenViking has a built-in memory self-iteration loop. At the end of each session

This allows the Agent to get "smarter with use" through interactions with the world, achieving self-evolution. Learn more: [Session Management](./docs/en/concepts/08-session.md)

---

## Project Architecture

The OpenViking project adopts a clear modular architecture design. The main directory structure is as follows:

```
OpenViking/
├── openviking/ # Core source code directory
│ ├── core/ # Core modules: client, engine, filesystem, etc.
│ ├── models/ # Model integration: VLM and Embedding model encapsulation
│ ├── parse/ # Resource parsing: file parsing, detection, OVPack handling
│ ├── retrieve/ # Retrieval module: semantic retrieval, directory recursive retrieval
│ ├── storage/ # Storage layer: vector DB, filesystem queue, observers
│ ├── session/ # Session management: history, memory extraction
│ ├── message/ # Message processing: formatting, conversion
│ ├── prompts/ # Prompt templates: templates for various tasks
│ ├── utils/ # Utilities: config, helpers
│ └── bin/ # Command line tools
├── docs/ # Project documentation
│ ├── zh/ # Chinese documentation
│ ├── en/ # English documentation
│ └── images/ # Documentation images
├── examples/ # Usage examples
├── tests/ # Test cases
│ ├── client/ # Client tests
│ ├── engine/ # Engine tests
│ ├── integration/ # Integration tests
│ ├── session/ # Session tests
│ └── vectordb/ # Vector DB tests
├── src/ # C++ extensions (high-performance index and storage)
│ ├── common/ # Common components
│ ├── index/ # Index implementation
│ └── store/ # Storage implementation
├── third_party/ # Third-party dependencies
├── pyproject.toml # Python project configuration
├── setup.py # Setup script
├── LICENSE # Open source license
├── CONTRIBUTING.md # Contributing guide
├── AGENT.md # Agent development guide
└── README.md # Project readme
```


---

## Advanced Reading

For more details, please visit our [Full Documentation](./docs/en/).

---

## Community & Team
### Documentation

### About Us

OpenViking is an open-source context database initiated and maintained by the **ByteDance Volcengine Viking Team**.

The Viking team focuses on unstructured information processing and intelligent retrieval, accumulating rich commercial practical experience in context engineering technology:
For more details, please visit our [Full Documentation](./docs/en/).

- **2019**: VikingDB vector database supported large-scale use across all ByteDance businesses.
- **2023**: VikingDB sold on Volcengine public cloud.
- **2024**: Launched developer product matrix: VikingDB, Viking KnowledgeBase, Viking MemoryBase.
- **2025**: Created upper-layer application products like AI Search and Vaka Knowledge Assistant.
- **Oct 2025**: Open-sourced [MineContext](https://github.com/volcengine/MineContext), exploring proactive AI applications.
- **Jan 2026**: Open-sourced OpenViking, providing underlying context database support for AI Agents.
### Community & Team

For more details, please see: **[About Us](./docs/en/about/01-about-us.md)**

---

### Join the Community

OpenViking is still in its early stages, and there are many areas for improvement and exploration. We sincerely invite every developer passionate about AI Agent technology:
Expand All @@ -621,14 +565,10 @@ OpenViking is still in its early stages, and there are many areas for improvemen

Let's work together to define and build the future of AI Agent context management. The journey has begun, looking forward to your participation!

---

### Star Trend

[![Star History Chart](https://api.star-history.com/svg?repos=volcengine/OpenViking&type=timeline&legend=top-left)](https://www.star-history.com/#volcengine/OpenViking&type=timeline&legend=top-left)

---

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](./LICENSE) file for details.
Expand Down
Loading