Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion VERSIONS/GO_VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.24.5
1.26.5
9 changes: 2 additions & 7 deletions content/docs/api/build_proto.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Build_proto_api"
date: 2024-08-08T00:23:48+09:00
date: 2026-08-12T01:33:11Z
draft: false
weight: 1100
description: How to build gRPC proto files for calling API to your Vald cluster
Expand Down Expand Up @@ -41,12 +41,10 @@ Let's build proto files using your favorite programming language.
There are 3 steps to building API proto:

1. Install gRPC tools

- gRPC official document provides [the way to install for each language](https://grpc.io/docs/languages/).<br>
If your favorite programming language is not there, you can find 3rd party tools for building.

1. Download Vald api proto files and external dependence

- [vald api proto](https://github.com/vdaas/vald/tree/main/apis/proto/v1/vald)
- [vald payload proto](https://github.com/vdaas/vald/tree/main/apis/proto/v1/payload)
- [googleapis](https://github.com/googleapis/googleapis)
Expand Down Expand Up @@ -96,7 +94,7 @@ There are many tools for building proto in Rust, we use [tonic](https://github.c
[package]
name = "vald-grpc"
version = "0.1.0"
edition = "2021"
edition = "2024"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

Expand Down Expand Up @@ -154,7 +152,6 @@ There are many tools for building proto in Rust, we use [tonic](https://github.c
```

1. Implement `build.rs` and Build proto

1. `build.rs`

```rust
Expand Down Expand Up @@ -207,7 +204,6 @@ There are many tools for building proto in Rust, we use [tonic](https://github.c
```

1. Implement code using client

1. `lib.rs`

Import build proto in `src/lib.rs`
Expand Down Expand Up @@ -241,7 +237,6 @@ There are many tools for building proto in Rust, we use [tonic](https://github.c
1. `src/client.rs`

There are 4 steps in `src/client.rs`:

1. Load dataset
1. Insert vector to Vald cluster
1. Search nearest neighbor vectors from Vald cluster after indexing finished
Expand Down
5 changes: 1 addition & 4 deletions content/docs/contributing/coding-style.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Coding Style_contributing"
date: 2024-10-16T16:03:39+09:00
date: 2026-08-12T01:33:10Z
draft: false
weight: 300
description: Coding style for Vald project
Expand Down Expand Up @@ -815,7 +815,6 @@ tests := map[string]func(t *testing.T) test {

Test case names should be readable, meaningful, and understandable easily.
If you create a new test, the test name should be named based on the below-naming templates

- Success cases:
- Start with `success` or `{verb} success` or `success {verb}`
- End with the condition `when {condition}` or `with {condition}
Expand Down Expand Up @@ -988,7 +987,6 @@ Still, in some cases, you may need to change the generated code to meet your req
1. goleak usage

There are two methods for valid goroutine leak:

1. Use `goleak.VerifyNone()` to validate it on each test cases.
1. Use `goleak.VerifyTestMain()` to validate it on each package.

Expand Down Expand Up @@ -1135,7 +1133,6 @@ Still, in some cases, you may need to change the generated code to meet your req

By default, when testing the function of the struct, the target struct initialization is implemented by setting the data from the `fields` defined in the test case.
This initialization method has a few disadvantages:

1. When there are many fields in the struct, it is hard to set them all.
1. The default value is the zero value of the type, not the struct default value from the struct initialization function.

Expand Down
36 changes: 18 additions & 18 deletions content/docs/contributing/contributing-guide.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Contributing Guide_contributing"
date: 2024-01-10T18:16:39+09:00
date: 2026-08-12T01:33:10Z
draft: false
weight: 100
description: Contribute guideline for the Vald project
Expand Down Expand Up @@ -30,11 +30,11 @@ We appreciate your help!

## Contributing Issue

We use [Github Issues](https://github.com/vdaas/vald/issues) to track issues within this repository.
If you can determine the problem you are facing is a bug or issue, you can easily submit the issues.
We use [GitHub Issues](https://github.com/vdaas/vald/issues) to track issues within this repository.
If you can determine the problem you are facing is a bug or issue, you can submit the issue.

If you are unsure whether you have found a bug or security-related issues, please consider asking in the [Vald Slack](https://join.slack.com/t/vald-community/shared_invite/zt-db2ky9o4-R_9p2sVp8xRwztVa8gfnPA) first.
If the behavior you are seeing is confirmed as a bug or issue, it can easily be re-raised in the issue tracker.
When unsure whether you have found a bug or security-related issue, please consider asking in the [Vald Slack](https://join.slack.com/t/vald-community/shared_invite/zt-db2ky9o4-R_9p2sVp8xRwztVa8gfnPA) first.
If the behavior you are seeing is confirmed as a bug or issue, it can be re-raised in the issue tracker.

### Bug Report

Expand All @@ -46,7 +46,7 @@ Please try to write as detailed as possible in your bug report.

When filing an issue, make sure to answer these five questions:

1. What version of Vald are you using (vald docker image version)?
1. What version of Vald are you using (Vald Docker image version)?
2. What operating system and processor architecture are you using?
3. What did you do?
4. What did you expect to see?
Expand All @@ -60,7 +60,7 @@ The Vald is being developed based on the design-driven process.<br>
The significant change to the library or the architecture should be discussed first.

We may ask for detailed documentation before your proposal is accepted.<br>
Your proposal will be reviewed, discussed, and decided whether it is accepted or declined.
Your proposal will be reviewed, discussed, and accepted or declined.

Please submit the proposal [here](https://github.com/vdaas/vald/issues/new?assignees=&labels=type%2Ffeature%2C+priority%2Flow%2C+team%2Fcore&template=feature_request&title=)

Expand Down Expand Up @@ -102,7 +102,7 @@ Follow these steps to make a contribution to any of our open source repositories
2. Set your name and email (these should match the information on your submitted CLA)

```bash
git config --global user.name "Firstname Lastname"
git config --global user.name "Your Name"
git config --global user.email "your_email@example.com"
```

Expand All @@ -115,9 +115,9 @@ Follow these steps to make a contribution to any of our open source repositories
5. Push to the forked branch (git push origin my-new-feature)
6. Create new Pull Request

Each pull request and commit should be small enough to contain only one purpose.
Each pull request and commit should be small enough to address a single purpose.

Your pull request is much more likely to be accepted if:
Your pull request has a higher chance of being accepted if:

- Your pull request includes tests

Expand All @@ -127,14 +127,14 @@ Your pull request is much more likely to be accepted if:

### How to contribute unit test code

To contribute unit test code, the steps are almost the same as [contribute source code](#how-to-contribute-source-code), but there are some extra steps when implementing a test function which is not implemented before:
To contribute unit test code, the steps are almost the same as [contribute source code](#how-to-contribute-source-code), but there are some extra steps when adding a test function that has not been added before:

1. Execute `make gotests/gen` command under Vald repository
1. Move the test function which you would like to implement above the placeholder `// NOT IMPLEMENTED BELOW`
1. Uncomment and implement the test function
1. Move the test function which you would like to write above the placeholder `// NOT IMPLEMENTED BELOW`
1. Remove the comment markers and write the test function

Vald implmentes unit test code to ensure the quality of Vald.
Each implementation file comes with its unit test file, with `*_test.go` postfix.
Vald implements unit test code to ensure the quality of Vald.
Each implementation file comes with its unit test file, with `*_test.go` suffix.
In Vald, unit test files are looks like:

```go
Expand Down Expand Up @@ -197,10 +197,10 @@ Name your branches with prefixes: `[type]/[area]/[description]`

| Field | Explanation | Naming Rule |
| :---------- | :------------------------------------ | :------------------------------------------------------------------------------------------------------------------------ |
| type | The PR type | The type of PR can be feature, bug, refactoring, benchmark, security, documentation, dependencies, ci, test, or etc... |
| type | The PR type | Accepted PR types: feature, bug, refactoring, benchmark, security, documentation, dependencies, ci, test, etc. |
| area | Area of context | The area of PR can be gateway, agent, agent-sidecar, lb-gateway, or etc... |
| description | Summarized description of your branch | The description must be hyphenated. Please use [a-zA-Z0-9] and hyphen as characters, and do not use any other characters. |

(\*) If you changed multiple areas, please list each area with "-".
(\*) If you changed more than one area, please list each area with `-`.

For example, when you add a new feature for internal/servers, the name of the branch will be `feature/internal/add-newfeature-for-servers`.
For example, when you add a new feature for internal/servers, the name of the branch will be `feature/internal/add-new-feature-for-servers`.
14 changes: 3 additions & 11 deletions content/docs/overview/about-vald.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "About Vald_overview"
date: 2025-09-02T04:53:44Z
date: 2026-08-12T01:33:10Z
draft: false
weight: 100
description: What is Vald and what you can do with Vald
Expand All @@ -21,7 +21,7 @@ Vald is a highly scalable distributed fast approximate nearest neighbor dense ve

Vald is designed and implemented based on Cloud-Native architecture.

It uses the fastest ANN Algorithm [NGT](https://github.com/yahoojapan/NGT) to search neighbors.
It uses the fastest ANN Algorithm [NGT](https://github.com/NGT-labs/NGT) to search neighbors.

Vald has automatic vector indexing and index backup, and horizontal scaling which made for searching from billions of feature vector data.

Expand All @@ -32,39 +32,31 @@ Vald is easy to use, feature-rich and highly customizable as you needed.
<!-- copied from README-->

- Asynchronous Auto Indexing

- Usually the graph requires locking during indexing, which causes stop-the-world. But Vald uses distributed index graphs so it continues to work during indexing.

- Customizable Ingress/Egress Filtering

- Vald implements it's own highly customizable Ingress/Egress filter.
- Which can be configured to fit the gRPC interface.
- Ingress Filter: Ability to Vectorize through filter on request.
- Egress Filter: rerank or filter the searching result with your own algorithm.

- Cloud-native based vector searching engine

- Horizontal scalable on memory and CPU for your demand.

- Auto Backup for Index data

- Vald supports to backup Vald Agent index data using Object Storage or Persistent Volume.

- Distributed Indexing

- Vald distributes vector index to multiple agents, and each agent stores different index.

- Index Replication

- Vald stores each index in multiple agents which enables index replicas.
- Automatically rebalancing the replica when some Vald agent goes down.

- Easy to use

- Vald can be easily installed in a few steps.

- Highly customizable

- You can configure the number of vector dimensions, the number of replica and etc.

- Multi language supported
Expand All @@ -85,7 +77,7 @@ Vald supports similarity searching.
Vald is based on Kubernetes and Cloud-Native architecture, which means Vald is highly scalable.
You can easily scale Vald by changing Vald's configuration.

Vald uses the fastest ANN Algorithm [NGT](https://github.com/yahoojapan/NGT) to search neighbors by default, but users can switch to another vector searching engine in Vald to support the best performance for your use case.
Vald uses the fastest ANN Algorithm [NGT](https://github.com/NGT-labs/NGT) to search neighbors by default, but users can switch to another vector searching engine in Vald to support the best performance for your use case.

Also, Vald supports auto-healing, to reduce running and maintenance costs. Vald implements the backup mechanism to support disaster recovery.
Whenever one of the Vald Agent instances is down, the new Vald Agent instance will be created automatically and the data will be recovered automatically.
Expand Down
7 changes: 3 additions & 4 deletions content/docs/overview/architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Architecture_overview"
date: 2024-08-08T00:23:48+09:00
date: 2026-08-12T01:33:10Z
draft: false
weight: 200
description: High-Level architecture design and overview of each component
Expand Down Expand Up @@ -35,7 +35,7 @@ Vald is based on the following technologies.
Vald contains multiple components and configurations.
Helm helps us to manage those manifests and provides a better and easy way to deploy and configure Vald.

- [NGT](https://github.com/yahoojapan/NGT)
- [NGT](https://github.com/NGT-labs/NGT)

NGT is one of the core components of Vald.
NGT is a super-fast vector search engine used by Vald to guarantee the high performance of Vald.
Expand Down Expand Up @@ -63,7 +63,6 @@ Here are the concepts of Vald.
- Kubernetes based

Vald can integrate with Kubernetes which enables the following features.

- Orchestrated

Kubernetes supports container orchestration. All components in Vald can be managed by Kubernetes automatically.
Expand Down Expand Up @@ -96,7 +95,7 @@ We will introduce each component and why it is needed in Vald.
### Vald Agent

Vald Agent is the core component of Vald, the approximate nearest neighbor search engine, and stores the graph tree construction on memory for indexing the vectors.
Vald Agent uses [yahoojapan/NGT](https://github.com/yahoojapan/ngt) as a core library.
Vald Agent uses [NGT-labs/NGT](https://github.com/NGT-labs/NGT) as a core library.

### Vald LB Gateway

Expand Down
4 changes: 2 additions & 2 deletions content/docs/overview/component/_index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Component_overview/Component"
date: 2024-02-15T17:10:08+09:00
date: 2026-08-12T01:33:10Z
draft: false
weight: 0
---
Expand Down Expand Up @@ -73,7 +73,7 @@ In this section, we will describe what is Vald Agent and the corresponding compo
### Vald Agent

Vald Agent provides functionalities to perform approximate nearest neighbor search.
Agent-NGT uses [yahoojapan/NGT](https://github.com/yahoojapan/NGT) as a core library.
Agent-NGT uses [NGT-labs/NGT](https://github.com/NGT-labs/NGT) as a core library.
And Agent-Faiss uses [facebookresearch/faiss](https://github.com/facebookresearch/faiss) as a core library.

Each Vald Agent pod has its own vector data space because only several Vald Agents are selected to be inserted/updated in a single insert/update request.
Expand Down
4 changes: 2 additions & 2 deletions content/docs/overview/component/agent.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Agent_overview/Component"
date: 2024-05-23T16:15:24+09:00
date: 2026-08-12T01:33:10Z
draft: false
weight: 100
description: Stores index data and Search ANN vectors
Expand Down Expand Up @@ -45,7 +45,7 @@ as core algorithm layer.

#### Vald Agent NGT

Vald Agent NGT uses [NGT](https://github.com/yahoojapan/NGT) as an algorithm.
Vald Agent NGT uses [NGT](https://github.com/NGT-labs/NGT) as an algorithm.

The main functions are the followings:

Expand Down
6 changes: 1 addition & 5 deletions content/docs/overview/component/mirror-gateway.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Mirror Gateway_overview/Component"
date: 2024-02-15T17:10:09+09:00
date: 2026-08-12T01:33:10Z
draft: false
weight: 600
description: Gateway for operating Vald cluster compatible with Multi-region and Multi AZ
Expand Down Expand Up @@ -66,27 +66,23 @@ Here's an overview of how the Mirror Gateway handles failures for each type of r
For more information about status code, please refer to [Mirror Gateway Troubleshooting](/docs/troubleshooting/mirror-gateway).

- Insert Request

- If the target host returns a status code of `ALREADY_EXISTS`, the Update request is sent to this host.
- If the target host returns a status code other than `OK`, `ALREADY_EXISTS`, the Mirror Gateway returns that status code without continuous processing.
- If all target hosts return a status code `ALREADY_EXISTS`, the Mirror Gateway returns `ALREADY_EXISTS`.
- If all target hosts return a status code `OK` or `ALREADY_EXISTS`, the Mirror Gateway returns `OK`.

- Update Request

- If the target host returns a status code `NOT_FOUND`, the Insert request is sent to this host.
- If the target host returns a status code other than `OK`, `ALREADY_EXISTS`, the Mirror Gateway returns that status code without continuous processing.
- If all target hosts return a status code `ALREADY_EXISTS`, the Mirror Gateway returns `ALREADY_EXISTS`.
- If all target hosts return a status code `OK` or `ALREADY_EXISTS`, the Mirror Gateway returns `OK`.

- Upsert Request

- If all target hosts return a status code `ALREADY_EXISTS`, the Mirror Gateway returns `ALREADY_EXISTS`.
- If the target host returns a status code other than `OK` or `ALREADY_EXISTS`, the Mirror Gateway returns that status code without continuous processing.
- If all target hosts return a status code `OK` or `ALREADY_EXISTS`, the Mirror Gateway returns `OK`.

- Remove/RemoveByTimestamp Request

- If all target hosts return a status code `NOT_FOUND`, the Mirror Gateway returns `NOT_FOUND`.
- If the target host returns a status code other than `OK` or `NOT_FOUND`, the Mirror Gateway returns that status code without continuous processing.
- If all target hosts return a status code `OK` or `NOT_FOUND`, the Mirror Gateway returns `OK`.
8 changes: 4 additions & 4 deletions content/docs/performance/continuous-benchmark.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Continuous Benchmark_performance"
date: 2024-10-16T16:03:40+09:00
date: 2026-08-12T01:33:09Z
draft: false
weight: 300
description: Performance verification using continuous benchmark tool
Expand Down Expand Up @@ -50,7 +50,7 @@ Benchmark Operator manages benchmark job according to the applied manifest.

### ValdBenchmarkJob

[`ValdBenchmarkJob`](https://github.com/vdaas/vald/blob/main/charts/vald-benchmark-operator/crds/valdbenchmarkjob.yaml) is used for executing single benchmark job.
[`ValdBenchmarkJob`](https://github.com/vdaas/vald/blob/main/charts/operator/benchmark/crds/valdbenchmarkjob.yaml) is used for executing single benchmark job.

And, Benchmark Operator also applies it to the Kubernetes cluster based on `ValdBenchmarkScenarioResource`.

Expand Down Expand Up @@ -184,7 +184,7 @@ And, Benchmark Operator also applies it to the Kubernetes cluster based on `Vald

### ValdBenchmarkScenario

[`ValdBenchmarkScenario`](https://github.com/vdaas/vald/blob/main/charts/vald-benchmark-operator/crds/valdbenchmarkscenario.yaml) is used for executing single or multiple benchmark job.
[`ValdBenchmarkScenario`](https://github.com/vdaas/vald/blob/main/charts/operator/benchmark/crds/valdbenchmarkscenario.yaml) is used for executing one or more benchmark jobs.

Benchmark Operator decomposes manifest and creates benchmark resources one by one.
The `target` and `dataset` property are the global config for scenario, they can be overwritten when each job has own config.
Expand Down Expand Up @@ -457,7 +457,7 @@ logging:

</details>

For more details of the configuration of `vald-benchmark-operator-release`, please refer to [here](https://github.com/vdaas/vald/blob/main/charts/vald-benchmark-operator/values.yaml)
For more details of the configuration of `vald-benchmark-operator-release`, please refer to [here](https://github.com/vdaas/vald/blob/main/charts/operator/benchmark/values.yaml)

1. Add Vald repo into the helm repo

Expand Down
Loading
Loading