I tried Knowledge Management in Kiro CLI

Introduction

Hello, I'm Jinno from the Consulting Department.

Recently, while looking at the Kiro CLI documentation, I noticed that Knowledge management has been added.
It seems like you can now index local files and directories, and search across chat sessions.

https://kiro.dev/docs/cli/experimental/knowledge-management/

It looked interesting so I tried it out right away.

Although Iwasa-san has already introduced this feature in the article below, I would like to explore this feature a little more deeply.

https://dev.classmethod.jp/articles/qdev-cli-experiment/

Knowledge management

Kiro CLI's knowledge feature allows you to index local files and directories as a persistent knowledge base that can be searched within chat sessions.

It recursively indexes files in the specified directory, and offers two types of indices (Fast and Best) that can be selected according to your use case.

Additionally, by separating agents, each can have an isolated knowledge base, avoiding context contamination.

This is essentially a local RAG system. I got the impression that it would be useful when you want to search local data to create answers and conduct investigations. I'd like to quickly utilize it when I have a large amount of documents locally but find it troublesome to search through them.

Index Types: The Difference Between Fast and Best

I mentioned there are two types, Fast and Best. Let me explain them.

Fast (Keyword Search - BM25)

This is a keyword-based fast search. The benefit is that both indexing and search result retrieval are quick due to the keyword search approach.

On the other hand, the drawback is that it requires exact keyword matching.
According to the documentation, it's suitable for searching log files, configuration files, and large code bases.

Best (Semantic Search - all-minilm-l6-v2)

This is a meaning-based search using an AI model.
The advantage is that it can understand context and meaning to search for corresponding documents, which is a nice feature. So it can potentially pick up related topics even without exact keywords.

However, the disadvantages include that indexing takes more time and consumes more CPU and memory. According to the documentation, this is suitable for searching documents and research materials.

By the way, it seems to be using models like all-minilm-l6-v2 for vectorization.

Usage Guidelines

Best seems suitable for documents and papers. On the other hand, Fast is recommended for log files and code that are likely to contain large amounts of data.
It would be good to subdivide knowledge bases and consider types according to use cases.

https://kiro.dev/docs/cli/experimental/knowledge-management/#when-to-use-each-type

Let's Try It Out

Now, let's try out Knowledge management.

Enabling Knowledge management

First, enable Knowledge management in settings.

kiro-cli settings chat.enableKnowledge true

Pull the Repository for Knowledge

This time, I'll add FastAPI documentation to the knowledge base as a sample.

mkdir sample-knowledge-management
cd sample-knowledge-management
git clone https://github.com/fastapi/fastapi.git

Add Project to Knowledge Base

Now I'll index the documentation in my local project. I specified --index-type Best because I want to use semantic search.

I'll target the docs/ja/docs folder that contains Japanese documentation.

# Start Kiro
kiro-cli

# Add knowledge base
/knowledge add --name "fastAPI-docs" --path fastapi/docs/ja/docs --index-type Best

I'm not specifying it this time, but if you want to exclude specific files, you can use the --exclude option.

/knowledge add --name "my-codebase" --path /path/to/project --exclude "target/**" --exclude "node_modules/**" --index-type Fast

Check Indexing Status

You can check the progress of indexing with the /knowledge show command.

> /knowledge show

👤 Agent (kiro_default):
    📂 fastAPI-docs (baa17fca)
       /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs
       67 items • Best • 12/13 02:58

If indexing is running in the background, progress and ETA will also be displayed. (I didn't see the ETA display because it finished instantly, probably due to the small amount of documents...)

Let's Search

Once indexing is complete, you can simply ask questions in natural language within the chat to search the knowledge base.

Search Prompt

「FastAPIの非同期実装のコツを教えてください、引用元ドキュメントも添えてください」

Response

Searching knowledge base for: async await 非同期 asynchronous in context: baa17fca-1187-4250-a6bf-5baa17f7287e (using tool: knowledge)
 ✓ Found 5 results for query 'async await 非同期 asynchronous'
 - Completed in 0.16s

Searching knowledge base for: async def database connection pool in context: baa17fca-1187-4250-a6bf-5baa17f7287e (using tool: knowledge)
 ✓ Found 5 results for query 'async def database connection pool'
 - Completed in 0.20s

> FastAPIの非同期実装のコツについて、引用元ドキュメントと共に説明します。

## FastAPIの非同期実装のコツ

### 1. 基本的な使い分け

#### awaitを使うライブラリがある場合
python
@app.get('/')
async def read_results():
    results = await some_library()
    return results

#### awaitをサポートしていないライブラリの場合
python
@app.get('/')
def results():
    results = some_library()
    return results

引用元: /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/async.md

### 2. データベース接続での実装

#### 非同期データベースライブラリ使用時
python
import asyncpg

async def get_database():
    return await asyncpg.connect("postgresql://...")

@app.get("/users/{user_id}")
async def read_user(user_id: int):
    conn = await get_database()
    user = await conn.fetchrow("SELECT * FROM users WHERE id = $1", user_id)
    await conn.close()
    return user

#### 同期データベースライブラリ使用時
python
import psycopg2

def get_database():
    return psycopg2.connect("postgresql://...")

@app.get("/users/{user_id}")
def read_user(user_id: int):
    conn = get_database()
    cursor = conn.cursor()
    cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))
    user = cursor.fetchone()
    conn.close()
    return user

### 3. 依存関係での非同期実装

python
from fastapi import Depends

async def get_db_connection():
    conn = await asyncpg.connect("postgresql://...")
    try:
        yield conn
    finally:
        await conn.close()

@app.get("/items/")
async def read_items(db = Depends(get_db_connection)):
    items = await db.fetch("SELECT * FROM items")
    return items

### 4. WebSocketでの非同期実装

python
from fastapi import WebSocket

@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    while True:
        data = await websocket.receive_text()
        await websocket.send_text(f"Message: {data}")

引用元: /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/advanced/websockets.md

### 5. パフォーマンス最適化のコツ

#### I/Oバウンド操作にはasync defを使用
- データベースクエリ
- API呼び出し
- ファイル読み書き

#### CPUバウンド操作には通常のdefを使用
- 複雑な計算処理
- 画像処理
- 機械学習の推論

### 6. 実装時の注意点

#### 混在使用が可能
python
@app.get("/sync")
def sync_endpoint():
    return {"message": "sync"}

@app.get("/async")
async def async_endpoint():
    await asyncio.sleep(1)
    return {"message": "async"}

#### エラーハンドリング
python
@app.get("/data")
async def get_data():
    try:
        result = await external_api_call()
        return result
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

### 7. よくある間違い

#### ❌ 間違い: async def内で同期関数を直接呼び出し
python
@app.get("/bad")
async def bad_example():
    # ブロッキング操作
    time.sleep(1)  # これは避ける
    return {"message": "bad"}

#### ✅ 正しい: 非同期版を使用
python
@app.get("/good")
async def good_example():
    await asyncio.sleep(1)  # 非同期版を使用
    return {"message": "good"}

引用元: /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/async.md

これらのコツを活用することで、FastAPIの非同期機能を最大限に活用し、高パフォーマンスなAPIを構築できます。

It created an answer quickly after searching, perhaps due to the small amount of documentation.
The source references are included, but there's no citation for database-related processes, so I asked about that.

> データベース関連はどこから引用しましたか？                                
> 申し訳ありません。データベース関連のコード例（asyncpgやpsycopg2を使った例）
は、私が一般的な知識から作成したもので、提供された知識ベースからの直接的な引                                         用ではありませんでした。
知識ベースから実際に引用した内容は以下の部分のみです：

## 実際の引用元

引用元: 
/Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/async.md

### 基本的な使い分けの部分：
python
# awaitを使うライブラリがある場合
@app.get('/')
async def read_results():
    results = await some_library()
    return results

# awaitをサポートしていないライブラリの場合  
@app.get('/')
def results():
    results = some_library()
    return results

### データベースに関する記述：
│ データベース、API、ファイルシステムなどと通信し、await の使用をサポート
していないサードパーティライブラリ (現在のほとんどのデータベースライブラリに
当てはまります) を使用している場合、次の様に、単に def を使用して通常通り 
path operation 関数 を宣言してください

WebSocketの例の引用元: 
/Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/advanced/websockets.md

データベース接続プールや具体的なデータベースライブラリの実装例については、知
識ベースに含まれていませんでした。

This isn't unique to this feature, but it seemed to be answering appropriately so I thought it was quoting, but there are areas that need caution.
If you want to get answers strictly from the documentation only, you need to control it with prompts. While it's difficult to completely prevent mixing, you could try something simple like including "Please don't create answers using knowledge outside the documentation" in your prompt.

> FastAPIの非同期実装のコツを教えてください、引用元ドキュメントも添えてください。ドキュメント外の知識では回答を作成しないでください。

Searching knowledge base for: FastAPI 非同期 async await 実装 コツ in context: baa17fca-1187-4250-a6bf-5baa17f7287e (using tool: knowledge)
Allow this action? Use 't' to trust (always allow) this tool for the session. [y/n/t]:

> t

 ✓ Found 5 results for query 'FastAPI 非同期 async await 実装 コツ'
 - Completed in 0.38s

> FastAPIの非同期実装のコツについて、ドキュメントから以下の要点をお伝えします：

## 基本的な使い分け

awaitを使用するライブラリを使っている場合：
python
@app.get('/')
async def read_results():
    results = await some_library()
    return results

awaitをサポートしていないライブラリを使っている場合：
python
@app.get('/')
def results():
    results = some_library()
    return results

## 重要なコツ

1. async defとdefの混在が可能
   - 必要に応じてdefとasync defを混在させることができ、FastAPIが適切に処理します

2. 迷った場合は通常のdefを使用
   - よく分からない場合は、通常のdefを使用することが推奨されています

3. awaitはasync def内でのみ使用可能
   - async defで定義された関数の内部でしかawaitは使用できません

## パフォーマンス最適化のポイント

I/Oバウンド操作（推奨：async def）：
- ネットワーク通信
- データベース操作
- ファイル読み書き
- リモートAPI呼び出し

CPUバウンド操作（推奨：通常のdef）：
- 複雑な数学計算
- 画像・音声処理
- 機械学習処理

## 技術的な詳細

- async defで定義された*path operation関数*を通常のdefで宣言すると、外部スレッドプールで実行されます
- ブロッキングI/Oを実行しない場合はasync defの使用が推奨されます
- FastAPIは両方の方式を適切に処理し、高いパフォーマンスを実現します

References:
[1] 並行処理と async / await - /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/async.md
[2] ベンチマーク - /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/benchmarks.md
[3] デプロイ - /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/deployment/index.md
[4] 環境変数 - /Users/xxx/sample-knowledge-management/fastapi/docs/ja/docs/environment-variables.md

Best Practices & Considerations

Here are some best practices mentioned in the documentation.

Organizing Knowledge Bases

It's important to name knowledge bases specifically, like "api-documentation" rather than "docs" to make it easier to determine which knowledge base to use.
Also, using --include and --exclude to focus on relevant files and reduce noise is important.
You can update your knowledge bases to the latest state using /knowledge update when documents are updated.

Tips for Effective Searching

It's best to avoid vague expressions and ask specific questions in natural language. If you don't get hits, try changing the wording.
This is a common but important point.

Managing Large Projects

It's good to divide large projects according to purpose. Separating by concerns works well. Add by directory unit rather than individual files, exclude build artifacts (target/**, node_modules/**, .git/**), and consider dividing into subdirectories according to the units you want to search and use cases.

Pattern Filtering

Setting default patterns is convenient. You want to filter out unnecessary items as much as possible as they can create noise in search results.

kiro-cli settings knowledge.defaultIncludePatterns '["**/*.rs", "**/*.md"]'
kiro-cli settings knowledge.defaultExcludePatterns '["target/**", "node_modules/**"]'

Supported Files

The following file types are supported.
Note that files with extensions other than these will not be indexed.

Text files: .txt, .log, .rtf, .tex, .rst

Markdown: .md, .markdown, .mdx

JSON: .json (treated as text for searchability)

Configuration: .ini, .conf, .cfg, .properties, .env

Data files: .csv, .tsv

Web formats: .svg (text-based)

Code files: .rs, .py, .js, .jsx, .ts, .tsx, .java, .c, .cpp, .h, .hpp, .go, .rb, .php, .swift, .kt, .kts, .cs, .sh, .bash, .zsh, .html, .htm, .xml, .css, .scss, .sass, .less, .sql, .yaml, .yml, .toml

Special files: Dockerfile, Makefile, LICENSE, CHANGELOG, README (files without extensions)

Agent Separation

This time I added a knowledge base to the default agent, but if you keep adding more and more, the data might become mixed, and searches might not be performed on the appropriate knowledge base. When you want to separate knowledge bases, use the custom agent feature to separate them according to purpose.

Knowledge bases are created on a per-agent basis.

~/.kiro/knowledge_bases/
├── kiro_cli_default/          # Default agent
│   ├── contexts.json
│   ├── context-id-1/
│   │   ├── data.json
│   │   └── bm25_data.json
│   └── context-id-2/
│       └── data.json
├── my-custom-agent_<code>/    # Custom agent
│   ├── contexts.json
│   └── context-id-3/
│       └── data.json

So, when you switch to a custom agent, the knowledge bases you've added so far won't be displayed.

# Specify custom agent
kiro-cli chat --agent  operations-review

[operations-review] > /knowledge show

👤 Agent (operations-review):
    <none>

I confirmed that they are separated by agent. It's also possible to add a unique knowledge base to this agent.

Conclusion

This time, I tried out Kiro CLI's Knowledge management.

It's convenient to be able to easily index local files and perform semantic searches from within a chat session!
It seems particularly useful for projects with lots of documentation when you want to quickly try something locally.
I'd like to introduce it in another blog if there are convenient scenarios. You're probably also curious about its accuracy.
However, it's worth noting that as it's currently an experimental feature, there's a possibility of breaking changes.

I hope this article was helpful. Thank you for reading until the end!