mirror of
https://gitcode.com/JianFeeeee/HomeAgent.git
synced 2026-09-21 17:38:10 +00:00
feat: 实现多语言文档架构
- 创建 docs/zh/ 和 docs/en/ 目录结构 - 迁移文档至多语言目录,添加中英文切换链接 - 创建 README_EN.md 英文版本 - 更新 README.md 添加语言切换链接 - 删除旧文档和临时文件
This commit is contained in:
104
docs/en/ADAPTER.md
Normal file
104
docs/en/ADAPTER.md
Normal file
@ -0,0 +1,104 @@
|
||||
**中文** | [English](../zh/ADAPTER.md)
|
||||
|
||||
# Lua Adapter — LLM Source Adaptation Guide
|
||||
|
||||
Each LLM API source corresponds to a Lua script, responsible for request transformation (Go unified format → API format) and response transformation (API format → Go unified format).
|
||||
|
||||
## Adapter Contract
|
||||
|
||||
The Lua script must return a table containing the following fields and functions:
|
||||
|
||||
```lua
|
||||
local adapter = {}
|
||||
|
||||
-- Metadata
|
||||
adapter.name = "my_provider" -- Unique identifier, matches adapter field in config
|
||||
adapter.version = "2.0.0"
|
||||
adapter.endpoint = "/v1/chat/completions" -- API path, appended to base_url
|
||||
adapter.headers = {} -- Additional HTTP request headers
|
||||
|
||||
-- Request transformation: Go → API
|
||||
function adapter.transform_request(raw_json)
|
||||
-- raw_json: Go's CompletionRequest JSON string
|
||||
-- Returns: JSON string to send to API
|
||||
return transformed_json
|
||||
end
|
||||
|
||||
-- Response transformation: API → Go
|
||||
function adapter.transform_response(raw_json)
|
||||
-- raw_json: API's raw response JSON string
|
||||
-- Returns: Unified CompletionResponse JSON string
|
||||
-- Unified format:
|
||||
-- { content: "", finish_reason: "", token_usage: { prompt: N, completion: N, total: N }, tool_calls?: [...] }
|
||||
return unified_json
|
||||
end
|
||||
|
||||
-- Stream chunk transformation (optional)
|
||||
function adapter.transform_stream_chunk(raw_line)
|
||||
-- raw_line: JSON string after data: in SSE
|
||||
-- Returns: JSON of { content: "", done: bool }, return "" to skip this chunk
|
||||
return chunk_json
|
||||
end
|
||||
|
||||
return adapter
|
||||
```
|
||||
|
||||
## Unified CompletionRequest Format (Go → Adapter)
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "deepseek-v4-flash",
|
||||
"messages": [
|
||||
{ "role": "system", "content": "..." },
|
||||
{ "role": "user", "content": "..." },
|
||||
{ "role": "assistant", "content": "...", "tool_calls": [...] }
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"max_tokens": 4096,
|
||||
"stream": false,
|
||||
"tools": [...],
|
||||
"tool_choice": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
## Unified CompletionResponse Format (Adapter → Go)
|
||||
|
||||
```json
|
||||
{
|
||||
"content": "Response content",
|
||||
"finish_reason": "stop",
|
||||
"token_usage": { "prompt": 10, "completion": 20, "total": 30 },
|
||||
"tool_calls": [
|
||||
{ "id": "call_xxx", "type": "function", "name": "tool_name", "arguments": { "key": "val" } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Lua VM Built-in Functions
|
||||
|
||||
`json.encode(table)` — Encode Lua table to JSON string
|
||||
|
||||
`json.decode(string)` — Decode JSON string to Lua table
|
||||
|
||||
`log(level, message)` — Output log (level: info/warn/error)
|
||||
|
||||
## Adapting Typical APIs
|
||||
|
||||
| API | endpoint | auth method | Format differences |
|
||||
|-----|----------|-------------|-------------------|
|
||||
| **OpenAI** | `/chat/completions` | `Authorization: Bearer <key>` | Standard OpenAI format |
|
||||
| **DeepSeek** | `/chat/completions` | `Authorization: Bearer <key>` | OpenAI compatible, forces temperature=0 |
|
||||
| **Anthropic** | `/v1/messages` | `x-api-key: <key>` | Messages API, system message separated, content as block array |
|
||||
| **Gemini** | `/v1/models/{model}:generateContent` | `?key=<key>` or Bearer | contents/parts format, role uses model instead of assistant |
|
||||
| **Mistral** | `/v1/chat/completions` | `Authorization: Bearer <key>` | OpenAI compatible |
|
||||
| **Groq** | `/openai/v1/chat/completions` | `Authorization: Bearer <key>` | OpenAI compatible |
|
||||
| **GitHub Models** | `/chat/completions` | `Authorization: Bearer <pat>` | OpenAI compatible |
|
||||
| **Ollama** | `/api/chat` | None | Different options format |
|
||||
|
||||
## Steps to Add a New Source
|
||||
|
||||
1. Create `<name>.lua` under `internal/lua/adapters/`
|
||||
2. Script defines `transform_request` and `transform_response`
|
||||
3. (Optional) Define `transform_stream_chunk` for streaming support
|
||||
4. Build verification: `go build ./cmd/homed/`
|
||||
5. Test verification: `go test ./...`
|
||||
347
docs/en/ARCHITECTURE.md
Normal file
347
docs/en/ARCHITECTURE.md
Normal file
@ -0,0 +1,347 @@
|
||||
**中文** | [English](../zh/ARCHITECTURE.md)
|
||||
|
||||
# HomeAgent Architecture
|
||||
|
||||
The kernel performs zero IO; all external interaction comes from plugins.
|
||||
|
||||
## Message Processing Flow
|
||||
|
||||
### Full Pipeline
|
||||
|
||||
```
|
||||
External input (via plugin InjectInput)
|
||||
│
|
||||
▼
|
||||
eventLoop() → processTextInput()
|
||||
│
|
||||
├── on_input stage Plugins can intercept/rewrite/short-circuit
|
||||
├── Context.Append Record to context window
|
||||
├── Context.Prune Low-relevance events archived to Document
|
||||
├── buildMemoryContext() Indexer recall → GraphDB BFS traversal
|
||||
│
|
||||
├── pre_action stage Plugins can inject system messages
|
||||
│
|
||||
├── [Tool Loop] process()
|
||||
│ ├── buildSystemPrompt Persona + Memory + Knowledge + Context
|
||||
│ ├── buildToolDefs Built-in tools + Plugin tools
|
||||
│ ├── provider.Chat() LLM call
|
||||
│ ├── post_action stage Plugins see LLM output + tool list
|
||||
│ ├── Has tools?
|
||||
│ │ ├── before_toolcall Plugins can reject/modify params
|
||||
│ │ ├── executeToolCall Route to plugin/built-in
|
||||
│ │ ├── after_toolcall Plugins can modify results
|
||||
│ │ └── → back to post_action
|
||||
│ └── No tools → exit loop
|
||||
│
|
||||
├── Context.Append(response)
|
||||
├── before_output stage Plugins can modify final text
|
||||
├── emitResponse() Send via output_send
|
||||
└── after_output stage Read-only, cleanup
|
||||
```
|
||||
|
||||
Code: `internal/agent/core/agent.go` — `process()` is the main tool loop
|
||||
|
||||
### 7 Stage Hooks
|
||||
|
||||
| Stage | Trigger | Plugin Capabilities |
|
||||
|-------|---------|---------------------|
|
||||
| `on_input` | Message arrives at Agent, zero processing | Blacklist/rate-limit/short-circuit reply |
|
||||
| `pre_action` | Context ready, before LLM call | Inject external data into context |
|
||||
| `post_action` | LLM returns text + tool list | Sensitive word filter/forced redirect |
|
||||
| `before_toolcall` | Before single tool execution | Audit/reject/modify params |
|
||||
| `after_toolcall` | After single tool execution | Desensitize/sort results |
|
||||
| `before_output` | Final text ready, before sending | Format adaptation |
|
||||
| `after_output` | Already sent | Statistics/logging |
|
||||
|
||||
Code: `internal/agent/core/stages.go` — `StageHost` orchestration
|
||||
|
||||
### Loop Rules
|
||||
|
||||
`post_action → [before_toolcall → execute → after_toolcall] → post_action` forms the inner loop.
|
||||
Exit conditions: LLM has no tool calls / all rejected / exceeded limit.
|
||||
|
||||
### Short-Circuit Rules
|
||||
|
||||
Setting `ctx.Response` at any stage jumps to `after_output`.
|
||||
|
||||
## Three-Layer Memory
|
||||
|
||||
### Memory Flow
|
||||
|
||||
```
|
||||
① Context (Working Window)
|
||||
RelevanceContext — In-memory events[] + JSON persistence
|
||||
Append: Each input, Vectorize(char 1-2gram TF-IDF)
|
||||
Prune: TF-IDF CosineSimilarity, keep topK + last 10
|
||||
├── Keep → timeline → system prompt (sorted by time)
|
||||
└── Low score → Document layer archive (ContextToDoc)
|
||||
Save: 5s debounce write to disk
|
||||
|
||||
↓ Prune archive ↑ LLM active recall
|
||||
|
||||
② Document (File Memory)
|
||||
DocStore — JSON files + TF-IDF InvertedIndex
|
||||
Write: Prune archive / doc_commit / Graph snapshot (syncGraphToDocs)
|
||||
Read:
|
||||
├── Auto-inject: Query(input, top3) → [Related Memory Docs] → system prompt (read-only, update AccessCount)
|
||||
└── LLM active: doc_query → Consume(read and delete)
|
||||
→ context.Append{Timestamp: d.CreatedAt, Source: "cold_storage"} per doc
|
||||
→ Docs written to context timeline with original timestamps, deleted from docStore
|
||||
Cold: FindColdDocs(72h, ≤2 accesses) → docToTriples → Graph
|
||||
|
||||
↓ Cold doc distillation ↑ Auto recall
|
||||
|
||||
③ Graph (Graph Database)
|
||||
SQLite — entities + relations tables
|
||||
Write: memory_commit / cold doc distillation / Pipeline rule distillation
|
||||
Read:
|
||||
├── Auto recall: Indexer.BuildContext(input)
|
||||
│ → TF-IDF entity name search → BFS depth=2
|
||||
│ → [Memory Index] → system prompt
|
||||
└── LLM active: memory_recall / doc_query
|
||||
Social: person_query / set_trait / relate (wraps GraphDB)
|
||||
|
||||
④ Distillation Pipeline (30min heartbeat)
|
||||
distillContext → window > 2×maxSize → force Prune
|
||||
syncGraphToDocs → Graph snapshot to Document (cross-layer searchable)
|
||||
reorgGraph:
|
||||
Step1: indexer.Sync — rebuild entity TF-IDF vector index
|
||||
Step2: docStore.Reindex — rebuild document TF-IDF vector index
|
||||
Step3: Cold docs → docToTriples → GraphDB.Commit
|
||||
Step4: Entity similarity (Bigram Jaccard > 0.75) → consolidation → LLM decides merge
|
||||
Step5: evaluateGraphQuality → LLM decides keep/delete
|
||||
|
||||
⑤ Pipeline Rule Distiller (every heartbeat)
|
||||
distillOnce → regex match personal info:
|
||||
我叫X / 我住在X / 我喜欢X / 我X岁 / 我的工作是X
|
||||
→ triples → GraphDB.Commit
|
||||
```
|
||||
|
||||
### TF-IDF Vectorization (char 1-2 gram)
|
||||
|
||||
TF-IDF is the core algorithm running through all three memory layers, used in 4 independent locations:
|
||||
|
||||
| Location | File | Purpose | Algorithm |
|
||||
|----------|------|---------|-----------|
|
||||
| Context Prune | `context.go:162` | Trim low-relevance context events | CosineSimilarity(queryVec, evt.Vector) |
|
||||
| DocStore Query | `document.go:205` | Recall related content from document memory | InvertedIndex + CosineSimilarity |
|
||||
| Indexer Entity Search | `indexer.go:149` | Recall related entities from Graph | InvertedIndex + CosineSimilarity |
|
||||
| Entity Similarity Detection | `agent.go:2297` | Detect similar entities in Graph | Bigram Jaccard (>0.75 → consolidation) |
|
||||
|
||||
### Context Layer
|
||||
|
||||
`internal/agent/core/context.go` — `RelevanceContext`
|
||||
- Maintains recent event list, writes JSON on each Append/Prune to prevent data loss
|
||||
- TF-IDF relevance scoring on user input, keeps topK
|
||||
|
||||
### Document Layer
|
||||
|
||||
`internal/memory/document/doc.go` — `Store`
|
||||
- Consume-on-read mode: deleted after `doc_query` retrieval
|
||||
- TF-IDF index with character bigram + inverted index
|
||||
|
||||
### Graph Layer
|
||||
|
||||
`internal/memory/graph.go` — `GraphDB`
|
||||
- SQLite WAL mode, two tables
|
||||
- `Commit(triples)` — UPSERT entities + INSERT relations
|
||||
- `Recall(keywords, depth)` — Keyword LIKE search + BFS traversal
|
||||
|
||||
### Memory Tools (LLM-callable)
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| `memory_recall` | Recall from Graph |
|
||||
| `memory_commit` | Write triples to Graph |
|
||||
| `memory_introspect` | View memory statistics |
|
||||
| `doc_query` | Search from Document |
|
||||
| `doc_commit` | Write to Document |
|
||||
|
||||
### Other Memory Layers
|
||||
|
||||
- **Social** (`internal/memory/social/social.go`) — Persona traits and relationship network, wraps GraphDB entity types
|
||||
- **Text Memory** (`internal/memory/text/text.go`) — Raw conversation JSONL logs, rotation strategy
|
||||
- **Memory Indexer** (`internal/memory/indexer.go`) — Entity vectorization, auto-inject into system prompt
|
||||
|
||||
### Distillation Pipeline
|
||||
|
||||
`internal/memory/pipeline/pipeline.go`
|
||||
- 10-minute tick, 7-day retention
|
||||
- Rule-based triple extraction (name / location / likes / age / job patterns)
|
||||
- Writes to GraphDB
|
||||
|
||||
### Context Pruning
|
||||
|
||||
```
|
||||
Heartbeat 30min:
|
||||
├── distillContext() — Distill current context
|
||||
├── syncGraphToDocs() — Graph → Document sync
|
||||
└── reorgGraph()
|
||||
├── Indexer.Sync()
|
||||
├── DocStore.Reindex()
|
||||
├── Cold docs → Graph
|
||||
└── Entity conflicts → enqueueConsolidationTask()
|
||||
│
|
||||
selfInputCh → LLM decides merge/skip
|
||||
```
|
||||
|
||||
Entity conflict detection heuristic (bigram Jaccard > 0.5), routed through `selfInputCh` internal channel, LLM makes the final merge decision.
|
||||
|
||||
## Knowledge Base
|
||||
|
||||
`internal/knowledge/knowledge.go`
|
||||
- File directory `knowledge/<name>/content.md`
|
||||
- Independent TF-IDF index, separate from memory system
|
||||
- `knowledge_search` / `knowledge_create` / `knowledge_list`
|
||||
|
||||
## Provider & Lua Adapter Layer
|
||||
|
||||
```
|
||||
Agent
|
||||
│
|
||||
▼
|
||||
Provider Interface (Name / Chat / ChatStream)
|
||||
│
|
||||
├── OpenAIProvider — Standard OpenAI API
|
||||
├── OllamaProvider — Local Ollama
|
||||
└── LuaAdaptedProvider (primary)
|
||||
├── Serialize CompletionRequest → JSON
|
||||
├── adapter.transform_request() → API format
|
||||
├── HTTP request + adapter.headers
|
||||
├── adapter.transform_response() → unified format
|
||||
└── Deserialize
|
||||
```
|
||||
|
||||
Code: `internal/agent/api/provider.go`
|
||||
|
||||
ProviderManager manages multiple sources, fallback in registration order. Lua adapters at `internal/lua/adapters/`, each `.lua` script defines `transform_request` / `transform_response` / `transform_stream_chunk`.
|
||||
|
||||
VM built-ins: `json.encode` / `json.decode` / `log` / `http_get` / `http_post`.
|
||||
|
||||
## Plugin System
|
||||
|
||||
### Three Loading Methods
|
||||
|
||||
| Method | Registration Mechanism | Compilation | Usage |
|
||||
|--------|----------------------|-------------|-------|
|
||||
| Built-in | `init()` → `RegisterFactory` | `internal/plugins/` compiled into kernel | webui/cli/timer/mcp etc. |
|
||||
| External `.so`/`.dll` | `plugin.Open` dynamic loading | `-buildmode=plugin` | qq/files/web/memo etc. |
|
||||
| Lua script plugin | Parse `main.lua` to register tools | No compilation, hot-reload | luaplugintest/testlua etc. |
|
||||
| SKILL plugin | Parse `SKILL.md` | Markdown definition | OpenClaw compatible |
|
||||
|
||||
Built-in plugin registration: `internal/plugins/all.go` blank imports → each plugin `init()` → `Registry.Load()` scans directory to match factory.
|
||||
External plugin loading: `internal/plugin/dynamic.go` → copy to SHA256 temp path (bypass `plugin.Open` path cache) → `Open` + `Lookup("NewPlugin")`.
|
||||
Lua script plugin loading: `internal/lua/` → parse `main.lua` via Lua VM, call `start()` to register tools.
|
||||
|
||||
### PluginSDK Three Channels
|
||||
|
||||
```
|
||||
Plugin ──→ Kernel
|
||||
|
||||
RegisterTool(name, fn) ──→ buildToolDefs() / executeToolCall()
|
||||
RegisterStage(stage, fn) ──→ runStage() called at corresponding phase
|
||||
Subscribe(event, fn) ──→ Publish() notify all subscribers
|
||||
```
|
||||
|
||||
`internal/sdk/` bridges external SDK interface to kernel, defines complete PluginSDK:
|
||||
|
||||
```go
|
||||
sdk.RegisterTool(name, def, handler)
|
||||
sdk.RegisterStage(stage, handler)
|
||||
sdk.Publish(event)
|
||||
sdk.InjectInput(source, channel, payload)
|
||||
sdk.InjectInterrupt(source, channel, payload)
|
||||
sdk.Memory().Recall/Commit
|
||||
sdk.Knowledge().Search/Create
|
||||
sdk.Settings().Get/Set/List
|
||||
```
|
||||
|
||||
### Plugin Interface
|
||||
|
||||
```go
|
||||
type Plugin interface {
|
||||
Name() string
|
||||
Start(sdk *PluginSDK) error
|
||||
Stop() error
|
||||
}
|
||||
```
|
||||
|
||||
## Interrupt Mechanism
|
||||
|
||||
```
|
||||
interceptLoop (goroutine)
|
||||
├── InputInterruptChan() ← Timer/message notifications
|
||||
├── (a) cancelLLM() → Cancel Provider HTTP request
|
||||
├── (b) interceptCh → process() pre-loop read [interrupt message]
|
||||
└── (c) InjectInput() → Trigger new processing when idle
|
||||
```
|
||||
|
||||
Three delivery paths:
|
||||
|
||||
| Path | Effect | Timing |
|
||||
|------|--------|--------|
|
||||
| cancelLLM | Cancel current HTTP request | On context.Canceled |
|
||||
| interceptCh | Insert `[interrupt message]` in process() | Before each LLM call |
|
||||
| InjectInput | Trigger new processing when eventLoop is idle | No ongoing request |
|
||||
|
||||
Code: `internal/agent/core/agent.go` — `interceptLoop` / `drainInterrupt`
|
||||
|
||||
## Configuration System
|
||||
|
||||
`internal/config/registry.go` — ConfigRegistry
|
||||
|
||||
- SQLite storage, `config` table + `config_<plugin>` independent tables
|
||||
- Namespaces: `core.*` / `plugin.<name>.*`
|
||||
- `RegisterDefault` inserts ~80 default keys (seeds for 8 LLM sources)
|
||||
- WebUI settings page `/api/v1/settings` for read/write
|
||||
|
||||
## Code Structure
|
||||
|
||||
```
|
||||
cmd/homed/main.go — Entry: assembles all subsystems
|
||||
cmd/waiter/main.go — CLI client (Unix socket)
|
||||
internal/
|
||||
├── agent/
|
||||
│ ├── core/ — Agent core (eventLoop/process/stages/context)
|
||||
│ ├── api/ — Provider interface + LuaAdaptedProvider
|
||||
│ ├── io/ — IOManager (queue/interrupt/output)
|
||||
│ └── personal.go — Persona loading
|
||||
├── plugin/
|
||||
│ ├── registry.go — Registry + lifecycle
|
||||
│ ├── dynamic.go — .so dynamic loader
|
||||
│ └── manifest.go — plugin.json metadata
|
||||
├── plugins/ — Built-in plugin implementations
|
||||
│ ├── all.go — Blank imports
|
||||
│ ├── webui/ — HTTP server + embedded SPA
|
||||
│ ├── cli/ — Unix socket CLI
|
||||
│ ├── timer/ — Timer
|
||||
│ ├── cmd/ — Command execution
|
||||
│ ├── mcp/ — MCP protocol
|
||||
│ ├── openclaw/ — OpenClaw compatible
|
||||
│ ├── agentcli/ — PTY terminal
|
||||
│ ├── healthcheck/ — Health check
|
||||
│ └── pluginmgr/ — Plugin manager
|
||||
├── internal/sdk/ — PluginSDK definitions
|
||||
│ ├── plugin.go — Plugin interface + PluginSDK
|
||||
│ ├── memory.go — MemoryAPI
|
||||
│ ├── knowledge.go — KnowledgeAPI
|
||||
│ ├── settings.go — SettingsAPI
|
||||
│ └── llm.go — LLMAPI
|
||||
├── memory/
|
||||
│ ├── graph.go — SQLite graph database
|
||||
│ ├── indexer.go — Graph → vector index
|
||||
│ ├── vector/store.go — TF-IDF vector engine
|
||||
│ ├── document/doc.go — Document memory
|
||||
│ ├── text/text.go — Text logs
|
||||
│ └── pipeline/ — Distiller
|
||||
├── knowledge/knowledge.go — Knowledge base
|
||||
├── lua/
|
||||
│ ├── vm.go — Lua VM (json/log/http)
|
||||
│ └── adapters/ — 8 LLM adapter scripts
|
||||
├── config/registry.go — SQLite config center
|
||||
├── events/bus.go — Event bus
|
||||
├── tracker/ — OverlayFS change tracking
|
||||
├── supervisor/ — Daemon management
|
||||
├── snapshot/ — Snapshots
|
||||
└── tokenizer/ — Chinese tokenization (jieba wrapper)
|
||||
```
|
||||
74
docs/en/OVERVIEW.md
Normal file
74
docs/en/OVERVIEW.md
Normal file
@ -0,0 +1,74 @@
|
||||
**中文** | [English](../zh/OVERVIEW.md)
|
||||
|
||||
# HomeAgent — Project Overview
|
||||
|
||||
## What Is This
|
||||
|
||||
HomeAgent is a continuously-running personal intelligent Agent framework.
|
||||
|
||||
Core architecture: a long-running kernel process (`homed`) that connects to various IO channels (QQ, Web, CLI, etc.) through a plugin system. The kernel handles LLM orchestration, memory management, and knowledge retrieval; plugins handle all external IO — sending/receiving messages, file operations, web search, etc.
|
||||
|
||||
### Key Innovations
|
||||
|
||||
**Separation of Core Domain and Application Domain** — This is the first Agent framework to explicitly make this distinction. The kernel (core domain) performs zero IO; all IO capabilities belong to plugins (application domain). The boundary is clearly defined through PluginSDK:
|
||||
- Plugins register tools (Tool) with the kernel for LLM invocation
|
||||
- Plugins hook into the processing pipeline (Stage) to intercept/rewrite message flow at various phases
|
||||
- Plugins subscribe/publish events (Event) for loosely-coupled communication
|
||||
- Plugins queue or interrupt input delivery through IO API
|
||||
|
||||
The significance: the kernel stays pure (zero IO, only orchestration and memory), plugins stay flexible (each does its job, hot-loadable), with no cross-contamination.
|
||||
|
||||
**Three-Layer Memory Architecture** — Solves the memory decay problem for long-running agents:
|
||||
- **Context Layer**: In-memory TF-IDF scored event window, maintains recent context in real-time, low-relevance events automatically sink to the next layer
|
||||
- **Document Layer**: JSON files + TF-IDF vector-indexed temporary memory, supports explicit submission and implicit archival, cold data distills to Graph
|
||||
- **Graph Layer**: SQLite graph database, persists entities and relations, BFS traversal recall, distillation pipeline extracts triples from conversations
|
||||
|
||||
Three progressive layers: context → cold archive → long-term graph memory, ensuring the agent doesn't degrade over time.
|
||||
|
||||
## What It Actually Does
|
||||
|
||||
Code is in the project root, implemented in Go.
|
||||
|
||||
**Kernel** (`internal/agent/core/agent.go`):
|
||||
- Maintains a message loop (`eventLoop`), queuing input from the IO layer
|
||||
- Each input goes through the full processing pipeline: memory recall → persona injection → LLM call → tool execution → output delivery
|
||||
- LLM calls abstracted through Provider interface, supports 8 LLM sources with automatic fallback
|
||||
- Context management (`context.go`) based on TF-IDF scoring, automatic pruning of low-relevance events
|
||||
|
||||
**Memory System** (`internal/memory/`):
|
||||
- **GraphDB** (`graph.go`) — SQLite, entities + relations tables, BFS traversal
|
||||
- **Document Store** (`document/doc.go`) — Temporary memory, JSON files + TF-IDF vector index, consume-on-read
|
||||
- **Text Memory** (`text/text.go`) — Raw conversation logs, JSONL file rotation
|
||||
- **Social Store** (`social/social.go`) — Persona traits + relationship network, wraps GraphDB
|
||||
- **Memory Indexer** (`indexer.go`) — Auto-vectorizes GraphDB entities, recalls and injects into system prompt on user input
|
||||
|
||||
**Knowledge Base** (`internal/knowledge/knowledge.go`):
|
||||
- File system directory `knowledge/<name>/content.md`
|
||||
- TF-IDF vector search, independent index instance from the memory system
|
||||
- LLM operates via three tools: `knowledge_search` / `knowledge_create` / `knowledge_list`
|
||||
|
||||
**Plugin System** (`internal/plugin/`):
|
||||
- Built-in plugins: Go `init()` self-registration, compiled into kernel
|
||||
- External plugins: Go `-buildmode=plugin` compiled to `.so`/`.dll`, dynamically loaded via `plugin.Open`; also supports Lua script plugins
|
||||
- PluginSDK (`internal/sdk/`) defines three channels: RegisterTool / RegisterStage / Subscribe
|
||||
- 7 stage hooks: on_input → pre_action → post_action → before_toolcall → after_toolcall → before_output → after_output
|
||||
|
||||
**LLM Provider** (`internal/agent/api/provider.go`):
|
||||
- Provider interface: Name / Chat / ChatStream
|
||||
- Three implementations: OpenAIProvider (standard OpenAI API), OllamaProvider (local), LuaAdaptedProvider (Lua adapter)
|
||||
- LuaAdapter located at `internal/lua/adapters/`, each LLM source has a corresponding `.lua` script
|
||||
- 8 built-in adapters: deepseek / openai / anthropic / gemini / mistral / groq / github / ollama
|
||||
|
||||
**WebUI** (`internal/plugins/webui/`):
|
||||
- Embedded SPA dashboard (`dashboard.html` packaged via `//go:embed`)
|
||||
- REST API: status query, configuration management, memory operations, knowledge management, plugin management
|
||||
- OpenAI API-compatible `/v1/chat/completions` endpoint
|
||||
- SSE event stream `/api/v1/chat/events`
|
||||
|
||||
## Project Status
|
||||
|
||||
Core functionality is operational. Plugin system and SDK are ready for independent external plugin development.
|
||||
|
||||
- Built-in plugins: webui / cli / timer / cmd / mcp / agentcli / healthcheck / pluginmgr / openclaw / files
|
||||
- External plugin examples ([homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo `example/`, both Go and Lua types): qq / files / web / memo / bili / editdoc / a2a / ocr / sanitizer / luaplugintest / testlua
|
||||
- Distribution: `.hmap` plugin package format, installable via WebUI
|
||||
477
docs/en/PLUGIN_DEV.md
Normal file
477
docs/en/PLUGIN_DEV.md
Normal file
@ -0,0 +1,477 @@
|
||||
**中文** | [English](../zh/PLUGIN_DEV.md)
|
||||
|
||||
# HomeAgent Plugin Development Guide
|
||||
|
||||
## Overview
|
||||
|
||||
All external interaction capabilities of HomeAgent comes from plugins. Plugins interact with the kernel through `PluginSDK` (Go API).
|
||||
|
||||
**SDK Repository**: Plugin development tools, template code, and example plugins are hosted in the [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repository.
|
||||
|
||||
```bash
|
||||
git clone https://gitcode.com/JianFeeeee/homeagent-sdk.git
|
||||
cd homeagent-sdk
|
||||
```
|
||||
|
||||
Each plugin implements a three-method interface:
|
||||
|
||||
```go
|
||||
type Plugin interface {
|
||||
Name() string
|
||||
Start(sdk *PluginSDK) error
|
||||
Stop() error
|
||||
}
|
||||
```
|
||||
|
||||
### Three Development Methods
|
||||
|
||||
| Method | Use Case | Complexity |
|
||||
|--------|----------|------------|
|
||||
| **Dynamic .so/.dll plugin (recommended)** | Independently distributed third-party plugins | Medium, generated using `plugindev` toolchain |
|
||||
| **Built-in plugin** | Released with HomeAgent | Simple, requires merging into main repo |
|
||||
| **Lua script plugin** | Lightweight rapid prototyping | Simple, generated using `plugindev init --lua` |
|
||||
|
||||
---
|
||||
|
||||
## 1. Quick Start: Using the plugindev Toolchain
|
||||
|
||||
`plugindev` is the unified plugin development toolchain provided in the SDK repository, supporting both Go and Lua plugin types.
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
cd homeagent-sdk/tools/plugindev
|
||||
go build -o plugindev.exe
|
||||
# Add plugindev.exe to PATH or use directly
|
||||
```
|
||||
|
||||
### Creating a Go Plugin
|
||||
|
||||
```bash
|
||||
plugindev init myplugin
|
||||
cd myplugin
|
||||
# Edit plugin code
|
||||
vim plugin.go
|
||||
# Build and package
|
||||
plugindev build
|
||||
# Output: dist/myplugin_linux_amd64.hmap (or windows_amd64)
|
||||
```
|
||||
|
||||
### Creating a Lua Plugin
|
||||
|
||||
```bash
|
||||
plugindev init myluaplugin --lua
|
||||
cd myluaplugin
|
||||
# Edit plugin code
|
||||
vim main.lua
|
||||
# Local test
|
||||
lua main.lua
|
||||
# Build and package
|
||||
plugindev build
|
||||
# Output: dist/myluaplugin_lua.hmap
|
||||
```
|
||||
|
||||
### Template Project Structure
|
||||
|
||||
**Go plugin**:
|
||||
|
||||
```
|
||||
myplugin/
|
||||
├── plg.json — Plugin metadata (name, version, entry, target platform)
|
||||
├── main.go — Entry point (compiled for non-Windows or non-cgo)
|
||||
├── plugin.go — Plugin implementation (Plugin interface)
|
||||
├── go.mod — Go module definition
|
||||
└── README.md — Documentation
|
||||
```
|
||||
|
||||
**Lua plugin**:
|
||||
|
||||
```
|
||||
myluaplugin/
|
||||
├── plg.json — Plugin metadata (entry: "main.lua", targets: "lua")
|
||||
├── main.lua — Plugin implementation (Lua version of Plugin interface)
|
||||
├── sdk.lua — SDK mock layer (supports `lua main.lua` standalone testing)
|
||||
└── README.md — Documentation
|
||||
```
|
||||
|
||||
### Build & Package
|
||||
|
||||
`plugindev build` automatically handles compilation and packaging:
|
||||
|
||||
```bash
|
||||
cd myplugin
|
||||
plugindev build
|
||||
```
|
||||
|
||||
Execution process:
|
||||
1. Reads `plg.json` to determine target platform
|
||||
2. **Go plugin**: Runs `go build -buildmode=plugin` (Linux) or `-buildmode=c-shared` (Windows)
|
||||
3. **Lua plugin**: Packages source code directly, no compilation needed
|
||||
4. Generates `plugin.json` manifest file
|
||||
5. Packages as `.hmap` distribution (zip format, containing `plugin.json` + `plugin.so`/`plugin.dll`/`main.lua`)
|
||||
|
||||
Output in `dist/` directory:
|
||||
```
|
||||
dist/
|
||||
├── myplugin_linux_amd64.hmap # Go plugin Linux version
|
||||
├── myplugin_windows_amd64.hmap # Go plugin Windows version
|
||||
└── myplugin_lua.hmap # Lua plugin
|
||||
```
|
||||
|
||||
### Deployment
|
||||
|
||||
Install via PluginMgr HTTP API:
|
||||
|
||||
```bash
|
||||
# Kernel PluginMgr listens on :9876
|
||||
curl -X POST http://127.0.0.1:9876/plugins \
|
||||
-F "file=@dist/myplugin_linux_amd64.hmap"
|
||||
```
|
||||
|
||||
Or upload via WebUI plugin management page.
|
||||
|
||||
---
|
||||
|
||||
## 2. Go Plugin Development in Detail
|
||||
|
||||
### Plugin Interface
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import "gitcode.com/JianFeeeee/homeagent-sdk/sdk"
|
||||
|
||||
type Plugin struct {
|
||||
name string
|
||||
sdk *sdk.PluginSDK
|
||||
}
|
||||
|
||||
func (p *Plugin) Name() string { return p.name }
|
||||
|
||||
func (p *Plugin) Start(s *sdk.PluginSDK) error {
|
||||
p.sdk = s
|
||||
// Register config items, tools, stage hooks, etc.
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Plugin) Stop() error {
|
||||
// Clean up resources
|
||||
return nil
|
||||
}
|
||||
|
||||
// NewPluginFactory creates plugin instance (called by main.go or Windows bridge)
|
||||
func NewPluginFactory(name string, config map[string]interface{}) (sdk.Plugin, error) {
|
||||
return &Plugin{name: name}, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Entry Point
|
||||
|
||||
`main.go` provides the `NewPlugin` export function, which is the entry point when the kernel loads the plugin:
|
||||
|
||||
```go
|
||||
//go:build !windows || !cgo
|
||||
|
||||
package main
|
||||
|
||||
import "gitcode.com/JianFeeeee/homeagent-sdk/sdk"
|
||||
|
||||
func NewPlugin(name string, config map[string]interface{}) (sdk.Plugin, error) {
|
||||
return NewPluginFactory(name, config)
|
||||
}
|
||||
```
|
||||
|
||||
For Windows `-buildmode=c-shared`, `plugindev build` auto-generates C ABI bridge code, no manual handling needed.
|
||||
|
||||
### PluginSDK Core API
|
||||
|
||||
#### Tool Registration — Make your capabilities callable by LLM
|
||||
|
||||
```go
|
||||
s.RegisterTool("weather_query", sdk.ToolDef{
|
||||
Name: "weather_query",
|
||||
Description: "Query weather for a specified city",
|
||||
Parameters: map[string]interface{}{
|
||||
"type": "object",
|
||||
"properties": map[string]interface{}{
|
||||
"city": map[string]interface{}{
|
||||
"type": "string",
|
||||
"description": "City name, e.g. Beijing",
|
||||
},
|
||||
},
|
||||
"required": []string{"city"},
|
||||
},
|
||||
}, func(args map[string]interface{}) (interface{}, error) {
|
||||
city, _ := args["city"].(string)
|
||||
return map[string]interface{}{
|
||||
"city": city,
|
||||
"temp": 25,
|
||||
"weather": "Sunny",
|
||||
}, nil
|
||||
})
|
||||
```
|
||||
|
||||
#### Stage Hooks — Intervene in message processing flow
|
||||
|
||||
7 stages:
|
||||
|
||||
| Stage | Timing | Purpose |
|
||||
|-------|--------|---------|
|
||||
| `on_input` | Message just arrived at Agent | Blacklist, rate-limit, short-circuit |
|
||||
| `pre_action` | About to call LLM | Inject context |
|
||||
| `post_action` | LLM returned results | Modify output/tool list |
|
||||
| `before_toolcall` | Before tool execution | Audit, reject, modify params |
|
||||
| `after_toolcall` | After tool execution | Desensitize, rewrite results |
|
||||
| `before_output` | Before output | Format adaptation |
|
||||
| `after_output` | After output | Statistics/logging |
|
||||
|
||||
```go
|
||||
s.RegisterStage(sdk.StagePreAction, func(ctx *sdk.StageContext) error {
|
||||
ctx.Lock()
|
||||
ctx.ContextMsgs = append(ctx.ContextMsgs, map[string]interface{}{
|
||||
"role": "system",
|
||||
"content": "Injected context content",
|
||||
})
|
||||
ctx.Unlock()
|
||||
return nil
|
||||
})
|
||||
```
|
||||
|
||||
#### Configuration Management
|
||||
|
||||
```go
|
||||
// Register config definition
|
||||
s.Settings().RegisterDef(sdk.ConfigDef{
|
||||
Key: "plugin.myplugin.api_key",
|
||||
Default: "",
|
||||
Type: "string",
|
||||
DisplayName: "API Key",
|
||||
Description: "API key",
|
||||
Category: "myplugin",
|
||||
})
|
||||
|
||||
// Read/write config
|
||||
val, err := s.Settings().Get("api_key")
|
||||
s.Settings().Set("api_key", "new-value")
|
||||
|
||||
// Read core config
|
||||
s.Settings().GetCore("llm.model")
|
||||
|
||||
// Read other plugin's config
|
||||
s.Settings().GetPlugin("other_plugin", "some_key")
|
||||
```
|
||||
|
||||
#### Input Delivery
|
||||
|
||||
```go
|
||||
// Queued delivery (processed in order)
|
||||
s.InjectInput(source, channel string, payload map[string]interface{})
|
||||
|
||||
// Interrupt delivery (can interrupt current LLM processing)
|
||||
s.InjectInterrupt(source, channel string, payload map[string]interface{})
|
||||
|
||||
// Shortcuts
|
||||
s.InjectText(source, channel, text string)
|
||||
s.InjectInterruptText(source, channel, text string)
|
||||
```
|
||||
|
||||
#### Event Subscription
|
||||
|
||||
```go
|
||||
unsub := s.Subscribe("tool_call", func(evt *events.Event) {
|
||||
log.Printf("Tool was called: %v", evt.Payload)
|
||||
})
|
||||
defer unsub()
|
||||
```
|
||||
|
||||
#### Capability Access
|
||||
|
||||
```go
|
||||
// Memory
|
||||
s.Memory().Recall(query string) ([]MemItem, error)
|
||||
s.Memory().Commit(triples []Triple) error
|
||||
|
||||
// Knowledge
|
||||
s.Knowledge().Search(query string) ([]string, error)
|
||||
|
||||
// LLM source management
|
||||
s.LLM().ListSources() []SourceInfo
|
||||
s.LLM().SetSource(name string) error
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Lua Plugin Development in Detail
|
||||
|
||||
Lua plugins are suitable for lightweight rapid prototyping, requiring no Go compilation environment. Changes take effect after kernel restart.
|
||||
|
||||
### Plugin Structure
|
||||
|
||||
```lua
|
||||
-- main.lua
|
||||
local plugin = {
|
||||
name = "myluaplugin"
|
||||
}
|
||||
|
||||
function plugin.start(sdk)
|
||||
sdk.log("info", "myluaplugin starting...")
|
||||
|
||||
sdk.register_tool("myluaplugin_hello", {
|
||||
description = "A hello world tool",
|
||||
parameters = {
|
||||
type = "object",
|
||||
properties = {}
|
||||
}
|
||||
}, function(args)
|
||||
return { content = "Hello from myluaplugin plugin!" }
|
||||
end)
|
||||
|
||||
sdk.log("info", "myluaplugin started")
|
||||
end
|
||||
|
||||
function plugin.stop()
|
||||
sdk.log("info", "myluaplugin stopped")
|
||||
end
|
||||
|
||||
return plugin
|
||||
```
|
||||
|
||||
### SDK Mock Layer
|
||||
|
||||
`sdk.lua` provides a pure Lua SDK mock implementation, supporting `lua main.lua` standalone testing:
|
||||
|
||||
```bash
|
||||
lua main.lua
|
||||
# Output:
|
||||
# [lua-plugin] info: myluaplugin starting...
|
||||
# [lua-plugin] register_tool: myluaplugin_hello
|
||||
# [lua-plugin] info: myluaplugin started
|
||||
```
|
||||
|
||||
When running inside the kernel, `sdk.*` global variables are injected by the Go layer, and all functions marked with `-- !impl` are replaced with real implementations.
|
||||
|
||||
### Lua SDK API
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `sdk.log(level, msg)` | Log output |
|
||||
| `sdk.register_tool(name, def, handler)` | Register tool |
|
||||
| `sdk.register_stage(stage, handler)` | Register stage hook |
|
||||
| `sdk.register_api(name)` | Register API |
|
||||
| `sdk.get_setting(key)` | Read config |
|
||||
| `sdk.set_setting(key, value)` | Write config |
|
||||
| `sdk.inject_text(source, channel, text)` | Deliver text message |
|
||||
| `sdk.inject_interrupt(source, channel, text)` | Interrupt delivery |
|
||||
| `sdk.json.encode(val)` | JSON encode |
|
||||
| `sdk.json.decode(str)` | JSON decode |
|
||||
| `sdk.http.get(url)` | HTTP GET request (`-- !impl`) |
|
||||
| `sdk.http.post(url, body, content_type)` | HTTP POST request (`-- !impl`) |
|
||||
|
||||
---
|
||||
|
||||
## 4. Built-in Plugins
|
||||
|
||||
Built-in plugins use `init()` self-registration, compiled into the kernel, no separate deployment needed.
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
internal/plugins/yourplugin/
|
||||
plugin.go — Plugin main file
|
||||
```
|
||||
|
||||
### Minimal Plugin Example
|
||||
|
||||
```go
|
||||
package yourplugin
|
||||
|
||||
import (
|
||||
"gitcode.com/JianFeeeee/HomeAgent/internal/plugin"
|
||||
sdk "gitcode.com/JianFeeeee/HomeAgent/internal/sdk"
|
||||
)
|
||||
|
||||
func init() {
|
||||
plugin.RegisterFactory("yourplugin", func(name string, config map[string]interface{}) (sdk.Plugin, error) {
|
||||
return New(name), nil
|
||||
})
|
||||
}
|
||||
|
||||
type Plugin struct {
|
||||
name string
|
||||
}
|
||||
|
||||
func New(name string) *Plugin {
|
||||
return &Plugin{name: name}
|
||||
}
|
||||
|
||||
func (p *Plugin) Name() string { return p.name }
|
||||
|
||||
func (p *Plugin) Start(s *sdk.PluginSDK) error {
|
||||
// Initialize plugin here: start goroutines, register tools, subscribe events, etc.
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Plugin) Stop() error {
|
||||
// Clean up resources
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Register with Kernel
|
||||
|
||||
Add blank import in `internal/plugins/all.go`:
|
||||
|
||||
```go
|
||||
package plugins
|
||||
|
||||
import (
|
||||
_ "gitcode.com/JianFeeeee/HomeAgent/internal/plugins/yourplugin"
|
||||
// ... other plugins
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Best Practices
|
||||
|
||||
1. `Start()` is non-blocking — start long tasks in goroutines, don't block Start
|
||||
2. `Stop()` cleans up resources — close connections, stop goroutines, cancel subscriptions
|
||||
3. Unique tool names — use plugin name prefix to avoid conflicts
|
||||
4. When handler returns `error`, LLM will receive it and may retry
|
||||
5. Use `InjectInterruptText` for interrupts, `InjectText` for normal delivery
|
||||
6. Use `Settings().Get/Set` for config, don't hardcode
|
||||
7. External Go plugins compile independently, not tied to kernel version; only built-in plugins need recompilation with kernel
|
||||
|
||||
---
|
||||
|
||||
## 6. Example Plugin Reference
|
||||
|
||||
### SDK Repository Examples (`homeagent-sdk/example/`)
|
||||
|
||||
| Example | Type | Features |
|
||||
|---------|------|----------|
|
||||
| [memo](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/memo) | Go | Memo management, PreAction injection + timed interrupt dual reminder |
|
||||
| [files](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/files) | Go | File system operations, 4 write modes, sandbox isolation |
|
||||
| [web](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/web) | Go | DuckDuckGo search + web scraping, SSRF protection |
|
||||
| [qq](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/qq) | Go | NapCat OneBot integration, 17 tools |
|
||||
| [bili](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/bili) | Go | Bilibili video download (you-get) |
|
||||
| [editdoc](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/editdoc) | Go | Office document editing and format conversion |
|
||||
| [a2a](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/a2a) | Go | Agent-to-Agent protocol |
|
||||
| [ocr](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/ocr) | Go | Offline text recognition (Tesseract) |
|
||||
| [sanitizer](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/sanitizer) | Go | Output sanitizer filter |
|
||||
| [luaplugintest](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/luaplugintest) | Lua | Lua plugin Hello World |
|
||||
| [testlua](https://gitcode.com/JianFeeeee/homeagent-sdk/tree/main/example/testlua) | Lua | Lua plugin example |
|
||||
|
||||
### Built-in Plugins
|
||||
|
||||
| Plugin | Location | Features |
|
||||
|--------|----------|----------|
|
||||
| Timer | `internal/plugins/timer/` | Simplest complete example, registers one tool + interrupt feedback |
|
||||
| CLI | `internal/plugins/cli/` | Unix socket listener + synchronous request-response |
|
||||
| WebUI | `internal/plugins/webui/` | HTTP service + dependency injection |
|
||||
|
||||
---
|
||||
|
||||
*Want to understand the project goals? See [OVERVIEW.md](OVERVIEW.md).*
|
||||
*Want to understand the architecture? See [ARCHITECTURE.md](ARCHITECTURE.md).*
|
||||
*SDK repository and development tools? See [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk).*
|
||||
Reference in New Issue
Block a user