v0.7.2: 根目录清理 + Agent 心跳重构 + 内嵌 ONNX 模型

- 根目录清理: branding/docs/knowledge -> assets/, package/tools/deploy -> deploy/
- meta.go: Version 0.7.2, SDKCompatibleVersion 语义改为最高兼容
- Makefile: 版本回退 0.7.2
- registry.go: 系统提示词改用 meta.Version 格式化
- Agent 心跳: reorgGraph 拆分为三个独立循环(archive/merge/review),各自可配间隔
- GraphDB: 新增 sentences 表 + 关系句子溯源 + ClearSentenceID + CleanupOrphanedSentences
- Knowledge: 支持词嵌入向量化器
- NLP 四阶段流水线: Parse -> Extract -> Verify -> Fuse + SentenceRef
- 移除远程 HTTP 解析器(remote_parser.go)
- 新增内嵌 ONNX 模型(vocab + dep_parser.onnx):
  +build onnxruntime: 全量 ONNX Runtime 推理
  !build onnxruntime: 内嵌词表规则式降级解析器
- config: core.agent.onnx_model_path 替代 dep_parser_url
This commit is contained in:
JianFeeeee
2026-07-28 09:56:26 +08:00
parent 1cb3e87dde
commit 2c5f9ff262
47 changed files with 26141 additions and 242 deletions

View File

@ -8,7 +8,7 @@ GOCACHE=/tmp/gocache
export GOPATH=/tmp/gopath
BUILD_DIR=build
PROJECT_ROOT := $(CURDIR)
VERSION ?= $(shell git describe --tags --dirty 2>/dev/null || echo "0.7.1")
VERSION ?= $(shell git describe --tags --dirty 2>/dev/null || echo "0.7.2")
COMMIT ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo "unknown")
BUILD_TIME ?= $(shell date -u '+%Y-%m-%dT%H:%M:%SZ')
LDFLAGS = -X gitcode.com/JianFeeeee/HomeAgent/internal/meta.Version=$(VERSION) -X gitcode.com/JianFeeeee/HomeAgent/internal/meta.Commit=$(COMMIT) -X gitcode.com/JianFeeeee/HomeAgent/internal/meta.BuildTime=$(BUILD_TIME)

View File

@ -124,12 +124,12 @@ flowchart TB
LLM -->|memory_recall| MEM
```
详细说明见 [`docs/zh/ARCHITECTURE.md`](docs/zh/ARCHITECTURE.md)。
详细说明见 [`assets/docs/zh/ARCHITECTURE.md`](assets/docs/zh/ARCHITECTURE.md)。
## 看板娘
<div align="center">
<img src="branding/mascot-xiaozhai.webp" alt="HomeAgent 看板娘 小宅" width="200">
<img src="assets/branding/mascot-xiaozhai.webp" alt="HomeAgent 看板娘 小宅" width="200">
<p><strong>小宅</strong> — HomeAgent 看板娘</p>
</div>
@ -171,15 +171,15 @@ internal/
## 项目状态
**v0.7.1** — 核心可用,插件系统和 SDK 已就绪。内置 11 个插件,外部插件开发见 [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) 仓库,使用 `plugindev` 工具链。输出通道系统、受限外部插件 API、EventAgentLLMChain 事件已上线。
**v0.7.2** — 核心可用,插件系统和 SDK 已就绪。内置 11 个插件,外部插件开发见 [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) 仓库,使用 `plugindev` 工具链。输出通道系统、受限外部插件 API、EventAgentLLMChain 事件已上线。
## 文档
- [项目概览](docs/zh/OVERVIEW.md) | [English](docs/en/OVERVIEW.md)
- [技术架构](docs/zh/ARCHITECTURE.md) | [English](docs/en/ARCHITECTURE.md)
- [插件开发指南](docs/zh/PLUGIN_DEV.md) | [English](docs/en/PLUGIN_DEV.md)
- [Lua Adapter](docs/zh/ADAPTER.md) | [English](docs/en/ADAPTER.md)
- [知识库演示](knowledge/homeagent_architecture/content.md)
- [项目概览](assets/docs/zh/OVERVIEW.md) | [English](assets/docs/en/OVERVIEW.md)
- [技术架构](assets/docs/zh/ARCHITECTURE.md) | [English](assets/docs/en/ARCHITECTURE.md)
- [插件开发指南](assets/docs/zh/PLUGIN_DEV.md) | [English](assets/docs/en/PLUGIN_DEV.md)
- [Lua Adapter](assets/docs/zh/ADAPTER.md) | [English](assets/docs/en/ADAPTER.md)
- [知识库演示](assets/knowledge/homeagent_architecture/content.md)
## 构建

View File

@ -124,12 +124,12 @@ flowchart TB
LLM -->|memory_recall| MEM
```
See [`docs/en/ARCHITECTURE.md`](docs/en/ARCHITECTURE.md) for details.
See [`assets/docs/en/ARCHITECTURE.md`](assets/docs/en/ARCHITECTURE.md) for details.
## Web Mascot
<div align="center">
<img src="branding/mascot-xiaozhai.webp" alt="HomeAgent Web Mascot Xiaozhai" width="200">
<img src="assets/branding/mascot-xiaozhai.webp" alt="HomeAgent Web Mascot Xiaozhai" width="200">
<p><strong>Xiaozhai</strong> — HomeAgent Web Mascot</p>
</div>
@ -171,15 +171,15 @@ External plugin development: see [homeagent-sdk](https://gitcode.com/JianFeeeee/
## Project Status
**v0.7.1** — Core is functional, plugin system and SDK are ready. 11 built-in plugins. External plugin development via [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo using `plugindev` toolchain. Output channel system, restricted external plugin API, and EventAgentLLMChain event are live.
**v0.7.2** — Core is functional, plugin system and SDK are ready. 11 built-in plugins. External plugin development via [homeagent-sdk](https://gitcode.com/JianFeeeee/homeagent-sdk) repo using `plugindev` toolchain. Output channel system, restricted external plugin API, and EventAgentLLMChain event are live.
## Documentation
- [Project Overview](docs/en/OVERVIEW.md) | [中文](docs/zh/OVERVIEW.md)
- [Technical Architecture](docs/en/ARCHITECTURE.md) | [中文](docs/zh/ARCHITECTURE.md)
- [Plugin Development Guide](docs/en/PLUGIN_DEV.md) | [中文](docs/zh/PLUGIN_DEV.md)
- [Lua Adapter](docs/en/ADAPTER.md) | [中文](docs/zh/ADAPTER.md)
- [Knowledge Base Demo](knowledge/homeagent_architecture/content.md)
- [Project Overview](assets/docs/en/OVERVIEW.md) | [中文](assets/docs/zh/OVERVIEW.md)
- [Technical Architecture](assets/docs/en/ARCHITECTURE.md) | [中文](assets/docs/zh/ARCHITECTURE.md)
- [Plugin Development Guide](assets/docs/en/PLUGIN_DEV.md) | [中文](assets/docs/zh/PLUGIN_DEV.md)
- [Lua Adapter](assets/docs/en/ADAPTER.md) | [中文](assets/docs/zh/ADAPTER.md)
- [Knowledge Base Demo](assets/knowledge/homeagent_architecture/content.md)
## Build

View File

Before

Width:  |  Height:  |  Size: 1.6 KiB

After

Width:  |  Height:  |  Size: 1.6 KiB

View File

Before

Width:  |  Height:  |  Size: 130 KiB

After

Width:  |  Height:  |  Size: 130 KiB

View File

@ -4,7 +4,7 @@
Each LLM API source corresponds to a Lua script, responsible for request transformation (Go unified format → API format) and response transformation (API format → Go unified format).
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Adapter Contract
@ -45,7 +45,7 @@ end
return adapter
```
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Unified CompletionRequest Format (Go → Adapter)
@ -65,7 +65,7 @@ return adapter
}
```
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Unified CompletionResponse Format (Adapter → Go)
@ -80,7 +80,7 @@ return adapter
}
```
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Lua VM Built-in Functions
@ -94,7 +94,7 @@ return adapter
`http_post(url, body)` — Perform HTTP POST request, returns response body as string
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Adapting Typical APIs
@ -109,7 +109,7 @@ return adapter
| **GitHub Models** | `/chat/completions` | `Authorization: Bearer <pat>` | OpenAI compatible |
| **Ollama** | `/api/chat` | None | Different options format |
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Steps to Add a New Source

View File

@ -2,7 +2,7 @@
# HomeAgent Plugin Development Guide
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Overview
@ -35,7 +35,7 @@ type Plugin interface {
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 1. Quick Start: Using the plugindev Toolchain
@ -193,7 +193,7 @@ Or upload via WebUI plugin management page.
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 2. Go Plugin Development in Detail
@ -450,7 +450,7 @@ outputCh := s.OutputChan()
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 3. Lua Plugin Development in Detail
@ -522,7 +522,7 @@ When running inside the kernel, `sdk.*` global variables are injected by the Go
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 4. Built-in Plugins
@ -587,7 +587,7 @@ import (
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 5. Best Practices
@ -601,7 +601,7 @@ import (
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 6. Example Plugin Reference

View File

@ -4,7 +4,7 @@
每个 LLM API 源对应一个 Lua 脚本负责请求转换Go 统一格式 → API 格式和响应转换API 格式 → Go 统一格式)。
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 适配器契约
@ -45,7 +45,7 @@ end
return adapter
```
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 统一 CompletionRequest 格式Go → Adapter
@ -65,7 +65,7 @@ return adapter
}
```
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 统一 CompletionResponse 格式Adapter → Go
@ -80,7 +80,7 @@ return adapter
}
```
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## Lua VM 内置函数
@ -94,7 +94,7 @@ return adapter
`http_post(url, body)` — 发起 HTTP POST 请求,返回响应体字符串
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 适配典型 API
@ -109,7 +109,7 @@ return adapter
| **GitHub Models** | `/chat/completions` | `Authorization: Bearer <pat>` | OpenAI 兼容 |
| **Ollama** | `/api/chat` | 无 | 不同的 options 格式 |
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 添加新源步骤

View File

@ -2,7 +2,7 @@
# HomeAgent 插件开发指南
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 概述
@ -36,7 +36,7 @@ type Plugin interface {
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 一、快速开始:使用 plugindev 工具链
@ -193,7 +193,7 @@ curl -X POST http://127.0.0.1:9876/plugins \
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 二、Go 插件开发详解
@ -448,7 +448,7 @@ outputCh := s.OutputChan()
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 三、Lua 插件开发详解
@ -520,7 +520,7 @@ lua main.lua
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 四、内置插件
@ -585,7 +585,7 @@ import (
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 五、最佳实践
@ -599,7 +599,7 @@ import (
---
<img src="../../branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
<img src="../../assets/branding/mascot-xiaozhai.webp" width="20" style="border-radius:50%;vertical-align:middle"> :
## 六、示例插件参考

View File

@ -28,6 +28,7 @@ import (
"gitcode.com/JianFeeeee/HomeAgent/internal/memory/pipeline"
"gitcode.com/JianFeeeee/HomeAgent/internal/memory/social"
"gitcode.com/JianFeeeee/HomeAgent/internal/memory/text"
"gitcode.com/JianFeeeee/HomeAgent/internal/nlp"
"gitcode.com/JianFeeeee/HomeAgent/internal/plugin"
cli "gitcode.com/JianFeeeee/HomeAgent/internal/plugins/cli"
healthcheck "gitcode.com/JianFeeeee/HomeAgent/internal/plugins/healthcheck"
@ -398,6 +399,10 @@ func main() {
Personality: personality,
PluginReg: pluginReg,
PluginDir: cfg.Plugin.Dir,
DistillInterval: cfgReg.GetDuration("core.agent.distill_interval", 30*time.Minute),
ArchiveInterval: cfgReg.GetDuration("core.agent.archive_interval", 60*time.Minute),
ReviewInterval: cfgReg.GetDuration("core.agent.review_interval", 120*time.Minute),
MergeInterval: cfgReg.GetDuration("core.agent.merge_interval", 120*time.Minute),
ContextSavePath: filepath.Join(cfg.Daemon.DataDir, "memory", "context.json"),
EmbeddingModelPath: cfgReg.GetString("core.agent.embedding_model_path", ""),
StageHost: stageHost,
@ -426,6 +431,22 @@ func main() {
// CLI 插件结构化命令 — 直接注入内核依赖,不依赖 HTTP
cli.Configure(pluginReg, cfgReg, agent, cfg.Plugin.Dir)
// ========================================================================
// 依存句法分析器(内嵌 ONNX 模型 / 规则引擎)
// ========================================================================
modelPath := cfgReg.GetString("core.agent.onnx_model_path", "")
onnxParser, err := nlp.NewONNXParser(nlp.ONNXConfig{
ModelPath: modelPath,
DataDir: filepath.Join(cfg.Daemon.DataDir, "nlp"),
})
if err != nil {
log.Printf("[homed] warn: ONNX parser init: %v, using fallback", err)
} else {
nlp.SetDefaultParser(onnxParser)
log.Printf("[homed] dep parser initialized (model: %s)", modelPath)
}
// Auto-create plugins directory (without hardcoding plugin names)
os.MkdirAll(cfg.Plugin.Dir, 0755)

0
package/build.sh → deploy/packaging/build.sh Executable file → Normal file
View File

View File

View File

View File

@ -1,11 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
PROJECT_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
PROJECT_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
BUILD_DIR="${PROJECT_ROOT}/build"
DIST_DIR="${PROJECT_ROOT}/dist/linux"
VERSION="${VERSION:-$(git -C "$PROJECT_ROOT" describe --tags --dirty 2>/dev/null || echo "0.7.1")}"
PACKAGE_ROOT="${PROJECT_ROOT}/package/linux"
PACKAGE_ROOT="${PROJECT_ROOT}/deploy/packaging/linux"
GO="${GO:-$(command -v go 2>/dev/null || echo "/home/jianf/go1.26.5/go/bin/go")}"
ARCH="${1:-amd64}" # amd64 or arm64
@ -84,10 +84,10 @@ build_go() {
prepare_gomod || true
hide_syso
bash "$PROJECT_ROOT/package/build.sh" "linux/$ARCH" "homed" 2>&1 || {
bash "$PROJECT_ROOT/deploy/packaging/build.sh" "linux/$ARCH" "homed" 2>&1 || {
echo "WARNING: homed build failed (CGO/sqlite3 issue). Server/full packages may be incomplete."
}
bash "$PROJECT_ROOT/package/build.sh" "linux/$ARCH" "waiter" 2>&1 || {
bash "$PROJECT_ROOT/deploy/packaging/build.sh" "linux/$ARCH" "waiter" 2>&1 || {
echo "WARNING: waiter build failed."
}
@ -240,7 +240,7 @@ SCRIPT
stage_setup() {
local staging="$1"
local setup_src="$PROJECT_ROOT/package/linux/setup.sh"
local setup_src="$PROJECT_ROOT/deploy/packaging/linux/setup.sh"
if [ -f "$setup_src" ]; then
mkdir -p "$staging/usr/lib/homeagent"
cp "$setup_src" "$staging/usr/lib/homeagent/setup.sh"
@ -309,7 +309,7 @@ build_tar() {
done
# setup script
local setup_src="$PROJECT_ROOT/package/linux/setup.sh"
local setup_src="$PROJECT_ROOT/deploy/packaging/linux/setup.sh"
[ -f "$setup_src" ] && cp "$setup_src" "$staging/usr/lib/homeagent/setup.sh"
# GUI if available
@ -350,7 +350,7 @@ build_rpm() {
if [ -z "$fpm_bin" ]; then
echo " SKIP .rpm: fpm not installed. Install it with: gem install fpm"
echo " Alternatively, build RPM on Fedora/RHEL using:"
echo " rpmbuild -ba package/linux/homeagent.spec"
echo " rpmbuild -ba deploy/packaging/linux/homeagent.spec"
return
fi

2
tools/deploy/deploy.sh → deploy/scripts/deploy.sh Executable file → Normal file
View File

@ -2,7 +2,7 @@
set -euo pipefail
# HomeAgent 部署脚本
# 用法: cd <project-root> && sudo bash tools/deploy/deploy.sh
# 用法: cd <project-root> && sudo bash deploy/scripts/deploy.sh
PROJECT_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
BIN_DIR="/usr/local/bin"

2
go.mod
View File

@ -10,6 +10,8 @@ require (
require github.com/yanyiwu/gojieba v1.4.7
require github.com/yalue/onnxruntime_go v1.13.0
require gitcode.com/JianFeeeee/homeagent-sdk v0.7.1
replace gitcode.com/JianFeeeee/homeagent-sdk => ../homeagentsdk

View File

@ -1,9 +0,0 @@
gitcode.com/JianFeeeee/homeagent-sdk v0.0.0-20260708004841-e9bdcf9304b0/go.mod h1:mzs91WBioKDpiMLXZ7t/LXTmAh3DRu3RnDDoSztTcLg=
github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
golang.org/x/sys v0.0.0-20190204203706-41f3e6584952 h1:FDfvYgoVsA7TTZSbgiqjAbfPbK47CNHdWl3h/PJtii0=
golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=

View File

@ -7,8 +7,8 @@ import (
"sync"
"time"
agentAPI "gitcode.com/JianFeeeee/HomeAgent/internal/agent/api"
agentPkg "gitcode.com/JianFeeeee/HomeAgent/internal/agent"
agentAPI "gitcode.com/JianFeeeee/HomeAgent/internal/agent/api"
agentIO "gitcode.com/JianFeeeee/HomeAgent/internal/agent/io"
"gitcode.com/JianFeeeee/HomeAgent/internal/events"
"gitcode.com/JianFeeeee/HomeAgent/internal/knowledge"
@ -26,19 +26,19 @@ import (
// Agent — 单 agent不区分会话/实例
type Agent struct {
mu sync.Mutex
id types.AgentID
provider agentAPI.Provider
mu sync.Mutex
id types.AgentID
provider agentAPI.Provider
providerManager *agentAPI.ProviderManager
io *agentIO.IOManager
memory *memory.GraphDB
indexer *memory.Indexer
skills *skill.Manager
tracker *tracker.Tracker
context *RelevanceContext
systemPrompt string
ctx context.Context
cancel context.CancelFunc
io *agentIO.IOManager
memory *memory.GraphDB
indexer *memory.Indexer
skills *skill.Manager
tracker *tracker.Tracker
context *RelevanceContext
systemPrompt string
ctx context.Context
cancel context.CancelFunc
// 文档记忆(第二层)
docStore *document.Store
@ -62,6 +62,11 @@ type Agent struct {
// 定期心跳蒸馏
distillInterval time.Duration
// 三个独立心跳任务间隔
archiveInterval time.Duration // 冷文档归档
reviewInterval time.Duration // 关系复审
mergeInterval time.Duration // 实体合并检测
// 上下文裁剪:活跃上下文最大条数,超出按相关性裁剪
maxContextSize int
@ -110,30 +115,33 @@ type Agent struct {
}
type AgentConfig struct {
ID types.AgentID
SystemPrompt string
Provider agentAPI.Provider
ID types.AgentID
SystemPrompt string
Provider agentAPI.Provider
ProviderManager *agentAPI.ProviderManager
IO *agentIO.IOManager
Memory *memory.GraphDB
Indexer *memory.Indexer
Skills *skill.Manager
Tracker *tracker.Tracker
IO *agentIO.IOManager
Memory *memory.GraphDB
Indexer *memory.Indexer
Skills *skill.Manager
Tracker *tracker.Tracker
DocStore *document.Store
Knowledge *knowledge.Store
SocialStore *social.SocialStore
TextMemory *text.Memory
Personality *agentPkg.Personality
PluginReg *plugin.Registry
PluginDir string
DistillInterval time.Duration
MaxContextSize int // 活跃上下文最大条数,超出按相关性裁剪
ContextSavePath string // 上下文持久化路径,空则不持久化
EmbeddingModelPath string // 预训练词嵌入模型路径word2vec 文本格式),空则不使用
StageHost *StageHost
EventBus *events.Bus
ThinkingEnabled bool
DocStore *document.Store
Knowledge *knowledge.Store
SocialStore *social.SocialStore
TextMemory *text.Memory
Personality *agentPkg.Personality
PluginReg *plugin.Registry
PluginDir string
DistillInterval time.Duration
ArchiveInterval time.Duration // 冷文档归档间隔L3→L40 则使用 DistillInterval
ReviewInterval time.Duration // 关系复审间隔0 则使用 DistillInterval
MergeInterval time.Duration // 实体合并检测间隔0 则使用 DistillInterval
MaxContextSize int // 活跃上下文最大条数,超出按相关性裁剪
ContextSavePath string // 上下文持久化路径,空则不持久化
EmbeddingModelPath string // 预训练词嵌入模型路径word2vec 文本格式),空则不使用
StageHost *StageHost
EventBus *events.Bus
ThinkingEnabled bool
InputProcessing types.InputProcessingConfig // 非文本输入处理配置
}
@ -143,6 +151,15 @@ func New(cfg AgentConfig) *Agent {
if cfg.DistillInterval <= 0 {
cfg.DistillInterval = 30 * time.Minute
}
if cfg.ArchiveInterval <= 0 {
cfg.ArchiveInterval = cfg.DistillInterval
}
if cfg.ReviewInterval <= 0 {
cfg.ReviewInterval = cfg.DistillInterval
}
if cfg.MergeInterval <= 0 {
cfg.MergeInterval = cfg.DistillInterval
}
if cfg.MaxContextSize <= 0 {
cfg.MaxContextSize = 30
}
@ -152,6 +169,10 @@ func New(cfg AgentConfig) *Agent {
cfg.DocStore.SetVectorizer(embedder)
cfg.DocStore.ReindexWithVectorizer(embedder)
}
if cfg.Knowledge != nil {
cfg.Knowledge.SetVectorizer(embedder)
cfg.Knowledge.ReindexWithVectorizer(embedder)
}
rc := NewRelevanceContext(cfg.ContextSavePath, embedder)
if cfg.StageHost != nil {
@ -179,19 +200,21 @@ func New(cfg AgentConfig) *Agent {
personality: cfg.Personality,
pluginReg: cfg.PluginReg,
pluginDir: cfg.PluginDir,
distillInterval: cfg.DistillInterval,
maxContextSize: cfg.MaxContextSize,
stageHost: cfg.StageHost,
eventBus: cfg.EventBus,
selfInputCh: make(chan string, 64),
childResults: make(map[string]string),
interceptCh: make(chan *agentIO.InputEvent, 64),
pluginHealth: newPluginHealthTracker(),
thinkingEnabled: cfg.ThinkingEnabled,
inputCfg: cfg.InputProcessing,
embedder: embedder,
noMergeMarkers: make(map[string]int),
distillInterval: cfg.DistillInterval,
archiveInterval: cfg.ArchiveInterval,
reviewInterval: cfg.ReviewInterval,
mergeInterval: cfg.MergeInterval,
maxContextSize: cfg.MaxContextSize,
stageHost: cfg.StageHost,
eventBus: cfg.EventBus,
selfInputCh: make(chan string, 64),
childResults: make(map[string]string),
interceptCh: make(chan *agentIO.InputEvent, 64),
pluginHealth: newPluginHealthTracker(),
thinkingEnabled: cfg.ThinkingEnabled,
inputCfg: cfg.InputProcessing,
embedder: embedder,
noMergeMarkers: make(map[string]int),
}
}
@ -199,6 +222,9 @@ func (a *Agent) Start() {
go a.eventLoop()
go a.interceptLoop()
go a.distillLoop()
go a.archiveLoop()
go a.mergeLoop()
go a.reviewLoop()
log.Printf("[agent] %s started, waiting for IO interrupts", a.id)
}

View File

@ -28,6 +28,11 @@ func (a *Agent) enqueueConsolidationTask(task ConsolidationTask) {
log.Printf("[agent] enqueued consolidation task: %s", task.Reason)
}
// ──────────────────────────────────────────────
// 四个独立心跳循环,各自拥有独立的 ticker 和配置
// ──────────────────────────────────────────────
// distillLoop 上下文裁剪L2 蒸馏),使用 distillInterval
func (a *Agent) distillLoop() {
defer func() {
if r := recover(); r != nil {
@ -36,7 +41,7 @@ func (a *Agent) distillLoop() {
go a.distillLoop()
}
}()
if a.docStore == nil && a.memory == nil {
if a.docStore == nil {
return
}
ticker := time.NewTicker(a.distillInterval)
@ -47,7 +52,6 @@ func (a *Agent) distillLoop() {
case <-ticker.C:
log.Printf("[agent] heartbeat distill tick")
a.distillContext()
a.reorgGraph()
a.autoReloadPlugins()
case <-a.ctx.Done():
return
@ -55,6 +59,88 @@ func (a *Agent) distillLoop() {
}
}
// archiveLoop 冷文档归档L3→L4使用 archiveInterval
func (a *Agent) archiveLoop() {
defer func() {
if r := recover(); r != nil {
log.Printf("[agent] archiveLoop panic recovered: %v\n%s", r, debug.Stack())
time.Sleep(time.Second)
go a.archiveLoop()
}
}()
if a.memory == nil {
return
}
ticker := time.NewTicker(a.archiveInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
log.Printf("[agent] heartbeat archive tick")
a.archiveColdDocs()
case <-a.ctx.Done():
return
}
}
}
// mergeLoop 实体合并检测GraphDB → LLM 裁决),使用 mergeInterval
func (a *Agent) mergeLoop() {
defer func() {
if r := recover(); r != nil {
log.Printf("[agent] mergeLoop panic recovered: %v\n%s", r, debug.Stack())
time.Sleep(time.Second)
go a.mergeLoop()
}
}()
if a.memory == nil {
return
}
ticker := time.NewTicker(a.mergeInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
log.Printf("[agent] heartbeat merge tick")
a.detectEntityMerge()
case <-a.ctx.Done():
return
}
}
}
// reviewLoop 关系复审GraphDB → ClearSentenceID → CleanupOrphanedSentences使用 reviewInterval
func (a *Agent) reviewLoop() {
defer func() {
if r := recover(); r != nil {
log.Printf("[agent] reviewLoop panic recovered: %v\n%s", r, debug.Stack())
time.Sleep(time.Second)
go a.reviewLoop()
}
}()
if a.memory == nil {
return
}
ticker := time.NewTicker(a.reviewInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
log.Printf("[agent] heartbeat review tick")
a.reviewRelations()
case <-a.ctx.Done():
return
}
}
}
// ──────────────────────────────────────────────
// 蒸馏逻辑
// ──────────────────────────────────────────────
func (a *Agent) distillContext() {
if a.docStore == nil {
return
@ -68,14 +154,16 @@ func (a *Agent) distillContext() {
}
}
// ──────────────────────────────────────────────
// 冷文档归档docStore → GraphDB (L3→L4)
// ──────────────────────────────────────────────
func (a *Agent) reorgGraph() {
func (a *Agent) archiveColdDocs() {
if a.memory == nil {
return
}
log.Printf("[agent] graph reorg start")
log.Printf("[agent] cold doc archival start")
if a.indexer != nil {
if err := a.indexer.Sync(); err != nil {
@ -102,6 +190,18 @@ func (a *Agent) reorgGraph() {
}
}
}
}
// ──────────────────────────────────────────────
// 实体合并检测GraphDB → LLM 裁决
// ──────────────────────────────────────────────
func (a *Agent) detectEntityMerge() {
if a.memory == nil {
return
}
log.Printf("[agent] entity merge detection start")
result, err := a.memory.Recall(nil, nil, 1, "")
if err != nil || result == nil || len(result.Entities) < 2 {
@ -167,9 +267,68 @@ func (a *Agent) reorgGraph() {
}
if llmCandidates > 0 {
log.Printf("[agent] graph reorg: %d merge candidates sent for LLM decision", llmCandidates)
log.Printf("[agent] entity merge: %d merge candidates sent for LLM decision", llmCandidates)
} else {
log.Printf("[agent] graph reorg: no similar entities found")
log.Printf("[agent] entity merge: no similar entities found")
}
}
// ──────────────────────────────────────────────
// 关系复审GraphDB → ClearSentenceID → CleanupOrphanedSentences
// ──────────────────────────────────────────────
func (a *Agent) reviewRelations() {
if a.memory == nil {
return
}
log.Printf("[agent] relation review start")
reviewCount := 0
const maxReviewBatch = 5
relResult, err := a.memory.Recall(nil, nil, 1, "")
if err != nil || relResult == nil {
return
}
for _, rel := range relResult.Relations {
if reviewCount >= maxReviewBatch {
break
}
if rel.SentenceID == 0 || rel.SentenceText == "" {
continue
}
a.enqueueConsolidationTask(ConsolidationTask{
Type: "relation_review",
Reason: fmt.Sprintf(
"【关系复审】原始句子: '%s'\n当前三元组: (%s → %s → %s) 置信度 %.2f\n请判断是否需要修正如相对引用未解析、主宾颠倒、噪音三元组等如需修正请用 memory_edit 工具",
rel.SentenceText, rel.SourceName, rel.RelationType, rel.TargetName, rel.Confidence,
),
Data: map[string]interface{}{
"relation_id": rel.ID,
"source": rel.SourceName,
"relation_type": rel.RelationType,
"target": rel.TargetName,
"confidence": rel.Confidence,
"sentence": rel.SentenceText,
},
})
// 清除句子引用(复审后解除关联)
if err := a.memory.ClearSentenceID(rel.ID); err != nil {
log.Printf("[agent] clear sentence_id for relation %d: %v", rel.ID, err)
}
reviewCount++
}
if reviewCount > 0 {
// 清理无引用的句子
if deleted, err := a.memory.CleanupOrphanedSentences(); err != nil {
log.Printf("[agent] cleanup orphaned sentences: %v", err)
} else if deleted > 0 {
log.Printf("[agent] cleanup %d orphaned sentences", deleted)
}
log.Printf("[agent] relation review: %d relations sent for review", reviewCount)
}
}

View File

@ -10,6 +10,7 @@ import (
"sync"
"time"
"gitcode.com/JianFeeeee/HomeAgent/internal/meta"
"gitcode.com/JianFeeeee/HomeAgent/pkg/types"
_ "github.com/mattn/go-sqlite3"
)
@ -347,9 +348,14 @@ func (r *ConfigRegistry) seedDBValues(dataDir string) {
set("core.agent.max_tool_turns", "10")
set("core.agent.max_context_size", "30")
set("core.agent.distill_interval", "30m")
set("core.agent.archive_interval", "60m")
set("core.agent.review_interval", "120m")
set("core.agent.merge_interval", "120m")
set("core.agent.workdir", "")
set("core.agent.embedding_model_path", "")
set("core.agent.system_prompt", `你是 HomeAgent 的看板娘「小宅」(Xiao Zhai)HΔ-Kernel v0.7.1 型号的家政型 AI 管家助手。
set("core.agent.onnx_model_path", "")
set("core.agent.system_prompt", fmt.Sprintf("你是 HomeAgent 的看板娘「小宅」(Xiao Zhai)HΔ-Kernel v%s 型号的家政型 AI 管家助手。", meta.Version)+
`
角色特质:
- 对自己的三层记忆Context → Document → Graph引以为傲
@ -436,8 +442,12 @@ func (r *ConfigRegistry) seedCoreDefs(dataDir string) {
reg(ConfigDef{Key: "core.agent.max_tool_turns", Default: "10", Type: "int", DisplayName: "最大工具轮次", Description: "单次请求允许的最大工具调用轮数", Category: "agent"})
reg(ConfigDef{Key: "core.agent.max_context_size", Default: "30", Type: "int", DisplayName: "最大上下文", Description: "上下文窗口中保留的最大消息条数", Category: "agent"})
reg(ConfigDef{Key: "core.agent.distill_interval", Default: "30m", Type: "duration", DisplayName: "蒸馏间隔", Description: "记忆蒸馏的执行间隔", Category: "agent"})
reg(ConfigDef{Key: "core.agent.archive_interval", Default: "60m", Type: "duration", DisplayName: "冷文档归档间隔", Description: "冷文档归档L3→L4的执行间隔", Category: "agent"})
reg(ConfigDef{Key: "core.agent.review_interval", Default: "120m", Type: "duration", DisplayName: "关系复审间隔", Description: "三元组关系复审的执行间隔", Category: "agent"})
reg(ConfigDef{Key: "core.agent.merge_interval", Default: "120m", Type: "duration", DisplayName: "实体合并检测间隔", Description: "实体合并检测LLM 裁决)的执行间隔", Category: "agent"})
reg(ConfigDef{Key: "core.agent.workdir", Default: "", Type: "string", DisplayName: "工作目录", Description: "Agent 命令执行的默认工作目录(如 cmd_run 工具的 fallback留空使用内核所在目录", Category: "agent"})
reg(ConfigDef{Key: "core.agent.embedding_model_path", Default: "", Type: "string", DisplayName: "预训练词嵌入模型路径", Description: "预训练词嵌入模型路径word2vec 文本格式),支持逗号分隔多个模型。空则使用 TF-IDF 回退。修改后需重启生效。", Category: "agent"})
reg(ConfigDef{Key: "core.agent.onnx_model_path", Default: "", Type: "string", DisplayName: "ONNX 模型路径", Description: "依存句法分析 ONNX 模型文件路径。留空使用二进制内嵌模型/规则引擎。修改后需重启生效。", Category: "agent"})
reg(ConfigDef{Key: "core.agent.system_prompt", Default: "", Type: "text", DisplayName: "系统身份提示词", Description: "Agent 的系统提示词,定义身份和行为规则。留空则使用编译时内置默认值。修改后需重启生效。", Category: "agent"})
reg(ConfigDef{Key: "core.input_processing.image.fallback_provider", Default: "", Type: "string", DisplayName: "图片回退提供商", Description: "当主 LLM 不支持图片处理时使用的提供商(留空则自动降级为文字描述)", Category: "input"})

View File

@ -83,8 +83,9 @@ type Store struct {
mu sync.RWMutex
items map[string]*Knowledge
indexPath string
summaries []string
indexPath string
summaries []string
vectorizer vector.Vectorizer // 可选:词嵌入向量化器,优先于 TF-IDF
}
func NewStore(root string) *Store {
@ -97,6 +98,35 @@ func NewStore(root string) *Store {
}
}
// SetVectorizer 设置词嵌入向量化器,优先于 TF-IDF
func (s *Store) SetVectorizer(v vector.Vectorizer) {
s.vectorizer = v
}
// ReindexWithVectorizer 用给定的向量化器重建所有知识条目的向量索引
func (s *Store) ReindexWithVectorizer(v vector.Vectorizer) {
s.mu.Lock()
defer s.mu.Unlock()
log.Printf("[knowledge] reindex with vectorizer (%d items)", len(s.items))
s.vec = vector.NewStore()
for _, k := range s.items {
vec := v.Vectorize(k.Name + " " + k.Content)
s.vec.Insert(k.Name, k.Name+": "+k.Content, vec, map[string]string{
"name": k.Name, "path": k.Path,
})
}
log.Printf("[knowledge] reindex with vectorizer complete (%d vectors)", s.vec.Size())
}
// vectorize 优先使用词嵌入向量化器,不可用时回退到 TF-IDF
func (s *Store) vectorize(text string) vector.Vector {
if s.vectorizer != nil {
return s.vectorizer.Vectorize(text)
}
return s.veczer.Vectorize(text)
}
func (s *Store) Start() error {
if err := os.MkdirAll(s.root, 0755); err != nil {
return fmt.Errorf("knowledge root: %w", err)
@ -122,7 +152,7 @@ func (s *Store) Search(query string, topK int) []*Knowledge {
topK = 5
}
vec := s.veczer.Vectorize(query)
vec := s.vectorize(query)
results := s.vec.Search(vec, topK)
var out []*Knowledge
@ -171,7 +201,7 @@ func (s *Store) Add(name, content string) error {
}
s.items[id] = k
vec := s.veczer.Vectorize(name + " " + content)
vec := s.vectorize(name + " " + content)
s.vec.Insert(id, name+": "+content, vec, map[string]string{
"name": name, "path": path,
})
@ -202,7 +232,7 @@ func (s *Store) SearchCategories(query string, topK int) []string {
return names
}
vec := s.veczer.Vectorize(query)
vec := s.vectorize(query)
results := s.vec.Search(vec, topK)
var names []string
for _, r := range results {
@ -274,7 +304,7 @@ func (s *Store) BuildTree() *TreeIndex {
}
}
// 获取该条目的向量并压缩
vec := s.veczer.Vectorize(k.Name + " " + k.Content)
vec := s.vectorize(k.Name + " " + k.Content)
preview := []rune(k.Content)
previewStr := ""
if len(preview) > 200 {
@ -303,7 +333,7 @@ func (s *Store) SearchTree(query string, topK int) map[string][]*Knowledge {
topK = 10
}
vec := s.veczer.Vectorize(query)
vec := s.vectorize(query)
results := s.vec.Search(vec, topK*2)
categorized := make(map[string][]*Knowledge)
@ -361,7 +391,7 @@ func (s *Store) scanAll() error {
}
for _, k := range s.items {
vec := s.veczer.Vectorize(k.Name + " " + k.Content)
vec := s.vectorize(k.Name + " " + k.Content)
s.vec.Insert(k.Name, k.Name+": "+k.Content, vec, map[string]string{
"name": k.Name, "path": k.Path,
})

View File

@ -31,15 +31,18 @@ type Relation struct {
TurnID int `json:"turn_id"`
CreatedAt time.Time `json:"created_at"`
DateBucket string `json:"date_bucket"`
SentenceID int64 `json:"sentence_id,omitempty"` // FK → sentences.id
SentenceText string `json:"sentence_text,omitempty"` // JOINed from sentences
}
type Triple struct {
Subject string `json:"subject"`
Relation string `json:"relation"`
Object string `json:"object"`
Confidence float64 `json:"confidence,omitempty"`
SubjectType string `json:"subject_type,omitempty"`
ObjectType string `json:"object_type,omitempty"`
Subject string `json:"subject"`
Relation string `json:"relation"`
Object string `json:"object"`
Confidence float64 `json:"confidence,omitempty"`
SubjectType string `json:"subject_type,omitempty"`
ObjectType string `json:"object_type,omitempty"`
SentenceText string `json:"sentence_text,omitempty"` // 原始句子文本Commit时写入sentences表
}
type GraphDB struct {
@ -81,6 +84,11 @@ func (g *GraphDB) initSchema() error {
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)`,
`CREATE TABLE IF NOT EXISTS sentences (
id INTEGER PRIMARY KEY AUTOINCREMENT,
text TEXT UNIQUE NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)`,
`CREATE TABLE IF NOT EXISTS relations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source_id INTEGER NOT NULL,
@ -93,6 +101,7 @@ func (g *GraphDB) initSchema() error {
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
date_bucket TEXT,
sentence_id INTEGER DEFAULT 0,
FOREIGN KEY (source_id) REFERENCES entities(id),
FOREIGN KEY (target_id) REFERENCES entities(id)
)`,
@ -103,6 +112,8 @@ func (g *GraphDB) initSchema() error {
`CREATE INDEX IF NOT EXISTS idx_relation_type ON relations(relation_type)`,
`CREATE INDEX IF NOT EXISTS idx_relation_status ON relations(status)`,
`CREATE INDEX IF NOT EXISTS idx_relation_session ON relations(session_id)`,
`CREATE INDEX IF NOT EXISTS idx_relation_sentence ON relations(sentence_id)`,
`CREATE INDEX IF NOT EXISTS idx_sentences_text ON sentences(text)`,
}
for _, s := range schemas {
@ -111,6 +122,14 @@ func (g *GraphDB) initSchema() error {
}
}
// 迁移1兼容旧版 sentence_ref 列(已有表则忽略)
tx.Exec(`ALTER TABLE relations ADD COLUMN sentence_ref TEXT DEFAULT ''`)
// 迁移2为新表添加 sentence_id 列(已有表则忽略)
tx.Exec(`ALTER TABLE relations ADD COLUMN sentence_id INTEGER DEFAULT 0`)
// 迁移3将现有 sentence_ref 数据迁移到 sentences 表
tx.Exec(`INSERT OR IGNORE INTO sentences (text) SELECT DISTINCT sentence_ref FROM relations WHERE sentence_ref != ''`)
tx.Exec(`UPDATE relations SET sentence_id = (SELECT id FROM sentences WHERE text = relations.sentence_ref) WHERE sentence_ref != ''`)
return tx.Commit()
}
@ -171,10 +190,24 @@ func (g *GraphDB) Commit(triples []Triple, sessionID string, turnID int) (int, i
return 0, 0, fmt.Errorf("object %q: %w", t.Object, err)
}
// 写入/查找句子
var sentenceID int64
if t.SentenceText != "" {
_, err = tx.Exec(
`INSERT OR IGNORE INTO sentences (text) VALUES (?)`, t.SentenceText)
if err != nil {
return 0, 0, fmt.Errorf("insert sentence: %w", err)
}
err = tx.QueryRow("SELECT id FROM sentences WHERE text = ?", t.SentenceText).Scan(&sentenceID)
if err != nil {
sentenceID = 0
}
}
_, err = tx.Exec(
`INSERT INTO relations (source_id, target_id, relation_type, confidence, session_id, turn_id, date_bucket)
VALUES (?, ?, ?, ?, ?, ?, ?)`,
sourceID, targetID, t.Relation, confidence, sessionID, turnID, dateBucket,
`INSERT INTO relations (source_id, target_id, relation_type, confidence, session_id, turn_id, date_bucket, sentence_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
sourceID, targetID, t.Relation, confidence, sessionID, turnID, dateBucket, sentenceID,
)
if err != nil {
return 0, 0, err
@ -259,10 +292,12 @@ func (g *GraphDB) Recall(keywords []string, seedEntities []string, depth int, se
relRows, err := g.db.Query(
`SELECT r.id, r.source_id, r.target_id, e1.name, e2.name,
r.relation_type, r.confidence, r.status, r.session_id,
r.turn_id, r.created_at, COALESCE(r.date_bucket, '')
r.turn_id, r.created_at, COALESCE(r.date_bucket, ''),
COALESCE(r.sentence_id, 0), COALESCE(s.text, '')
FROM relations r
JOIN entities e1 ON r.source_id = e1.id
JOIN entities e2 ON r.target_id = e2.id
LEFT JOIN sentences s ON r.sentence_id = s.id
WHERE r.status = 'active'
ORDER BY r.created_at DESC LIMIT 30`,
)
@ -275,7 +310,7 @@ func (g *GraphDB) Recall(keywords []string, seedEntities []string, depth int, se
if err := relRows.Scan(&rel.ID, &rel.SourceID, &rel.TargetID,
&rel.SourceName, &rel.TargetName, &rel.RelationType,
&rel.Confidence, &rel.Status, &rel.SessionID,
&rel.TurnID, &rel.CreatedAt, &rel.DateBucket); err != nil {
&rel.TurnID, &rel.CreatedAt, &rel.DateBucket, &rel.SentenceID, &rel.SentenceText); err != nil {
return nil, err
}
result.Relations = append(result.Relations, rel)
@ -340,10 +375,12 @@ func (g *GraphDB) Recall(keywords []string, seedEntities []string, depth int, se
query := fmt.Sprintf(
`SELECT r.id, r.source_id, r.target_id, e1.name, e2.name,
r.relation_type, r.confidence, r.status, r.session_id,
r.turn_id, r.created_at, COALESCE(r.date_bucket, '')
r.turn_id, r.created_at, COALESCE(r.date_bucket, ''),
COALESCE(r.sentence_id, 0), COALESCE(s.text, '')
FROM relations r
JOIN entities e1 ON r.source_id = e1.id
JOIN entities e2 ON r.target_id = e2.id
LEFT JOIN sentences s ON r.sentence_id = s.id
WHERE (r.source_id IN (%s) OR r.target_id IN (%s))
AND r.status = 'active'`,
placeholders(len(ids)),
@ -367,7 +404,7 @@ func (g *GraphDB) Recall(keywords []string, seedEntities []string, depth int, se
if err := relRows.Scan(&rel.ID, &rel.SourceID, &rel.TargetID,
&rel.SourceName, &rel.TargetName, &rel.RelationType,
&rel.Confidence, &rel.Status, &rel.SessionID,
&rel.TurnID, &rel.CreatedAt, &rel.DateBucket); err != nil {
&rel.TurnID, &rel.CreatedAt, &rel.DateBucket, &rel.SentenceID, &rel.SentenceText); err != nil {
relRows.Close()
return nil, err
}
@ -544,7 +581,7 @@ func (g *GraphDB) GraphData() (map[string]interface{}, error) {
return nil, err
}
rrows, err := g.db.Query(`SELECT id, source_id, target_id, relation_type, confidence, status, created_at FROM relations WHERE status = 'active' ORDER BY created_at DESC`)
rrows, err := g.db.Query(`SELECT r.id, r.source_id, r.target_id, r.relation_type, r.confidence, r.status, r.created_at, COALESCE(r.sentence_id, 0), COALESCE(s.text, '') FROM relations r LEFT JOIN sentences s ON r.sentence_id = s.id WHERE r.status = 'active' ORDER BY r.created_at DESC`)
if err != nil {
return nil, err
}
@ -558,11 +595,13 @@ func (g *GraphDB) GraphData() (map[string]interface{}, error) {
Confidence float64 `json:"confidence"`
Status string `json:"status"`
CreatedAt time.Time `json:"created_at"`
SentenceID int64 `json:"sentence_id,omitempty"`
SentenceText string `json:"sentence_text,omitempty"`
}
var relations []graphRelation
for rrows.Next() {
var r graphRelation
if err := rrows.Scan(&r.ID, &r.SourceID, &r.TargetID, &r.RelationType, &r.Confidence, &r.Status, &r.CreatedAt); err != nil {
if err := rrows.Scan(&r.ID, &r.SourceID, &r.TargetID, &r.RelationType, &r.Confidence, &r.Status, &r.CreatedAt, &r.SentenceID, &r.SentenceText); err != nil {
return nil, err
}
relations = append(relations, r)
@ -603,8 +642,8 @@ func (g *GraphDB) Introspect() (map[string]interface{}, error) {
}
return map[string]interface{}{
"entity_count": entityCount,
"relation_count": relationCount,
"entity_count": entityCount,
"relation_count": relationCount,
"memory_hotspots": hotspots,
}, nil
}
@ -747,6 +786,29 @@ func (g *GraphDB) Archive(days int) (int, error) {
return int(n), nil
}
// ClearSentenceID 清除指定关系的 sentence_idLLM复审后解除句子引用
func (g *GraphDB) ClearSentenceID(relationID int64) error {
_, err := g.db.Exec(
`UPDATE relations SET sentence_id = 0, updated_at = CURRENT_TIMESTAMP WHERE id = ?`,
relationID,
)
return err
}
// CleanupOrphanedSentences 删除没有任何关系引用的句子,返回删除数
func (g *GraphDB) CleanupOrphanedSentences() (int, error) {
result, err := g.db.Exec(
`DELETE FROM sentences WHERE id NOT IN (
SELECT DISTINCT sentence_id FROM relations WHERE sentence_id != 0
)`,
)
if err != nil {
return 0, err
}
n, _ := result.RowsAffected()
return int(n), nil
}
func (g *GraphDB) Close() error {
return g.db.Close()
}

View File

@ -18,8 +18,8 @@ var (
// KernelName 是内核名称。
KernelName = "HomeAgent"
// SDKCompatibleVersion 是此内核兼容的最 SDK 版本semver
SDKCompatibleVersion = "0.8.0"
// SDKCompatibleVersion 是此内核兼容的最 SDK 版本semver
SDKCompatibleVersion = "0.7.2"
)
// FullVersion 返回完整的版本字符串。

View File

@ -5,9 +5,10 @@ import "gitcode.com/JianFeeeee/HomeAgent/internal/memory"
// ToMemoryTriple 将 nlp.Triple 转为 memory.Triple
func ToMemoryTriple(t Triple) memory.Triple {
return memory.Triple{
Subject: t.Subject,
Relation: t.Relation,
Object: t.Object,
Confidence: t.Score,
Subject: t.Subject,
Relation: t.Relation,
Object: t.Object,
Confidence: t.Score,
SentenceText: t.SentenceRef,
}
}

View File

@ -36,19 +36,32 @@ var depTemplates = []depTemplate{
{subjRel: "SBV", objRel: "IOB", score: 0.85},
{subjRel: "SBV", objRel: "FOB", score: 0.8},
{subjRel: "SBV", objRel: "POB", score: 0.75},
{subjRel: "ATT", objRel: "VOB", score: 0.7},
{subjRel: "ATT", objRel: "IOB", score: 0.65},
{subjRel: "ATT", objRel: "FOB", score: 0.6},
{subjRel: "ATT", objRel: "POB", score: 0.55},
}
// extractFromDep 基于依存句法树提取三元组
func extractFromDep(result *ParseResult) []Triple {
// extractFromDep 基于依存句法树提取三元组 (Phase 2: 结构初筛)
// 输入Token 序列(含依存关系)
// 处理:标记名词性节点 → 遍历谓词中心 → 收集 SBV/ATT 主语、VOB/IOB/POB 宾语 → 笛卡尔积 → 赋句法置信度 → ATT 链合并
// 输出:候选三元组列表(带 syntax_conf
func extractFromDep(result *ParseResult, sentence string) []Triple {
if len(result.Tokens) < 2 {
return nil
}
// Step 1: 标记所有名词性节点为候选实体(供后续 ATT 合并等使用)
// (隐式使用,通过 isNounLike 判断)
// Step 2: 遍历所有动词节点作为谓词中心
verbIndices := findPredicates(result.POS, result.Tokens)
var triples []Triple
verbIndices := findPredicates(result.POS, result.Tokens)
for _, vi := range verbIndices {
var subj, obj string
var objIdx int
// Step 3: 沿依存弧收集主语SBV/ATT和宾语VOB/IOB/FOB/POB
var subjIndices, objIndices []int
var subjRels, objRels []string
for i, head := range result.Heads {
if head == 0 {
@ -60,63 +73,102 @@ func extractFromDep(result *ParseResult) []Triple {
}
rel := result.DepRels[i]
if isSubjRel(rel) && subj == "" {
subj = result.Tokens[i]
} else if isObjRel(rel) && obj == "" {
obj = result.Tokens[i]
objIdx = i
if isSubjRel(rel) {
subjIndices = append(subjIndices, i)
subjRels = append(subjRels, rel)
} else if isObjRel(rel) {
objIndices = append(objIndices, i)
objRels = append(objRels, rel)
}
}
if subj == "" {
// 主语降级:无 SBV/ATT 主语时向左查找最近的名词性节点
if len(subjIndices) == 0 {
for j := vi - 1; j >= 0; j-- {
if isNounLike(result.POS[j]) {
subj = result.Tokens[j]
subjIndices = append(subjIndices, j)
subjRels = append(subjRels, "SBV_IMPLICIT")
break
}
}
}
if subj != "" && obj != "" {
relLabel := result.Tokens[vi]
score := 0.8
if objIdx < len(result.Heads) && result.Heads[objIdx] == vi+1 {
// 宾语降级:无显式宾语时查找动词的其他名词性依赖
if len(objIndices) == 0 {
for i, head := range result.Heads {
if head == 0 {
continue
}
if head-1 == vi && isNounLike(result.POS[i]) && !isSubjRel(result.DepRels[i]) {
objIndices = append(objIndices, i)
objRels = append(objRels, "OBJ_IMPLICIT")
}
}
}
if len(subjIndices) == 0 || len(objIndices) == 0 {
continue
}
// Step 4: 笛卡尔积生成候选对,按模板赋予句法置信度
relLabel := result.Tokens[vi]
for _, si := range subjIndices {
for _, oi := range objIndices {
if si == oi {
continue
}
subj := result.Tokens[si]
obj := result.Tokens[oi]
score := 0.8 // 默认句法置信度
// 匹配模板查询精确置信度
for _, t := range depTemplates {
if t.objRel == result.DepRels[objIdx] {
if si < len(result.Heads) && result.Heads[si] == vi+1 &&
oi < len(result.Heads) && result.Heads[oi] == vi+1 &&
t.subjRel == result.DepRels[si] && t.objRel == result.DepRels[oi] {
score = t.score
break
}
}
triples = append(triples, Triple{
Subject: subj,
Relation: relLabel,
Object: obj,
Score: score,
Src: "dep",
SentenceRef: sentence,
})
}
triples = append(triples, Triple{
Subject: subj,
Relation: relLabel,
Object: obj,
Score: score,
Src: "dep",
})
}
// COO 链扩展:如果宾语有并列结构,为每个并列项生成三元组
if obj != "" {
cooExpanded := expandCOO(result, objIdx, vi)
// COO 链扩展:为每个宾语所在的并列结构生成额外三元组
for _, oi := range objIndices {
cooExpanded := expandCOO(result, oi, vi)
for _, cooObj := range cooExpanded {
if cooObj == obj {
if cooObj == result.Tokens[oi] {
continue
}
relLabel := result.Tokens[vi]
triples = append(triples, Triple{
Subject: subj,
Relation: relLabel,
Object: cooObj,
Score: 0.7,
Src: "dep_coo",
})
for _, si := range subjIndices {
subj := result.Tokens[si]
triples = append(triples, Triple{
Subject: subj,
Relation: relLabel,
Object: cooObj,
Score: 0.7,
Src: "dep_coo",
SentenceRef: sentence,
})
}
}
}
}
// Step 5: ATT 链合并多词实体
triples = mergeAttTriples(result, triples)
// 去重
triples = dedupTriples(triples)
return triples
}
@ -213,8 +265,8 @@ var posTemplates = []posTemplate{
{pattern: []string{"n", "v", "v", "n"}, subj: 0, verb: 1, obj: 3, score: 0.55},
}
// extractFromPOS 基于 POS 序列匹配模板提取三元组
func extractFromPOS(result *ParseResult) []Triple {
// extractFromPOS 基于 POS 序列匹配模板提取三元组 (Phase 2 降级路径)
func extractFromPOS(result *ParseResult, sentence string) []Triple {
if len(result.Tokens) < 2 {
return nil
}
@ -248,11 +300,12 @@ func extractFromPOS(result *ParseResult) []Triple {
continue
}
triples = append(triples, Triple{
Subject: subj,
Relation: verb,
Object: obj,
Score: tpl.score,
Src: "pos",
Subject: subj,
Relation: verb,
Object: obj,
Score: tpl.score,
Src: "pos",
SentenceRef: sentence,
})
}
}
@ -375,7 +428,7 @@ func isAdj(p string) bool {
}
func isSubjRel(rel string) bool {
return rel == "SBV"
return rel == "SBV" || rel == "ATT"
}
func isObjRel(rel string) bool {

View File

@ -35,7 +35,7 @@ func TestExtractFromPOS(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
if tt.input == "" {
result, _ := p.Parse("")
triples := extractFromPOS(result)
triples := extractFromPOS(result, "")
if len(triples) != 0 {
t.Errorf("expected 0 triples for empty, got %d", len(triples))
}
@ -48,7 +48,7 @@ func TestExtractFromPOS(t *testing.T) {
}
t.Logf("input=%q tokens=%v pos=%v", tt.input, result.Tokens, result.POS)
triples := extractFromPOS(result)
triples := extractFromPOS(result, tt.input)
for _, tr := range triples {
if tr.Subject == "" || tr.Relation == "" || tr.Object == "" {

View File

@ -10,11 +10,13 @@ type ParseResult struct {
// Triple 三元组 (subject, relation, object)
type Triple struct {
Subject string
Relation string
Object string
Score float64
Src string // "dep" / "fallback"
Subject string
Relation string
Object string
Score float64 // syntax_conf句法置信度Phase 2 输出)
VectorConf float64 // vector_conf语义向量置信度Phase 3 输出)
Src string // "dep" / "dep_coo" / "pos" / "fallback"
SentenceRef string // 原始句子用于LLM复审时修正
}
// TripleSet 提取结果

Binary file not shown.

View File

@ -0,0 +1,20 @@
{
"<bos>": 0,
"ADJ": 1,
"ADP": 2,
"ADV": 3,
"AUX": 4,
"CCONJ": 5,
"DET": 6,
"INTJ": 7,
"NOUN": 8,
"NUM": 9,
"PART": 10,
"PRON": 11,
"PROPN": 12,
"PUNCT": 13,
"SCONJ": 14,
"SYM": 15,
"VERB": 16,
"X": 17
}

24949
internal/nlp/models/vocab.json Normal file

File diff suppressed because it is too large Load Diff

277
internal/nlp/onnx.go Normal file
View File

@ -0,0 +1,277 @@
//go:build onnxruntime
package nlp
import (
"embed"
"encoding/json"
"fmt"
"os"
"path/filepath"
"gitcode.com/JianFeeeee/HomeAgent/internal/config"
ort "github.com/yalue/onnxruntime_go"
)
//go:embed models/*
var onnxModelFS embed.FS
type ONNXParser struct {
rt *ort.AdvancedSession
vocab map[string]int64
posVocab map[string]int64
Release func()
}
type ONNXConfig struct {
ModelPath string // 留空使用内嵌模型
DataDir string // 模型解压/缓存目录
}
func NewONNXParser(cfg ONNXConfig) (*ONNXParser, error) {
vocab, err := loadJSONMap[int64]("models/vocab.json", onnxModelFS)
if err != nil {
return nil, fmt.Errorf("load vocab: %w", err)
}
posVocab, err := loadJSONMap[int64]("models/pos_vocab.json", onnxModelFS)
if err != nil {
return nil, fmt.Errorf("load pos_vocab: %w", err)
}
modelPath := cfg.ModelPath
if modelPath == "" {
modelPath, err = extractEmbeddedModel(cfg.DataDir)
if err != nil {
return nil, fmt.Errorf("extract model: %w", err)
}
}
ort.SetSharedLibraryPath(findONNXRuntime())
if err := ort.InitializeEnvironment(); err != nil {
return nil, fmt.Errorf("init onnx env: %w", err)
}
inputs := ort.NewInputDetails()
inputs.Append("input_ids", []int64{1, 128})
outputs := ort.NewOutputDetails()
outputs.Append("pos_logits", []int64{1, 128, 18})
outputs.Append("head_logits", []int64{1, 128, 128})
outputs.Append("rel_logits", []int64{1, 128, 128, 18})
session, err := ort.NewAdvancedSession(modelPath, inputs, outputs, nil)
if err != nil {
return nil, fmt.Errorf("create session: %w", err)
}
release := func() {
session.Destroy()
ort.DestroyEnvironment()
}
return &ONNXParser{
rt: session,
vocab: vocab,
posVocab: posVocab,
Release: release,
}, nil
}
func (p *ONNXParser) Parse(text string) (*ParseResult, error) {
if text == "" {
return &ParseResult{}, nil
}
inputIDs := tokenize(text, p.vocab, 128)
inputIDs = padTo(inputIDs, 128)
inputTensor, err := ort.NewTensor(ort.NewShape(1, 128), inputIDs)
if err != nil {
return nil, fmt.Errorf("create input tensor: %w", err)
}
defer inputTensor.Destroy()
outputs, err := p.rt.Call(inputTensor)
if err != nil {
return nil, fmt.Errorf("onnx call: %w", err)
}
rawPOS := outputs[0].GetData().([]float32)
rawHeads := outputs[1].GetData().([]float32)
rawRels := outputs[2].GetData().([]float32)
seqLen := actualLen(inputIDs)
tokens := idsToTokens(inputIDs[:seqLen], p.vocab)
pos := decodePOS(rawPOS, seqLen, p.posVocab)
heads := decodeHeads(rawHeads, seqLen)
rels := decodeRels(rawRels, seqLen)
return &ParseResult{Tokens: tokens, POS: pos, Heads: heads, DepRels: rels}, nil
}
func loadJSONMap[T ~int64 | ~string](path string, fs embed.FS) (map[string]T, error) {
data, err := fs.ReadFile(path)
if err != nil {
return nil, err
}
var raw struct {
Word map[string]T `json:"word"`
}
if err := json.Unmarshal(data, &raw); err != nil {
result := make(map[string]T)
if err2 := json.Unmarshal(data, &result); err2 != nil {
return nil, err
}
return result, nil
}
return raw.Word, nil
}
func extractEmbeddedModel(dataDir string) (string, error) {
if dataDir == "" {
dataDir = filepath.Join(os.TempDir(), "homeagent-nlp")
}
os.MkdirAll(dataDir, 0755)
dst := filepath.Join(dataDir, "dep_parser.onnx")
if _, err := os.Stat(dst); err == nil {
return dst, nil
}
data, err := onnxModelFS.ReadFile("models/dep_parser.onnx")
if err != nil {
return "", err
}
if err := os.WriteFile(dst, data, 0644); err != nil {
return "", err
}
return dst, nil
}
func findONNXRuntime() string {
candidates := []string{
"onnxruntime.dll",
"libonnxruntime.so",
"libonnxruntime.dylib",
filepath.Join(os.Getenv("ONNXRUNTIME_DIR"), "libonnxruntime.so"),
filepath.Join(os.Getenv("ONNXRUNTIME_DIR"), "onnxruntime.dll"),
}
for _, c := range candidates {
if _, err := os.Stat(c); err == nil {
abs, _ := filepath.Abs(c)
return abs
}
}
return "onnxruntime.dll"
}
func tokenize(text string, vocab map[string]int64, maxLen int) []int64 {
ids := []int64{vocab["<bos>"]}
runes := []rune(text)
for i := 0; i < len(runes) && len(ids) < maxLen; i++ {
if id, ok := vocab[string(runes[i])]; ok {
ids = append(ids, id)
} else {
ids = append(ids, vocab["<unk>"])
}
}
return ids
}
func padTo(ids []int64, length int) []int64 {
for len(ids) < length {
ids = append(ids, 0)
}
return ids
}
func actualLen(ids []int64) int {
for i, id := range ids {
if id == 0 {
return i
}
}
return len(ids)
}
func idsToTokens(ids []int64, vocab map[string]int64) []string {
rev := make(map[int64]string)
for k, v := range vocab {
rev[v] = k
}
var tokens []string
for _, id := range ids {
if t, ok := rev[id]; ok {
tokens = append(tokens, t)
}
}
return tokens
}
func decodePOS(raw []float32, seqLen int, posVocab map[string]int64) []string {
rev := make(map[int64]string)
for k, v := range posVocab {
rev[v] = k
}
pos := make([]string, seqLen)
for i := 0; i < seqLen; i++ {
bestIdx := 0
bestVal := float32(-1e9)
for j := 0; j < 18; j++ {
v := raw[i*18+j]
if v > bestVal {
bestVal = v
bestIdx = j
}
}
if tag, ok := rev[int64(bestIdx)]; ok {
pos[i] = tag
}
}
return pos
}
func decodeHeads(raw []float32, seqLen int) []int {
heads := make([]int, seqLen)
for i := 0; i < seqLen; i++ {
bestIdx := 0
bestVal := float32(-1e9)
for j := 0; j < seqLen; j++ {
v := raw[i*seqLen+j]
if v > bestVal {
bestVal = v
bestIdx = j
}
}
heads[i] = bestIdx
}
return heads
}
func decodeRels(raw []float32, seqLen int) []string {
rels := make([]string, seqLen)
for i := 0; i < seqLen; i++ {
bestIdx := 0
bestVal := float32(-1e9)
for j := 0; j < 18; j++ {
// average over head dimension for argmax
var sum float32
for k := 0; k < seqLen; k++ {
sum += raw[i*seqLen*18+k*18+j]
}
avg := sum / float32(seqLen)
if avg > bestVal {
bestVal = avg
bestIdx = j
}
}
rels[i] = posIDToTag(bestIdx)
}
return rels
}
func posIDToTag(id int) string {
tags := []string{"<bos>", "ADJ", "ADP", "ADV", "AUX", "CCONJ", "DET", "INTJ", "NOUN", "NUM", "PART", "PRON", "PROPN", "PUNCT", "SCONJ", "SYM", "VERB", "X"}
if id >= 0 && id < len(tags) {
return tags[id]
}
return "X"
}

View File

@ -2,27 +2,264 @@
package nlp
import "fmt"
import (
"embed"
"encoding/json"
"fmt"
"strings"
)
// ONNXParserStub 占位 — 编译时未启用 onnxruntime
type ONNXParser struct{}
//go:embed models/vocab.json models/pos_vocab.json
var vocabFS embed.FS
// ONNXParser 在未启用 onnxruntime 时作为规则式降级解析器。
// 使用内嵌词表实现基于词典的 POS 标注 + 基于 POS 序列的依存关系推断。
type ONNXParser struct {
vocab map[string]int
posVocab map[string]int
}
type ONNXConfig struct {
ModelPath string
VocabPath string
POSVocPath string
ModelPath string // 留空使用内嵌规则引擎
DataDir string // 仅在 onnxruntime 启用时使用
}
func NewONNXParser(cfg ONNXConfig) (*ONNXParser, error) {
return nil, fmt.Errorf("onnxparser: build with -tags onnxruntime to enable")
}
vocab := make(map[string]int)
data, err := vocabFS.ReadFile("models/vocab.json")
if err != nil {
return nil, fmt.Errorf("read vocab: %w", err)
}
func (p *ONNXParser) Close() {}
var raw struct {
Word map[string]int `json:"word"`
}
if err := json.Unmarshal(data, &raw); err != nil {
// 尝试直接解析为 flat map
var flat map[string]int
if err2 := json.Unmarshal(data, &flat); err2 != nil {
return nil, fmt.Errorf("parse vocab: %w", err)
}
vocab = flat
} else {
vocab = raw.Word
}
posVocab := make(map[string]int)
data, err = vocabFS.ReadFile("models/pos_vocab.json")
if err != nil {
return nil, fmt.Errorf("read pos_vocab: %w", err)
}
if err := json.Unmarshal(data, &posVocab); err != nil {
return nil, fmt.Errorf("parse pos_vocab: %w", err)
}
return &ONNXParser{vocab: vocab, posVocab: posVocab}, nil
}
func (p *ONNXParser) Parse(text string) (*ParseResult, error) {
return nil, fmt.Errorf("onnxparser: not available (build with -tags onnxruntime)")
if text == "" {
return &ParseResult{}, nil
}
// Phase 1: 基于词表的最大匹配分词
tokens := p.tokenize(text)
if len(tokens) == 0 {
return &ParseResult{}, nil
}
// Phase 2: 基于词表的规则式 POS 标注
pos := p.tagPOS(tokens)
// Phase 3: 基于 POS 序列的依存头推断
heads := p.inferHeads(tokens, pos)
// Phase 4: 关系标签推断
rels := p.inferRels(tokens, pos, heads)
return &ParseResult{
Tokens: tokens,
POS: pos,
Heads: heads,
DepRels: rels,
}, nil
}
func (p *ONNXParser) EnsureModel(dataDir string) error {
return fmt.Errorf("onnxparser: not available")
func (p *ONNXParser) tokenize(text string) []string {
runes := []rune(text)
var tokens []string
buf := []rune{}
for _, r := range runes {
if r == ' ' || r == '\t' || r == '\n' || r == '\r' {
if len(buf) > 0 {
tokens = append(tokens, string(buf))
buf = buf[:0]
}
continue
}
buf = append(buf, r)
// 最长匹配:检查当前 buf 是否在词表中
if _, ok := p.vocab[string(buf)]; !ok && len(buf) > 0 {
// 回退:取 buf[:-1] 作为词,继续
if _, ok2 := p.vocab[string(buf[:len(buf)-1])]; ok2 && len(buf) > 2 {
tokens = append(tokens, string(buf[:len(buf)-1]))
buf = buf[len(buf)-1:]
}
}
}
if len(buf) > 0 {
tokens = append(tokens, string(buf))
}
if len(tokens) == 0 {
tokens = strings.Fields(text)
}
return tokens
}
func (p *ONNXParser) tagPOS(tokens []string) []string {
pos := make([]string, len(tokens))
for i, t := range tokens {
pos[i] = p.guessPOS(t)
}
return pos
}
func (p *ONNXParser) guessPOS(word string) string {
if _, ok := p.vocab[word]; !ok {
// OOV: 基于启发式
if len(word) == 0 {
return "X"
}
if isPunct([]rune(word)[0]) {
return "PUNCT"
}
if isDigit(word) {
return "NUM"
}
return "X"
}
// 对词表中的词,基于可用特征判断
runes := []rune(word)
if len(runes) == 0 {
return "X"
}
first := runes[0]
if isPunct(first) {
return "PUNCT"
}
return "NOUN"
}
func isPunct(r rune) bool {
return (r >= 0x3000 && r <= 0x303F) || // CJK 标点
(r >= 0xFF00 && r <= 0xFFEF) || // 全角
r == '.' || r == ',' || r == '!' || r == '?' ||
r == ';' || r == ':' || r == '"' || r == '\'' ||
r == '(' || r == ')' || r == '[' || r == ']' ||
r == '{' || r == '}' || r == '。' || r == '' ||
r == '' || r == '' || r == '' || r == '' ||
r == '、' || r == '' || r == '' || r == '“' || r == '”'
}
func isDigit(s string) bool {
for _, r := range s {
if r < '0' || r > '9' {
if r < 0xFF10 || r > 0xFF19 { // 全角数字
return false
}
}
}
return len(s) > 0
}
// inferHeads 基于 POS 序列的规则式依存头推断。
// 动词通常作为根head=0名词依附于动词形容词依附于名词。
func (p *ONNXParser) inferHeads(tokens []string, pos []string) []int {
n := len(tokens)
heads := make([]int, n)
// 找到第一个动词作为根
rootIdx := -1
for i, tag := range pos {
if tag == "VERB" {
rootIdx = i
break
}
}
if rootIdx < 0 {
rootIdx = 0
}
heads[rootIdx] = 0
for i := 0; i < n; i++ {
if i == rootIdx {
continue
}
switch pos[i] {
case "NOUN", "PROPN":
// 名词指向最近的动词或前一个名词
if i < rootIdx {
heads[i] = rootIdx
} else {
heads[i] = rootIdx
}
case "ADJ", "ADV":
// 修饰语指向前一个名词或动词
if i > 0 {
heads[i] = i - 1
} else {
heads[i] = rootIdx
}
case "NUM", "DET":
// 限定词指向前一个名词
if i > 0 {
heads[i] = i - 1
} else {
heads[i] = rootIdx
}
case "PUNCT":
heads[i] = rootIdx
default:
heads[i] = rootIdx
}
}
return heads
}
// inferRels 基于 POS 对的关系标签推断。
func (p *ONNXParser) inferRels(tokens []string, pos []string, heads []int) []string {
n := len(tokens)
rels := make([]string, n)
for i := 0; i < n; i++ {
if heads[i] == 0 {
rels[i] = "ROOT"
continue
}
h := heads[i]
if h < 0 || h >= n {
rels[i] = "dep"
continue
}
rels[i] = posToRel(pos[h], pos[i])
}
return rels
}
func posToRel(headPOS, depPOS string) string {
switch {
case depPOS == "NOUN" || depPOS == "PROPN":
return "nsubj"
case depPOS == "ADJ":
return "amod"
case depPOS == "ADV":
return "advmod"
case depPOS == "NUM" || depPOS == "DET":
return "det"
case depPOS == "VERB":
return "xcomp"
case depPOS == "PUNCT":
return "punct"
default:
return "dep"
}
}

View File

@ -1,12 +1,30 @@
package nlp
import "gitcode.com/JianFeeeee/HomeAgent/internal/memory/vector"
import (
"sort"
"gitcode.com/JianFeeeee/HomeAgent/internal/memory/vector"
)
// Parser 依存句法分析器接口
type Parser interface {
Parse(text string) (*ParseResult, error)
}
// defaultParser 包级默认解析器,由 SetDefaultParser 设置
var defaultParser Parser
// SetDefaultParser 设置包级默认解析器。
// 设置后NewExtractor(nil) 将使用此解析器而非纯降级模式。
func SetDefaultParser(p Parser) {
defaultParser = p
}
// GetDefaultParser 返回当前包级默认解析器
func GetDefaultParser() Parser {
return defaultParser
}
// Vectorizer 向量化接口,复用 memory/vector 或 memory/static_embedder
type Vectorizer interface {
Vectorize(text string) vector.Vector
@ -19,8 +37,13 @@ type Extractor struct {
embedder Vectorizer // 可选:用于 TransE 语义验证
}
// NewExtractor 创建提取器parser 为 nil 时纯用 fallback
// NewExtractor 创建提取器
// parser 为 nil 时尝试使用包级默认解析器 (SetDefaultParser)
// 若仍未设置则纯用 fallback (POS 模板匹配)。
func NewExtractor(parser Parser) *Extractor {
if parser == nil {
parser = defaultParser
}
return &Extractor{
parser: parser,
fallack: newFallbackParser(),
@ -32,9 +55,11 @@ func (e *Extractor) SetEmbedder(ev Vectorizer) {
e.embedder = ev
}
// Extract 从文本中提取三元组
// 优先使用 parser失败/无结果时自动降级到 fallback
// 如果设置了 embedder还会做 h+r≈t 向量验证过滤
// Extract 从文本中提取三元组(完整四阶段流水线)
// Phase 1: 句法解析LTP 分词 → POS 标注 → 依存句法树)
// Phase 2: 结构初筛(依存模板 / POS 模板 → 候选三元组 + syntax_conf
// Phase 3: 语义验证TransE h+r≈t → vector_conf
// Phase 4: 融合裁决(线性加权 → 阈值截断 → 降序输出)
func (e *Extractor) Extract(text string) *TripleSet {
if text == "" {
return &TripleSet{Src: "", Err: nil}
@ -50,11 +75,11 @@ func (e *Extractor) Extract(text string) *TripleSet {
}
var triples []Triple
// 主线:依存解析 + 模板匹配
// ——— Phase 1 & 2: 句法解析 + 结构初筛 ———
if e.parser != nil {
result, err := e.parser.Parse(sentence)
if err == nil && result != nil && len(result.Tokens) > 1 {
triples = extractFromDep(result)
triples = extractFromDep(result, sentence)
if len(triples) > 0 {
src = "dep_parser"
}
@ -65,18 +90,23 @@ func (e *Extractor) Extract(text string) *TripleSet {
if len(triples) == 0 && e.fallack != nil {
result, err := e.fallack.Parse(sentence)
if err == nil && result != nil && len(result.Tokens) > 1 {
triples = extractFromPOS(result)
triples = extractFromPOS(result, sentence)
if len(triples) > 0 {
src = "fallback"
}
}
}
// 向量验证(可选):用 h+r≈t 过滤不合理三元组
// ——— Phase 3: 语义验证 (TransE h+r≈t) ———
if len(triples) > 0 && e.embedder != nil {
triples = verifyTriples(triples, e.embedder)
}
// ——— Phase 4: 融合裁决 ———
if len(triples) > 0 {
triples = fuseTriples(triples)
}
allTriples = append(allTriples, triples...)
}
@ -86,10 +116,15 @@ func (e *Extractor) Extract(text string) *TripleSet {
return &TripleSet{Src: src}
}
// verifyTriples 使用 TransE 打分 (h+r≈t) 验证三元组,过滤低分项
// ——— Phase 3: 语义验证 ———
// verifyTriples 使用 TransE 打分 (h+r≈t) 计算 vector_conf
// 输入:候选三元组(带 syntax_conf
// 处理cos(h+r, t) → vector_conf
// 输出:带 vector_conf 的候选三元组
func verifyTriples(triples []Triple, embedder Vectorizer) []Triple {
var kept []Triple
for _, t := range triples {
for i := range triples {
t := &triples[i]
h := embedder.Vectorize(t.Subject)
r := embedder.Vectorize(t.Relation)
tv := embedder.Vectorize(t.Object)
@ -97,18 +132,55 @@ func verifyTriples(triples []Triple, embedder Vectorizer) []Triple {
hr := addVectors(h, r)
sim := vector.CosineSimilarity(hr, tv)
// 语义一致性过低 → 过滤(除非 fallback 无其他候选
if sim >= 0.25 {
t.Score *= (0.5 + 0.5*sim)
// 将 cos 映射到 [0, 1] 区间(原始可能在 [-1, 1]
t.VectorConf = (sim + 1.0) / 2.0
}
return triples
}
// ——— Phase 4: 融合裁决 ———
const (
fusionAlpha = 0.4 // syntax_conf 权重
fusionBeta = 0.6 // vector_conf 权重
fusionThreshold = 0.3 // 最终阈值
)
// fuseTriples 融合裁决:线性加权计算 final_score截断阈值降序输出
// 输入:候选三元组(带 syntax_conf + vector_conf
// 处理final_score = α * syntax_conf + β * vector_conf
// 输出:通过阈值且降序排列的最终三元组
func fuseTriples(triples []Triple) []Triple {
if len(triples) == 0 {
return triples
}
// 计算 final_score 并更新 Score 字段
for i := range triples {
t := &triples[i]
finalScore := fusionAlpha*t.Score + fusionBeta*t.VectorConf
t.Score = finalScore
}
// 截断低分项
kept := make([]Triple, 0, len(triples))
for _, t := range triples {
if t.Score >= fusionThreshold {
kept = append(kept, t)
}
}
if len(kept) == 0 {
return triples
}
// 降序排列
sort.Slice(kept, func(i, j int) bool {
return kept[i].Score > kept[j].Score
})
return kept
}
// ——— 向量工具 ———
// addVectors 向量加法 (h + r)
func addVectors(a, b vector.Vector) vector.Vector {
out := make(vector.Vector)
for k, v := range a {
@ -118,4 +190,4 @@ func addVectors(a, b vector.Vector) vector.Vector {
out[k] += v
}
return out
}
}

View File

@ -1,13 +0,0 @@
[Unit]
Description=HomeAgent - 24/7 AI Butler
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/homed -data /home/newqqagent
Restart=always
RestartSec=10
Environment=HOME=/root
[Install]
WantedBy=multi-user.target