diff --git a/.env.example b/.env.example deleted file mode 100644 index 21ade5f..0000000 --- a/.env.example +++ /dev/null @@ -1,21 +0,0 @@ -# CamTalk 环境变量模板 -# 复制为 .env 并填入实际值：cp .env.example .env -# .env 已在 .gitignore 中，不会提交到版本控制 - -# ---- AI 服务 API Key ---- -CAMTALK_AI_LLM_API_KEY=sk-xxx -CAMTALK_AI_STT_API_KEY= -CAMTALK_AI_TTS_API_KEY= - -# ---- 可选覆盖（默认值见 config.yaml）---- -# CAMTALK_AI_LLM_MODEL=gpt-4o -# CAMTALK_AI_LLM_ENDPOINT=https://api.openai.com/v1 -# CAMTALK_AI_LLM_TIMEOUT=10 -# CAMTALK_AI_STT_ENDPOINT=https://api.xiaomimimo.com/v1 -# CAMTALK_AI_TTS_ENDPOINT=https://api.openai.com/v1 -# CAMTALK_AI_TTS_VOICE=alloy -# CAMTALK_AI_TTS_SPEED=1.0 -# CAMTALK_AI_TTS_TIMEOUT=5 - -# ---- 应用 ---- -# APP_ENV=dev diff --git a/.gitea/workflows/deploy.yml b/.gitea/workflows/deploy.yml index f910ceb..65aa676 100644 --- a/.gitea/workflows/deploy.yml +++ b/.gitea/workflows/deploy.yml @@ -2,20 +2,34 @@ name: Deploy on: push: - branches: [main] + branches: [main, v2] jobs: deploy: runs-on: aliyun steps: - - name: Checkout - uses: "http://8.161.227.145:3000/huanghaosheng/checkout@releases/v4" - - - name: Install Docker CLI - run: apk add --no-cache docker-cli docker-cli-compose - - - name: Build and Deploy + - name: Deploy run: | - chmod +x deploy.sh - ./deploy.sh build - ./deploy.sh restart + sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories + apk add --no-cache rsync docker-cli docker-cli-compose + GIT_URL="http://8.161.227.145:3000/XEngineers/CamTalk.git" + if [ -d /root/camtalk/.git ]; then + cd /root/camtalk + git fetch "$GIT_URL" ${GITHUB_REF_NAME} --depth=1 + git reset --hard FETCH_HEAD + else + rm -rf /tmp/camtalk-deploy + git clone --depth=1 --branch ${GITHUB_REF_NAME} \ + http://8.161.227.145:3000/XEngineers/CamTalk.git /tmp/camtalk-deploy + mkdir -p /root/camtalk + rsync -a --delete \ + --exclude='.env' \ + --exclude='pgdata' \ + --exclude='redisdata' \ + /tmp/camtalk-deploy/ /root/camtalk/ + rm -rf /tmp/camtalk-deploy + fi + + chmod +x /root/camtalk/deploy.sh + /root/camtalk/deploy.sh build + /root/camtalk/deploy.sh restart \ No newline at end of file diff --git a/.gitignore b/.gitignore index 4bb1e75..ee73c2d 100644 --- a/.gitignore +++ b/.gitignore @@ -24,3 +24,6 @@ Thumbs.db # ---- Obsidian ---- .obsidian/ +.claudian/ +修改过程笔记/ +学习复盘/ diff --git a/CLAUDE.md b/CLAUDE.md index 8a8a26a..9780660 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,106 +1,73 @@ # CLAUDE.md -本文件为 Claude Code (claude.ai/code) 在本仓库中工作时提供指引。 +CamTalk — 多模态实时 AI 视觉对话助手（摄像头 + 麦克风 + 视觉 + 语音 AI） -## 项目概述 - -CamTalk 是一款多模态实时 AI 视觉对话助手。用户通过摄像头和麦克风与 AI 交互，AI 理解视觉场景和语音输入后，以文字和语音形式给出自然回应。项目目前处于设计文档阶段，源代码正在逐步构建。 - -> **文档优先原则：** 执行任何开发任务前，先读取 `docs/` 下的相关设计文档（架构、接口、技术选型等），以文档为最高依据。代码实现应与文档一致；若有偏差，优先更新文档（尤其是接口文档）。 +> **文档优先原则：** 开发前先读 `docs/` 设计文档，以文档为准；若代码与文档不一致，优先更新文档（尤其接口文档）。详细设计见 `docs/01-13` 系列文档。**注意**：`docs/Eino/` 框架文档内容庞大（~75 个文件），仅在需要了解 Eino Graph/节点/Callback 等框架细节时才读取。 ## 架构 -三层系统： +三层系统：前端（React + Vite）→ Go 网关（Gin + WebSocket + Eino Graph AI 编排）→ AI 服务（DashScope LLM, MiMo STT/TTS） -1. **浏览器客户端**（React 18 + TypeScript, Vite）—— 媒体采集、边缘预处理（VAD 通过 `@ricky0123/vad-web`、关键帧检测通过 Canvas 像素比较）、UI 渲染。核心 Hook：`useVisionSession()` -2. **Go 网关**（Gin, gorilla/websocket, Viper, Zap）—— WebSocket 服务器、会话管理、AI 编排。每个 WebSocket 连接一个 goroutine。 -3. **云端 AI 服务** —— 通过 OpenAI 兼容接口可灵活切换。默认：GPT-4o（LLM）、Deepgram（STT）、OpenAI TTS。仅通过 Go 网关访问，浏览器不直连。 +**AI 编排流水线**（Eino Graph 7 节点 DAG）：`STT → History → ChatModel → Msg2Str → Splitter → TTS → Done`。LLM token 通过 Callback 实时推送，TTS 逐句并行合成。 -**关键模式**：LLM 文本流和 TTS 音频流并行推送给客户端，以最小化感知延迟。 +**会话存储**（TieredManager）：L1 Memory → L2 Redis → L3 PostgreSQL 三级存储，30 分钟 TTL，Redis 故障自动降级。 -**存储**：MVP 阶段使用进程内存（`MemoryManager`），Redis 实现已就绪可通过配置切换，PostgreSQL 为规划中。Repository 接口模式（`HistoryRepository`、`UsageRepository`），MVP 用内存实现。 +**鉴权**：JWT 双 token 轮转（Access 120min + Refresh 7d），重放攻击检测（DB hash 校验），Redis 缓存装饰器。 ## 技术栈 -| 层级 | 技术 | -|------|------| -| 前端 | React 18, TypeScript, Vite, @ricky0123/vad-web | -| 后端 | Go, Gin, gorilla/websocket, Viper, Zap | -| LLM | GPT-4o（默认，通过 OpenAI 兼容接口可切换） | -| STT | Deepgram（默认） / MiMo ASR | -| TTS | OpenAI TTS（默认） / MiMo TTS | +前端：React 18 + TypeScript + Vite，VAD（@ricky0123/vad-web），ONNX Runtime +后端：Go 1.25+, Gin, WebSocket, Viper, Zap, CloudWeGo Eino Graph +AI：DashScope qwen3-vl-plus, MiMo ASR/TTS（可切换 Deepgram/OpenAI TTS） +存储：PostgreSQL 15 + Redis 7 -## 构建与运行命令 +## 快速启动 ```bash -# 前端 -cd frontend && npm install -npm run dev # Vite 开发服务器 -npm run build # 生产构建 -npm run lint # ESLint 检查 -npm run test # Vitest 测试 - -# 后端 -cd backend && go mod download -go run ./cmd/server # 启动网关，监听 :8080 -go build -o bin/camtalk ./cmd/server -go test ./... # 运行所有测试 -go test -run TestName ./path # 运行单个测试 -go vet ./... # 静态分析 +# 前端：npm run dev（Vite，代理 /ws 和 /api 到 :8080） +# 后端：go run ./cmd/server（监听 :8080） +# 生产：./deploy.sh up（4 容器：frontend/backend/postgres/redis） ``` -基础设施：MVP 使用进程内存管理会话状态。Redis 已实现可通过配置切换，PostgreSQL 为规划中。 +核心环境变量（`.env.example`）：`CAMTALK_AI_LLM_API_KEY`, `CAMTALK_AI_STT_API_KEY`, `CAMTALK_AUTH_JWT_SECRET`, `CAMTALK_STORAGE_DSN` -## WebSocket 协议 +配置优先级：环境变量 > `config.{APP_ENV}.yaml` > `config.yaml` +环境切换：`APP_ENV=dev|prod`（dev 默认，prod 启用限流 + 严格 CORS） -端点：`ws://localhost:8080/ws` +## 协议与 API -所有消息为 JSON 文本帧，统一信封格式 `{type, request_id?, timestamp?}`。完整契约见 `docs/03-接口文档.md`。 +**WebSocket**：`ws://localhost:8080/ws?token=&conversation_id=` +- 客户端：`query`（图像/音频 Base64）, `config`, `interrupt`, `ping` +- 服务端：`connected`, `stt_result`, `llm_chunk`, `llm_done`, `tts_audio`, `error`, `pong` +- 心跳：客户端 30s ping，服务端 60s 超时断连；重连：指数退避 1s→30s +- 实现：`CamTalkWebSocket` 单例（`frontend/src/lib/websocket.ts`），订阅模式，自动重连 -**客户端 → 服务端**：`query`（图像 Base64 + 音频 Base64）、`config`、`interrupt`、`ping` -**服务端 → 客户端**：`connected`、`stt_result`、`llm_chunk`、`llm_done`、`tts_audio`、`error`、`pong` +**REST API**：`/api/auth/*`（注册/登录/刷新/登出），`/api/conversations/*`（CRUD + 消息分页），`/api/health` -**心跳**：客户端每 30 秒 ping，服务端 60 秒无 ping 断开连接。 -**重连**：指数退避 + 抖动 —— 1s, 2s, 4s, 8s… 最大 30s。 +**错误码**：`INVALID_MESSAGE`, `SESSION_NOT_FOUND`, `RATE_LIMITED`, `IMAGE_TOO_LARGE`, `LLM_TIMEOUT`, `STT/TTS/LLM_ERROR`, `INVALID_TOKEN`, 等 -## REST API（辅助） +## 关键文件路径 -- `GET /api/health` — 健康检查（版本、运行时间、活跃会话数） -- `POST /api/sessions` — 创建会话（可选，MVP 在 WS 连接时自动创建） -- `DELETE /api/sessions/{id}` — 销毁会话 +**后端核心**： +- `backend/internal/eino/` — Graph 定义、节点、Callback、Adapter、State +- `backend/internal/session/tiered.go` — 三级会话存储 +- `backend/internal/store/` — Repository 实现（PG + 内存 + Redis 缓存） +- `backend/internal/ws/handler.go` — WebSocket 连接管理 +- `backend/migrations/` — SQL 迁移文件 -## 错误码 - -`INVALID_MESSAGE`、`SESSION_NOT_FOUND`、`RATE_LIMITED`、`IMAGE_TOO_LARGE`、`AUDIO_TOO_SHORT`、`LLM_TIMEOUT`、`LLM_ERROR`、`STT_ERROR`、`TTS_ERROR`、`INTERNAL_ERROR` - -## 前端组件结构 - -| 组件 | 职责 | -|------|------| -| `CameraManager` | 摄像头流采集 | -| `MicManager` | 麦克风音频采集 | -| `EdgeProcessor` | VAD + 关键帧检测（Canvas 像素比较） | -| `WebSocketManager` | WebSocket 连接生命周期管理 | -| `ChatPanel` | 消息展示 | -| `VideoPreview` | 摄像头画面预览 | - -## 后端模块结构 - -| 模块 | 职责 | -|------|------| -| WebSocket Handler | 连接管理、单播消息推送 | -| Session Manager | 会话状态、对话历史（Memory/Redis，30 分钟 TTL） | -| AI Orchestrator | STT→LLM→TTS 流式并行管道编排 | -| AI Service Layer | AI 服务抽象层（STT/LLM/TTS 多 provider） | -| REST API | 健康检查、会话管理（Gin 路由） | -| Models | 数据模型定义 | -| Model Router | 按请求选择 AI 模型（规划中） | -| Rate Limiter | 按用户的令牌桶速率限制（规划中） | +**前端核心**： +- `frontend/src/hooks/useVisionSession.ts` — 核心会话 Hook（~500 行） +- `frontend/src/lib/websocket.ts` — WebSocket 客户端单例 +- `frontend/src/lib/auth.tsx` — JWT 自动刷新 + AuthProvider +- `frontend/src/lib/api.ts` — REST 客户端（401 拦截 + token 刷新） +- `frontend/src/lib/ttsPlayer.ts` — 流式 TTS 音频播放队列 +- `frontend/vite.config.ts` — VAD 模型文件自动复制 + 代理配置 ## 编码规范 -- **Go**：遵循标准 Go 规范。所有 AI 调用使用 `context.Context` 做取消/超时。并发 map 访问使用 `sync.RWMutex`。结构体标签用 `json:"snake_case"`。 -- **TypeScript**：严格模式。所有数据模型用接口定义。WebSocket 消息类型用可辨识联合类型（`type` 字段）。 -- **提交信息**：Conventional Commits 格式，描述用中文。示例：`feat: 添加 WebSocket 连接管理`、`fix: 修复心跳超时判断`、`docs: 更新接口文档` -- **禁止自动 push**：除非用户明确要求。 -- **文档优先**：实现功能前先读取 `docs/` 下的相关设计文档。实现与文档不一致时，优先更新 `docs/` 下的接口文档。 +- **Go**：标准规范，`context.Context` 超时控制，`sync.RWMutex` 并发保护，`json:"snake_case"` 标签，编译期接口检查 `var _ Interface = (*Impl)(nil)` +- **TypeScript**：严格模式，接口定义数据模型，WebSocket 消息用可辨识联合类型（`type` 字段区分） +- **CORS**：禁止后端代码/配置文件配置 CORS，统一由代理层处理（开发环境 Vite proxy，生产环境 Nginx） +- **提交信息**：Conventional Commits，中文描述（如 `feat: 添加 WebSocket 心跳`） +- **禁止自动 push**：除非用户明确要求 +- **文档优先**：开发前先读 `docs/` 设计文档，代码与文档不一致时优先更新文档 diff --git a/README.md b/README.md index dd9d270..0cbdd85 100644 --- a/README.md +++ b/README.md @@ -1,167 +1,561 @@ # CamTalk -多模态实时 AI 视觉对话助手。用户通过摄像头和麦克风与 AI 交互，AI 理解视觉场景和语音输入后，以文字和语音形式给出自然回应。 +
-- **路演视频**：[哔哩哔哩弹幕网——七牛云第四批议题1](https://www.bilibili.com/video/BV1dDJK6cE5S/) -- **线上体验**：http://8.161.227.145:9000 +**多模态实时 AI 视觉对话助手** -> ⚠️ **注意**：由于线上地址使用 HTTP 协议，浏览器默认禁止在非 HTTPS 环境下调用摄像头和麦克风。需要按以下步骤配置 Chrome 浏览器： +用户通过摄像头和麦克风与 AI 交互，AI 理解视觉场景和语音输入后，以文字和语音形式给出自然回应 + +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![Go Version](https://img.shields.io/badge/Go-1.25+-00ADD8?logo=go)](https://go.dev/) +[![React](https://img.shields.io/badge/React-18-61DAFB?logo=react)](https://react.dev/) +[![TypeScript](https://img.shields.io/badge/TypeScript-5-3178C6?logo=typescript)](https://www.typescriptlang.org/) + +[路演视频](https://www.bilibili.com/video/BV1dDJK6cE5S/) • [在线体验](http://8.161.227.145:9000) • [文档](docs/README.md) + +
+ +--- + + +> ⚠️ **在线体验提示**：由于演示环境使用 HTTP 协议，需配置 Chrome 允许非 HTTPS 下访问摄像头/麦克风： > -> 1. 在浏览器地址栏中输入 `chrome://flags/#unsafely-treat-insecure-origin-as-secure`，回车 -> 2. 将 **Insecure origins treated as secure** 选项设置为 **Enabled**（已启用） -> 3. 在输入框中输入 `http://8.161.227.145:9000` 地址 -> 4. 点击右下角弹出的 **Relaunch** 按钮，自动重启浏览器 -> -> 重启后即可在该 HTTP 地址下正常调用摄像头和麦克风。 +> 1. 访问 `chrome://flags/#unsafely-treat-insecure-origin-as-secure` +> 2. 启用该选项，并在输入框填入 `http://8.161.227.145:9000` +> 3. 点击 **Relaunch** 重启浏览器 -![](docs/pictures/1.png) +![CamTalk 界面截图](docs/pictures/1.png) -## 架构 +## ✨ 核心特性 -三层系统，前端做轻量预处理，后端做智能编排，云端 AI 服务按需调用： +- 🎥 **多模态理解**：摄像头视觉 + 麦克风语音双输入，AI 理解完整场景 +- 🗣️ **自然对话**：基于 VAD 的端到端语音交互，低延迟流式响应 +- 🚀 **实时推送**：LLM 文本流 + TTS 音频流并行推送，感知延迟 < 0.5 秒 +- 🎭 **情景模式**：自由对话、面试官、英语老师等多场景支持 +- 💾 **对话历史**：自动保存会话，支持搜索、重命名、删除、时间分组 +- 🔐 **安全认证**：JWT 双 token 轮转 + Refresh Token Rotation 防重放 +- 📊 **三级存储**：Memory → Redis → PostgreSQL 自动降级，保障可靠性 +- 🌐 **国际化**：支持中文、英文、日文界面 + +## 🏗️ 系统架构 + +CamTalk 采用**三层架构**：前端轻量预处理 → Go 网关智能编排 → 云端 AI 按需调用 ```mermaid graph TB - subgraph client[浏览器客户端] - A1[媒体采集] - A2[VAD 语音检测] - A3[关键帧检测] - A4[UI 渲染] + subgraph Browser["🌐 浏览器客户端"] + UI["React UI 渲染"] + VAD["VAD 语音检测"] + Media["媒体采集"] end - subgraph gateway[Go 网关 :8080] - B1[WebSocket Handler] - B2[Session Manager] - B3[AI Orchestrator] - B4[REST API] + subgraph Gateway["⚙️ Go 网关 (Eino Graph)"] + WS["WebSocket Handler"] + Auth["JWT 认证"] + Session["会话管理 (三级存储)"] + Orch["AI 编排器 (7节点DAG)"] end - subgraph cloud[云端 AI 服务] - C1[STT 语音识别] - C2[LLM 多模态推理] - C3[TTS 语音合成] + subgraph AI["☁️ 云端 AI 服务"] + STT["STT (MiMo/Deepgram)"] + LLM["LLM (qwen3-vl-plus)"] + TTS["TTS (MiMo/OpenAI)"] end - client <-->|WebSocket| gateway - gateway <-->|HTTP| cloud + Browser <-->|"WebSocket
(JWT + query/config)"| Gateway + Orch --> STT + Orch --> LLM + Orch --> TTS ``` -**关键模式**：LLM 文本流和 TTS 音频流并行推送，用户先看到文字、紧接着听到语音，感知延迟 < 0.5 秒。 +### AI 编排流水线（Eino Graph） -## 技术栈 +基于 [CloudWeGo Eino](https://github.com/cloudwego/eino) 框架的声明式 7 节点 DAG： -| 层级 | 技术 | -|------|------| -| 前端 | React 18, TypeScript, Vite, @ricky0123/vad-web | -| 后端 | Go, Gin, gorilla/websocket, Viper, Zap | -| STT | Deepgram（默认） / MiMo ASR | -| LLM | GPT-4o（默认，通过 OpenAI 兼容接口可切换） | -| TTS | OpenAI TTS（默认） / MiMo TTS | +``` +START → STT → History → ChatModel → Msg2Str → Splitter → TTS → Done → END +``` -## 项目结构 +**核心优势**： +- **流式处理**：ChatModel 逐 token 推送，Callback AOP 机制实时转发客户端 +- **句子级 TTS**：Splitter 实时切分句子，TTS 逐句并行合成，无需等待完整回复 +- **类型安全**：Go 泛型 + 编译期检查，Graph 拓扑错误在编译时发现 + +## 🛠️ 技术栈 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
层级技术选型说明
前端React 18 + TypeScript + Vite组件化开发，类型安全，快速热更新
VAD@ricky0123/vad-web (ONNX Runtime)浏览器端语音活动检测，零延迟
后端Go 1.25+ + Gin + gorilla/websocket高并发 goroutine，长连接管理
AI 编排CloudWeGo Eino Graph声明式 DAG，Stream 模式，Callback AOP
STTMiMo ASR（默认）/ Deepgram实时语音识别，多语言支持
LLMDashScope qwen3-vl-plus多模态推理（通过 eino-ext OpenAI 接入）
TTSMiMo TTS（默认）/ OpenAI TTS自然语音合成
存储PostgreSQL 15 + Redis 7三级存储架构：Memory → Redis → PG
认证JWT (HS256) + bcrypt双 token 轮转 + Refresh Token Rotation
配置Viper + godotenvYAML + .env + 环境变量覆盖
日志Zap高性能结构化日志 + Trace ID 追踪
+ +## 📁 项目结构 ``` CamTalk/ -├── frontend/ # 浏览器客户端 +├── frontend/ # 🌐 浏览器客户端 │ └── src/ │ ├── components/ # UI 组件 +│ │ ├── LandingPage/ # 登录着陆页 + LoginModal │ │ ├── CameraManager/ # 摄像头流采集 -│ │ ├── MicManager/ # 麦克风音频采集 -│ │ ├── EdgeProcessor/ # VAD + 关键帧检测 -│ │ ├── WebSocketManager/ # WS 连接管理 -│ │ ├── ChatPanel/ # 消息展示 -│ │ ├── VideoPreview/ # 摄像头画面预览 -│ │ ├── ConfigPanel/ # 配置面板 -│ │ └── Toast/ # 通知提示 +│ │ ├── MicManager/ # 麦克风音频采集 + VAD +│ │ ├── WebSocketManager/ # WS 连接生命周期 +│ │ ├── ChatPanel/ # 消息展示 + 流式回复 +│ │ ├── SessionSidebar/ # 对话历史侧边栏 +│ │ └── ConfigPanel/ # 配置面板（主题/TTS/语言/场景） │ ├── hooks/ # 自定义 Hooks -│ │ ├── useVisionSession.ts # 核心会话 Hook +│ │ ├── useVisionSession.ts # 核心会话 Hook (~500 行) +│ │ ├── useSessionList.ts # 对话列表管理 │ │ └── useObservationMode.ts # 观察模式 │ ├── lib/ # 工具库 -│ │ ├── websocket.ts # WebSocket 连接管理 -│ │ ├── audio.ts # 音频编码 -│ │ ├── ttsPlayer.ts # TTS 播放器 -│ │ └── sampling.ts # 采样策略 +│ │ ├── websocket.ts # WebSocket 单例（心跳/重连/订阅） +│ │ ├── api.ts # REST 客户端（401拦截+刷新） +│ │ ├── auth.tsx # AuthProvider（JWT 自动刷新） +│ │ ├── ttsPlayer.ts # TTS 流式播放队列 +│ │ └── i18n/ # 国际化（zh-CN/en-US/ja-JP） │ └── types/ # TypeScript 类型定义 -├── backend/ # Go 网关 -│ ├── cmd/server/ # 入口 +├── backend/ # ⚙️ Go 网关 +│ ├── cmd/server/ # 服务入口（main.go） │ └── internal/ +│ ├── eino/ # 🔥 Eino Graph 编排层（7节点DAG） +│ │ ├── graph.go # Graph 构建与编译 +│ │ ├── adapter.go # EinoOrchestrator 适配器 +│ │ ├── callback.go # LLM token 推送回调 +│ │ ├── state.go # 跨节点状态管理 +│ │ └── nodes_*.go # STT/History/Splitter/TTS/Done 节点 +│ ├── session/ # 会话管理（TieredManager 三级存储） +│ ├── store/ # 持久化层（Repository 接口 + PG/内存实现） +│ │ ├── user_pg.go # PostgreSQL 实现 +│ │ └── cached_user.go # Redis 缓存装饰器 +│ ├── auth/ # 认证（JWT/bcrypt/中间件） │ ├── ai/ # AI 服务抽象层 -│ │ ├── llm/ # LLM 服务（OpenAI 兼容） -│ │ ├── stt/ # STT 服务（Deepgram/MiMo） -│ │ └── tts/ # TTS 服务（OpenAI/MiMo） -│ ├── orchestrator/ # AI 编排器（STT→LLM→TTS 管道） -│ ├── session/ # 会话管理（Memory/Redis） +│ │ ├── llm/ # LLM 提示词与场景 +│ │ ├── stt/ # STT 服务（MiMo/Deepgram） +│ │ └── tts/ # TTS 服务（MiMo/OpenAI） │ ├── ws/ # WebSocket Handler -│ ├── api/ # REST API -│ ├── config/ # 配置管理 -│ ├── models/ # 数据模型 -│ ├── errors/ # 错误码 -│ └── logger/ # 日志 -├── docs/ # 设计文档 -└── CLAUDE.md # Claude Code 指引 +│ ├── api/ # REST API（Auth/Conversation） +│ ├── config/ # 配置管理（Viper） +│ └── logger/ # 日志（Zap + Trace ID） +├── migrations/ # 📊 数据库迁移（嵌入式 SQL） +├── docs/ # 📚 设计文档 +│ ├── 01-架构设计.md +│ ├── 02-接口文档.md +│ ├── 08-Eino框架与编排设计.md +│ ├── 10-鉴权体系.md +│ └── 13-日志追踪.md +├── deploy.sh # 🐳 部署脚本（Docker Compose） +├── docker-compose.yml # 容器编排配置 +└── CLAUDE.md # 🤖 Claude Code 开发指引 ``` -## 快速开始 +## 🚀 快速开始 ### 前置条件 -- Node.js >= 18 -- Go >= 1.24 +- **Node.js** >= 18 +- **Go** >= 1.25 +- **PostgreSQL** >= 15（可选 Docker） +- **Redis** >= 7（可选，用于缓存加速） -### 前端 +### 本地开发 + +#### 1. 克隆项目 ```bash -cd frontend -npm install -npm run dev # Vite 开发服务器 http://localhost:5173 +git clone https://github.com/yourusername/CamTalk.git +cd CamTalk ``` -### 后端 +#### 2. 配置环境变量 + +```bash +# 复制环境变量模板 +cp backend/.env.example backend/.env + +# 编辑 .env 文件，填入以下必需配置： +# - CAMTALK_AUTH_JWT_SECRET（使用 openssl rand -hex 32 生成） +# - CAMTALK_STORAGE_DSN（PostgreSQL 连接字符串） +# - CAMTALK_AI_LLM_API_KEY（DashScope API Key） +# - CAMTALK_AI_STT_API_KEY（MiMo/Deepgram API Key） +# - CAMTALK_AI_TTS_API_KEY（MiMo/OpenAI API Key） +``` + +#### 3. 启动后端 ```bash cd backend + +# 安装依赖 go mod download -go run ./cmd/server # 启动网关 :8080 -``` -### 配置 +# 运行数据库迁移（自动创建表） +go run ./cmd/server migrate -后端配置文件位于 `backend/config.yaml`，支持环境变量覆盖（前缀 `CAMTALK_`）。 - -```bash -# 最小启动（需要至少一个 AI 服务的 API Key） -cd backend -CAMTALK_AI_LLM_API_KEY=sk-xxx \ -CAMTALK_AI_STT_API_KEY=xxx \ +# 启动服务（监听 :8080） go run ./cmd/server ``` -配置优先级：环境变量 > `config.{env}.yaml` > `config.yaml` > `.env` +#### 4. 启动前端 -## WebSocket 协议 +```bash +cd frontend -连接地址：`ws://localhost:8080/ws` +# 安装依赖 +npm install -所有消息为 JSON 文本帧，统一信封格式 `{type, request_id?, timestamp?}`。 +# 启动开发服务器（http://localhost:5173） +npm run dev +``` -**客户端 → 服务端**：`query`、`config`、`interrupt`、`ping` -**服务端 → 客户端**：`connected`、`stt_result`、`llm_chunk`、`llm_done`、`tts_audio`、`error`、`pong` +#### 5. 访问应用 -完整协议见 [docs/03-接口文档.md](docs/03-接口文档.md)。 +打开浏览器访问 [http://localhost:5173](http://localhost:5173)，注册账号后即可开始使用。 -## 文档 +#### 6. 代码检查与测试 + +```bash +# 安装 Go 代码检查工具 +go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest + +# 运行后端代码检查 +cd backend +golangci-lint run + +# 后端单元测试 +go test ./... + +# 后端集成测试（需要 PostgreSQL） +go test -tags=integration ./... + +# 前端代码检查 +cd frontend +npm run lint + +# 前端测试 +npm test +``` + +### 远程部署 + +#### 方式一：Docker Compose（推荐） + +```bash +# 1. 克隆代码到服务器 +git clone https://github.com/yourusername/CamTalk.git +cd CamTalk + +# 2. 配置环境变量 +cp backend/.env.example backend/.env +# 编辑 .env 文件，填入生产环境配置 + +# 3. 一键部署（frontend + backend + postgres + redis） +./deploy.sh up + +# 4. 查看日志 +./deploy.sh logs + +# 5. 停止服务 +./deploy.sh down +``` + +部署完成后访问 [http://localhost:9000](http://localhost:9000) + +#### 方式二：手动部署 + +```bash +# 1. 构建前端 +cd frontend +npm install +npm run build # 输出到 dist/ + +# 2. 构建后端 +cd backend +go build -o camtalk ./cmd/server + +# 3. 配置 Nginx +# 参考 nginx.conf.example 配置反向代理 + +# 4. 启动服务 +APP_ENV=prod ./camtalk + +# 5. 使用 systemd 管理（可选） +sudo systemctl enable camtalk +sudo systemctl start camtalk +``` + +#### 环境变量检查清单 + +部署前确保已配置以下环境变量： + +- ✅ `CAMTALK_AUTH_JWT_SECRET`（使用 `openssl rand -hex 32` 生成） +- ✅ `CAMTALK_STORAGE_DSN`（PostgreSQL 连接字符串） +- ✅ `CAMTALK_AI_LLM_API_KEY`（DashScope API Key） +- ✅ `CAMTALK_AI_STT_API_KEY`（STT 服务 API Key） +- ✅ `CAMTALK_AI_TTS_API_KEY`（TTS 服务 API Key） +- ✅ `APP_ENV=prod`（启用生产环境配置） + +### 配置优先级 + +``` +环境变量 > config.{APP_ENV}.yaml > config.yaml > .env +``` + +通过 `APP_ENV=prod` 切换生产环境配置（启用限流 + 严格 CORS） + +## 📡 WebSocket 协议 + +连接地址：`ws://localhost:8080/ws?token=&conversation_id=` + +所有消息为 JSON 文本帧，统一信封格式： + +```typescript +interface BaseMessage { + type: string; + request_id?: string; + timestamp?: number; +} +``` + +### 客户端 → 服务端 + +| 消息类型 | 说明 | 示例 | +|---------|------|------| +| `query` | 发送视觉+语音查询 | `{type: "query", image: "base64...", audio: "base64..."}` | +| `config` | 更新会话配置 | `{type: "config", scenario: "interviewer", language: "en"}` | +| `interrupt` | 中断当前响应 | `{type: "interrupt", request_id: "xxx"}` | +| `ping` | 心跳保活 | `{type: "ping"}` | + +### 服务端 → 客户端 + +| 消息类型 | 说明 | 触发时机 | +|---------|------|---------| +| `connected` | 连接成功 | WebSocket 握手后 | +| `stt_result` | STT 识别结果 | STT 节点完成 | +| `llm_chunk` | LLM 文本增量 | ChatModel 逐 token（Callback） | +| `llm_done` | LLM 推理完成 | Done 节点执行 | +| `tts_audio` | TTS 音频片段 | TTS 节点逐句合成 | +| `error` | 错误通知 | 任意节点失败 | +| `pong` | 心跳响应 | 响应 `ping` | + +**心跳机制**： +- 客户端每 30 秒发送 `ping` +- 服务端 60 秒无消息自动断连 +- 断连后自动重连（指数退避 1s → 30s） + +完整协议定义见 [docs/02-接口文档.md](docs/02-接口文档.md) + +## 🔐 认证体系 + +CamTalk 采用 **JWT 双 token 轮转 + Refresh Token Rotation** 安全机制： + +### 双 Token 设计 + +| Token | 有效期 | 存储位置 | 用途 | +|-------|-------|---------|------| +| `access_token` | 120 分钟 | 前端内存（推荐）/ localStorage | 访问受保护资源 | +| `refresh_token` | 7 天 | httpOnly Cookie（推荐）/ localStorage | 刷新 access_token | + +### Refresh Token Rotation + +每次刷新 token 时： +1. 验证 `refresh_token` 签名和有效期 +2. 查询数据库中的 SHA256 哈希 +3. **如果哈希不存在** → 检测到 token 复用 → **吊销该用户所有 token** +4. 删除旧 refresh_token，生成新 token pair +5. 返回新 access_token + refresh_token + +**防重放攻击**：旧 refresh_token 立即失效，复用时触发全局吊销，强制所有设备重新登录。 + +### REST API 端点 + +- `POST /api/auth/register` — 用户注册 +- `POST /api/auth/login` — 用户登录 +- `POST /api/auth/refresh` — 刷新 token +- `POST /api/auth/logout` — 登出（需认证） +- `GET /api/conversations` — 获取对话列表（需认证） +- `POST /api/conversations` — 创建对话（需认证） +- `GET /api/health` — 健康检查 + +详细设计见 [docs/10-鉴权体系.md](docs/10-鉴权体系.md) + +## 💾 三级存储架构 + +**TieredManager** 实现会话状态的三级存储，平衡性能与可靠性： + +``` +┌─────────────┐ +│ L1 Memory │ ← 微秒级读写，进程内缓存 +├─────────────┤ +│ L2 Redis │ ← 毫秒级访问，跨实例共享 +├─────────────┤ +│ L3 PostgreSQL│ ← 持久化存储，数据可靠性 +└─────────────┘ +``` + +**特性**： +- ✅ **自动降级**：Redis 故障时自动切换到 Memory + PostgreSQL 模式 +- ✅ **灵活配置**：支持单级（Memory）、双级（Memory + PG）、完整三级 +- ✅ **TTL 管理**：会话默认 30 分钟过期，自动清理 +- ✅ **写穿透**：数据先写 L1，异步同步到 L2/L3 + +## 📊 数据库设计 + +系统使用 PostgreSQL 存储持久化数据： + +### 核心表 + +| 表名 | 说明 | 关键字段 | +|------|------|---------| +| `users` | 用户账户 | `id (UUID)`, `username (UNIQUE)`, `password_hash (bcrypt)` | +| `sessions` | 对话会话 | `id (UUID)`, `user_id (FK)`, `title`, `config (JSONB)` | +| `messages` | 消息记录 | `id (BIGSERIAL)`, `session_id (FK)`, `role`, `content`, `tokens_used` | +| `refresh_tokens` | 刷新令牌 | `token_hash (PK, SHA256)`, `user_id (FK)`, `expires_at` | + +**关系**：`users 1:N sessions 1:N messages`，`users 1:N refresh_tokens` + +**迁移管理**：使用嵌入式 SQL 文件（`backend/migrations/`），应用启动时自动执行。 + +## 🛡️ 安全特性 + +- 🔒 **密码安全**：bcrypt (cost=10) 哈希，自动生成盐值 +- 🔑 **Token 安全**：JWT HS256 签名，refresh_token SHA256 哈希存储 +- 🚫 **防重放攻击**：Refresh Token Rotation + 复用检测自动吊销 +- 🌐 **传输安全**：生产环境强制 HTTPS，开发环境 Vite proxy 同源代理 +- 🚦 **限流保护**：令牌桶算法（生产环境启用），防暴力破解 +- 🔍 **日志追踪**：全链路 Trace ID，请求/响应/错误统一记录 + +## 🌍 部署架构 + +``` +┌─────────────┐ +│ Nginx │ ← 反向代理（静态资源 + API + WebSocket） +└──────┬──────┘ + │ +┌──────┴───────────────────┐ +│ Go Gateway 集群 │ +│ ├─ Gateway-1 │ +│ ├─ Gateway-2 │ +│ └─ Gateway-N │ +└───┬────────────┬─────────┘ + │ │ +┌───┴────┐ ┌───┴────────┐ +│ Redis │ │ PostgreSQL │ +└────────┘ └────────────┘ + │ +┌───┴────────────────────┐ +│ 外部 AI 服务 │ +│ ├─ DashScope (LLM) │ +│ ├─ MiMo (STT/TTS) │ +│ └─ Deepgram (可选) │ +└───────────────────────┘ +``` + +**跨域策略**：Nginx 统一反代前后端到同一域名，无跨域问题。 + +**水平扩展**：Gateway 无状态设计，会话状态存储在 Redis/PostgreSQL，支持多实例部署。 + +## 📖 文档 + +### 核心设计文档 | 文档 | 内容 | |------|------| -| [01-项目概述](docs/01-项目概述.md) | 项目目标与核心挑战 | -| [02-系统架构](docs/02-系统架构.md) | 三层架构、技术栈、部署方案 | -| [03-接口文档](docs/03-接口文档.md) | WebSocket 协议、REST API、配置管理 | -| [04-技术选型](docs/04-技术选型.md) | AI 服务栈、持久化层、前端边缘处理选型 | -| [05-用户故事](docs/05-用户故事.md) | 用户场景与优先级 | -| [06-语音交互](docs/06-语音交互.md) | VAD → STT → LLM → TTS 全链路 | -| [07-视觉理解](docs/07-视觉理解.md) | 帧采样、关键帧检测、多模态输入 | -| [08-成本控制](docs/08-成本控制.md) | 采样策略、端云协同、模型分级 | +| [01-架构设计](docs/01-架构设计.md) | 三层架构、技术栈、数据库设计、部署方案 | +| [02-接口文档](docs/02-接口文档.md) | WebSocket 协议、REST API、AI 服务层、编排器、配置管理 | +| [08-Eino框架与编排设计](docs/08-Eino框架与编排设计.md) | Eino Graph 7 节点 DAG、节点实现、流式处理、Callback AOP | +| [10-鉴权体系](docs/10-鉴权体系.md) | JWT 双 token 轮转、Refresh Token Rotation、密码安全、中间件 | +| [11-令牌桶限流](docs/11-令牌桶限流.md) | 限流算法、配置策略、生产环境保护 | +| [13-日志追踪](docs/13-日志追踪.md) | Zap 日志、Trace ID 全链路追踪、日志级别 | -## License +### 功能文档 -[MIT](LICENSE) © XEngineers +| 文档 | 内容 | +|------|------| +| [03-技术选型](docs/03-技术选型.md) | AI 服务栈、持久化层、前端边缘处理选型 | +| [04-用户故事](docs/04-用户故事.md) | 用户场景与优先级 | +| [05-语音交互](docs/05-语音交互.md) | VAD → STT → LLM → TTS 全链路 | +| [06-视觉理解](docs/06-视觉理解.md) | 帧采样、关键帧检测、多模态输入 | +| [07-成本控制](docs/07-成本控制.md) | 采样策略、端云协同、模型分级 | +| [09-情景切换](docs/09-情景切换.md) | 情景模式设计与实现 | +| [12-自定义情景](docs/12-自定义情景.md) | 用户自定义情景功能（规划中） | + + +## 🐛 问题反馈 + +遇到问题？请提交 [Issue](https://github.com/yourusername/CamTalk/issues)，并提供以下信息： + +- 操作系统版本 +- Go / Node.js 版本 +- 错误日志（后端日志 + 浏览器控制台） +- 复现步骤 + +## 📝 版权声明 + +MIT License © 2024 XEngineers + +--- + +
+ +**Built with ❤️ using Go, React, and AI** + +[⬆️ 回到顶部](#camtalk) + +
diff --git a/backend/.env.example b/backend/.env.example new file mode 100644 index 0000000..092827d --- /dev/null +++ b/backend/.env.example @@ -0,0 +1,32 @@ +# 运行环境 +# dev：本地开发环境（debug 日志、关闭限流、允许所有 CORS） +# prod：生产环境（info 日志、启用限流、严格 CORS 白名单） +# 本地开发保持 dev，生产部署会被 docker-compose.yml 覆盖为 prod +APP_ENV=dev + +# AI 服务 API Key +CAMTALK_AI_STT_API_KEY=sk-your-stt-key +CAMTALK_AI_LLM_API_KEY=sk-your-llm-key +CAMTALK_AI_TTS_API_KEY=sk-your-tts-key + +# JWT 认证 +CAMTALK_AUTH_JWT_SECRET=your-jwt-secret-here + +# 三级存储配置 +# L2: Redis（热数据分布式会话层） +CAMTALK_STORAGE_REDIS_ENABLED=true +# 开发时填写远程服务器地址，部署时 docker-compose 会覆盖为容器内网地址 +CAMTALK_REDIS_ADDR=your-remote-server:6379 +CAMTALK_REDIS_PASSWORD=your-redis-password + +# L3: PostgreSQL（冷数据持久化层） +CAMTALK_STORAGE_PERSISTENCE_ENABLED=true +CAMTALK_STORAGE_PERSISTENCE_DRIVER=postgres +POSTGRES_USER=camtalk +POSTGRES_PASSWORD=your-postgres-password +# 开发时填写远程服务器地址，部署时 docker-compose 会覆盖为容器内网地址 +CAMTALK_STORAGE_DSN=postgres://camtalk:your-postgres-password@your-remote-server:5432/camtalk?sslmode=disable + +# 可选覆盖（默认值见 config.yaml） +# CAMTALK_SERVER_PORT=8080 +# CAMTALK_LOG_LEVEL=info diff --git a/backend/.gitignore b/backend/.gitignore index ac13b7b..a230058 100644 --- a/backend/.gitignore +++ b/backend/.gitignore @@ -3,8 +3,7 @@ bin/ # 环境配置 -config.dev.yaml -config.prod.yaml +.env # 临时文件 tmp/ diff --git a/backend/Dockerfile b/backend/Dockerfile index 7036eb6..5551224 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -8,11 +8,19 @@ ENV GOPROXY=https://goproxy.cn,https://goproxy.io,direct # 先复制依赖清单，利用 Docker 缓存层 COPY go.mod go.sum ./ -RUN go mod download + +# --mount=type=cache 复用 Go module 缓存，依赖不变时跳过下载 +RUN --mount=type=cache,target=/go/pkg/mod \ + go mod download # 复制源码并构建 COPY . . -RUN CGO_ENABLED=0 GOOS=linux go build -o /camtalk ./cmd/server + +# 复用 module 缓存 + 编译缓存；-ldflags="-s -w" 裁剪符号表减小 ~30% 体积 +RUN --mount=type=cache,target=/go/pkg/mod \ + --mount=type=cache,target=/root/.cache/go-build \ + CGO_ENABLED=0 GOOS=linux \ + go build -ldflags="-s -w" -o /camtalk ./cmd/server # ---- 运行阶段 ---- FROM alpine:3.20 @@ -21,9 +29,9 @@ RUN apk add --no-cache ca-certificates tzdata WORKDIR /app -# 复制二进制和配置 +# 复制二进制和配置文件（敏感配置通过 docker-compose env_file 注入覆盖） COPY --from=builder /camtalk . -COPY config.yaml . +COPY config/ ./config/ EXPOSE 8080 diff --git a/backend/cmd/server/main.go b/backend/cmd/server/main.go index 224f666..f4fca48 100644 --- a/backend/cmd/server/main.go +++ b/backend/cmd/server/main.go @@ -10,17 +10,20 @@ import ( "time" "github.com/gin-gonic/gin" + "github.com/jackc/pgx/v5/pgxpool" + "github.com/redis/go-redis/v9" "github.com/hhs/camtalk/internal/api" "github.com/hhs/camtalk/internal/auth" - "github.com/hhs/camtalk/internal/ai/llm" "github.com/hhs/camtalk/internal/ai/stt" "github.com/hhs/camtalk/internal/ai/tts" "github.com/hhs/camtalk/internal/config" + eino "github.com/hhs/camtalk/internal/eino" "github.com/hhs/camtalk/internal/logger" - "github.com/hhs/camtalk/internal/orchestrator" + "github.com/hhs/camtalk/internal/ratelimit" "github.com/hhs/camtalk/internal/session" "github.com/hhs/camtalk/internal/store" + "github.com/hhs/camtalk/internal/trace" "github.com/hhs/camtalk/internal/ws" migrations "github.com/hhs/camtalk/migrations" ) @@ -32,8 +35,8 @@ var Version string var startTime = time.Now() func main() { - // 加载配置 - cfg, err := config.Load() + // 加载配置（工作目录用于定位 .env 和 config.yaml） + cfg, err := config.Load(".") if err != nil { panic("failed to load config: " + err.Error()) } @@ -47,20 +50,27 @@ func main() { "addr", cfg.Server.Addr(), ) - // 初始化存储层（条件初始化 PostgreSQL） + // 初始化存储层（三级存储架构：L1 内存 → L2 Redis → L3 PostgreSQL） ctx, cancel := context.WithCancel(context.Background()) defer cancel() var userRepo store.UserRepository var msgRepo store.MessageRepository var sessRepo store.SessionRepository + var pool *pgxpool.Pool // 数据库连接池 - if cfg.Storage.Driver == "postgres" { - if cfg.Storage.DSN == "" { - logger.Log.Fatalw("storage.dsn is required when storage.driver is postgres", + // L3: PostgreSQL（冷数据持久化层） + dsn := cfg.Storage.Persistence.DSN + if dsn == "" { + dsn = cfg.Storage.DSN // 兼容旧配置 + } + if cfg.Storage.Persistence.Enabled && cfg.Storage.Persistence.Driver == "postgres" { + if dsn == "" { + logger.Log.Fatalw("storage.persistence.dsn is required when persistence is enabled", "hint", "set CAMTALK_STORAGE_DSN environment variable") } - pool, err := store.NewPostgresPool(ctx, cfg.Storage.DSN) + var err error + pool, err = store.NewPostgresPool(ctx, dsn) if err != nil { logger.Log.Fatalw("failed to connect to postgres", "error", err) } @@ -74,27 +84,76 @@ func main() { userRepo = store.NewPgUserRepository(pool) msgRepo = store.NewPgMessageRepository(pool) sessRepo = store.NewPgSessionRepository(pool) - logger.Log.Infow("postgres storage initialized", "driver", cfg.Storage.Driver) + logger.Log.Infow("L3 PostgreSQL storage initialized", "driver", cfg.Storage.Persistence.Driver) } else { userRepo = store.NewMemUserRepository() - logger.Log.Info("using in-memory storage") + logger.Log.Info("using in-memory user storage") } - // 初始化 Session Manager + // L2: Redis（热数据分布式会话层） + var rdb *redis.Client + var redisMgr *session.RedisManager + if cfg.Storage.Redis.Enabled { + rdb = redis.NewClient(&redis.Options{ + Addr: cfg.Redis.Addr, + Password: cfg.Redis.Password, + DB: cfg.Redis.DB, + }) + // 验证 Redis 连接 + if err := rdb.Ping(ctx).Err(); err != nil { + logger.Log.Fatalw("failed to connect to redis", "error", err) + } + redisMgr = session.NewRedisManager( + rdb, + time.Duration(cfg.Session.TTL)*time.Minute, + cfg.Session.MaxHistory, + ) + // 包装 userRepo 为带 Redis 缓存的版本（refresh token 二级缓存） + userRepo = store.NewCachedUserRepository(userRepo, rdb, time.Duration(cfg.Auth.RefreshTTL)*time.Minute) + logger.Log.Infow("L2 Redis storage initialized", + "addr", cfg.Redis.Addr, + "db", cfg.Redis.DB, + "cached_user_repo", true) + } + + // 初始化 Session Manager（三级存储） var sessionMgr session.Manager - var sessionOpts []session.Option - if msgRepo != nil { - sessionOpts = append(sessionOpts, session.WithMessageRepository(msgRepo)) + if cfg.Storage.Redis.Enabled { + // L1 + L2 + L3 三级存储 + var tieredOpts []session.TieredOption + if sessRepo != nil { + tieredOpts = append(tieredOpts, session.WithTieredSessionRepository(sessRepo)) + } + if msgRepo != nil { + tieredOpts = append(tieredOpts, session.WithTieredMessageRepository(msgRepo)) + } + tieredMgr := session.NewTieredManager( + time.Duration(cfg.Session.TTL)*time.Minute, + cfg.Session.MaxHistory, + redisMgr, + tieredOpts..., + ) + sessionMgr = tieredMgr + defer tieredMgr.Stop() + logger.Log.Info("session manager initialized with L1+L2+L3 tiered storage") + } else { + // L1 + L3 两级存储（无 Redis） + var sessionOpts []session.Option + if msgRepo != nil { + sessionOpts = append(sessionOpts, session.WithMessageRepository(msgRepo)) + } + if sessRepo != nil { + sessionOpts = append(sessionOpts, session.WithSessionRepository(sessRepo)) + } + memMgr := session.NewMemoryManager( + time.Duration(cfg.Session.TTL)*time.Minute, + cfg.Session.MaxHistory, + sessionOpts..., + ) + sessionMgr = memMgr + defer memMgr.Stop() + logger.Log.Info("session manager initialized with L1+L3 storage (Redis disabled)") } - if sessRepo != nil { - sessionOpts = append(sessionOpts, session.WithSessionRepository(sessRepo)) - } - sessionMgr = session.NewMemoryManager( - time.Duration(cfg.Session.TTL)*time.Minute, - cfg.Session.MaxHistory, - sessionOpts..., - ) - defer sessionMgr.(*session.MemoryManager).Stop() // 初始化 AI 服务 logger.Log.Infow("initializing AI services", @@ -116,9 +175,6 @@ func main() { sttService = stt.NewDeepgramService(cfg.AI.STT.APIKey, cfg.AI.STT.Model, cfg.AI.STT.Endpoint, cfg.AI.STT.Timeout, logger.Log) logger.Log.Infow("STT service initialized", "provider", "deepgram", "model", cfg.AI.STT.Model) } - llmService := llm.NewOpenAIService(cfg.AI.LLM.APIKey, cfg.AI.LLM.Model, cfg.AI.LLM.Endpoint, cfg.AI.LLM.Timeout, cfg.AI.LLM.HTTPClientTimeout, logger.Log) - logger.Log.Infow("LLM service initialized", "provider", cfg.AI.LLM.Provider, "model", cfg.AI.LLM.Model, "endpoint", cfg.AI.LLM.Endpoint, "timeout", cfg.AI.LLM.Timeout) - var ttsService tts.Service switch strings.ToLower(cfg.AI.TTS.Provider) { case "mimo", "xiaomi": @@ -129,8 +185,16 @@ func main() { logger.Log.Infow("TTS service initialized", "provider", "openai", "model", cfg.AI.TTS.Model, "voice", cfg.AI.TTS.Voice, "speed", cfg.AI.TTS.Speed) } - // 初始化 Orchestrator - orch := orchestrator.New(sttService, llmService, ttsService, sessionMgr, cfg) + // 初始化 Eino Graph + Orchestrator + var userScenarioRepo store.UserScenarioRepository + if pool != nil { + userScenarioRepo = store.NewPostgresUserScenarioRepo(pool) + } + pipelineGraph, err := eino.NewPipelineGraph(ctx, cfg, sttService, ttsService, sessionMgr, userScenarioRepo) + if err != nil { + logger.Log.Fatalw("failed to create eino pipeline graph", "error", err) + } + orch := eino.NewEinoOrchestrator(pipelineGraph, sessionMgr, cfg.AI.LLM.Model) // 初始化认证服务 tokenMgr := auth.NewTokenManager( @@ -140,13 +204,32 @@ func main() { ) authService := auth.NewAuthService(tokenMgr, userRepo) + // 初始化限流器 + var limiter ratelimit.Limiter + if cfg.RateLimit.Enabled { + if rdb != nil { + // 多实例：使用 Redis 令牌桶 + limiter = ratelimit.NewRedisLimiter(rdb, cfg.RateLimit) + logger.Log.Info("rate limiter initialized with Redis backend") + } else { + // 单实例：使用内存令牌桶 + limiter = ratelimit.NewMemoryLimiter(cfg.RateLimit) + logger.Log.Info("rate limiter initialized with in-memory backend") + } + defer limiter.Stop() + } else { + logger.Log.Info("rate limiter disabled") + } + // Gin 模式 if cfg.App.Env == "prod" { gin.SetMode(gin.ReleaseMode) } r := gin.New() - r.Use(gin.Recovery()) + r.Use(trace.TraceMiddleware()) // 第一层：生成 trace ID + r.Use(trace.GinLogger()) // 第二层：记录请求 + r.Use(trace.GinRecovery()) // 第三层：panic 恢复 // REST API apiGroup := r.Group("/api") @@ -160,14 +243,29 @@ func main() { // Auth REST 端点 authHandler := api.NewAuthHandler(authService, tokenMgr) - authHandler.RegisterRoutes(apiGroup) + authHandler.RegisterRoutes(apiGroup, limiter) // Conversation REST 端点 convHandler := api.NewConversationHandler(sessionMgr, tokenMgr, msgRepo) convHandler.RegisterRoutes(apiGroup) + // UserScenario REST 端点 + if pool != nil { + userScenarioRepo := store.NewPostgresUserScenarioRepo(pool) + userScenarioHandler := api.NewUserScenarioHandler(userScenarioRepo) + scenarioGroup := apiGroup.Group("/scenarios") + scenarioGroup.Use(auth.AuthMiddleware(tokenMgr)) + { + scenarioGroup.GET("", userScenarioHandler.List) + scenarioGroup.POST("", userScenarioHandler.Create) + scenarioGroup.GET("/:id", userScenarioHandler.Get) + scenarioGroup.PATCH("/:id", userScenarioHandler.Update) + scenarioGroup.DELETE("/:id", userScenarioHandler.Delete) + } + } + // WebSocket - r.GET("/ws", ws.ServeWS(sessionMgr, orch, cfg, tokenMgr)) + r.GET("/ws", ws.ServeWS(sessionMgr, orch, cfg, tokenMgr, limiter, userScenarioRepo)) // HTTP Server srv := &http.Server{ diff --git a/backend/config.yaml b/backend/config.yaml deleted file mode 100644 index 6e427d1..0000000 --- a/backend/config.yaml +++ /dev/null @@ -1,42 +0,0 @@ -# config.yaml — 默认配置 -app: - env: dev - -server: - host: "0.0.0.0" - port: 8080 - read_timeout: 30 - write_timeout: 30 - -redis: - addr: "localhost:6379" - password: "" - db: 0 - -ai: - stt: - provider: mimo - model: mimo-v2.5-asr - endpoint: "https://api.xiaomimimo.com/v1" - api_key: "sk-c3jhv58rr5djhxw398w2rrij5tfpnpdgxqq1bojagshzviah" - llm: - provider: dashscope - model: qwen3-vl-plus - endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1" - api_key: "sk-ws-H.REHELLY.C4s3.MEUCIQCRee37XWEKp2szaxVLFDtR1rxNNsf372zMvCR0Xl6UvQIgZgvhRTvaa1FmhbCQJgaHu4Jny29AQkn01-3hX9CWBOg" - timeout: 30 - tts: - provider: mimo - model: mimo-v2.5-tts - voice: mimo_default - speed: 1.0 - endpoint: "https://token-plan-cn.xiaomimimo.com/v1" - api_key: "tp-c9e7scwfx94qvqyhpnahnw8uaiya01za2qzvg4xe24rp3xiv" - timeout: 5 - -storage: - driver: memory - -log: - level: info - format: console diff --git a/backend/config/config.dev.yaml b/backend/config/config.dev.yaml new file mode 100644 index 0000000..259879a --- /dev/null +++ b/backend/config/config.dev.yaml @@ -0,0 +1,68 @@ +# CamTalk 开发环境配置 +# 通过 APP_ENV=dev 加载此文件，覆盖 config.yaml 中的配置 + +server: + host: "0.0.0.0" + port: 8080 + heartbeat_interval: 30 + heartbeat_timeout: 60 + allowed_origins: [] # 开发环境允许所有来源 + +session: + ttl: 30 # 开发环境会话较短，方便测试过期逻辑 + max_history: 20 + +ai: + stt: + provider: mimo # 与生产环境一致 + model: mimo-v2.5-asr + endpoint: "https://api.xiaomimimo.com/v1" + timeout: 10 # 开发环境超时较长，方便调试 + http_client_timeout: 30 + llm: + provider: dashscope # 与生产环境一致 + model: qwen3-vl-plus + endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1" + timeout: 60 # 开发环境 LLM 超时较长 + http_client_timeout: 120 + tts: + provider: mimo # 与生产环境一致 + model: mimo-v2.5-tts + voice: mimo_default + speed: 1.0 + endpoint: "https://token-plan-cn.xiaomimimo.com/v1" + timeout: 10 + http_client_timeout: 30 + output_format: mp3 + sample_rate: 24000 + +storage: + redis: + enabled: true # 开发环境启用 Redis，测试三级存储 + persistence: + enabled: true # 开发环境启用持久化 + +redis: + addr: "localhost:6379" # 本地 Redis + password: "" + db: 0 + +auth: + access_ttl: 120 # 开发环境 Access Token 2 小时，方便调试 + refresh_ttl: 10080 # 7 天 + +ratelimit: + enabled: false # 开发环境关闭限流，方便测试 + query: + capacity: 10 + rate: 0.2 + login: + capacity: 5 + rate: 0.1 + register: + capacity: 3 + rate: 0.05 + +log: + level: debug # 开发环境 debug 日志 + format: console # 控制台格式，易读 diff --git a/backend/config/config.prod.yaml b/backend/config/config.prod.yaml new file mode 100644 index 0000000..84195a8 --- /dev/null +++ b/backend/config/config.prod.yaml @@ -0,0 +1,71 @@ +# CamTalk 生产环境配置 +# 通过 APP_ENV=prod 加载此文件，覆盖 config.yaml 中的配置 + +server: + host: "0.0.0.0" + port: 8080 + read_timeout: 30 + write_timeout: 30 + shutdown_timeout: 15 # 生产环境优雅关闭时间稍长 + heartbeat_interval: 30 + heartbeat_timeout: 60 + +session: + ttl: 60 # 生产环境会话 1 小时 + max_history: 20 + +ai: + stt: + provider: mimo # 生产环境推荐 MiMo，性价比高 + model: mimo-v2.5-asr + endpoint: "https://api.xiaomimimo.com/v1" + timeout: 5 # 生产环境严格超时控制 + http_client_timeout: 30 + llm: + provider: dashscope # 生产环境推荐通义千问，稳定性好 + model: qwen3-vl-plus + endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1" + timeout: 30 + http_client_timeout: 60 + tts: + provider: mimo # 生产环境推荐 MiMo TTS + model: mimo-v2.5-tts + voice: mimo_default + speed: 1.0 + endpoint: "https://token-plan-cn.xiaomimimo.com/v1" + timeout: 5 + http_client_timeout: 30 + output_format: mp3 + sample_rate: 24000 + +storage: + redis: + enabled: true # 生产环境必须启用 Redis + persistence: + enabled: true # 生产环境必须启用持久化 + driver: postgres + +redis: + addr: "redis:6379" # Docker Compose 内部服务名 + password: "" # 密码通过 CAMTALK_REDIS_PASSWORD 环境变量设置 + db: 0 + +auth: + access_ttl: 120 # 生产环境 Access Token 2 小时 + refresh_ttl: 10080 # Refresh Token 7 天 + +ratelimit: + enabled: true # 生产环境启用限流 + query: + capacity: 10 # 允许突发 10 个请求 + rate: 0.2 # 每 5 秒恢复 1 个令牌 + login: + capacity: 5 # 防暴力破解 + rate: 0.1 # 每 10 秒恢复 1 次 + register: + capacity: 3 # 防批量注册 + rate: 0.05 # 每 20 秒恢复 1 次 + +log: + level: info # 生产环境 info 级别 + format: json # JSON 格式，便于日志收集和分析 diff --git a/backend/config/config.yaml b/backend/config/config.yaml new file mode 100644 index 0000000..f31a9cb --- /dev/null +++ b/backend/config/config.yaml @@ -0,0 +1,80 @@ +# CamTalk 后端配置 + +app: + env: dev # dev / prod，可通过 APP_ENV 环境变量覆盖 + +server: + host: "0.0.0.0" + port: 8080 + read_timeout: 30 # 秒 + write_timeout: 30 # 秒 + shutdown_timeout: 10 # 优雅关闭超时（秒） + heartbeat_interval: 30 # 心跳检查间隔（秒） + heartbeat_timeout: 60 # 心跳超时断开（秒） + allowed_origins: [] # CORS 白名单，空=允许所有 + +session: + ttl: 30 # 会话过期时间（分钟） + max_history: 20 # 对话历史上限（条） + +ai: + stt: + provider: mimo # mimo / deepgram + model: mimo-v2.5-asr + endpoint: "https://api.xiaomimimo.com/v1" + timeout: 5 # STT 请求超时（秒） + http_client_timeout: 30 # HTTP 客户端超时（秒） + llm: + provider: dashscope # dashscope / openai + model: qwen3-vl-plus + endpoint: "https://dashscope.aliyuncs.com/compatible-mode/v1" + timeout: 30 # LLM 请求超时（秒） + http_client_timeout: 60 # HTTP 客户端超时（秒） + tts: + provider: mimo # mimo / openai + model: mimo-v2.5-tts + voice: mimo_default + speed: 1.0 + endpoint: "https://token-plan-cn.xiaomimimo.com/v1" + timeout: 5 # TTS 请求超时（秒） + http_client_timeout: 30 # HTTP 客户端超时（秒） + output_format: mp3 # 输出格式：mp3 / wav + sample_rate: 24000 # 输出采样率 + +storage: + # 三级存储架构：L1 内存 → L2 Redis → L3 PostgreSQL + redis: + enabled: true # 是否启用 Redis（L2 热数据层） + persistence: + enabled: true # 是否启用持久化（L3 冷数据层） + driver: postgres # postgres + # dsn 通过环境变量 CAMTALK_STORAGE_DSN 设置 + +redis: + addr: "localhost:6379" + password: "" + db: 0 + +auth: + # jwt_secret 通过环境变量 CAMTALK_AUTH_JWT_SECRET 设置 + access_ttl: 120 # Access Token 过期时间（分钟） + refresh_ttl: 10080 # Refresh Token 过期时间（分钟），7 天 + +ratelimit: + enabled: false # 是否启用限流 + # WebSocket query 消息限流（核心，控制 AI 成本） + query: + capacity: 10 # 突发容量：允许连续发 10 个 query + rate: 0.2 # 填充速率：每 5 秒补充 1 个令牌 + # REST API 登录限流（防暴力破解） + login: + capacity: 5 # 突发容量：允许连续 5 次登录尝试 + rate: 0.1 # 填充速率：每 10 秒补充 1 次 + # REST API 注册限流 + register: + capacity: 3 # 突发容量：允许连续 3 次注册 + rate: 0.05 # 填充速率：每 20 秒补充 1 次 + +log: + level: info # debug / info / warn / error + format: console # console / json diff --git a/backend/go.mod b/backend/go.mod index d14bd93..65fe5c3 100644 --- a/backend/go.mod +++ b/backend/go.mod @@ -3,23 +3,36 @@ module github.com/hhs/camtalk go 1.25.0 require ( + github.com/alicebob/miniredis/v2 v2.38.0 + github.com/cloudwego/eino v0.9.9 + github.com/cloudwego/eino-ext/components/model/openai v0.1.13 github.com/gin-gonic/gin v1.10.0 + github.com/golang-jwt/jwt/v5 v5.3.1 github.com/google/uuid v1.6.0 github.com/gorilla/websocket v1.5.3 github.com/jackc/pgx/v5 v5.10.0 + github.com/joho/godotenv v1.5.1 + github.com/oklog/ulid/v2 v2.1.1 github.com/redis/go-redis/v9 v9.20.1 github.com/spf13/viper v1.21.0 github.com/stretchr/testify v1.11.1 go.uber.org/zap v1.28.0 + golang.org/x/crypto v0.31.0 ) require ( - github.com/bytedance/sonic v1.11.6 // indirect - github.com/bytedance/sonic/loader v0.1.1 // indirect + github.com/bahlo/generic-list-go v0.2.0 // indirect + github.com/buger/jsonparser v1.1.1 // indirect + github.com/bytedance/gopkg v0.1.3 // indirect + github.com/bytedance/sonic v1.15.0 // indirect + github.com/bytedance/sonic/loader v0.5.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/cloudwego/base64x v0.1.4 // indirect - github.com/cloudwego/iasm v0.2.0 // indirect + github.com/cloudwego/base64x v0.1.6 // indirect + github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17 // indirect github.com/davecgh/go-spew v1.1.1 // indirect + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/eino-contrib/jsonschema v1.0.3 // indirect + github.com/evanphx/json-patch v0.5.2 // indirect github.com/fsnotify/fsnotify v1.9.0 // indirect github.com/gabriel-vasile/mimetype v1.4.3 // indirect github.com/gin-contrib/sse v0.1.0 // indirect @@ -28,19 +41,25 @@ require ( github.com/go-playground/validator/v10 v10.20.0 // indirect github.com/go-viper/mapstructure/v2 v2.4.0 // indirect github.com/goccy/go-json v0.10.2 // indirect - github.com/golang-jwt/jwt/v5 v5.3.1 // indirect + github.com/goph/emperror v0.17.2 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect github.com/json-iterator/go v1.1.12 // indirect github.com/klauspost/cpuid/v2 v2.2.10 // indirect github.com/leodido/go-urn v1.4.0 // indirect + github.com/mailru/easyjson v0.7.7 // indirect github.com/mattn/go-isatty v0.0.20 // indirect + github.com/meguminnnnnnnnn/go-openai v0.1.2 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/nikolalohinski/gonja v1.5.3 // indirect github.com/pelletier/go-toml/v2 v2.2.4 // indirect + github.com/pkg/errors v0.9.1 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/sagikazarmark/locafero v0.11.0 // indirect + github.com/sirupsen/logrus v1.9.3 // indirect + github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f // indirect github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect github.com/spf13/afero v1.15.0 // indirect github.com/spf13/cast v1.10.0 // indirect @@ -49,11 +68,14 @@ require ( github.com/subosito/gotenv v1.6.0 // indirect github.com/twitchyliquid64/golang-asm v0.15.1 // indirect github.com/ugorji/go/codec v1.2.12 // indirect + github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect + github.com/yargevad/filepathx v1.0.0 // indirect + github.com/yuin/gopher-lua v1.1.1 // indirect go.uber.org/atomic v1.11.0 // indirect go.uber.org/multierr v1.10.0 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect - golang.org/x/arch v0.8.0 // indirect - golang.org/x/crypto v0.23.0 // indirect + golang.org/x/arch v0.11.0 // indirect + golang.org/x/exp v0.0.0-20230713183714-613f0c0eb8a1 // indirect golang.org/x/net v0.25.0 // indirect golang.org/x/sync v0.17.0 // indirect golang.org/x/sys v0.30.0 // indirect diff --git a/backend/go.sum b/backend/go.sum index 38b2a4b..87fc168 100644 --- a/backend/go.sum +++ b/backend/go.sum @@ -1,30 +1,60 @@ +github.com/airbrake/gobrake v3.6.1+incompatible/go.mod h1:wM4gu3Cn0W0K7GUuVWnlXZU11AGBXMILnrdOU8Kn00o= +github.com/alicebob/miniredis/v2 v2.38.0 h1:nZAzCR+Lj+Vxk4ZXzm2NuKq2O33RXj1XxJ2e2uP9jiw= +github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM= +github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk= +github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg= +github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA= +github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4= github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= -github.com/bytedance/sonic v1.11.6 h1:oUp34TzMlL+OY1OUWxHqsdkgC/Zfc85zGqw9siXjrc0= -github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4= -github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM= -github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU= +github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs= +github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0= +github.com/bugsnag/bugsnag-go v1.4.0/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqREvZLWFrtK8= +github.com/bugsnag/panicwrap v1.2.0/go.mod h1:D/8v3kj0zr8ZAKg1AQ6crr+5VwKN5eIywRkfhyM/+dE= +github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M= +github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM= +github.com/bytedance/mockey v1.3.0 h1:ONLRdvhqmCfr9rTasUB8ZKCfvbdD2tohOg4u+4Q/ed0= +github.com/bytedance/mockey v1.3.0/go.mod h1:1BPHF9sol5R1ud/+0VEHGQq/+i2lN+GTsr3O2Q9IENY= +github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE= +github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k= +github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE= +github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo= +github.com/certifi/gocertifi v0.0.0-20190105021004-abcd57078448/go.mod h1:GJKEexRPVJrBSOjoqN5VNOIKJ5Q3RViH6eu3puDRwx4= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y= -github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w= -github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg= -github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY= +github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M= +github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU= +github.com/cloudwego/eino v0.9.9 h1:x63hvRif6ANPh9YEPoTIrp1potEeoLQFAjOclKaX/Kg= +github.com/cloudwego/eino v0.9.9/go.mod h1:OBD1mrkfkt/pJa4rkg1P0VnaMeOVl7l8IAdEqY//3IQ= +github.com/cloudwego/eino-ext/components/model/openai v0.1.13 h1:5XHRTiTD5bt9KQrMHcfvuWNklEC3tpm3XHejdozt9vM= +github.com/cloudwego/eino-ext/components/model/openai v0.1.13/go.mod h1:mgIoqYYOc0eECCqvLbEYpOJrQNTNxkwXzSJzFU+v5sQ= +github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17 h1:EeVcR1TslRA2IdNW1h/2LaGbPlffwGhQm99jM3zWZiI= +github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17/go.mod h1:Zkcx6DPTR2NfWmtSXbhItswGw6hqUezNPhNcke0pOG8= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/eino-contrib/jsonschema v1.0.3 h1:2Kfsm1xlMV0ssY2nuxshS4AwbLFuqmPmzIjLVJ1Fsp0= +github.com/eino-contrib/jsonschema v1.0.3/go.mod h1:cpnX4SyKjWjGC7iN2EbhxaTdLqGjCi0e9DxpLYxddD4= +github.com/evanphx/json-patch v0.5.2 h1:xVCHIVMUu1wtM/VkR9jVZ45N3FhZfYMMYGorLCR8P3k= +github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= +github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0= github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk= +github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ= github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI= github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU= github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y= +github.com/go-check/check v0.0.0-20180628173108-788fd7840127 h1:0gkP6mzaMqkmpcJYCFOLkIBwI7xFExG03bbkOkCvUPI= +github.com/go-check/check v0.0.0-20180628173108-788fd7840127/go.mod h1:9ES+weclKsC9YodN5RgxqK/VD9HM9JsCSh7rNhMZE98= github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s= github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= @@ -37,15 +67,22 @@ github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9L github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU= github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I= +github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM= github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY= github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= +github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/goph/emperror v0.17.2 h1:yLapQcmEsO0ipe9p5TaN22djm3OFV/TfM/fcYP0/J18= +github.com/goph/emperror v0.17.2/go.mod h1:+ZbQ+fUNO/6FNiUo0ujtMjhgad9Xa6fQL9KhH4LNHic= +github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g= +github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k= github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM= github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg= github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo= @@ -54,35 +91,73 @@ github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0= github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= +github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= +github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0= +github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4= +github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= -github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= +github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= +github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= +github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0/go.mod h1:1NbS8ALrpOvjt0rHPNLyCIeMtbizbir8U//inJ+zuB8= github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE= github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= -github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M= +github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI= +github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= +github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU= +github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/meguminnnnnnnnn/go-openai v0.1.2 h1:iXombGGjqjBrmE9WaSidUhhi3YQhf42QTHvHLMkgvCA= +github.com/meguminnnnnnnnn/go-openai v0.1.2/go.mod h1:qs96ysDmxhE4BZoU45I43zcyfnaYxU3X+aRzLko/htY= +github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4= +github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/nikolalohinski/gonja v1.5.3 h1:GsA+EEaZDZPGJ8JtpeGN78jidhOlxeJROpqMT9fTj9c= +github.com/nikolalohinski/gonja v1.5.3/go.mod h1:RmjwxNiXAEqcq1HeK5SSMmqFJvKOfTfXhkJv6YBtPa4= +github.com/oklog/ulid/v2 v2.1.1 h1:suPZ4ARWLOJLegGFiZZ1dFAkqzhMjL3J1TzI+5wHz8s= +github.com/oklog/ulid/v2 v2.1.1/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ= +github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= +github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o= github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= +github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/redis/go-redis/v9 v9.20.1 h1:sfCU6A8P3dXbKyWes02uxA2baehGux9dZHfEKtsTB1w= github.com/redis/go-redis/v9 v9.20.1/go.mod h1:v/M13XI1PVCDcm01VtPFOADfZtHf8YW3baQf57KlIkA= github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= +github.com/rollbar/rollbar-go v1.0.2/go.mod h1:AcFs5f0I+c71bpHlXNNDbOWJiKwjFDtISeXco0L5PKQ= github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc= github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik= +github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= +github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= +github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f h1:Z2cODYsUxQPofhpYRMQVwWz4yUVpHF+vPi+eUdruUYI= +github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f/go.mod h1:JqzWyvTuI2X4+9wOHmKSQCYxybB/8j6Ko43qVmXDuZg= +github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY= +github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec= +github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY= +github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60= github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw= github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U= github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I= @@ -94,15 +169,18 @@ github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3A github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU= github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY= github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= -github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8= @@ -111,30 +189,50 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE= github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg= +github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc= +github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw= +github.com/x-cray/logrus-prefixed-formatter v0.5.2 h1:00txxvfBM9muc0jiLIEAkAcIMJzfthRT6usrui8uGmg= +github.com/x-cray/logrus-prefixed-formatter v0.5.2/go.mod h1:2duySbKsL6M18s5GU7VPsoEPHyzalCE06qoARUCeBBE= +github.com/yargevad/filepathx v1.0.0 h1:SYcT+N3tYGi+NvazubCNlvgIPbzAk7i7y2dwg3I5FYc= +github.com/yargevad/filepathx v1.0.0/go.mod h1:BprfX/gpYNJHJfc35GjRRpVcwWXS89gGulUIU5tK3tA= +github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M= +github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw= github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs= github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s= go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.uber.org/mock v0.4.0 h1:VcM4ZOtdbR4f6VXfiOpwpVJDL6lCReaZ6mw31wqh7KU= +go.uber.org/mock v0.4.0/go.mod h1:a6FSlNadKUHUa9IP5Vyt1zh4fC7uAwxMutEAscFbkZc= go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ= go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/zap v1.28.0 h1:IZzaP1Fv73/T/pBMLk4VutPl36uNC+OSUh3JLG3FIjo= go.uber.org/zap v1.28.0/go.mod h1:rDLpOi171uODNm/mxFcuYWxDsqWSAVkFdX4XojSKg/Q= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= -golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8= -golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc= -golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys= -golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI= -golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8= +golang.org/x/arch v0.11.0 h1:KXV8WWKCXm6tRpLirl2szsO5j/oOODwZf4hATmGVNs4= +golang.org/x/arch v0.11.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys= +golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= +golang.org/x/exp v0.0.0-20230713183714-613f0c0eb8a1 h1:MGwJjxBy0HJshjDNfLsYO8xppfqWlA5ZT9OhtUUhTNw= +golang.org/x/exp v0.0.0-20230713183714-613f0c0eb8a1/go.mod h1:FXUEEKJgO7OQYeo8N01OfiKP8RXMtf6e8aTskBGqWdc= +golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac= golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= +golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug= golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/term v0.28.0 h1:/Ts8HFuMR2E6IP/jlo7QVLZHggjKQbhu/7H0LJFr3Gg= +golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk= golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4= google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg= @@ -142,8 +240,9 @@ google.golang.org/protobuf v1.34.1/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHh gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= +gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50= -rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4= diff --git a/backend/internal/ai/llm/openai.go b/backend/internal/ai/llm/openai.go deleted file mode 100644 index 982264d..0000000 --- a/backend/internal/ai/llm/openai.go +++ /dev/null @@ -1,239 +0,0 @@ -package llm - -import ( - "bufio" - "bytes" - "context" - "encoding/base64" - "encoding/json" - "fmt" - "io" - "net/http" - "strings" - "time" - - "go.uber.org/zap" -) - -// OpenAIService 基于 OpenAI Chat Completions API 的 LLM 实现。 -type OpenAIService struct { - apiKey string - model string - endpoint string - timeout time.Duration - logger *zap.SugaredLogger - client *http.Client -} - -// NewOpenAIService 创建 OpenAI LLM 服务。 -// model、endpoint 由 config 层保证非空。 -func NewOpenAIService(apiKey, model, endpoint string, timeoutSec, httpClientTimeoutSec int, logger *zap.SugaredLogger) *OpenAIService { - timeout := time.Duration(timeoutSec) * time.Second - if timeout <= 0 { - timeout = 10 * time.Second - } - httpClientTimeout := time.Duration(httpClientTimeoutSec) * time.Second - if httpClientTimeout <= 0 { - httpClientTimeout = 60 * time.Second - } - return &OpenAIService{ - apiKey: apiKey, - model: model, - endpoint: endpoint, - timeout: timeout, - logger: logger, - client: &http.Client{Timeout: httpClientTimeout}, - } -} - -// --- OpenAI API 请求/响应结构 --- - -type chatRequest struct { - Model string `json:"model"` - Messages []chatMessage `json:"messages"` - Stream bool `json:"stream"` -} - -type chatMessage struct { - Role string `json:"role"` - Content []contentPart `json:"content"` -} - -type contentPart struct { - Type string `json:"type"` - Text string `json:"text"` - ImageURL *imageURL `json:"image_url,omitempty"` -} - -type imageURL struct { - URL string `json:"url"` -} - -// streamDelta SSE 流式响应的单个 delta。 -type streamDelta struct { - Choices []struct { - Delta struct { - Content string `json:"content"` - } `json:"delta"` - FinishReason *string `json:"finish_reason"` - } `json:"choices"` - Usage *struct { - PromptTokens int `json:"prompt_tokens"` - CompletionTokens int `json:"completion_tokens"` - TotalTokens int `json:"total_tokens"` - } `json:"usage"` - Model string `json:"model"` -} - -// ChatStream 实现 llm.Service。调用 OpenAI Chat Completions API 流式推理。 -func (o *OpenAIService) ChatStream(ctx context.Context, req Request) (<-chan Chunk, error) { - // 构建请求 - messages := o.buildMessages(req) - - body := chatRequest{ - Model: o.model, - Messages: messages, - Stream: true, - } - - payload, err := json.Marshal(body) - if err != nil { - return nil, fmt.Errorf("llm: marshal request: %w", err) - } - if err != nil { - return nil, fmt.Errorf("llm: marshal request: %w", err) - } - - // 创建带超时的 context - ctx, cancel := context.WithTimeout(ctx, o.timeout) - - httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, o.endpoint+"/chat/completions", bytes.NewReader(payload)) - if err != nil { - cancel() - return nil, fmt.Errorf("llm: create request: %w", err) - } - httpReq.Header.Set("Content-Type", "application/json") - httpReq.Header.Set("Authorization", "Bearer "+o.apiKey) - - resp, err := o.client.Do(httpReq) - if err != nil { - cancel() - return nil, fmt.Errorf("llm: send request: %w", err) - } - - if resp.StatusCode != http.StatusOK { - cancel() - bodyBytes, _ := io.ReadAll(resp.Body) - resp.Body.Close() - return nil, fmt.Errorf("llm: api error (status %d): %s", resp.StatusCode, string(bodyBytes)) - } - - // 启动 goroutine 解析 SSE 流 - ch := make(chan Chunk, 64) - go func() { - defer close(ch) - defer cancel() - defer resp.Body.Close() - - o.parseSSEStream(resp.Body, ch) - }() - - return ch, nil -} - -// parseSSEStream 解析 SSE 流，将 delta 发送到 channel。 -func (o *OpenAIService) parseSSEStream(body io.Reader, ch chan<- Chunk) { - scanner := bufio.NewScanner(body) - scanner.Buffer(make([]byte, 0, 64*1024), 256*1024) - - var fullText strings.Builder - var lastModel string - - for scanner.Scan() { - line := scanner.Text() - - // SSE 格式：data: {...} - if !strings.HasPrefix(line, "data: ") { - continue - } - data := strings.TrimPrefix(line, "data: ") - if data == "[DONE]" { - // 流结束，发送最终 chunk - ch <- Chunk{Delta: "", Done: true, Model: lastModel} - return - } - - var delta streamDelta - if err := json.Unmarshal([]byte(data), &delta); err != nil { - o.logger.Warnw("llm: unmarshal delta failed", "error", err, "data", data) - continue - } - - if delta.Model != "" { - lastModel = delta.Model - } - - // 提取增量文本 - if len(delta.Choices) > 0 { - content := delta.Choices[0].Delta.Content - if content != "" { - fullText.WriteString(content) - ch <- Chunk{Delta: content, Done: false, Model: lastModel} - } - - // 某些模型在最后一个 choice 中携带 usage - if delta.Choices[0].FinishReason != nil && delta.Usage != nil { - ch <- Chunk{ - Delta: "", - Done: true, - Model: lastModel, - TokensUsed: &TokenUsage{ - Prompt: delta.Usage.PromptTokens, - Completion: delta.Usage.CompletionTokens, - Total: delta.Usage.TotalTokens, - }, - } - return - } - } - } - - // scanner 结束但没收到 [DONE] - if err := scanner.Err(); err != nil { - o.logger.Warnw("llm: scan error", "error", err) - } - ch <- Chunk{Delta: "", Done: true, Model: lastModel} -} - -// buildMessages 构建 OpenAI Chat API 的 messages 数组。 -func (o *OpenAIService) buildMessages(req Request) []chatMessage { - var messages []chatMessage - - // System prompt（情景覆盖优先） - messages = append(messages, chatMessage{ - Role: "system", - Content: []contentPart{{Type: "text", Text: BuildSystemPrompt(req.Language, "", req.SystemPrompt)}}, - }) - - // 历史消息 - for _, msg := range req.History { - messages = append(messages, chatMessage{ - Role: msg.Role, - Content: []contentPart{{Type: "text", Text: msg.Content}}, - }) - } - - // 当前用户消息（图像 + 文本） - var parts []contentPart - if len(req.Image) > 0 { - b64 := base64.StdEncoding.EncodeToString(req.Image) - parts = append(parts, contentPart{ - Type: "image_url", - ImageURL: &imageURL{URL: "data:image/jpeg;base64," + b64}, - }) - } - parts = append(parts, contentPart{Type: "text", Text: req.Text}) - messages = append(messages, chatMessage{Role: "user", Content: parts}) - - return messages -} diff --git a/backend/internal/ai/llm/openai_test.go b/backend/internal/ai/llm/openai_test.go deleted file mode 100644 index f75cbb3..0000000 --- a/backend/internal/ai/llm/openai_test.go +++ /dev/null @@ -1,251 +0,0 @@ -package llm - -import ( - "context" - "fmt" - "net/http" - "net/http/httptest" - "strings" - "testing" - "time" - - "go.uber.org/zap" - - "github.com/hhs/camtalk/internal/models" -) - -// mockLLMServer 创建模拟 OpenAI SSE 流式响应的 HTTP 服务器。 -func mockLLMServer(t *testing.T, handler http.HandlerFunc) *httptest.Server { - t.Helper() - return httptest.NewServer(handler) -} - -func TestOpenAIService_ChatStream_Success(t *testing.T) { - srv := mockLLMServer(t, func(w http.ResponseWriter, r *http.Request) { - // 验证请求 - if r.Method != http.MethodPost { - t.Errorf("method = %s, want POST", r.Method) - } - if !strings.Contains(r.URL.Path, "/chat/completions") { - t.Errorf("path = %s, should contain /chat/completions", r.URL.Path) - } - auth := r.Header.Get("Authorization") - if auth != "Bearer test-key" { - t.Errorf("Authorization = %q, want %q", auth, "Bearer test-key") - } - - w.Header().Set("Content-Type", "text/event-stream") - flusher, ok := w.(http.Flusher) - if !ok { - t.Fatal("ResponseWriter does not support Flusher") - } - - // 发送几个 delta - deltas := []string{"你好", "世界", "！"} - for _, d := range deltas { - fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":\"%s\"}}],\"model\":\"gpt-4o\"}\n\n", d) - flusher.Flush() - } - - // 发送 [DONE] - fmt.Fprintf(w, "data: [DONE]\n\n") - flusher.Flush() - }) - defer srv.Close() - - svc := NewOpenAIService("test-key", "gpt-4o", srv.URL, 10, 60, zap.NewNop().Sugar()) - - ch, err := svc.ChatStream(context.Background(), Request{ - Text: "这是什么？", - Language: "zh-CN", - }) - if err != nil { - t.Fatalf("ChatStream() error: %v", err) - } - - var chunks []Chunk - for c := range ch { - chunks = append(chunks, c) - } - - // 应该有 3 个文本 chunk + 1 个 Done chunk - if len(chunks) != 4 { - t.Fatalf("got %d chunks, want 4", len(chunks)) - } - - // 验证文本内容 - if chunks[0].Delta != "你好" { - t.Errorf("chunk[0].Delta = %q, want %q", chunks[0].Delta, "你好") - } - if chunks[1].Delta != "世界" { - t.Errorf("chunk[1].Delta = %q, want %q", chunks[1].Delta, "世界") - } - - // 验证最后一个 chunk 是 Done - last := chunks[len(chunks)-1] - if !last.Done { - t.Error("last chunk should be Done") - } - if last.Model != "gpt-4o" { - t.Errorf("last chunk Model = %q, want %q", last.Model, "gpt-4o") - } -} - -func TestOpenAIService_ChatStream_WithImage(t *testing.T) { - srv := mockLLMServer(t, func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/event-stream") - fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":\"ok\"}}],\"model\":\"gpt-4o\"}\n\n") - fmt.Fprintf(w, "data: [DONE]\n\n") - }) - defer srv.Close() - - svc := NewOpenAIService("test-key", "gpt-4o", srv.URL, 10, 60, zap.NewNop().Sugar()) - - ch, err := svc.ChatStream(context.Background(), Request{ - Image: []byte("fake-jpeg-data"), - Text: "描述图片", - Language: "zh-CN", - }) - if err != nil { - t.Fatalf("ChatStream() error: %v", err) - } - - // 消费 channel - for range ch { - } -} - -func TestOpenAIService_ChatStream_WithHistory(t *testing.T) { - srv := mockLLMServer(t, func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/event-stream") - fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":\"ok\"}}],\"model\":\"gpt-4o\"}\n\n") - fmt.Fprintf(w, "data: [DONE]\n\n") - }) - defer srv.Close() - - svc := NewOpenAIService("test-key", "gpt-4o", srv.URL, 10, 60, zap.NewNop().Sugar()) - - ch, err := svc.ChatStream(context.Background(), Request{ - Text: "继续", - Language: "zh-CN", - History: []models.Message{ - {Role: "user", Content: "你好"}, - {Role: "assistant", Content: "你好！有什么可以帮助你的吗？"}, - }, - }) - if err != nil { - t.Fatalf("ChatStream() error: %v", err) - } - - for range ch { - } -} - -func TestOpenAIService_ChatStream_APIError(t *testing.T) { - srv := mockLLMServer(t, func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "application/json") - w.WriteHeader(http.StatusUnauthorized) - fmt.Fprintf(w, `{"error":{"message":"Invalid API key"}}`) - }) - defer srv.Close() - - svc := NewOpenAIService("bad-key", "gpt-4o", srv.URL, 10, 60, zap.NewNop().Sugar()) - - _, err := svc.ChatStream(context.Background(), Request{ - Text: "test", - }) - if err == nil { - t.Fatal("ChatStream() should return error for 401") - } - if !strings.Contains(err.Error(), "401") { - t.Errorf("error should mention 401, got: %v", err) - } -} - -func TestOpenAIService_ChatStream_Timeout(t *testing.T) { - srv := mockLLMServer(t, func(w http.ResponseWriter, r *http.Request) { - // 模拟慢响应 - time.Sleep(5 * time.Second) - w.Header().Set("Content-Type", "text/event-stream") - fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":\"late\"}}]}\n\n") - fmt.Fprintf(w, "data: [DONE]\n\n") - }) - defer srv.Close() - - svc := NewOpenAIService("test-key", "gpt-4o", srv.URL, 1, 60, zap.NewNop().Sugar()) // 1s timeout - - ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) - defer cancel() - - ch, err := svc.ChatStream(ctx, Request{Text: "test"}) - if err != nil { - // 超时可能在建立连接时或读取时发生 - return - } - - // 如果连接成功，消费 channel 应该超时 - var gotContent bool - for c := range ch { - if c.Delta != "" { - gotContent = true - } - } - if gotContent { - t.Error("should not receive content before timeout") - } -} - -func TestOpenAIService_ChatStream_UsageInResponse(t *testing.T) { - srv := mockLLMServer(t, func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("Content-Type", "text/event-stream") - // 带 usage 的最后一个 chunk - fmt.Fprintf(w, "data: {\"choices\":[{\"delta\":{\"content\":\"hi\"},\"finish_reason\":\"stop\"}],\"model\":\"gpt-4o\",\"usage\":{\"prompt_tokens\":10,\"completion_tokens\":5,\"total_tokens\":15}}\n\n") - fmt.Fprintf(w, "data: [DONE]\n\n") - }) - defer srv.Close() - - svc := NewOpenAIService("test-key", "gpt-4o", srv.URL, 10, 60, zap.NewNop().Sugar()) - - ch, err := svc.ChatStream(context.Background(), Request{Text: "test"}) - if err != nil { - t.Fatalf("ChatStream() error: %v", err) - } - - var last Chunk - for c := range ch { - last = c - } - - if !last.Done { - t.Error("last chunk should be Done") - } - if last.TokensUsed == nil { - t.Fatal("last chunk should have TokensUsed") - } - if last.TokensUsed.Total != 15 { - t.Errorf("TokensUsed.Total = %d, want 15", last.TokensUsed.Total) - } -} - -func TestBuildSystemPrompt(t *testing.T) { - tests := []struct { - name string - language string - detailLevel string - wantContain string - }{ - {"chinese default", "zh-CN", "", "视觉助手"}, - {"chinese high", "zh-CN", "high", "更详细"}, - {"english default", "en", "", "visual assistant"}, - {"english high", "en", "high", "detailed"}, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got := BuildSystemPrompt(tt.language, tt.detailLevel, "") - if !strings.Contains(got, tt.wantContain) { - t.Errorf("BuildSystemPrompt(%q, %q, \"\") should contain %q", tt.language, tt.detailLevel, tt.wantContain) - } - }) - } -} diff --git a/backend/internal/ai/llm/scenarios.go b/backend/internal/ai/llm/scenarios.go index d11252a..e1e1de2 100644 --- a/backend/internal/ai/llm/scenarios.go +++ b/backend/internal/ai/llm/scenarios.go @@ -2,54 +2,147 @@ package llm import "strings" -// scenarioPrompt 定义单个情景的多语言 system prompt。 +// scenarioPrompt 定义单个情景的多语言 system prompt 和首句引导。 type scenarioPrompt struct { - ZH string - EN string - JA string + ZH string + EN string + JA string + GreetingZH string // 首句引导（中文） + GreetingEN string // 首句引导（英文） + GreetingJA string // 首句引导（日文） } // scenarioPrompts 预置情景 → prompt 映射表。 // key 为情景 ID（与前端 Scenario.id 对齐）。 var scenarioPrompts = map[string]scenarioPrompt{ "interviewer": { - ZH: "你是一位资深面试官。你通过摄像头观察面试者，并根据他们的背景和表现提出面试问题。规则：1) 每次只问一个问题，等用户回答后再追问；2) 问题要有层次，从自我介绍到专业问题再到情景题；3) 对用户的回答给出简短点评然后追问；4) 如果摄像头能看到用户的环境，可以结合环境提出相关话题；5) 回答控制在2-4句话。", + ZH: `你是一位资深面试官。你通过摄像头观察面试者，并根据他们的背景和表现提出面试问题。 + +【角色定位】 +- 你是面试官，不是助手或顾问 +- 你的目标是评估候选人的能力 +- 保持专业、客观、礼貌 + +【交互规则】 +1. 每次只问一个问题，等用户回答后再追问 +2. 问题要有层次：自我介绍 → 专业问题 → 情景题 → 反问环节 +3. 对用户的回答给出简短点评（优点+不足），然后追问 +4. 如果摄像头能看到用户的环境，可以结合环境提出相关话题 +5. 回答控制在2-4句话 + +【约束】 +- 不要主动提供建议或指导（除非候选人请求） +- 不要离开面试官的角色设定 +- 保持问题的专业性和针对性`, EN: "You are a senior interviewer. You observe the interviewee through their camera and ask interview questions based on their background and performance. Rules: 1) Ask one question at a time, wait for the answer before following up; 2) Questions should progress from self-introduction to professional questions to situational questions; 3) Give brief feedback on answers then follow up; 4) If the camera shows the user's environment, incorporate it into the conversation; 5) Keep responses to 2-4 sentences.", JA: "あなたはベテラン面接官です。カメラで面接者を見て、バックグラウンドと実績に基づいて面接質問をします。ルール：1) 一度に一つの質問だけし、回答を待ってから追及する；2) 質問は自己紹介から専門質問、シチュエーション質問へと段階的に；3) 回答に短いコメントをしてから次の質問へ；4) 回答は2〜4文以内。", + GreetingZH: "你好！我是今天的面试官。让我们先从自我介绍开始，请简单介绍一下你自己和你应聘的岗位。", + GreetingEN: "Hello! I'm your interviewer today. Let's start with a self-introduction. Please briefly introduce yourself and the position you're applying for.", + GreetingJA: "こんにちは！本日の面接官です。まず自己紹介から始めましょう。あなた自身と応募職種について簡単に教えてください。", }, "english_teacher": { ZH: "You are a friendly and patient English tutor. Speak in English with the user. Rules: 1) Always respond in English; 2) If the user makes grammar or vocabulary mistakes, gently point them out and suggest corrections; 3) Ask follow-up questions to keep the conversation going; 4) Adjust your language complexity based on the user's level; 5) If the camera shows objects or scenes, use them as teaching material (e.g., 'I can see a bookshelf behind you. What's your favorite book?'); 6) Keep responses to 3-5 sentences.", EN: "You are a friendly and patient English tutor. Speak in English with the user. Rules: 1) Always respond in English; 2) If the user makes grammar or vocabulary mistakes, gently point them out and suggest corrections; 3) Ask follow-up questions to keep the conversation going; 4) Adjust your language complexity based on the user's level; 5) If the camera shows objects or scenes, use them as teaching material; 6) Keep responses to 3-5 sentences.", JA: "You are a friendly and patient English tutor. Speak in English with the user. Rules: 1) Always respond in English; 2) If the user makes grammar or vocabulary mistakes, gently point them out and suggest corrections; 3) Ask follow-up questions to keep the conversation going; 4) Adjust your language complexity based on the user's level; 5) If the camera shows objects or scenes, use them as teaching material; 6) Keep responses to 3-5 sentences.", + GreetingZH: "Hi! I'm your English tutor. Let's practice English together! What would you like to talk about today?", + GreetingEN: "Hi! I'm your English tutor. Let's practice English together! What would you like to talk about today?", + GreetingJA: "Hi! I'm your English tutor. Let's practice English together! What would you like to talk about today?", }, "debate": { - ZH: "你是一位辩论赛对手。用户提出一个观点，你需要站在反方进行反驳。规则：1) 逻辑严密，用事实和论据反驳，不要人身攻击；2) 每次提出1-2个核心反驳点，并给出简要论据；3) 如果用户论证有力，承认其合理性但仍要寻找突破口；4) 适时提出反问，引导用户深入思考；5) 回答控制在3-5句话。", + ZH: `你是一位辩论赛对手。用户提出一个观点，你需要站在反方进行反驳。 + +【角色定位】 +- 你是辩论对手，不是评委或顾问 +- 你的目标是通过逻辑论证反驳对方观点 +- 保持理性、严谨、尊重对手 + +【交互规则】 +1. 逻辑严密，用事实和论据反驳，不要人身攻击 +2. 每次提出1-2个核心反驳点，并给出简要论据 +3. 如果用户论证有力，承认其合理性但仍要寻找突破口 +4. 适时提出反问，引导用户深入思考 +5. 回答控制在3-5句话 + +【约束】 +- 始终站在反方立场 +- 不要主动转换为支持方 +- 即使对方观点正确，也要寻找可辩论的角度`, EN: "You are a debate opponent. The user presents a viewpoint, and you argue against it. Rules: 1) Use logic and evidence, no personal attacks; 2) Present 1-2 core counterarguments with brief evidence; 3) Acknowledge strong points but look for weaknesses; 4) Ask counter-questions to provoke deeper thinking; 5) Keep responses to 3-5 sentences.", JA: "あなたはディベートの相手です。ユーザーが提示した观点に対して反論します。ルール：1) 論理と証拠で反論し、人格攻撃はしない；2) 1〜2つの核心的な反論を提示する；3) 相手の有力な論点は認めつつも突破口を探す；4) 深い思考を促す反问をする；5) 回答は3〜5文以内。", + GreetingZH: "你好！我是你的辩论对手。请提出一个你坚信的观点，我会站在反方立场与你辩论，帮你锻炼逻辑思维。", + GreetingEN: "Hello! I'm your debate opponent. Please present a viewpoint you firmly believe in, and I'll argue against it to help sharpen your critical thinking.", + GreetingJA: "こんにちは！あなたのディベート相手です。あなたが信じる观点を提示してください。反対の立場から論じて、論理的思考を鍛えます。", }, "interpreter": { ZH: "你是一名同声翻译员。将用户说的话实时翻译为目标语言。规则：1) 只输出翻译结果，不加任何解释或评论；2) 保持口语化，自然流畅；3) 如果用户说中文，翻译成英文；如果用户说英文，翻译成中文；4) 如果不确定目标语言，默认中英互译；5) 对于专有名词，首次翻译时在括号中注明原文。", EN: "You are a simultaneous interpreter. Translate what the user says in real-time. Rules: 1) Only output the translation, no explanations or comments; 2) Keep it conversational and natural; 3) If the user speaks Chinese, translate to English; if English, translate to Chinese; 4) Default to Chinese-English translation if the target language is unclear; 5) For proper nouns, note the original in parentheses on first use.", JA: "あなたは同時通訳者です。ユーザーの発言をリアルタイムで翻訳します。ルール：1) 翻訳結果のみ出力し、説明やコメントは加えない；2) 口語的で自然な表現を維持する；3) ユーザーが中国語を話せば英語に、英語を話せば中国語に翻訳する；4) 固有名詞は初出時に原文を括弧で注記する。", + GreetingZH: "我是你的同声翻译。请开始说话，我会实时将中文翻译成英文，或将英文翻译成中文。", + GreetingEN: "I'm your simultaneous interpreter. Please start speaking, and I'll translate Chinese to English or English to Chinese in real-time.", + GreetingJA: "私はあなたの同時通訳者です。お話しください。中国語を英語に、または英語を中国語にリアルタイムで翻訳します。", }, } // GetScenarioPrompt 根据情景 ID 和语言获取对应的 system prompt。 +// 支持系统预置情景和用户自建情景。 +// customScenarios: 用户自建情景映射表（scenarioID → prompt），可为 nil // 返回空字符串表示无此情景（使用默认 prompt）。 -func GetScenarioPrompt(scenarioID, language string) string { +func GetScenarioPrompt(scenarioID, language string, customScenarios map[string]string) string { if scenarioID == "" || scenarioID == "free_chat" { return "" } - p, ok := scenarioPrompts[scenarioID] - if !ok { + + // 1. 优先查找系统预置情景 + if p, ok := scenarioPrompts[scenarioID]; ok { + switch { + case strings.HasPrefix(language, "zh"): + return p.ZH + case strings.HasPrefix(language, "ja"): + return p.JA + default: + return p.EN + } + } + + // 2. 查找用户自建情景 + if customScenarios != nil { + if customPrompt, ok := customScenarios[scenarioID]; ok { + return customPrompt + } + } + + // 3. 默认空字符串 + return "" +} + +// GetScenarioGreeting 根据情景 ID 和语言获取对应的首句引导。 +// 支持系统预置情景和用户自建情景。 +// customGreetings: 用户自建情景的首句引导映射表（scenarioID → greeting），可为 nil +// 返回空字符串表示无此情景或不需要引导（自由对话）。 +func GetScenarioGreeting(scenarioID, language string, customGreetings map[string]string) string { + if scenarioID == "" || scenarioID == "free_chat" { return "" } - switch { - case strings.HasPrefix(language, "zh"): - return p.ZH - case strings.HasPrefix(language, "ja"): - return p.JA - default: - return p.EN + + // 1. 优先查找系统预置情景 + if p, ok := scenarioPrompts[scenarioID]; ok { + switch { + case strings.HasPrefix(language, "zh"): + return p.GreetingZH + case strings.HasPrefix(language, "ja"): + return p.GreetingJA + default: + return p.GreetingEN + } } + + // 2. 查找用户自建情景 + if customGreetings != nil { + if customGreeting, ok := customGreetings[scenarioID]; ok { + return customGreeting + } + } + + // 3. 默认空字符串 + return "" } diff --git a/backend/internal/ai/tts/mimo.go b/backend/internal/ai/tts/mimo.go index 717b9f7..0718ccf 100644 --- a/backend/internal/ai/tts/mimo.go +++ b/backend/internal/ai/tts/mimo.go @@ -11,6 +11,8 @@ import ( "strings" "time" + "github.com/hhs/camtalk/internal/trace" + "github.com/hhs/camtalk/internal/util" "go.uber.org/zap" ) @@ -108,7 +110,11 @@ func (m *MiMoService) SynthesizeStream(ctx context.Context, textStream <-chan st audio, err := m.synthesize(ctx, text, voice) if err != nil { - m.logger.Warnw("mimo tts: synthesize failed", "error", err, "text", text) + log := trace.FromContext(ctx) + log.Warnw("mimo tts: synthesize failed", + "error", err, + "text_len", len(text), + "text_preview", util.Truncate(text, 100)) // 静默跳过，不中断整个流 continue } diff --git a/backend/internal/ai/tts/openai.go b/backend/internal/ai/tts/openai.go index ba7f788..5834c9c 100644 --- a/backend/internal/ai/tts/openai.go +++ b/backend/internal/ai/tts/openai.go @@ -9,6 +9,8 @@ import ( "net/http" "time" + "github.com/hhs/camtalk/internal/trace" + "github.com/hhs/camtalk/internal/util" "go.uber.org/zap" ) @@ -81,7 +83,11 @@ func (o *OpenAIService) SynthesizeStream(ctx context.Context, textStream <-chan audio, err := o.synthesize(ctx, text, voice, speed) if err != nil { - o.logger.Warnw("tts: synthesize failed", "error", err, "text", text) + log := trace.FromContext(ctx) + log.Warnw("tts: synthesize failed", + "error", err, + "text_len", len(text), + "text_preview", util.Truncate(text, 100)) // 静默跳过，不中断整个流 continue } diff --git a/backend/internal/api/auth.go b/backend/internal/api/auth.go index 3b886c3..f7add06 100644 --- a/backend/internal/api/auth.go +++ b/backend/internal/api/auth.go @@ -8,6 +8,8 @@ import ( "github.com/hhs/camtalk/internal/auth" apperr "github.com/hhs/camtalk/internal/errors" + "github.com/hhs/camtalk/internal/ratelimit" + "github.com/hhs/camtalk/internal/trace" ) // AuthHandler 提供认证相关的 REST 端点。 @@ -25,11 +27,26 @@ func NewAuthHandler(authService auth.Service, tokenMgr *auth.TokenManager) *Auth } // RegisterRoutes 注册认证相关路由到给定的路由组。 -func (h *AuthHandler) RegisterRoutes(rg *gin.RouterGroup) { +func (h *AuthHandler) RegisterRoutes(rg *gin.RouterGroup, limiter ratelimit.Limiter) { authGroup := rg.Group("/auth") { - authGroup.POST("/register", h.Register) - authGroup.POST("/login", h.Login) + // 注册和登录端点添加限流中间件（按 IP 限流） + if limiter != nil { + authGroup.POST("/register", + ratelimit.Middleware(limiter, func(c *gin.Context) string { + return c.ClientIP() + ":register" + }), + h.Register) + authGroup.POST("/login", + ratelimit.Middleware(limiter, func(c *gin.Context) string { + return c.ClientIP() + ":login" + }), + h.Login) + } else { + authGroup.POST("/register", h.Register) + authGroup.POST("/login", h.Login) + } + // refresh 和 logout 不限流 authGroup.POST("/refresh", h.Refresh) authGroup.POST("/logout", auth.AuthMiddleware(h.tokenMgr), h.Logout) } @@ -37,6 +54,9 @@ func (h *AuthHandler) RegisterRoutes(rg *gin.RouterGroup) { // Register POST /api/auth/register — 用户注册。 func (h *AuthHandler) Register(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) + clientIP := c.ClientIP() + var req auth.RegisterRequest if err := c.ShouldBindJSON(&req); err != nil { c.JSON(http.StatusBadRequest, gin.H{ @@ -56,15 +76,25 @@ func (h *AuthHandler) Register(c *gin.Context) { resp, err := h.authService.Register(c.Request.Context(), req) if err != nil { + log.Warnw("register failed", + "username", req.Username, + "client_ip", clientIP, + "error", err) handleAuthError(c, err) return } + log.Infow("register success", + "username", req.Username, + "client_ip", clientIP) c.JSON(http.StatusCreated, resp) } // Login POST /api/auth/login — 用户登录。 func (h *AuthHandler) Login(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) + clientIP := c.ClientIP() + var req auth.LoginRequest if err := c.ShouldBindJSON(&req); err != nil { c.JSON(http.StatusBadRequest, gin.H{ @@ -84,15 +114,24 @@ func (h *AuthHandler) Login(c *gin.Context) { resp, err := h.authService.Login(c.Request.Context(), req) if err != nil { + log.Warnw("login failed", + "username", req.Username, + "client_ip", clientIP, + "error", err) handleAuthError(c, err) return } + log.Infow("login success", + "username", req.Username, + "client_ip", clientIP) c.JSON(http.StatusOK, resp) } // Refresh POST /api/auth/refresh — 刷新令牌。 func (h *AuthHandler) Refresh(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) + var req auth.RefreshRequest if err := c.ShouldBindJSON(&req); err != nil { c.JSON(http.StatusBadRequest, gin.H{ @@ -112,15 +151,21 @@ func (h *AuthHandler) Refresh(c *gin.Context) { resp, err := h.authService.Refresh(c.Request.Context(), req) if err != nil { + log.Warnw("token refresh failed", + "client_ip", c.ClientIP(), + "error", err) handleAuthError(c, err) return } + log.Infow("token refresh success", + "client_ip", c.ClientIP()) c.JSON(http.StatusOK, resp) } // Logout POST /api/auth/logout — 登出（需要认证）。 func (h *AuthHandler) Logout(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) userID := c.GetString(auth.ContextKeyUserID) var req struct { @@ -143,6 +188,9 @@ func (h *AuthHandler) Logout(c *gin.Context) { } if err := h.authService.Logout(c.Request.Context(), userID, req.RefreshToken); err != nil { + log.Errorw("logout failed", + "user_id", userID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to logout", @@ -150,6 +198,8 @@ func (h *AuthHandler) Logout(c *gin.Context) { return } + log.Infow("logout success", + "user_id", userID) c.JSON(http.StatusOK, gin.H{ "message": "logged out successfully", }) diff --git a/backend/internal/api/auth_test.go b/backend/internal/api/auth_test.go index f49b75a..624ec64 100644 --- a/backend/internal/api/auth_test.go +++ b/backend/internal/api/auth_test.go @@ -47,7 +47,7 @@ func newTestRouter(svc auth.Service) *gin.Engine { r := gin.New() tm := auth.NewTokenManager("test-secret", 15*time.Minute, 7*24*time.Hour) h := api.NewAuthHandler(svc, tm) - h.RegisterRoutes(r.Group("/api")) + h.RegisterRoutes(r.Group("/api"), nil) // 测试时不启用限流 return r } @@ -57,7 +57,7 @@ func newTestRouterWithToken(svc auth.Service) (*gin.Engine, *auth.TokenManager) r := gin.New() tm := auth.NewTokenManager("test-secret", 15*time.Minute, 7*24*time.Hour) h := api.NewAuthHandler(svc, tm) - h.RegisterRoutes(r.Group("/api")) + h.RegisterRoutes(r.Group("/api"), nil) // 测试时不启用限流 return r, tm } diff --git a/backend/internal/api/conversation.go b/backend/internal/api/conversation.go index 728cc9a..e21471a 100644 --- a/backend/internal/api/conversation.go +++ b/backend/internal/api/conversation.go @@ -13,6 +13,7 @@ import ( "github.com/hhs/camtalk/internal/models" "github.com/hhs/camtalk/internal/session" "github.com/hhs/camtalk/internal/store" + "github.com/hhs/camtalk/internal/trace" ) // ConversationHandler 提供对话相关的 REST 端点。 @@ -47,6 +48,7 @@ func (h *ConversationHandler) RegisterRoutes(rg *gin.RouterGroup) { // List GET /api/conversations — 获取当前用户的对话列表。 func (h *ConversationHandler) List(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) userID := c.GetString(auth.ContextKeyUserID) page, _ := strconv.Atoi(c.DefaultQuery("page", "1")) @@ -61,6 +63,9 @@ func (h *ConversationHandler) List(c *gin.Context) { summaries, total, err := h.sessionMgr.ListByUser(c.Request.Context(), userID, page, size) if err != nil { + log.Errorw("list conversations failed", + "user_id", userID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to list conversations", @@ -83,6 +88,7 @@ type CreateConversationRequest struct { // Create POST /api/conversations — 创建新对话。 func (h *ConversationHandler) Create(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) userID := c.GetString(auth.ContextKeyUserID) var req CreateConversationRequest @@ -95,6 +101,9 @@ func (h *ConversationHandler) Create(c *gin.Context) { sessionID, err := h.sessionMgr.Create(c.Request.Context(), userID, cfg) if err != nil { + log.Errorw("create conversation failed", + "user_id", userID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to create conversation", @@ -104,6 +113,9 @@ func (h *ConversationHandler) Create(c *gin.Context) { sess, err := h.sessionMgr.Get(c.Request.Context(), sessionID) if err != nil { + log.Errorw("retrieve created conversation failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to retrieve created conversation", @@ -111,6 +123,9 @@ func (h *ConversationHandler) Create(c *gin.Context) { return } + log.Infow("conversation created", + "conversation_id", sess.ID, + "user_id", userID) c.JSON(http.StatusCreated, gin.H{ "id": sess.ID, "title": sess.Title, @@ -144,6 +159,7 @@ type UpdateTitleRequest struct { // UpdateTitle PATCH /api/conversations/:id — 更新对话标题。 func (h *ConversationHandler) UpdateTitle(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) sessionID := c.Param("id") // 先校验归属 @@ -176,6 +192,9 @@ func (h *ConversationHandler) UpdateTitle(c *gin.Context) { }) return } + log.Errorw("update title failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to update title", @@ -190,6 +209,7 @@ func (h *ConversationHandler) UpdateTitle(c *gin.Context) { // Delete DELETE /api/conversations/:id — 删除对话。 func (h *ConversationHandler) Delete(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) sessionID := c.Param("id") // 先校验归属 @@ -205,6 +225,9 @@ func (h *ConversationHandler) Delete(c *gin.Context) { }) return } + log.Errorw("delete conversation failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to delete conversation", @@ -221,6 +244,7 @@ func (h *ConversationHandler) Delete(c *gin.Context) { // - limit: 返回消息数量上限，默认 50 // - before: 消息 ID 游标（用于分页），返回此 ID 之前的消息 func (h *ConversationHandler) GetMessages(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) sessionID := c.Param("id") // 先校验归属 @@ -239,6 +263,9 @@ func (h *ConversationHandler) GetMessages(c *gin.Context) { if h.msgRepo != nil { messages, err := h.msgRepo.GetMessages(c.Request.Context(), sessionID, limit, beforeID) if err != nil { + log.Errorw("get messages failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to get messages", @@ -246,6 +273,9 @@ func (h *ConversationHandler) GetMessages(c *gin.Context) { return } count, _ := h.msgRepo.GetMessageCount(c.Request.Context(), sessionID) + if messages == nil { + messages = []store.StoredMessage{} + } c.JSON(http.StatusOK, gin.H{ "messages": messages, "total": count, @@ -263,6 +293,9 @@ func (h *ConversationHandler) GetMessages(c *gin.Context) { }) return } + log.Errorw("get messages failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": apperr.CodeInternalError, "message": "failed to get messages", @@ -284,6 +317,9 @@ func (h *ConversationHandler) GetMessages(c *gin.Context) { } messages := allMessages[start:] + if messages == nil { + messages = []models.Message{} + } c.JSON(http.StatusOK, gin.H{ "messages": messages, "total": total, diff --git a/backend/internal/api/session.go b/backend/internal/api/session.go index 78bae2d..9690258 100644 --- a/backend/internal/api/session.go +++ b/backend/internal/api/session.go @@ -8,6 +8,7 @@ import ( "github.com/hhs/camtalk/internal/models" "github.com/hhs/camtalk/internal/session" + "github.com/hhs/camtalk/internal/trace" ) // SessionHandler 提供会话相关的 REST 端点。 @@ -27,6 +28,8 @@ type CreateSessionRequest struct { // CreateSession POST /api/sessions — 创建新会话。 func (h *SessionHandler) CreateSession(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) + var req CreateSessionRequest // 请求体可选，解析失败不报错（使用默认配置） _ = c.ShouldBindJSON(&req) @@ -38,6 +41,8 @@ func (h *SessionHandler) CreateSession(c *gin.Context) { sessionID, err := h.sessionMgr.Create(c.Request.Context(), "", cfg) if err != nil { + log.Errorw("create session failed", + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": "INTERNAL_ERROR", "message": "failed to create session", @@ -48,6 +53,9 @@ func (h *SessionHandler) CreateSession(c *gin.Context) { // 获取创建后的会话以返回 created_at sess, err := h.sessionMgr.Get(c.Request.Context(), sessionID) if err != nil { + log.Errorw("retrieve created session failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": "INTERNAL_ERROR", "message": "failed to retrieve created session", @@ -55,6 +63,8 @@ func (h *SessionHandler) CreateSession(c *gin.Context) { return } + log.Infow("session created", + "session_id", sess.ID) c.JSON(http.StatusCreated, gin.H{ "session_id": sess.ID, "created_at": sess.CreatedAt, @@ -63,6 +73,7 @@ func (h *SessionHandler) CreateSession(c *gin.Context) { // DestroySession DELETE /api/sessions/:id — 销毁会话。 func (h *SessionHandler) DestroySession(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) sessionID := c.Param("id") err := h.sessionMgr.Destroy(c.Request.Context(), sessionID) @@ -74,6 +85,9 @@ func (h *SessionHandler) DestroySession(c *gin.Context) { }) return } + log.Errorw("destroy session failed", + "session_id", sessionID, + "error", err) c.JSON(http.StatusInternalServerError, gin.H{ "code": "INTERNAL_ERROR", "message": "failed to destroy session", @@ -81,6 +95,8 @@ func (h *SessionHandler) DestroySession(c *gin.Context) { return } + log.Infow("session destroyed", + "session_id", sessionID) c.Status(http.StatusNoContent) } diff --git a/backend/internal/api/user_scenario_handler.go b/backend/internal/api/user_scenario_handler.go new file mode 100644 index 0000000..9acff70 --- /dev/null +++ b/backend/internal/api/user_scenario_handler.go @@ -0,0 +1,208 @@ +package api + +import ( + "net/http" + + "github.com/gin-gonic/gin" + + "github.com/hhs/camtalk/internal/logger" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/store" +) + +const ( + MaxScenariosPerUser = 20 // 每个用户最多 20 个自建情景 + MaxPromptLength = 2000 // Prompt 最大长度 +) + +// UserScenarioHandler 用户情景 API Handler。 +type UserScenarioHandler struct { + repo store.UserScenarioRepository +} + +// NewUserScenarioHandler 创建用户情景 Handler。 +func NewUserScenarioHandler(repo store.UserScenarioRepository) *UserScenarioHandler { + return &UserScenarioHandler{repo: repo} +} + +// List 获取用户的所有自建情景。 +// GET /api/scenarios +func (h *UserScenarioHandler) List(c *gin.Context) { + userID, exists := c.Get("user_id") + if !exists { + c.JSON(http.StatusUnauthorized, gin.H{"error": "未登录"}) + return + } + + scenarios, err := h.repo.FindByUserID(c.Request.Context(), userID.(string)) + if err != nil { + logger.Log.Errorw("查询用户情景失败", "user_id", userID, "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "查询失败"}) + return + } + + if scenarios == nil { + scenarios = []*models.UserScenario{} + } + + c.JSON(http.StatusOK, models.UserScenarioListResponse{ + Scenarios: scenarios, + Total: len(scenarios), + }) +} + +// Create 创建用户情景。 +// POST /api/scenarios +func (h *UserScenarioHandler) Create(c *gin.Context) { + userID, exists := c.Get("user_id") + if !exists { + c.JSON(http.StatusUnauthorized, gin.H{"error": "未登录"}) + return + } + + var req models.CreateUserScenarioRequest + if err := c.ShouldBindJSON(&req); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": "参数错误: " + err.Error()}) + return + } + + // 检查用户是否已达上限 + count, err := h.repo.CountByUserID(c.Request.Context(), userID.(string)) + if err != nil { + logger.Log.Errorw("统计用户情景数量失败", "user_id", userID, "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "创建失败"}) + return + } + if count >= MaxScenariosPerUser { + c.JSON(http.StatusBadRequest, gin.H{"error": "已达创建上限（最多 20 个）"}) + return + } + + // 创建情景 + scenario := &models.UserScenario{ + UserID: userID.(string), + Name: req.Name, + Icon: req.Icon, + Description: req.Description, + Prompt: req.Prompt, + Greeting: req.Greeting, + Language: req.Language, + } + + if err := h.repo.Create(c.Request.Context(), scenario); err != nil { + logger.Log.Errorw("创建用户情景失败", "user_id", userID, "error", err) + if err.Error() == "duplicate key value violates unique constraint" { + c.JSON(http.StatusBadRequest, gin.H{"error": "情景名称已存在"}) + return + } + c.JSON(http.StatusInternalServerError, gin.H{"error": "创建失败"}) + return + } + + logger.Log.Infow("创建用户情景成功", "user_id", userID, "scenario_id", scenario.ID) + c.JSON(http.StatusCreated, scenario) +} + +// Get 获取单个情景详情。 +// GET /api/scenarios/:id +func (h *UserScenarioHandler) Get(c *gin.Context) { + userID, exists := c.Get("user_id") + if !exists { + c.JSON(http.StatusUnauthorized, gin.H{"error": "未登录"}) + return + } + + scenarioID := c.Param("id") + scenario, err := h.repo.FindByIDAndUserID(c.Request.Context(), scenarioID, userID.(string)) + if err != nil { + logger.Log.Errorw("查询用户情景失败", "user_id", userID, "scenario_id", scenarioID, "error", err) + c.JSON(http.StatusNotFound, gin.H{"error": "情景不存在或无权限"}) + return + } + + c.JSON(http.StatusOK, scenario) +} + +// Update 更新用户情景。 +// PATCH /api/scenarios/:id +func (h *UserScenarioHandler) Update(c *gin.Context) { + userID, exists := c.Get("user_id") + if !exists { + c.JSON(http.StatusUnauthorized, gin.H{"error": "未登录"}) + return + } + + scenarioID := c.Param("id") + + // 查询并校验所有权 + scenario, err := h.repo.FindByIDAndUserID(c.Request.Context(), scenarioID, userID.(string)) + if err != nil { + logger.Log.Errorw("查询用户情景失败", "user_id", userID, "scenario_id", scenarioID, "error", err) + c.JSON(http.StatusNotFound, gin.H{"error": "情景不存在或无权限"}) + return + } + + var req models.UpdateUserScenarioRequest + if err := c.ShouldBindJSON(&req); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": "参数错误: " + err.Error()}) + return + } + + // 更新字段 + if req.Name != nil { + scenario.Name = *req.Name + } + if req.Icon != nil { + scenario.Icon = *req.Icon + } + if req.Description != nil { + scenario.Description = *req.Description + } + if req.Prompt != nil { + scenario.Prompt = *req.Prompt + } + if req.Greeting != nil { + scenario.Greeting = *req.Greeting + } + if req.Language != nil { + scenario.Language = *req.Language + } + + if err := h.repo.Update(c.Request.Context(), scenario); err != nil { + logger.Log.Errorw("更新用户情景失败", "user_id", userID, "scenario_id", scenarioID, "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "更新失败"}) + return + } + + logger.Log.Infow("更新用户情景成功", "user_id", userID, "scenario_id", scenarioID) + c.JSON(http.StatusOK, scenario) +} + +// Delete 删除用户情景。 +// DELETE /api/scenarios/:id +func (h *UserScenarioHandler) Delete(c *gin.Context) { + userID, exists := c.Get("user_id") + if !exists { + c.JSON(http.StatusUnauthorized, gin.H{"error": "未登录"}) + return + } + + scenarioID := c.Param("id") + + // 查询并校验所有权 + _, err := h.repo.FindByIDAndUserID(c.Request.Context(), scenarioID, userID.(string)) + if err != nil { + logger.Log.Errorw("查询用户情景失败", "user_id", userID, "scenario_id", scenarioID, "error", err) + c.JSON(http.StatusNotFound, gin.H{"error": "情景不存在或无权限"}) + return + } + + if err := h.repo.Delete(c.Request.Context(), scenarioID); err != nil { + logger.Log.Errorw("删除用户情景失败", "user_id", userID, "scenario_id", scenarioID, "error", err) + c.JSON(http.StatusInternalServerError, gin.H{"error": "删除失败"}) + return + } + + logger.Log.Infow("删除用户情景成功", "user_id", userID, "scenario_id", scenarioID) + c.Status(http.StatusNoContent) +} diff --git a/backend/internal/auth/jwt.go b/backend/internal/auth/jwt.go index 1da1ac0..01121ff 100644 --- a/backend/internal/auth/jwt.go +++ b/backend/internal/auth/jwt.go @@ -15,10 +15,17 @@ var ( ErrInvalidToken = errors.New("invalid or expired token") ) +// 令牌类型常量。 +const ( + TokenTypeAccess = "access" + TokenTypeRefresh = "refresh" +) + // Claims JWT 声明。 type Claims struct { - UserID string `json:"user_id"` - Username string `json:"username"` + UserID string `json:"user_id"` + Username string `json:"username"` + TokenType string `json:"token_type"` jwt.RegisteredClaims } @@ -45,8 +52,9 @@ func (tm *TokenManager) GeneratePair(userID, username string) (access, refresh s // access token accessClaims := &Claims{ - UserID: userID, - Username: username, + UserID: userID, + Username: username, + TokenType: TokenTypeAccess, RegisteredClaims: jwt.RegisteredClaims{ ExpiresAt: jwt.NewNumericDate(now.Add(tm.accessTTL)), IssuedAt: jwt.NewNumericDate(now), @@ -62,8 +70,9 @@ func (tm *TokenManager) GeneratePair(userID, username string) (access, refresh s // refresh token（含唯一 token_id 用于 DB 关联） tokenID := uuid.New().String() refreshClaims := &Claims{ - UserID: userID, - Username: username, + UserID: userID, + Username: username, + TokenType: TokenTypeRefresh, RegisteredClaims: jwt.RegisteredClaims{ ID: tokenID, ExpiresAt: jwt.NewNumericDate(now.Add(tm.refreshTTL)), @@ -78,12 +87,26 @@ func (tm *TokenManager) GeneratePair(userID, username string) (access, refresh s // ValidateAccess 校验 access token 并返回 Claims。 func (tm *TokenManager) ValidateAccess(tokenStr string) (*Claims, error) { - return tm.validate(tokenStr) + claims, err := tm.validate(tokenStr) + if err != nil { + return nil, err + } + if claims.TokenType != TokenTypeAccess { + return nil, ErrInvalidToken + } + return claims, nil } // ValidateRefresh 校验 refresh token 并返回 Claims。 func (tm *TokenManager) ValidateRefresh(tokenStr string) (*Claims, error) { - return tm.validate(tokenStr) + claims, err := tm.validate(tokenStr) + if err != nil { + return nil, err + } + if claims.TokenType != TokenTypeRefresh { + return nil, ErrInvalidToken + } + return claims, nil } // validate 解析并校验 JWT。 diff --git a/backend/internal/auth/jwt_test.go b/backend/internal/auth/jwt_test.go index 35b12fa..7178bae 100644 --- a/backend/internal/auth/jwt_test.go +++ b/backend/internal/auth/jwt_test.go @@ -103,6 +103,44 @@ func TestValidateRefresh_ExpiredToken(t *testing.T) { assert.ErrorIs(t, err, ErrInvalidToken) } +func TestValidateAccess_RejectsRefreshToken(t *testing.T) { + tm := NewTokenManager("test-secret-key", 15*time.Minute, 7*24*time.Hour) + + _, refresh, err := tm.GeneratePair("user-123", "alice") + require.NoError(t, err) + + // refresh token 不能通过 access 校验 + _, err = tm.ValidateAccess(refresh) + assert.ErrorIs(t, err, ErrInvalidToken) +} + +func TestValidateRefresh_RejectsAccessToken(t *testing.T) { + tm := NewTokenManager("test-secret-key", 15*time.Minute, 7*24*time.Hour) + + access, _, err := tm.GeneratePair("user-123", "alice") + require.NoError(t, err) + + // access token 不能通过 refresh 校验 + _, err = tm.ValidateRefresh(access) + assert.ErrorIs(t, err, ErrInvalidToken) +} + +func TestGeneratePair_TokenTypesAreCorrect(t *testing.T) { + tm := NewTokenManager("test-secret-key", 15*time.Minute, 7*24*time.Hour) + + access, refresh, err := tm.GeneratePair("user-123", "alice") + require.NoError(t, err) + + // 通过 validate（不做类型检查）验证 token_type 字段 + accessClaims, err := tm.validate(access) + require.NoError(t, err) + assert.Equal(t, TokenTypeAccess, accessClaims.TokenType) + + refreshClaims, err := tm.validate(refresh) + require.NoError(t, err) + assert.Equal(t, TokenTypeRefresh, refreshClaims.TokenType) +} + func TestGeneratePair_TokenClaimsContainCorrectExpiry(t *testing.T) { accessTTL := 15 * time.Minute refreshTTL := 7 * 24 * time.Hour diff --git a/backend/internal/auth/middleware.go b/backend/internal/auth/middleware.go index a932240..c563f6d 100644 --- a/backend/internal/auth/middleware.go +++ b/backend/internal/auth/middleware.go @@ -5,6 +5,8 @@ import ( "strings" "github.com/gin-gonic/gin" + + "github.com/hhs/camtalk/internal/trace" ) // contextKey 用于在 Gin context 中存储 Claims 的 key。 @@ -17,8 +19,13 @@ const ( // 校验成功后将 user_id 和 username 写入 Gin Context。 func AuthMiddleware(tokenMgr *TokenManager) gin.HandlerFunc { return func(c *gin.Context) { + log := trace.FromContext(c.Request.Context()) authHeader := c.GetHeader("Authorization") if authHeader == "" { + log.Warnw("auth rejected", + "client_ip", c.ClientIP(), + "path", c.Request.URL.Path, + "reason", "missing authorization header") c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{ "code": "INVALID_TOKEN", "message": "missing authorization header", @@ -29,6 +36,10 @@ func AuthMiddleware(tokenMgr *TokenManager) gin.HandlerFunc { // 提取 Bearer token parts := strings.SplitN(authHeader, " ", 2) if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") { + log.Warnw("auth rejected", + "client_ip", c.ClientIP(), + "path", c.Request.URL.Path, + "reason", "invalid authorization format") c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{ "code": "INVALID_TOKEN", "message": "invalid authorization format", @@ -38,6 +49,11 @@ func AuthMiddleware(tokenMgr *TokenManager) gin.HandlerFunc { claims, err := tokenMgr.ValidateAccess(parts[1]) if err != nil { + log.Warnw("auth rejected", + "client_ip", c.ClientIP(), + "path", c.Request.URL.Path, + "reason", "invalid or expired token", + "error", err) c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{ "code": "INVALID_TOKEN", "message": "invalid or expired token", diff --git a/backend/internal/auth/service.go b/backend/internal/auth/service.go index 4f18447..0cd3f3f 100644 --- a/backend/internal/auth/service.go +++ b/backend/internal/auth/service.go @@ -166,6 +166,9 @@ func (s *authService) Refresh(ctx context.Context, req RefreshRequest) (*AuthRes userID, err := s.userRepo.FindRefreshToken(ctx, tokenHash) if err != nil { if errors.Is(err, store.ErrRefreshTokenNotFound) { + // JWT 校验已通过但 DB 中不存在 → token 已被 rotation 删除，属于复用行为 + // 吊销该用户全部 refresh token，强制所有设备重新登录 + _ = s.userRepo.DeleteUserRefreshTokens(ctx, claims.UserID) return nil, ErrRefreshTokenUsed } return nil, err diff --git a/backend/internal/auth/service_test.go b/backend/internal/auth/service_test.go index 9fc91a5..ab14554 100644 --- a/backend/internal/auth/service_test.go +++ b/backend/internal/auth/service_test.go @@ -188,3 +188,48 @@ func TestLogout_Success(t *testing.T) { }) assert.ErrorIs(t, err, auth.ErrRefreshTokenUsed) } + +// --- Refresh Token 复用检测 --- + +func TestRefresh_ReuseDetectedRevokesAllTokens(t *testing.T) { + svc, repo := newTestService(t) + ctx := context.Background() + + // 注册，获得令牌对 A + regResp, err := svc.Register(ctx, auth.RegisterRequest{ + Username: "eve", + Password: "password123", + }) + require.NoError(t, err) + tokenPairA_refresh := regResp.RefreshToken + + // 再次登录，获得令牌对 B + loginResp, err := svc.Login(ctx, auth.LoginRequest{ + Username: "eve", + Password: "password123", + }) + require.NoError(t, err) + tokenPairB_refresh := loginResp.RefreshToken + + // 用令牌对 A 的 refresh token 正常刷新 → 成功 + refreshResp, err := svc.Refresh(ctx, auth.RefreshRequest{ + RefreshToken: tokenPairA_refresh, + }) + require.NoError(t, err) + assert.NotEmpty(t, refreshResp.AccessToken) + + // 用令牌对 A 的旧 refresh token 再次刷新 → 复用检测，应失败 + _, err = svc.Refresh(ctx, auth.RefreshRequest{ + RefreshToken: tokenPairA_refresh, + }) + assert.ErrorIs(t, err, auth.ErrRefreshTokenUsed) + + // 令牌对 B 的 refresh token 也应被吊销（全量吊销） + _, err = svc.Refresh(ctx, auth.RefreshRequest{ + RefreshToken: tokenPairB_refresh, + }) + assert.ErrorIs(t, err, auth.ErrRefreshTokenUsed) + + // 确认 DB 中该用户已无 refresh token + _ = repo // repo 用于确认，但 MemUserRepository 无直接查询方法，通过 Refresh 失败已间接验证 +} diff --git a/backend/internal/config/config.go b/backend/internal/config/config.go index b61ad4f..e0ba227 100644 --- a/backend/internal/config/config.go +++ b/backend/internal/config/config.go @@ -2,22 +2,23 @@ package config import ( "fmt" - "os" - "strings" + "path/filepath" + "github.com/joho/godotenv" "github.com/spf13/viper" ) // Config 应用配置。 type Config struct { - App AppConfig `mapstructure:"app"` - Server ServerConfig `mapstructure:"server"` - Session SessionConfig `mapstructure:"session"` - Redis RedisConfig `mapstructure:"redis"` - AI AIConfig `mapstructure:"ai"` - Storage StorageConfig `mapstructure:"storage"` - Log LogConfig `mapstructure:"log"` - Auth AuthConfig `mapstructure:"auth"` + App AppConfig `mapstructure:"app"` + Server ServerConfig `mapstructure:"server"` + Session SessionConfig `mapstructure:"session"` + Redis RedisConfig `mapstructure:"redis"` + AI AIConfig `mapstructure:"ai"` + Storage StorageConfig `mapstructure:"storage"` + Log LogConfig `mapstructure:"log"` + Auth AuthConfig `mapstructure:"auth"` + RateLimit RateLimitConfig `mapstructure:"ratelimit"` } // SessionConfig 会话管理配置。 @@ -91,10 +92,23 @@ type TTSConfig struct { } type StorageConfig struct { + Redis RedisStorageConfig `mapstructure:"redis"` + Persistence PersistenceConfig `mapstructure:"persistence"` + // Deprecated: 使用 Redis 和 Persistence 替代 Driver string `mapstructure:"driver"` DSN string `mapstructure:"dsn"` } +type RedisStorageConfig struct { + Enabled bool `mapstructure:"enabled"` +} + +type PersistenceConfig struct { + Enabled bool `mapstructure:"enabled"` + Driver string `mapstructure:"driver"` + DSN string `mapstructure:"dsn"` +} + type LogConfig struct { Level string `mapstructure:"level"` Format string `mapstructure:"format"` @@ -107,90 +121,152 @@ type AuthConfig struct { RefreshTTL int `mapstructure:"refresh_ttl"` // Refresh Token 过期时间（分钟），默认 10080（7天） } -// Load 加载配置。优先级：环境变量 > config.{env}.yaml > config.yaml。 -func Load() (*Config, error) { +// RateLimitConfig 限流配置。 +type RateLimitConfig struct { + Enabled bool `mapstructure:"enabled"` + Query BucketConfig `mapstructure:"query"` + Login BucketConfig `mapstructure:"login"` + Register BucketConfig `mapstructure:"register"` +} + +// BucketConfig 令牌桶配置。 +type BucketConfig struct { + Capacity int `mapstructure:"capacity"` // 桶容量（突发上限） + Rate float64 `mapstructure:"rate"` // 每秒填充令牌数 +} + +// Load 加载配置。优先级：环境变量 > config.{env}.yaml > config.yaml > 默认值。 +// workDir 为项目根目录或 backend 目录，用于定位 .env 和 config/config.yaml。 +func Load(workDir string) (*Config, error) { + // 1. 加载 .env 文件（敏感信息） + envFile := filepath.Join(workDir, ".env") + _ = godotenv.Load(envFile) // 文件不存在也不报错 + v := viper.New() v.SetConfigName("config") v.SetConfigType("yaml") - v.AddConfigPath(".") - v.AddConfigPath("./config") - v.AddConfigPath("./backend") - v.AddConfigPath("..") // 兼容从 backend/cmd/ 启动 - v.AddConfigPath("../..") // 兼容从 backend/cmd/server/ 启动 + v.AddConfigPath(filepath.Join(workDir, "config")) // 配置文件在 config/ 目录下 + v.AddConfigPath(workDir) // 兼容旧路径 - // 默认值 - v.SetDefault("app.env", "dev") - v.SetDefault("app.version", "dev") - v.SetDefault("server.host", "0.0.0.0") - v.SetDefault("server.port", 8080) - v.SetDefault("server.read_timeout", 30) - v.SetDefault("server.write_timeout", 30) - v.SetDefault("server.heartbeat_interval", 30) - v.SetDefault("server.heartbeat_timeout", 60) - v.SetDefault("server.shutdown_timeout", 10) - v.SetDefault("session.ttl", 30) - v.SetDefault("session.max_history", 20) - v.SetDefault("redis.addr", "localhost:6379") - v.SetDefault("redis.db", 0) - v.SetDefault("ai.stt.provider", "deepgram") - v.SetDefault("ai.stt.model", "nova-2") - v.SetDefault("ai.stt.endpoint", "wss://api.deepgram.com/v1/listen") - v.SetDefault("ai.stt.timeout", 5) - v.SetDefault("ai.stt.http_client_timeout", 30) - v.SetDefault("ai.llm.provider", "openai") - v.SetDefault("ai.llm.model", "gpt-4o") - v.SetDefault("ai.llm.endpoint", "https://api.openai.com/v1") - v.SetDefault("ai.llm.timeout", 10) - v.SetDefault("ai.llm.http_client_timeout", 60) - v.SetDefault("ai.tts.provider", "openai") - v.SetDefault("ai.tts.model", "tts-1") - v.SetDefault("ai.tts.voice", "mimo_default") - v.SetDefault("ai.tts.speed", 1.0) - v.SetDefault("ai.tts.endpoint", "https://api.openai.com/v1") - v.SetDefault("ai.tts.timeout", 5) - v.SetDefault("ai.tts.http_client_timeout", 30) - v.SetDefault("ai.tts.output_format", "mp3") - v.SetDefault("ai.tts.sample_rate", 24000) - v.SetDefault("storage.driver", "memory") - v.SetDefault("storage.dsn", "") - v.SetDefault("log.level", "info") - v.SetDefault("log.format", "console") - v.SetDefault("auth.access_ttl", 15) - v.SetDefault("auth.refresh_ttl", 10080) + // 2. 设置默认值（与 config.yaml 保持一致，仅作为兜底） + setDefaults(v) - // 读取基础配置文件 - _ = v.ReadInConfig() // 文件不存在不报错 - - // 根据 APP_ENV 覆盖 - env := os.Getenv("APP_ENV") - if env == "" { - env = v.GetString("app.env") + // 3. 读取 config.yaml + if err := v.ReadInConfig(); err != nil { + return nil, fmt.Errorf("config: read config.yaml: %w", err) } + + // 4. 合并环境专属配置 config.{env}.yaml（可选） + env := v.GetString("app.env") if env != "" { v.SetConfigName("config." + env) - _ = v.MergeInConfig() + _ = v.MergeInConfig() // 文件不存在也不报错 } - // 环境变量覆盖 - v.SetEnvPrefix("CAMTALK") - v.SetEnvKeyReplacer(strings.NewReplacer(".", "_")) - v.AutomaticEnv() + // 5. 显式绑定敏感信息环境变量（不用 AutomaticEnv，避免隐式映射） + bindEnvVars(v) var cfg Config if err := v.Unmarshal(&cfg); err != nil { - return nil, fmt.Errorf("config unmarshal: %w", err) - } - - // 填充默认值 - if cfg.Server.Host == "" { - cfg.Server.Host = "0.0.0.0" - } - if cfg.Server.Port == 0 { - cfg.Server.Port = 8080 - } - if cfg.App.Env == "" { - cfg.App.Env = "dev" + return nil, fmt.Errorf("config: unmarshal: %w", err) } return &cfg, nil } + +// setDefaults 设置兜底默认值，与 config.yaml 保持一致。 +func setDefaults(v *viper.Viper) { + // app + v.SetDefault("app.env", "dev") + v.SetDefault("app.version", "dev") + + // server + v.SetDefault("server.host", "0.0.0.0") + v.SetDefault("server.port", 8080) + v.SetDefault("server.read_timeout", 30) + v.SetDefault("server.write_timeout", 30) + v.SetDefault("server.shutdown_timeout", 10) + v.SetDefault("server.heartbeat_interval", 30) + v.SetDefault("server.heartbeat_timeout", 60) + + // session + v.SetDefault("session.ttl", 30) + v.SetDefault("session.max_history", 20) + + // ai — 默认值与 config.yaml 一致（mimo/dashscope） + v.SetDefault("ai.stt.provider", "mimo") + v.SetDefault("ai.stt.model", "mimo-v2.5-asr") + v.SetDefault("ai.stt.endpoint", "https://api.xiaomimimo.com/v1") + v.SetDefault("ai.stt.timeout", 5) + v.SetDefault("ai.stt.http_client_timeout", 30) + + v.SetDefault("ai.llm.provider", "dashscope") + v.SetDefault("ai.llm.model", "qwen3-vl-plus") + v.SetDefault("ai.llm.endpoint", "https://dashscope.aliyuncs.com/compatible-mode/v1") + v.SetDefault("ai.llm.timeout", 30) + v.SetDefault("ai.llm.http_client_timeout", 60) + + v.SetDefault("ai.tts.provider", "mimo") + v.SetDefault("ai.tts.model", "mimo-v2.5-tts") + v.SetDefault("ai.tts.voice", "mimo_default") + v.SetDefault("ai.tts.speed", 1.0) + v.SetDefault("ai.tts.endpoint", "https://token-plan-cn.xiaomimimo.com/v1") + v.SetDefault("ai.tts.timeout", 5) + v.SetDefault("ai.tts.http_client_timeout", 30) + v.SetDefault("ai.tts.output_format", "mp3") + v.SetDefault("ai.tts.sample_rate", 24000) + + // storage + v.SetDefault("storage.driver", "memory") + v.SetDefault("storage.redis.enabled", false) + v.SetDefault("storage.persistence.enabled", false) + v.SetDefault("storage.persistence.driver", "postgres") + + // redis + v.SetDefault("redis.addr", "localhost:6379") + v.SetDefault("redis.password", "") + v.SetDefault("redis.db", 0) + + // auth + v.SetDefault("auth.access_ttl", 15) + v.SetDefault("auth.refresh_ttl", 10080) + + // log + v.SetDefault("log.level", "info") + v.SetDefault("log.format", "console") + + // ratelimit + v.SetDefault("ratelimit.enabled", false) + v.SetDefault("ratelimit.query.capacity", 10) + v.SetDefault("ratelimit.query.rate", 0.2) + v.SetDefault("ratelimit.login.capacity", 5) + v.SetDefault("ratelimit.login.rate", 0.1) + v.SetDefault("ratelimit.register.capacity", 3) + v.SetDefault("ratelimit.register.rate", 0.05) +} + +// bindEnvVars 显式绑定敏感信息环境变量。 +// 只绑定不应出现在 config.yaml 中的敏感字段，非敏感配置通过 config.yaml 管理。 +func bindEnvVars(v *viper.Viper) { + // app.env 特殊处理：环境变量 APP_ENV 覆盖 config.yaml 中的 app.env + v.BindEnv("app.env", "APP_ENV") + + // AI API Key + v.BindEnv("ai.stt.api_key", "CAMTALK_AI_STT_API_KEY") + v.BindEnv("ai.llm.api_key", "CAMTALK_AI_LLM_API_KEY") + v.BindEnv("ai.tts.api_key", "CAMTALK_AI_TTS_API_KEY") + + // JWT + v.BindEnv("auth.jwt_secret", "CAMTALK_AUTH_JWT_SECRET") + + // 数据库 + v.BindEnv("storage.dsn", "CAMTALK_STORAGE_DSN") + v.BindEnv("storage.persistence.dsn", "CAMTALK_STORAGE_DSN") + v.BindEnv("storage.redis.enabled", "CAMTALK_STORAGE_REDIS_ENABLED") + v.BindEnv("storage.persistence.enabled", "CAMTALK_STORAGE_PERSISTENCE_ENABLED") + v.BindEnv("storage.persistence.driver", "CAMTALK_STORAGE_PERSISTENCE_DRIVER") + + // Redis（密码可能包含特殊字符，通过环境变量设置更安全） + v.BindEnv("redis.addr", "CAMTALK_REDIS_ADDR") + v.BindEnv("redis.password", "CAMTALK_REDIS_PASSWORD") +} diff --git a/backend/internal/eino/adapter.go b/backend/internal/eino/adapter.go new file mode 100644 index 0000000..665b391 --- /dev/null +++ b/backend/internal/eino/adapter.go @@ -0,0 +1,172 @@ +package eino + +import ( + "context" + "encoding/base64" + "io" + "time" + + "github.com/cloudwego/eino/compose" + + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/orchestrator" + "github.com/hhs/camtalk/internal/session" + "github.com/hhs/camtalk/internal/trace" +) + +// EinoOrchestrator 实现 orchestrator.Orchestrator 接口。 +// 将 Eino Graph 包装为现有接口，WS Handler 几乎不用改。 +type EinoOrchestrator struct { + graph *PipelineGraph + sessionMgr session.Manager + model string + callbacks compose.Option // 运行时 Callback option +} + +// NewEinoOrchestrator 创建 Eino 编排器适配器。 +func NewEinoOrchestrator(graph *PipelineGraph, sessionMgr session.Manager, model string) *EinoOrchestrator { + return &EinoOrchestrator{ + graph: graph, + sessionMgr: sessionMgr, + model: model, + callbacks: compose.WithCallbacks(BuildCallbackHandler()), + } +} + +// ProcessQuery 实现 orchestrator.Orchestrator 接口。 +func (e *EinoOrchestrator) ProcessQuery( + ctx context.Context, + sessionID string, + req models.WsQuery, + sender orchestrator.Sender, +) error { + log := trace.FromContext(ctx) + startTime := time.Now() + + // 1. 设置活跃请求 + if err := e.sessionMgr.SetActiveRequest(ctx, sessionID, req.RequestID); err != nil { + return err + } + defer e.sessionMgr.ClearActiveRequest(ctx, sessionID) + + // 2. 获取会话配置 + sess, err := e.sessionMgr.Get(ctx, sessionID) + if err != nil { + log.Errorw("get session failed", "error", err) + sender.SendError(models.WsError{ + Type: "error", + RequestID: req.RequestID, + Code: "SESSION_NOT_FOUND", + Message: "会话不存在", + }) + return err + } + + // 3. 解码音频和图片 + var audioData []byte + if req.Text == "" && req.Audio != "" { + audioData, err = base64.StdEncoding.DecodeString(req.Audio) + if err != nil { + log.Errorw("audio decode failed", "error", err) + sender.SendError(models.WsError{ + Type: "error", + RequestID: req.RequestID, + Code: "INVALID_MESSAGE", + Message: "音频数据解码失败", + }) + return err + } + } + + var imageData []byte + if req.Image != "" { + imageData, err = base64.StdEncoding.DecodeString(req.Image) + if err != nil { + log.Errorw("image decode failed", "error", err) + sender.SendError(models.WsError{ + Type: "error", + RequestID: req.RequestID, + Code: "INVALID_MESSAGE", + Message: "图片数据解码失败", + }) + return err + } + } + + // 4. 构建 Graph 输入 + input := buildPipelineInput(req, sessionID, sess, audioData, imageData) + + // 5. 注入 context 值（供 Callback 和 Lambda 节点使用） + ctx = WithSender(ctx, sender) + ctx = WithRequestID(ctx, req.RequestID) + ctx = trace.WithSessionID(ctx, sessionID) + ctx = WithStartTime(ctx, startTime) + + // 创建 State 并从 input 复制元数据 + state := genLocalState(ctx) + state.SessionID = input.SessionID + state.RequestID = input.RequestID + state.ImageData = input.ImageData + state.Scenario = input.Scenario + state.Language = input.Language + state.DetailLevel = sess.Config.DetailLevel + state.TTSEnabled = input.TTSEnabled + state.UserID = input.UserID + ctx = WithPipelineState(ctx, state) + + // 6. 调用 Graph（Stream 模式 + 运行时 Callback） + streamReader, err := e.graph.Runnable.Stream(ctx, input, e.callbacks) + if err != nil { + log.Errorw("graph stream start failed", "error", err) + sender.SendError(models.WsError{ + Type: "error", + RequestID: req.RequestID, + Code: "INTERNAL_ERROR", + Message: "编排器启动失败", + }) + return err + } + + // 7. 消费 StreamReader（触发整条链路执行，side effects 推送消息到客户端） + var output PipelineOutput + for { + o, err := streamReader.Recv() + if err != nil { + if err == io.EOF { + break + } + log.Errorw("graph stream consume error", "error", err) + break + } + output = o + } + + // 8. 追加用户消息到历史（使用 STT 结果，兼容文本输入和语音输入） + userText := output.TranscribedText + if userText == "" { + userText = req.Text // fallback 到原始文本输入 + } + if userText != "" { + if err := e.sessionMgr.AppendMessage(ctx, sessionID, models.Message{ + Role: "user", + Content: userText, + }); err != nil { + log.Errorw("append user message failed", "error", err) + } + } + + // 9. 追加助手消息到历史 + if output.FullResponse != "" { + if err := e.sessionMgr.AppendMessage(ctx, sessionID, models.Message{ + Role: "assistant", + Content: output.FullResponse, + }); err != nil { + log.Errorw("append assistant message failed", "error", err) + } + } + + latency := time.Since(startTime).Milliseconds() + log.Infow("eino pipeline completed", "latency_ms", latency) + + return nil +} diff --git a/backend/internal/eino/callback.go b/backend/internal/eino/callback.go new file mode 100644 index 0000000..d65799e --- /dev/null +++ b/backend/internal/eino/callback.go @@ -0,0 +1,131 @@ +package eino + +import ( + "context" + "io" + + "github.com/cloudwego/eino/callbacks" + "github.com/cloudwego/eino/components/model" + "github.com/cloudwego/eino/schema" + callbacksHelper "github.com/cloudwego/eino/utils/callbacks" + + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/orchestrator" + "github.com/hhs/camtalk/internal/trace" +) + +// context key 类型，避免与其他包冲突。 +type ctxKeySender struct{} +type ctxKeyState struct{} + +// WithSender 将 Sender 注入 context。 +func WithSender(ctx context.Context, sender orchestrator.Sender) context.Context { + return context.WithValue(ctx, ctxKeySender{}, sender) +} + +// WithRequestID 将 requestID 注入 context（使用 trace 包）。 +func WithRequestID(ctx context.Context, requestID string) context.Context { + return trace.WithRequestID(ctx, requestID) +} + +// WithPipelineState 将 PipelineState 注入 context。 +func WithPipelineState(ctx context.Context, state *PipelineState) context.Context { + return context.WithValue(ctx, ctxKeyState{}, state) +} + +// senderFromCtx 从 context 获取 Sender。 +func senderFromCtx(ctx context.Context) orchestrator.Sender { + s, _ := ctx.Value(ctxKeySender{}).(orchestrator.Sender) + return s +} + +// requestIDFromCtx 从 context 获取 requestID（使用 trace 包）。 +func requestIDFromCtx(ctx context.Context) string { + return trace.GetRequestID(ctx) +} + +// stateFromCtx 从 context 获取 PipelineState。 +func stateFromCtx(ctx context.Context) *PipelineState { + s, _ := ctx.Value(ctxKeyState{}).(*PipelineState) + return s +} + +// BuildCallbackHandler 构建 Eino Callback Handler。 +// +// 核心职责：ChatModel 节点通过 OnEndWithStreamOutput 逐 token 推送 llm_chunk 到客户端， +// 同时累积完整文本到 PipelineState。 +// +// 其他节点的消息推送（stt_result、tts_audio、llm_done）由各 Lambda 内部直接调用 Sender。 +func BuildCallbackHandler() callbacks.Handler { + return callbacksHelper.NewHandlerHelper(). + ChatModel(&callbacksHelper.ModelCallbackHandler{ + OnEndWithStreamOutput: func(ctx context.Context, info *callbacks.RunInfo, output *schema.StreamReader[*model.CallbackOutput]) context.Context { + log := trace.FromContext(ctx) + sender := senderFromCtx(ctx) + requestID := requestIDFromCtx(ctx) + state := stateFromCtx(ctx) + + if sender == nil || requestID == "" { + log.Warnw("ModelCallback: missing sender or request_id in context", + "node", info.Name) + return ctx + } + + // 异步消费流，避免阻塞框架的下游处理。 + // 框架对流做了内部拷贝，此 goroutine 读取独立副本。 + go func() { + defer output.Close() + + for { + chunk, err := output.Recv() + if err != nil { + if err == io.EOF { + return + } + log.Errorw("ModelCallback: stream recv error", + "node", info.Name, "error", err) + return + } + + if chunk == nil || chunk.Message == nil { + continue + } + + delta := chunk.Message.Content + if delta == "" { + continue + } + + // 推送 llm_chunk 到客户端 + if err := sender.SendLLMChunk(models.WsLLMChunk{ + Type: "llm_chunk", + RequestID: requestID, + Delta: delta, + Role: "assistant", + }); err != nil { + log.Errorw("ModelCallback: send llm_chunk failed", "error", err) + } + + // 累积完整文本到 State + if state != nil { + state.AppendText(delta) + } + + // 记录 token 用量（流的最后一帧携带） + if chunk.TokenUsage != nil && state != nil { + state.mu.Lock() + state.TokenUsage = &TokenUsage{ + Prompt: chunk.TokenUsage.PromptTokens, + Completion: chunk.TokenUsage.CompletionTokens, + Total: chunk.TokenUsage.TotalTokens, + } + state.mu.Unlock() + } + } + }() + + return ctx + }, + }). + Handler() +} diff --git a/backend/internal/eino/graph.go b/backend/internal/eino/graph.go new file mode 100644 index 0000000..b440d34 --- /dev/null +++ b/backend/internal/eino/graph.go @@ -0,0 +1,119 @@ +package eino + +import ( + "context" + "time" + + openaiImpl "github.com/cloudwego/eino-ext/components/model/openai" + "github.com/cloudwego/eino/compose" + + "github.com/hhs/camtalk/internal/ai/stt" + "github.com/hhs/camtalk/internal/ai/tts" + "github.com/hhs/camtalk/internal/config" + "github.com/hhs/camtalk/internal/logger" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/session" + "github.com/hhs/camtalk/internal/store" +) + +const ( + nodeSTT = "stt" + nodeHistory = "history" + nodeLLM = "llm" + nodeMessageToString = "msg2str" + nodeSplitter = "splitter" + nodeTTS = "tts" + nodeDone = "done" +) + +// PipelineGraph 封装编译后的 Eino Graph。 +type PipelineGraph struct { + Runnable compose.Runnable[PipelineInput, PipelineOutput] +} + +// NewPipelineGraph 构建 CamTalk AI 编排 Graph。 +// +// 拓扑：START → STT → History → ChatModel → Splitter → TTS → Done → END +// +// Graph 使用 Stream 模式调用，ChatModel 实现真正的 token 级流式输出。 +// LLM token 通过 Callback 的 OnEndWithStreamOutput 实时推送到客户端。 +func NewPipelineGraph( + ctx context.Context, + cfg *config.Config, + sttService stt.Service, + ttsService tts.Service, + sessionMgr session.Manager, + scenarioRepo store.UserScenarioRepository, +) (*PipelineGraph, error) { + log := logger.Log + + // 1. 创建 eino-ext ChatModel（对接 DashScope OpenAI 兼容接口） + chatModel, err := openaiImpl.NewChatModel(ctx, &openaiImpl.ChatModelConfig{ + APIKey: cfg.AI.LLM.APIKey, + Model: cfg.AI.LLM.Model, + BaseURL: cfg.AI.LLM.Endpoint, + Timeout: time.Duration(cfg.AI.LLM.Timeout) * time.Second, + }) + if err != nil { + return nil, err + } + log.Infow("Eino ChatModel 初始化成功", + "model", cfg.AI.LLM.Model, + "endpoint", cfg.AI.LLM.Endpoint) + + // 2. 构建 Graph（值类型，非指针） + g := compose.NewGraph[PipelineInput, PipelineOutput]( + compose.WithGenLocalState(genLocalState), + ) + + // 3. 添加节点 + maxHistory := cfg.Session.MaxHistory + + _ = g.AddLambdaNode(nodeSTT, NewSTTLambda(sttService)) + _ = g.AddLambdaNode(nodeHistory, NewHistoryLambda(sessionMgr.GetHistory, scenarioRepo, maxHistory)) + _ = g.AddChatModelNode(nodeLLM, chatModel) + _ = g.AddLambdaNode(nodeMessageToString, NewMessageToStringLambda()) + _ = g.AddLambdaNode(nodeSplitter, NewSplitterLambda()) + _ = g.AddLambdaNode(nodeTTS, NewTTSLambda( + ttsService, + cfg.AI.TTS.Voice, + cfg.AI.TTS.Speed, + cfg.AI.TTS.OutputFormat, + cfg.AI.TTS.SampleRate, + )) + _ = g.AddLambdaNode(nodeDone, NewDoneLambda(cfg.AI.LLM.Model)) + + // 4. 连接边 + _ = g.AddEdge(compose.START, nodeSTT) + _ = g.AddEdge(nodeSTT, nodeHistory) + _ = g.AddEdge(nodeHistory, nodeLLM) + _ = g.AddEdge(nodeLLM, nodeMessageToString) + _ = g.AddEdge(nodeMessageToString, nodeSplitter) + _ = g.AddEdge(nodeSplitter, nodeTTS) + _ = g.AddEdge(nodeTTS, nodeDone) + _ = g.AddEdge(nodeDone, compose.END) + + // 5. 编译（回调在运行时通过 Stream option 传入） + runnable, err := g.Compile(ctx) + if err != nil { + return nil, err + } + + log.Infow("Eino Graph 编译成功", "nodes", 7) + return &PipelineGraph{Runnable: runnable}, nil +} + +// buildPipelineInput 从 WebSocket 请求和会话配置构建 Graph 输入。 +func buildPipelineInput(req models.WsQuery, sessionID string, sess *models.Session, audioData, imageData []byte) PipelineInput { + return PipelineInput{ + AudioData: audioData, + ImageData: imageData, + Text: req.Text, + SessionID: sessionID, + RequestID: req.RequestID, + Language: sess.Config.Language, + Scenario: sess.Config.Scenario, + TTSEnabled: sess.Config.TTSEnabled, + UserID: sess.UserID, + } +} diff --git a/backend/internal/eino/graph_test.go b/backend/internal/eino/graph_test.go new file mode 100644 index 0000000..a5ed074 --- /dev/null +++ b/backend/internal/eino/graph_test.go @@ -0,0 +1,236 @@ +package eino + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/mock" + "github.com/stretchr/testify/require" + + "github.com/hhs/camtalk/internal/ai/stt" + "github.com/hhs/camtalk/internal/ai/tts" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/orchestrator" + "github.com/hhs/camtalk/internal/trace" +) + +// --- Mock STT Service --- + +type mockSTTService struct { + mock.Mock +} + +func (m *mockSTTService) Recognize(ctx context.Context, audio []byte, opts stt.Options) (string, error) { + args := m.Called(ctx, audio, opts) + return args.String(0), args.Error(1) +} + +// --- Mock TTS Service --- + +type mockTTSService struct { + mock.Mock +} + +func (m *mockTTSService) SynthesizeStream(ctx context.Context, textStream <-chan string, opts tts.Options) (<-chan tts.Chunk, error) { + args := m.Called(ctx, textStream, opts) + return args.Get(0).(<-chan tts.Chunk), args.Error(1) +} + +// --- Mock Sender --- + +type mockSender struct { + mock.Mock + STTResults []models.WsSTTResult + LLMChunks []models.WsLLMChunk + LLMDones []models.WsLLMDone + TTSAudios []models.WsTTSAudio + Errors []models.WsError +} + +func (m *mockSender) SendSTTResult(result models.WsSTTResult) error { + m.STTResults = append(m.STTResults, result) + return m.Called(result).Error(0) +} + +func (m *mockSender) SendLLMChunk(chunk models.WsLLMChunk) error { + m.LLMChunks = append(m.LLMChunks, chunk) + return m.Called(chunk).Error(0) +} + +func (m *mockSender) SendLLMDone(done models.WsLLMDone) error { + m.LLMDones = append(m.LLMDones, done) + return m.Called(done).Error(0) +} + +func (m *mockSender) SendTTSAudio(audio models.WsTTSAudio) error { + m.TTSAudios = append(m.TTSAudios, audio) + return m.Called(audio).Error(0) +} + +func (m *mockSender) SendError(err models.WsError) error { + m.Errors = append(m.Errors, err) + return m.Called(err).Error(0) +} + +// --- Tests --- + +func TestDetectImageMimeType(t *testing.T) { + tests := []struct { + name string + data []byte + expected string + }{ + {"JPEG", []byte{0xFF, 0xD8, 0xFF, 0xE0}, "image/jpeg"}, + {"PNG", []byte{0x89, 0x50, 0x4E, 0x47}, "image/png"}, + {"GIF", []byte{0x47, 0x49, 0x46, 0x38}, "image/gif"}, + {"WebP", []byte{0x52, 0x49, 0x46, 0x46}, "image/webp"}, + {"Unknown", []byte{0x00, 0x00, 0x00}, "image/jpeg"}, + {"Short", []byte{0xFF}, "image/jpeg"}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + result := detectImageMimeType(tt.data) + assert.Equal(t, tt.expected, result) + }) + } +} + +func TestBuildPipelineInput(t *testing.T) { + req := models.WsQuery{ + Text: "你好", + RequestID: "req-1", + } + sess := &models.Session{ + Config: models.SessionConfig{ + Language: "zh-CN", + Scenario: "free_chat", + TTSEnabled: true, + }, + } + + input := buildPipelineInput(req, "sess-1", sess, nil, nil) + require.Equal(t, "你好", input.Text) + require.Equal(t, "sess-1", input.SessionID) + require.Equal(t, "req-1", input.RequestID) + require.Equal(t, "zh-CN", input.Language) + require.Equal(t, "free_chat", input.Scenario) + require.True(t, input.TTSEnabled) +} + +func TestBuildPipelineInput_WithAudioData(t *testing.T) { + req := models.WsQuery{ + Audio: "base64audio", + RequestID: "req-2", + } + sess := &models.Session{ + Config: models.SessionConfig{ + Language: "en", + Scenario: "free_chat", + TTSEnabled: false, + }, + } + + audioData := []byte("fake-audio-bytes") + imageData := []byte("fake-image-bytes") + + input := buildPipelineInput(req, "sess-2", sess, audioData, imageData) + require.Equal(t, audioData, input.AudioData) + require.Equal(t, imageData, input.ImageData) + require.False(t, input.TTSEnabled) + require.Equal(t, "en", input.Language) +} + +func TestPipelineState_AppendAndGet(t *testing.T) { + state := genLocalState(context.Background()) + + state.AppendText("Hello ") + state.AppendText("World") + + require.Equal(t, "Hello World", state.GetFullResponse()) +} + +func TestPipelineState_ConcurrentAccess(t *testing.T) { + state := genLocalState(context.Background()) + + done := make(chan struct{}) + go func() { + for i := 0; i < 100; i++ { + state.AppendText("a") + } + close(done) + }() + + for i := 0; i < 100; i++ { + _ = state.GetFullResponse() + } + + <-done + require.Equal(t, 100, len(state.GetFullResponse())) +} + +func TestContextInjection(t *testing.T) { + ctx := context.Background() + + sender := &mockSender{} + ctx = WithSender(ctx, sender) + ctx = WithRequestID(ctx, "req-123") + ctx = trace.WithSessionID(ctx, "sess-456") + ctx = WithStartTime(ctx, time.Now()) + ctx = WithPipelineState(ctx, genLocalState(ctx)) + + require.NotNil(t, senderFromCtx(ctx)) + require.Equal(t, "req-123", requestIDFromCtx(ctx)) + require.NotNil(t, stateFromCtx(ctx)) +} + +func TestLatencyFromCtx(t *testing.T) { + ctx := context.Background() + + // No start time set + require.Equal(t, int64(0), latencyFromCtx(ctx)) + + // With start time + start := time.Now().Add(-100 * time.Millisecond) + ctx = WithStartTime(ctx, start) + latency := latencyFromCtx(ctx) + require.Greater(t, latency, int64(0)) + require.Less(t, latency, int64(1000)) // should be < 1 second +} + +func TestEinoOrchestrator_ImplementsInterface(t *testing.T) { + // Compile-time check that EinoOrchestrator implements orchestrator.Orchestrator + var _ orchestrator.Orchestrator = (*EinoOrchestrator)(nil) +} + +func TestNewSTTLambda_ReturnsNonNil(t *testing.T) { + mockSTT := &mockSTTService{} + lambda := NewSTTLambda(mockSTT) + require.NotNil(t, lambda) +} + +func TestNewHistoryLambda_ReturnsNonNil(t *testing.T) { + fetcher := func(ctx context.Context, sessionID string, limit int) ([]models.Message, error) { + return nil, nil + } + lambda := NewHistoryLambda(fetcher, nil, 10) + require.NotNil(t, lambda) +} + +func TestNewSplitterLambda_ReturnsNonNil(t *testing.T) { + lambda := NewSplitterLambda() + require.NotNil(t, lambda) +} + +func TestNewTTSLambda_ReturnsNonNil(t *testing.T) { + mockTTS := &mockTTSService{} + lambda := NewTTSLambda(mockTTS, "alloy", 1.0, "mp3", 24000) + require.NotNil(t, lambda) +} + +func TestNewDoneLambda_ReturnsNonNil(t *testing.T) { + lambda := NewDoneLambda("test-model") + require.NotNil(t, lambda) +} diff --git a/backend/internal/eino/nodes_done.go b/backend/internal/eino/nodes_done.go new file mode 100644 index 0000000..140b056 --- /dev/null +++ b/backend/internal/eino/nodes_done.go @@ -0,0 +1,86 @@ +package eino + +import ( + "context" + "time" + + "github.com/cloudwego/eino/compose" + + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/trace" +) + +// ctxKeyStartTime 请求开始时间的 context key。 +type ctxKeyStartTime struct{} + +// WithStartTime 将请求开始时间注入 context。 +func WithStartTime(ctx context.Context, t time.Time) context.Context { + return context.WithValue(ctx, ctxKeyStartTime{}, t) +} + +// latencyFromCtx 从 context 获取开始时间并计算延迟（毫秒）。 +func latencyFromCtx(ctx context.Context) int64 { + if startTime, ok := ctx.Value(ctxKeyStartTime{}).(time.Time); ok { + return time.Since(startTime).Milliseconds() + } + return 0 +} + +// NewDoneLambda 创建 Done Lambda 节点。 +// 输入: struct{}（TTS 完成信号）→ 输出: *PipelineOutput +// +// 从 PipelineState 读取完整回复和 token 用量，发送 llm_done 到客户端。 +// 历史消息追加由适配器负责（避免重复写入）。 +func NewDoneLambda(defaultModel string) *compose.Lambda { + return compose.InvokableLambda(func(ctx context.Context, _ struct{}) (PipelineOutput, error) { + log := trace.FromContext(ctx) + sender := senderFromCtx(ctx) + state := stateFromCtx(ctx) + + if state == nil { + return PipelineOutput{}, nil + } + + state.mu.Lock() + fullResponse := state.FullResponse.String() + transcribedText := state.TranscribedText + tokenUsage := state.TokenUsage + requestID := state.RequestID + modelName := defaultModel + state.mu.Unlock() + + // 发送 llm_done + if sender != nil && requestID != "" { + done := models.WsLLMDone{ + Type: "llm_done", + RequestID: requestID, + FullText: fullResponse, + Model: modelName, + LatencyMs: latencyFromCtx(ctx), + } + if tokenUsage != nil { + done.TokensUsed = struct { + Prompt int `json:"prompt"` + Completion int `json:"completion"` + Total int `json:"total"` + }{ + Prompt: tokenUsage.Prompt, + Completion: tokenUsage.Completion, + Total: tokenUsage.Total, + } + } + if err := sender.SendLLMDone(done); err != nil { + log.Errorw("send llm_done failed", "error", err) + } + } + + log.Infow("query processing completed", "response_length", len(fullResponse)) + + return PipelineOutput{ + TranscribedText: transcribedText, + FullResponse: fullResponse, + Model: modelName, + TokenUsage: tokenUsage, + }, nil + }) +} diff --git a/backend/internal/eino/nodes_history.go b/backend/internal/eino/nodes_history.go new file mode 100644 index 0000000..8099c98 --- /dev/null +++ b/backend/internal/eino/nodes_history.go @@ -0,0 +1,151 @@ +package eino + +import ( + "context" + "encoding/base64" + + "github.com/cloudwego/eino/compose" + "github.com/cloudwego/eino/schema" + + "github.com/hhs/camtalk/internal/ai/llm" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/store" + "github.com/hhs/camtalk/internal/trace" +) + +// NewHistoryLambda 创建历史组装 Lambda 节点。 +// 输入: *STTOutput → 输出: []*schema.Message +// +// 从 PipelineState 读取请求元数据（SessionID、Scenario、ImageData 等）， +// 构建系统提示词，组装历史消息和当前用户输入（含多模态图片）。 +func NewHistoryLambda( + historyFetcher func(ctx context.Context, sessionID string, limit int) ([]models.Message, error), + scenarioRepo store.UserScenarioRepository, + maxHistory int, +) *compose.Lambda { + return compose.InvokableLambda(func(ctx context.Context, sttOut STTOutput) ([]*schema.Message, error) { + log := trace.FromContext(ctx) + + // 从 State 读取请求元数据 + state := stateFromCtx(ctx) + if state == nil { + return []*schema.Message{}, nil + } + + state.mu.Lock() + sessionID := state.SessionID + requestID := state.RequestID + imageData := state.ImageData + scenario := state.Scenario + detailLevel := state.DetailLevel + language := sttOut.Language + userID := state.UserID + state.mu.Unlock() + + // 加载用户自建情景（如果有 userID 和 scenarioRepo） + var customScenarios map[string]string + var customGreetings map[string]string + if userID != "" && scenarioRepo != nil { + scenarios, err := scenarioRepo.FindByUserID(ctx, userID) + if err != nil { + log.Warnw("load user scenarios failed", "user_id", userID, "error", err) + } else if len(scenarios) > 0 { + customScenarios = make(map[string]string, len(scenarios)) + customGreetings = make(map[string]string, len(scenarios)) + for _, s := range scenarios { + customScenarios[s.ID] = s.Prompt + if s.Greeting != "" { + customGreetings[s.ID] = s.Greeting + } + } + log.Debugw("loaded user scenarios", "user_id", userID, "count", len(scenarios)) + } + } + + // 构建系统提示词（支持用户自建情景） + scenarioPrompt := llm.GetScenarioPrompt(scenario, language, customScenarios) + systemPrompt := llm.BuildSystemPrompt(language, detailLevel, scenarioPrompt) + + // 构建 system message（仅文本，多模态内容只能放在 user 角色） + systemMsg := &schema.Message{ + Role: schema.System, + Content: systemPrompt, + } + + messages := []*schema.Message{systemMsg} + + // 获取并追加历史消息 + if historyFetcher != nil && sessionID != "" { + history, err := historyFetcher(ctx, sessionID, maxHistory) + if err != nil { + log.Warnw("fetch history failed, continuing", "error", err, "request_id", requestID) + } else { + for _, msg := range history { + messages = append(messages, &schema.Message{ + Role: schema.RoleType(msg.Role), + Content: msg.Content, + }) + } + } + } + + // 追加当前用户输入（含图片，多模态内容只能放在 user 角色） + // 注意：不能同时设置 Content 和 UserInputMultiContent，需要统一放到 MultiContent 中 + if len(imageData) > 0 { + base64Str := base64.StdEncoding.EncodeToString(imageData) + mimeType := detectImageMimeType(imageData) + parts := []schema.MessageInputPart{ + { + Type: schema.ChatMessagePartTypeText, + Text: sttOut.Text, + }, + { + Type: schema.ChatMessagePartTypeImageURL, + Image: &schema.MessageInputImage{ + MessagePartCommon: schema.MessagePartCommon{ + Base64Data: &base64Str, + MIMEType: mimeType, + }, + Detail: schema.ImageURLDetailAuto, + }, + }, + } + messages = append(messages, &schema.Message{ + Role: schema.User, + UserInputMultiContent: parts, + }) + } else { + messages = append(messages, &schema.Message{ + Role: schema.User, + Content: sttOut.Text, + }) + } + + log.Debugw("history assembled", + "message_count", len(messages), + "has_image", len(imageData) > 0, + "scenario", scenario) + + return messages, nil + }) +} + +// detectImageMimeType 简单检测图片 MIME 类型。 +func detectImageMimeType(data []byte) string { + if len(data) < 4 { + return "image/jpeg" + } + if data[0] == 0xFF && data[1] == 0xD8 && data[2] == 0xFF { + return "image/jpeg" + } + if data[0] == 0x89 && data[1] == 0x50 && data[2] == 0x4E && data[3] == 0x47 { + return "image/png" + } + if data[0] == 0x47 && data[1] == 0x49 && data[2] == 0x46 { + return "image/gif" + } + if data[0] == 0x52 && data[1] == 0x49 && data[2] == 0x46 && data[3] == 0x46 { + return "image/webp" + } + return "image/jpeg" +} diff --git a/backend/internal/eino/nodes_splitter.go b/backend/internal/eino/nodes_splitter.go new file mode 100644 index 0000000..e3cfafb --- /dev/null +++ b/backend/internal/eino/nodes_splitter.go @@ -0,0 +1,102 @@ +package eino + +import ( + "context" + "io" + "strings" + + "github.com/cloudwego/eino/compose" + "github.com/cloudwego/eino/schema" +) + +// sentenceDelimiters 句子分隔符集合。 +var sentenceDelimiters = map[rune]bool{ + '。': true, + '！': true, + '？': true, + '\n': true, + '.': true, + '!': true, + '?': true, +} + +// NewMessageToStringLambda 创建 Message → String 转换 Lambda 节点。 +// 输入: *schema.Message → 输出: string +// +// 提取 Message.Content 文本，供 Splitter 节点消费。 +func NewMessageToStringLambda() *compose.Lambda { + return compose.TransformableLambda(func(ctx context.Context, input *schema.StreamReader[*schema.Message]) (*schema.StreamReader[string], error) { + sr, sw := schema.Pipe[string](8) + + go func() { + defer sw.Close() + defer input.Close() + + for { + msg, err := input.Recv() + if err != nil { + if err == io.EOF { + return + } + sw.Send("", err) + return + } + if msg != nil && msg.Content != "" { + sw.Send(msg.Content, nil) + } + } + }() + + return sr, nil + }) +} + +// NewSplitterLambda 创建句子分割 Transform Lambda 节点。 +// 输入: StreamReader[string]（LLM token 流）→ 输出: StreamReader[string]（完整句子流） +// +// 逐字符累积，按句子分隔符切分。每切出一个完整句子就输出一次， +// 供下游 TTS 节点实时合成。 +func NewSplitterLambda() *compose.Lambda { + return compose.TransformableLambda(func(ctx context.Context, input *schema.StreamReader[string]) (*schema.StreamReader[string], error) { + sr, sw := schema.Pipe[string](8) + + go func() { + defer sw.Close() + defer input.Close() + + var buffer strings.Builder + + for { + chunk, err := input.Recv() + if err != nil { + if err == io.EOF { + // 流结束，flush 剩余缓冲 + if buffer.Len() > 0 { + text := strings.TrimSpace(buffer.String()) + if text != "" { + sw.Send(text, nil) + } + } + return + } + sw.Send("", err) + return + } + + // 逐字符累积，按句子分隔符切分 + for _, r := range chunk { + buffer.WriteRune(r) + if sentenceDelimiters[r] { + text := strings.TrimSpace(buffer.String()) + if text != "" { + sw.Send(text, nil) + } + buffer.Reset() + } + } + } + }() + + return sr, nil + }) +} diff --git a/backend/internal/eino/nodes_stt.go b/backend/internal/eino/nodes_stt.go new file mode 100644 index 0000000..1a4a6b6 --- /dev/null +++ b/backend/internal/eino/nodes_stt.go @@ -0,0 +1,135 @@ +package eino + +import ( + "context" + "fmt" + "strings" + + "github.com/cloudwego/eino/compose" + + "github.com/hhs/camtalk/internal/ai/stt" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/trace" + "github.com/hhs/camtalk/internal/util" +) + +// NewSTTLambda 创建 STT Lambda 节点。 +// 输入: PipelineInput → 输出: STTOutput +// +// 文本输入模式：跳过 STT，直接返回用户输入文本。 +// 语音模式：调用 sttService.Recognize() 进行语音识别。 +// 识别结果通过 Sender 发送 stt_result 到客户端。 +func NewSTTLambda(sttService stt.Service) *compose.Lambda { + return compose.InvokableLambda(func(ctx context.Context, input PipelineInput) (STTOutput, error) { + log := trace.FromContext(ctx) + sender := senderFromCtx(ctx) + requestID := requestIDFromCtx(ctx) + + // 将输入元数据写入 State，供下游节点（History、Done）读取 + if state := stateFromCtx(ctx); state != nil { + state.mu.Lock() + state.SessionID = input.SessionID + state.RequestID = input.RequestID + state.ImageData = input.ImageData + state.Scenario = input.Scenario + state.DetailLevel = "low" + state.Language = input.Language + state.TTSEnabled = input.TTSEnabled + state.mu.Unlock() + } + + // 文本输入模式：跳过 STT + if input.Text != "" { + log.Debugw("text input mode, skipping stt", + "text_len", len(input.Text), + "text_preview", util.Truncate(input.Text, 50)) + + // 发送 stt_result 保持前端消息流一致性 + if sender != nil { + if err := sender.SendSTTResult(models.WsSTTResult{ + Type: "stt_result", + RequestID: requestID, + Text: input.Text, + IsFinal: true, + }); err != nil { + log.Errorw("send stt_result failed", "error", err) + } + } + + // 写入 State + if state := stateFromCtx(ctx); state != nil { + state.mu.Lock() + state.TranscribedText = input.Text + state.mu.Unlock() + } + + return STTOutput{ + Text: input.Text, + Language: input.Language, + IsSkipped: true, + }, nil + } + + // 语音模式：解码音频 + if len(input.AudioData) == 0 { + return STTOutput{}, fmt.Errorf("stt: no audio data provided") + } + + log.Debugw("stt recognition started", "audio_bytes", len(input.AudioData)) + + // 调用 STT 服务 + text, err := sttService.Recognize(ctx, input.AudioData, stt.Options{ + Encoding: "pcm_s16le", + SampleRate: 16000, + Language: input.Language, + }) + if err != nil { + log.Errorw("stt recognition failed", "error", err) + if sender != nil { + sender.SendError(models.WsError{ + Type: "error", + RequestID: requestID, + Code: "STT_ERROR", + Message: "语音识别失败: " + err.Error(), + }) + } + return STTOutput{}, fmt.Errorf("stt: recognize: %w", err) + } + + // STT 返回空文本 + if strings.TrimSpace(text) == "" { + log.Infow("stt returned empty text") + text = "（未识别到语音）" + } + + log.Debugw("stt recognition completed", + "text_len", len(text), + "text_preview", util.Truncate(text, 50)) + + // 发送 stt_result + if sender != nil { + if err := sender.SendSTTResult(models.WsSTTResult{ + Type: "stt_result", + RequestID: requestID, + Text: text, + IsFinal: true, + }); err != nil { + log.Errorw("send stt_result failed", "error", err) + } + } + + // 写入 State + if state := stateFromCtx(ctx); state != nil { + state.mu.Lock() + state.TranscribedText = text + state.mu.Unlock() + } + + return STTOutput{ + Text: text, + Language: input.Language, + IsSkipped: false, + }, nil + }) +} + diff --git a/backend/internal/eino/nodes_tts.go b/backend/internal/eino/nodes_tts.go new file mode 100644 index 0000000..0f13a4d --- /dev/null +++ b/backend/internal/eino/nodes_tts.go @@ -0,0 +1,116 @@ +package eino + +import ( + "context" + "encoding/base64" + "io" + + "github.com/cloudwego/eino/compose" + "github.com/cloudwego/eino/schema" + + "github.com/hhs/camtalk/internal/ai/tts" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/trace" +) + +// NewTTSLambda 创建 TTS Transform Lambda 节点。 +// 输入: StreamReader[string]（句子流）→ 输出: StreamReader[struct{}]（结果流） +// +// 流式消费每个句子，调用 ttsService.SynthesizeStream() 合成， +// 逐 chunk 推送 tts_audio 到客户端。TTS 失败静默跳过。 +func NewTTSLambda(ttsService tts.Service, ttsVoice string, ttsSpeed float64, ttsOutputFmt string, ttsSampleRate int) *compose.Lambda { + return compose.TransformableLambda(func(ctx context.Context, input *schema.StreamReader[string]) (*schema.StreamReader[struct{}], error) { + sr, sw := schema.Pipe[struct{}](8) + + go func() { + defer sw.Close() + defer input.Close() + + log := trace.FromContext(ctx) + sender := senderFromCtx(ctx) + requestID := requestIDFromCtx(ctx) + + if sender == nil || requestID == "" { + // 消费并丢弃流 + for { + _, err := input.Recv() + if err != nil { + return + } + } + } + + // 收集句子，按批次合成 TTS + var sentences []string + for { + sentence, err := input.Recv() + if err != nil { + if err == io.EOF { + break + } + log.Errorw("TTS: stream recv error", "error", err) + break + } + if sentence != "" { + sentences = append(sentences, sentence) + } + } + + if len(sentences) == 0 { + sw.Send(struct{}{}, nil) + return + } + + log.Infow("开始 TTS 合成", "sentence_count", len(sentences)) + + // 将句子数组转为 channel + sentenceCh := make(chan string, len(sentences)) + for _, s := range sentences { + sentenceCh <- s + } + close(sentenceCh) + + // 调用 TTS 服务 + ttsStream, err := ttsService.SynthesizeStream(ctx, sentenceCh, tts.Options{ + Voice: ttsVoice, + Speed: ttsSpeed, + OutputFmt: ttsOutputFmt, + SampleRate: ttsSampleRate, + }) + if err != nil { + log.Errorw("TTS 合成启动失败（已跳过）", "error", err) + sw.Send(struct{}{}, nil) + return + } + + // 消费 TTS 音频流，推送到客户端 + for chunk := range ttsStream { + select { + case <-ctx.Done(): + log.Debugw("tts stream interrupted") + sw.Send(struct{}{}, ctx.Err()) + return + default: + } + + audioBase64 := base64.StdEncoding.EncodeToString(chunk.Audio) + + if err := sender.SendTTSAudio(models.WsTTSAudio{ + Type: "tts_audio", + RequestID: requestID, + Audio: audioBase64, + MimeType: "audio/mp3", + IsLast: chunk.IsLast, + Final: chunk.Final, + }); err != nil { + log.Errorw("发送 tts_audio 失败", "error", err) + } + } + + log.Infow("TTS 合成完成") + sw.Send(struct{}{}, nil) + }() + + return sr, nil + }) +} diff --git a/backend/internal/eino/state.go b/backend/internal/eino/state.go new file mode 100644 index 0000000..b45957e --- /dev/null +++ b/backend/internal/eino/state.go @@ -0,0 +1,46 @@ +package eino + +import ( + "context" + "strings" + "sync" +) + +// PipelineState Graph 全局状态，用于跨节点收集数据。 +// 通过 compose.WithGenLocalState 注册，各节点通过 compose.ProcessState 读写。 +type PipelineState struct { + mu sync.Mutex + FullResponse strings.Builder // LLM 完整回复（由 Callback 累积） + TranscribedText string // STT 识别文本 + Model string // 实际使用的模型名 + TokenUsage *TokenUsage // token 用量 + + // 从 PipelineInput 复制的元数据，供下游节点（History、Done）读取 + SessionID string + RequestID string + ImageData []byte + Scenario string + DetailLevel string + Language string + TTSEnabled bool + UserID string // 新增：用户 ID，用于加载自建情景 +} + +// genLocalState 创建每请求的 PipelineState 实例。 +func genLocalState(ctx context.Context) *PipelineState { + return &PipelineState{} +} + +// AppendText 追加文本到 FullResponse（线程安全）。 +func (s *PipelineState) AppendText(text string) { + s.mu.Lock() + defer s.mu.Unlock() + s.FullResponse.WriteString(text) +} + +// GetFullResponse 获取完整回复文本（线程安全）。 +func (s *PipelineState) GetFullResponse() string { + s.mu.Lock() + defer s.mu.Unlock() + return s.FullResponse.String() +} diff --git a/backend/internal/eino/types.go b/backend/internal/eino/types.go new file mode 100644 index 0000000..99a8ed9 --- /dev/null +++ b/backend/internal/eino/types.go @@ -0,0 +1,38 @@ +// Package eino 基于 CloudWeGo Eino 框架的 AI 编排层。 +// 使用 Eino Graph 替代手写 goroutine 管道，实现声明式 STT → LLM → TTS 编排。 +package eino + +// PipelineInput Graph 统一输入。 +type PipelineInput struct { + AudioData []byte // base64 解码后的音频（可选） + ImageData []byte // base64 解码后的图像（可选） + Text string // 直接文本输入（可选，跳过 STT） + SessionID string + RequestID string + Language string // zh / en + Scenario string // free_chat, interviewer, etc. + TTSEnabled bool + UserID string // 用户 ID，用于加载自建情景 +} + +// PipelineOutput Graph 统一输出。 +type PipelineOutput struct { + TranscribedText string // STT 结果 + FullResponse string // LLM 完整回复 + Model string // 实际使用的模型名 + TokenUsage *TokenUsage // token 用量 +} + +// STTOutput STT 节点输出。 +type STTOutput struct { + Text string + Language string + IsSkipped bool // 文本输入模式跳过了 STT +} + +// TokenUsage token 用量统计。 +type TokenUsage struct { + Prompt int + Completion int + Total int +} diff --git a/backend/internal/models/user_scenario.go b/backend/internal/models/user_scenario.go new file mode 100644 index 0000000..9edaa55 --- /dev/null +++ b/backend/internal/models/user_scenario.go @@ -0,0 +1,43 @@ +package models + +import "time" + +// UserScenario 用户自建情景。 +type UserScenario struct { + ID string `json:"id"` + UserID string `json:"user_id"` + Name string `json:"name"` + Icon string `json:"icon"` + Description string `json:"description"` + Prompt string `json:"prompt"` + Greeting string `json:"greeting,omitempty"` + Language string `json:"language"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} + +// CreateUserScenarioRequest 创建用户情景请求。 +type CreateUserScenarioRequest struct { + Name string `json:"name" binding:"required,min=2,max=50"` + Icon string `json:"icon,omitempty"` + Description string `json:"description,omitempty" binding:"omitempty,max=100"` + Prompt string `json:"prompt" binding:"required,min=10,max=2000"` + Greeting string `json:"greeting,omitempty" binding:"omitempty,max=500"` + Language string `json:"language,omitempty"` +} + +// UpdateUserScenarioRequest 更新用户情景请求。 +type UpdateUserScenarioRequest struct { + Name *string `json:"name,omitempty" binding:"omitempty,min=2,max=50"` + Icon *string `json:"icon,omitempty"` + Description *string `json:"description,omitempty" binding:"omitempty,max=100"` + Prompt *string `json:"prompt,omitempty" binding:"omitempty,min=10,max=2000"` + Greeting *string `json:"greeting,omitempty" binding:"omitempty,max=500"` + Language *string `json:"language,omitempty"` +} + +// UserScenarioListResponse 用户情景列表响应。 +type UserScenarioListResponse struct { + Scenarios []*UserScenario `json:"scenarios"` + Total int `json:"total"` +} diff --git a/backend/internal/orchestrator/orchestrator.go b/backend/internal/orchestrator/orchestrator.go index d038603..d15e8bf 100644 --- a/backend/internal/orchestrator/orchestrator.go +++ b/backend/internal/orchestrator/orchestrator.go @@ -14,13 +14,11 @@ type Orchestrator interface { // ctx 用于整体超时和中断控制。 // sessionID 用于会话管理和历史获取。 // req 包含图像和音频数据。 - // history 是最近的对话历史。 // sender 用于向客户端推送消息。 ProcessQuery( ctx context.Context, sessionID string, req models.WsQuery, - history []models.Message, sender Sender, ) error } diff --git a/backend/internal/orchestrator/pipeline.go b/backend/internal/orchestrator/pipeline.go deleted file mode 100644 index 5ed94e6..0000000 --- a/backend/internal/orchestrator/pipeline.go +++ /dev/null @@ -1,403 +0,0 @@ -package orchestrator - -import ( - "context" - "encoding/base64" - "strings" - "sync" - "time" - "unicode/utf8" - - "github.com/hhs/camtalk/internal/ai/llm" - "github.com/hhs/camtalk/internal/ai/stt" - "github.com/hhs/camtalk/internal/ai/tts" - "github.com/hhs/camtalk/internal/config" - "github.com/hhs/camtalk/internal/logger" - "github.com/hhs/camtalk/internal/models" - "github.com/hhs/camtalk/internal/session" -) - -// Pipeline 实现 Orchestrator 接口，管理 STT → LLM → TTS 流式管道。 -type Pipeline struct { - sttService stt.Service - llmService llm.Service - ttsService tts.Service - sessionMgr session.Manager - model string // LLM 模型名，用于 llm_done 上报 - ttsVoice string // TTS 音色 - ttsSpeed float64 // TTS 语速 - ttsOutputFmt string // TTS 输出格式 - ttsSampleRate int // TTS 输出采样率 -} - -// New 创建 Pipeline 实例。 -func New( - sttService stt.Service, - llmService llm.Service, - ttsService tts.Service, - sessionMgr session.Manager, - cfg *config.Config, -) *Pipeline { - return &Pipeline{ - sttService: sttService, - llmService: llmService, - ttsService: ttsService, - sessionMgr: sessionMgr, - model: cfg.AI.LLM.Model, - ttsVoice: cfg.AI.TTS.Voice, - ttsSpeed: cfg.AI.TTS.Speed, - ttsOutputFmt: cfg.AI.TTS.OutputFormat, - ttsSampleRate: cfg.AI.TTS.SampleRate, - } -} - -// ProcessQuery 实现 Orchestrator 接口。 -func (p *Pipeline) ProcessQuery( - ctx context.Context, - sessionID string, - req models.WsQuery, - history []models.Message, - sender Sender, -) error { - log := logger.Log - startTime := time.Now() - - // 解码音频数据（文本输入模式可跳过） - var audio []byte - if req.Text == "" && req.Audio != "" { - var err error - audio, err = base64.StdEncoding.DecodeString(req.Audio) - if err != nil { - log.Errorw("音频解码失败", "error", err) - sender.SendError(models.WsError{ - Type: "error", - RequestID: req.RequestID, - Code: "INVALID_MESSAGE", - Message: "音频数据解码失败", - }) - return err - } - } - - // 解码图片数据（可选） - var image []byte - if req.Image != "" { - var err error - image, err = base64.StdEncoding.DecodeString(req.Image) - if err != nil { - log.Errorw("图片解码失败", "error", err) - sender.SendError(models.WsError{ - Type: "error", - RequestID: req.RequestID, - Code: "INVALID_MESSAGE", - Message: "图片数据解码失败", - }) - return err - } - } - - // 设置活跃请求 - if err := p.sessionMgr.SetActiveRequest(ctx, sessionID, req.RequestID); err != nil { - log.Errorw("设置活跃请求失败", "error", err) - } - defer p.sessionMgr.ClearActiveRequest(ctx, sessionID) - - // 获取会话配置 - sess, err := p.sessionMgr.Get(ctx, sessionID) - if err != nil { - log.Errorw("获取会话失败", "error", err) - sender.SendError(models.WsError{ - Type: "error", - RequestID: req.RequestID, - Code: "SESSION_NOT_FOUND", - Message: "会话不存在", - }) - return err - } - - // Step 1: 获取用户文本（语音识别或直接使用输入文本） - var userText string - if req.Text != "" { - // 文本输入模式：跳过 STT，直接使用用户输入的文本 - log.Infow("使用文本输入", "request_id", req.RequestID, "text", req.Text) - userText = req.Text - - // 发送 stt_result 以保持前端消息流一致性 - if err := sender.SendSTTResult(models.WsSTTResult{ - Type: "stt_result", - RequestID: req.RequestID, - Text: userText, - IsFinal: true, - }); err != nil { - log.Errorw("发送 STT 结果失败", "error", err) - } - } else { - // 语音模式：执行 STT 语音识别 - log.Infow("开始语音识别", "request_id", req.RequestID, "audio_bytes", len(audio)) - sttResult, err := p.sttService.Recognize(ctx, audio, stt.Options{ - Encoding: "pcm_s16le", - SampleRate: 16000, - Language: sess.Config.Language, - }) - if err != nil { - log.Errorw("语音识别失败", "error", err, "audio_bytes", len(audio)) - sender.SendError(models.WsError{ - Type: "error", - RequestID: req.RequestID, - Code: "STT_ERROR", - Message: "语音识别失败: " + err.Error(), - }) - return err - } - userText = sttResult - - // STT 返回空文本：未识别到语音，发送结果后直接返回（不调 LLM） - if strings.TrimSpace(userText) == "" { - log.Infow("语音识别结果为空", "request_id", req.RequestID) - userText = "（未识别到语音）" - if err := sender.SendSTTResult(models.WsSTTResult{ - Type: "stt_result", - RequestID: req.RequestID, - Text: userText, - IsFinal: true, - }); err != nil { - log.Errorw("发送 STT 结果失败", "error", err) - } - // 发送空的 llm_done 以结束本轮处理 - latency := time.Since(startTime).Milliseconds() - _ = sender.SendLLMDone(models.WsLLMDone{ - Type: "llm_done", - RequestID: req.RequestID, - FullText: "", - Model: p.model, - LatencyMs: latency, - }) - return nil - } - - // 发送 STT 结果 - if err := sender.SendSTTResult(models.WsSTTResult{ - Type: "stt_result", - RequestID: req.RequestID, - Text: userText, - IsFinal: true, - }); err != nil { - log.Errorw("发送 STT 结果失败", "error", err) - } - } - - // 追加用户消息到历史 - p.sessionMgr.AppendMessage(ctx, sessionID, models.Message{ - Role: "user", - Content: userText, - }) - - // Step 2+3: LLM 流式推理 + TTS 并行合成 - log.Infow("开始 LLM 推理", "request_id", req.RequestID, "scenario", sess.Config.Scenario) - llmReq := llm.Request{ - Image: image, - Text: userText, - History: history, - Language: sess.Config.Language, - SystemPrompt: llm.GetScenarioPrompt(sess.Config.Scenario, sess.Config.Language), - } - - llmStream, err := p.llmService.ChatStream(ctx, llmReq) - if err != nil { - log.Errorw("LLM 流式推理启动失败", "error", err) - sender.SendError(models.WsError{ - Type: "error", - RequestID: req.RequestID, - Code: "LLM_ERROR", - Message: "LLM 推理失败", - }) - return err - } - - // 创建句子切分器 - sentenceCh := make(chan string, 4) - splitter := NewSplitter(sentenceCh) - - // 并行：LLM 消费 + TTS 合成 - var wg sync.WaitGroup - var fullText string - var ttsErr error - - // goroutine 1: 消费 LLM token + 句子切分 - var tokenUsage *llm.TokenUsage - wg.Add(1) - go func() { - defer wg.Done() - defer close(sentenceCh) - fullText, tokenUsage = p.consumeLLMStream(ctx, llmStream, req.RequestID, sender, splitter) - }() - - // goroutine 2: TTS 合成（如果启用） - if sess.Config.TTSEnabled { - wg.Add(1) - go func() { - defer wg.Done() - log.Infow("开始 TTS 合成", "request_id", req.RequestID) - ttsErr = p.synthesizeTTS(ctx, sentenceCh, req.RequestID, sender) - }() - } else { - // 如果 TTS 未启用，需要消费 sentenceCh 防止阻塞 - go func() { - for range sentenceCh { - } - }() - } - - // 等待所有 goroutine 完成 - wg.Wait() - - // TTS 失败静默跳过 - if ttsErr != nil { - log.Warnw("TTS 合成失败（已跳过）", "error", ttsErr) - } - - // 追加助手消息到历史 - p.sessionMgr.AppendMessage(ctx, sessionID, models.Message{ - Role: "assistant", - Content: fullText, - }) - - // 发送 llm_done - latency := time.Since(startTime).Milliseconds() - done := models.WsLLMDone{ - Type: "llm_done", - RequestID: req.RequestID, - FullText: fullText, - Model: p.model, - LatencyMs: latency, - } - if tokenUsage != nil { - done.TokensUsed = struct { - Prompt int `json:"prompt"` - Completion int `json:"completion"` - Total int `json:"total"` - }{ - Prompt: tokenUsage.Prompt, - Completion: tokenUsage.Completion, - Total: tokenUsage.Total, - } - } - if err := sender.SendLLMDone(done); err != nil { - log.Errorw("发送 llm_done 失败", "error", err) - } - - log.Infow("查询处理完成", - "request_id", req.RequestID, - "latency_ms", latency, - "text_length", utf8.RuneCountInString(fullText), - ) - - return nil -} - -// consumeLLMStream 消费 LLM 流式输出，发送 llm_chunk 并进行句子切分。 -// 返回完整文本和 token 用量。 -func (p *Pipeline) consumeLLMStream( - ctx context.Context, - stream <-chan llm.Chunk, - requestID string, - sender Sender, - splitter *Splitter, -) (string, *llm.TokenUsage) { - log := logger.Log - var fullText strings.Builder - var tokenUsage *llm.TokenUsage - - for chunk := range stream { - // 检查上下文是否已取消 - select { - case <-ctx.Done(): - log.Infow("LLM 流被中断", "request_id", requestID) - return fullText.String(), tokenUsage - default: - } - - if chunk.Done { - // 流结束，记录 token 用量 - if chunk.TokensUsed != nil { - tokenUsage = chunk.TokensUsed - log.Infow("LLM 用量统计", - "request_id", requestID, - "prompt_tokens", tokenUsage.Prompt, - "completion_tokens", tokenUsage.Completion, - "total_tokens", tokenUsage.Total, - ) - } - break - } - - // 累积全文 - fullText.WriteString(chunk.Delta) - - // 发送 llm_chunk - if err := sender.SendLLMChunk(models.WsLLMChunk{ - Type: "llm_chunk", - RequestID: requestID, - Delta: chunk.Delta, - Role: "assistant", - }); err != nil { - log.Errorw("发送 llm_chunk 失败", "error", err) - } - - // 句子切分 - splitter.Feed(chunk.Delta) - } - - // 刷新切分器中的剩余文本 - splitter.Flush() - - return fullText.String(), tokenUsage -} - -// synthesizeTTS 从句子 channel 读取文本，进行 TTS 合成并发送音频。 -func (p *Pipeline) synthesizeTTS( - ctx context.Context, - sentenceCh <-chan string, - requestID string, - sender Sender, -) error { - log := logger.Log - - ttsStream, err := p.ttsService.SynthesizeStream(ctx, sentenceCh, tts.Options{ - Voice: p.ttsVoice, - Speed: p.ttsSpeed, - OutputFmt: p.ttsOutputFmt, - SampleRate: p.ttsSampleRate, - }) - if err != nil { - log.Errorw("TTS 合成启动失败", "error", err) - return err - } - - // 消费 TTS 音频流 - for chunk := range ttsStream { - // 检查上下文是否已取消 - select { - case <-ctx.Done(): - log.Infow("TTS 流被中断", "request_id", requestID) - return ctx.Err() - default: - } - - // Base64 编码音频数据 - audioBase64 := base64.StdEncoding.EncodeToString(chunk.Audio) - - if err := sender.SendTTSAudio(models.WsTTSAudio{ - Type: "tts_audio", - RequestID: requestID, - Audio: audioBase64, - MimeType: "audio/mp3", - IsLast: chunk.IsLast, - Final: chunk.Final, - }); err != nil { - log.Errorw("发送 tts_audio 失败", "error", err) - } - } - - return nil -} diff --git a/backend/internal/orchestrator/pipeline_test.go b/backend/internal/orchestrator/pipeline_test.go deleted file mode 100644 index 9000e96..0000000 --- a/backend/internal/orchestrator/pipeline_test.go +++ /dev/null @@ -1,713 +0,0 @@ -package orchestrator - -import ( - "context" - "encoding/base64" - "errors" - "testing" - "time" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/mock" - - "github.com/hhs/camtalk/internal/ai/llm" - "github.com/hhs/camtalk/internal/ai/stt" - "github.com/hhs/camtalk/internal/ai/tts" - "github.com/hhs/camtalk/internal/config" - "github.com/hhs/camtalk/internal/logger" - "github.com/hhs/camtalk/internal/models" - "github.com/hhs/camtalk/internal/session" -) - -func init() { - logger.Init("debug", "console") -} - -// MockSTTService mock STT 服务 -type MockSTTService struct { - mock.Mock -} - -func (m *MockSTTService) Recognize(ctx context.Context, audio []byte, opts stt.Options) (string, error) { - args := m.Called(ctx, audio, opts) - return args.String(0), args.Error(1) -} - -// MockLLMService mock LLM 服务 -type MockLLMService struct { - mock.Mock -} - -func (m *MockLLMService) ChatStream(ctx context.Context, req llm.Request) (<-chan llm.Chunk, error) { - args := m.Called(ctx, req) - if args.Get(0) == nil { - return nil, args.Error(1) - } - return args.Get(0).(<-chan llm.Chunk), args.Error(1) -} - -// MockTTSService mock TTS 服务 -type MockTTSService struct { - mock.Mock -} - -func (m *MockTTSService) SynthesizeStream(ctx context.Context, textStream <-chan string, opts tts.Options) (<-chan tts.Chunk, error) { - args := m.Called(ctx, textStream, opts) - if args.Get(0) == nil { - return nil, args.Error(1) - } - return args.Get(0).(<-chan tts.Chunk), args.Error(1) -} - -// MockSessionManager mock 会话管理器 -type MockSessionManager struct { - mock.Mock -} - -func (m *MockSessionManager) Create(ctx context.Context, userID string, config models.SessionConfig) (string, error) { - args := m.Called(ctx, userID, config) - return args.String(0), args.Error(1) -} - -func (m *MockSessionManager) UpdateTitle(ctx context.Context, sessionID string, title string) error { - args := m.Called(ctx, sessionID, title) - return args.Error(0) -} - -func (m *MockSessionManager) ListByUser(ctx context.Context, userID string, page, size int) ([]session.ConversationSummary, int, error) { - args := m.Called(ctx, userID, page, size) - return args.Get(0).([]session.ConversationSummary), args.Int(1), args.Error(2) -} - -func (m *MockSessionManager) Get(ctx context.Context, sessionID string) (*models.Session, error) { - args := m.Called(ctx, sessionID) - if args.Get(0) == nil { - return nil, args.Error(1) - } - return args.Get(0).(*models.Session), args.Error(1) -} - -func (m *MockSessionManager) UpdateConfig(ctx context.Context, sessionID string, patch models.SessionConfigPatch) error { - args := m.Called(ctx, sessionID, patch) - return args.Error(0) -} - -func (m *MockSessionManager) GetHistory(ctx context.Context, sessionID string, limit int) ([]models.Message, error) { - args := m.Called(ctx, sessionID, limit) - return args.Get(0).([]models.Message), args.Error(1) -} - -func (m *MockSessionManager) AppendMessage(ctx context.Context, sessionID string, msg models.Message) error { - args := m.Called(ctx, sessionID, msg) - return args.Error(0) -} - -func (m *MockSessionManager) SetActiveRequest(ctx context.Context, sessionID string, requestID string) error { - args := m.Called(ctx, sessionID, requestID) - return args.Error(0) -} - -func (m *MockSessionManager) GetActiveRequestID(ctx context.Context, sessionID string) (string, error) { - args := m.Called(ctx, sessionID) - return args.String(0), args.Error(1) -} - -func (m *MockSessionManager) ClearActiveRequest(ctx context.Context, sessionID string) error { - args := m.Called(ctx, sessionID) - return args.Error(0) -} - -func (m *MockSessionManager) Touch(ctx context.Context, sessionID string) error { - args := m.Called(ctx, sessionID) - return args.Error(0) -} - -func (m *MockSessionManager) Destroy(ctx context.Context, sessionID string) error { - args := m.Called(ctx, sessionID) - return args.Error(0) -} - -func (m *MockSessionManager) ActiveCount() int { - args := m.Called() - return args.Int(0) -} - -// MockSender mock WebSocket 发送器 -type MockSender struct { - mock.Mock - STTResults []models.WsSTTResult - LLMChunks []models.WsLLMChunk - LLMDones []models.WsLLMDone - TTSAudios []models.WsTTSAudio - Errors []models.WsError -} - -func NewMockSender() *MockSender { - return &MockSender{ - STTResults: make([]models.WsSTTResult, 0), - LLMChunks: make([]models.WsLLMChunk, 0), - LLMDones: make([]models.WsLLMDone, 0), - TTSAudios: make([]models.WsTTSAudio, 0), - Errors: make([]models.WsError, 0), - } -} - -func (m *MockSender) SendSTTResult(result models.WsSTTResult) error { - m.STTResults = append(m.STTResults, result) - args := m.Called(result) - return args.Error(0) -} - -func (m *MockSender) SendLLMChunk(chunk models.WsLLMChunk) error { - m.LLMChunks = append(m.LLMChunks, chunk) - args := m.Called(chunk) - return args.Error(0) -} - -func (m *MockSender) SendLLMDone(done models.WsLLMDone) error { - m.LLMDones = append(m.LLMDones, done) - args := m.Called(done) - return args.Error(0) -} - -func (m *MockSender) SendTTSAudio(audio models.WsTTSAudio) error { - m.TTSAudios = append(m.TTSAudios, audio) - args := m.Called(audio) - return args.Error(0) -} - -func (m *MockSender) SendError(err models.WsError) error { - m.Errors = append(m.Errors, err) - args := m.Called(err) - return args.Error(0) -} - -// 辅助函数：创建 LLM 流式响应 -func createLLMStream(chunks []llm.Chunk) <-chan llm.Chunk { - ch := make(chan llm.Chunk, len(chunks)) - for _, chunk := range chunks { - ch <- chunk - } - close(ch) - return ch -} - -// 辅助函数：创建 TTS 流式响应 -func createTTSStream(chunks []tts.Chunk) <-chan tts.Chunk { - ch := make(chan tts.Chunk, len(chunks)) - for _, chunk := range chunks { - ch <- chunk - } - close(ch) - return ch -} - -// TestProcessQuery_Success 测试完整流程 -func TestProcessQuery_Success(t *testing.T) { - // 准备测试数据 - audioData := []byte("test audio") - imageData := []byte("test image") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - imageBase64 := base64.StdEncoding.EncodeToString(imageData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Image: imageBase64, - Audio: audioBase64, - } - - session := &models.Session{ - ID: "session-123", - Config: models.SessionConfig{ - TTSEnabled: true, - DetailLevel: "low", - Language: "zh-CN", - }, - } - - // 创建 mock - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - // 设置 mock 期望 - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(session, nil) - mockSession.On("AppendMessage", mock.Anything, "session-123", mock.Anything).Return(nil) - - mockSTT.On("Recognize", mock.Anything, audioData, stt.Options{ - Encoding: "pcm_s16le", - SampleRate: 16000, - Language: "zh-CN", - }).Return("你好，世界", nil) - - mockSender.On("SendSTTResult", mock.Anything).Return(nil) - - llmChunks := []llm.Chunk{ - {Delta: "你好"}, - {Delta: "，世界！"}, - {Done: true, TokensUsed: &llm.TokenUsage{Prompt: 10, Completion: 5, Total: 15}}, - } - mockLLM.On("ChatStream", mock.Anything, mock.Anything).Return(createLLMStream(llmChunks), nil) - - mockSender.On("SendLLMChunk", mock.Anything).Return(nil) - mockSender.On("SendLLMDone", mock.Anything).Return(nil) - - ttsChunks := []tts.Chunk{ - {Audio: []byte("audio1"), IsLast: false}, - {Audio: []byte("audio2"), IsLast: true}, - } - mockTTS.On("SynthesizeStream", mock.Anything, mock.Anything, mock.Anything).Return(createTTSStream(ttsChunks), nil) - - mockSender.On("SendTTSAudio", mock.Anything).Return(nil) - - // 创建 Pipeline - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - // 执行 - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - // 验证 - assert.NoError(t, err) - assert.Len(t, mockSender.STTResults, 1) - assert.Equal(t, "你好，世界", mockSender.STTResults[0].Text) - assert.Len(t, mockSender.LLMChunks, 2) - assert.Len(t, mockSender.LLMDones, 1) - assert.Len(t, mockSender.TTSAudios, 2) - - mockSTT.AssertExpectations(t) - mockLLM.AssertExpectations(t) - mockTTS.AssertExpectations(t) - mockSession.AssertExpectations(t) -} - -// TestProcessQuery_STTError 测试 STT 失败降级 -func TestProcessQuery_STTError(t *testing.T) { - audioData := []byte("test audio") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: audioBase64, - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(&models.Session{ - ID: "session-123", - Config: models.SessionConfig{ - Language: "zh-CN", - }, - }, nil) - - mockSTT.On("Recognize", mock.Anything, audioData, mock.Anything). - Return("", errors.New("STT service unavailable")) - - mockSender.On("SendError", mock.Anything).Return(nil) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - assert.Error(t, err) - assert.Len(t, mockSender.Errors, 1) - assert.Equal(t, "STT_ERROR", mockSender.Errors[0].Code) - - mockSTT.AssertExpectations(t) - mockLLM.AssertNotCalled(t, "ChatStream") - mockTTS.AssertNotCalled(t, "SynthesizeStream") -} - -// TestProcessQuery_LLMError 测试 LLM 失败降级 -func TestProcessQuery_LLMError(t *testing.T) { - audioData := []byte("test audio") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: audioBase64, - } - - session := &models.Session{ - ID: "session-123", - Config: models.SessionConfig{ - TTSEnabled: true, - Language: "zh-CN", - }, - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(session, nil) - mockSession.On("AppendMessage", mock.Anything, "session-123", mock.Anything).Return(nil) - - mockSTT.On("Recognize", mock.Anything, audioData, mock.Anything).Return("你好", nil) - mockSender.On("SendSTTResult", mock.Anything).Return(nil) - - mockLLM.On("ChatStream", mock.Anything, mock.Anything). - Return(nil, errors.New("LLM service unavailable")) - - mockSender.On("SendError", mock.Anything).Return(nil) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - assert.Error(t, err) - assert.Len(t, mockSender.Errors, 1) - assert.Equal(t, "LLM_ERROR", mockSender.Errors[0].Code) - - mockSTT.AssertExpectations(t) - mockLLM.AssertExpectations(t) - mockTTS.AssertNotCalled(t, "SynthesizeStream") -} - -// TestProcessQuery_TTSError 测试 TTS 失败静默跳过 -func TestProcessQuery_TTSError(t *testing.T) { - audioData := []byte("test audio") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: audioBase64, - } - - session := &models.Session{ - ID: "session-123", - Config: models.SessionConfig{ - TTSEnabled: true, - Language: "zh-CN", - }, - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(session, nil) - mockSession.On("AppendMessage", mock.Anything, "session-123", mock.Anything).Return(nil) - - mockSTT.On("Recognize", mock.Anything, audioData, mock.Anything).Return("你好", nil) - mockSender.On("SendSTTResult", mock.Anything).Return(nil) - - llmChunks := []llm.Chunk{ - {Delta: "你好"}, - {Done: true}, - } - mockLLM.On("ChatStream", mock.Anything, mock.Anything).Return(createLLMStream(llmChunks), nil) - mockSender.On("SendLLMChunk", mock.Anything).Return(nil) - mockSender.On("SendLLMDone", mock.Anything).Return(nil) - - mockTTS.On("SynthesizeStream", mock.Anything, mock.Anything, mock.Anything). - Return(nil, errors.New("TTS service unavailable")) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - // TTS 失败应该静默跳过，不返回错误 - assert.NoError(t, err) - assert.Len(t, mockSender.LLMDones, 1) - assert.Len(t, mockSender.TTSAudios, 0) - - mockSTT.AssertExpectations(t) - mockLLM.AssertExpectations(t) - mockTTS.AssertExpectations(t) -} - -// TestProcessQuery_ContextCancelled 测试上下文取消（Interrupt） -func TestProcessQuery_ContextCancelled(t *testing.T) { - audioData := []byte("test audio") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: audioBase64, - } - - session := &models.Session{ - ID: "session-123", - Config: models.SessionConfig{ - TTSEnabled: true, - Language: "zh-CN", - }, - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(session, nil) - mockSession.On("AppendMessage", mock.Anything, "session-123", mock.Anything).Return(nil) - - mockSTT.On("Recognize", mock.Anything, audioData, mock.Anything).Return("你好", nil) - mockSender.On("SendSTTResult", mock.Anything).Return(nil) - - // 创建一个会延迟的 LLM 流，以便我们可以取消上下文 - llmCh := make(chan llm.Chunk) - go func() { - time.Sleep(100 * time.Millisecond) - llmCh <- llm.Chunk{Delta: "你"} - time.Sleep(100 * time.Millisecond) - llmCh <- llm.Chunk{Delta: "好"} - close(llmCh) - }() - - mockLLM.On("ChatStream", mock.Anything, mock.Anything).Return((<-chan llm.Chunk)(llmCh), nil) - mockSender.On("SendLLMChunk", mock.Anything).Return(nil) - mockSender.On("SendLLMDone", mock.Anything).Return(nil) - - // 创建一个会延迟的 TTS 流 - ttsCh := make(chan tts.Chunk) - go func() { - time.Sleep(200 * time.Millisecond) - close(ttsCh) - }() - mockTTS.On("SynthesizeStream", mock.Anything, mock.Anything, mock.Anything).Return((<-chan tts.Chunk)(ttsCh), nil) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - // 创建可取消的上下文 - ctx, cancel := context.WithCancel(context.Background()) - - // 在 50ms 后取消 - go func() { - time.Sleep(50 * time.Millisecond) - cancel() - }() - - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - // 上下文取消后，流程应该正常完成（中断流但不返回错误） - assert.NoError(t, err) - - mockSTT.AssertExpectations(t) -} - -// TestProcessQuery_DisabledTTS 测试 TTS 未启用的情况 -func TestProcessQuery_DisabledTTS(t *testing.T) { - audioData := []byte("test audio") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: audioBase64, - } - - session := &models.Session{ - ID: "session-123", - Config: models.SessionConfig{ - TTSEnabled: false, // TTS 未启用 - Language: "zh-CN", - }, - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(session, nil) - mockSession.On("AppendMessage", mock.Anything, "session-123", mock.Anything).Return(nil) - - mockSTT.On("Recognize", mock.Anything, audioData, mock.Anything).Return("你好", nil) - mockSender.On("SendSTTResult", mock.Anything).Return(nil) - - llmChunks := []llm.Chunk{ - {Delta: "你好"}, - {Done: true}, - } - mockLLM.On("ChatStream", mock.Anything, mock.Anything).Return(createLLMStream(llmChunks), nil) - mockSender.On("SendLLMChunk", mock.Anything).Return(nil) - mockSender.On("SendLLMDone", mock.Anything).Return(nil) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - assert.NoError(t, err) - assert.Len(t, mockSender.LLMDones, 1) - assert.Len(t, mockSender.TTSAudios, 0) - - // TTS 不应该被调用 - mockTTS.AssertNotCalled(t, "SynthesizeStream") -} - -// TestSplitter 测试句子切分器 -func TestSplitter(t *testing.T) { - ch := make(chan string, 10) - splitter := NewSplitter(ch) - - // 输入包含多个句子的文本 - splitter.Feed("你好。") - splitter.Feed("世界！") - splitter.Feed("这是") - splitter.Feed("一个测试。") - splitter.Flush() - - // 应该有 3 个句子 - assert.Equal(t, 3, len(ch)) - assert.Equal(t, "你好。", <-ch) - assert.Equal(t, "世界！", <-ch) - assert.Equal(t, "这是一个测试。", <-ch) -} - -// TestSplitter_NoDelimiter 测试没有分隔符的情况 -func TestSplitter_NoDelimiter(t *testing.T) { - ch := make(chan string, 10) - splitter := NewSplitter(ch) - - splitter.Feed("没有分隔符的文本") - splitter.Flush() - - // 应该有 1 个句子（Flush 会发送剩余内容） - assert.Equal(t, 1, len(ch)) - assert.Equal(t, "没有分隔符的文本", <-ch) -} - -// TestSplitter_Empty 测试空输入 -func TestSplitter_Empty(t *testing.T) { - ch := make(chan string, 10) - splitter := NewSplitter(ch) - - splitter.Flush() - - // 应该没有句子 - assert.Equal(t, 0, len(ch)) -} - -// TestProcessQuery_InvalidAudio 测试无效音频数据 -func TestProcessQuery_InvalidAudio(t *testing.T) { - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: "invalid-base64!!!", - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSender.On("SendError", mock.Anything).Return(nil) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - assert.Error(t, err) - assert.Len(t, mockSender.Errors, 1) - assert.Equal(t, "INVALID_MESSAGE", mockSender.Errors[0].Code) -} - -// TestProcessQuery_SessionNotFound 测试会话不存在 -func TestProcessQuery_SessionNotFound(t *testing.T) { - audioData := []byte("test audio") - audioBase64 := base64.StdEncoding.EncodeToString(audioData) - - req := models.WsQuery{ - Type: "query", - RequestID: "req-123", - Audio: audioBase64, - } - - mockSTT := new(MockSTTService) - mockLLM := new(MockLLMService) - mockTTS := new(MockTTSService) - mockSession := new(MockSessionManager) - mockSender := NewMockSender() - - mockSession.On("SetActiveRequest", mock.Anything, "session-123", "req-123").Return(nil) - mockSession.On("ClearActiveRequest", mock.Anything, "session-123").Return(nil) - mockSession.On("Get", mock.Anything, "session-123").Return(nil, errors.New("session not found")) - - mockSender.On("SendError", mock.Anything).Return(nil) - - pipeline := New(mockSTT, mockLLM, mockTTS, mockSession, &config.Config{ - AI: config.AIConfig{ - LLM: config.LLMConfig{Model: "gpt-4o"}, - TTS: config.TTSConfig{Voice: "alloy", Speed: 1.0, OutputFormat: "mp3", SampleRate: 24000}, - }, - }) - - ctx := context.Background() - err := pipeline.ProcessQuery(ctx, "session-123", req, nil, mockSender) - - assert.Error(t, err) - assert.Len(t, mockSender.Errors, 1) - assert.Equal(t, "SESSION_NOT_FOUND", mockSender.Errors[0].Code) -} diff --git a/backend/internal/orchestrator/splitter.go b/backend/internal/orchestrator/splitter.go deleted file mode 100644 index 57b8b26..0000000 --- a/backend/internal/orchestrator/splitter.go +++ /dev/null @@ -1,55 +0,0 @@ -package orchestrator - -import "strings" - -// sentenceDelimiters 句子分隔符集合。 -var sentenceDelimiters = map[rune]bool{ - '。': true, - '！': true, - '？': true, - '\n': true, - '.': true, - '!': true, - '?': true, -} - -// Splitter 句子切分器。 -// 将流式文本按句子边界切分，发送到 channel 供 TTS 合成。 -type Splitter struct { - ch chan<- string - buffer strings.Builder -} - -// NewSplitter 创建句子切分器。 -// ch 用于接收切分后的句子文本。 -func NewSplitter(ch chan<- string) *Splitter { - return &Splitter{ - ch: ch, - } -} - -// Feed 输入增量文本，遇到句子分隔符时发送完整句子。 -func (s *Splitter) Feed(delta string) { - for _, r := range delta { - s.buffer.WriteRune(r) - if sentenceDelimiters[r] { - s.flushBuffer() - } - } -} - -// Flush 刷新缓冲区中的剩余文本（即使没有句子分隔符）。 -func (s *Splitter) Flush() { - if s.buffer.Len() > 0 { - s.flushBuffer() - } -} - -// flushBuffer 将缓冲区内容发送到 channel 并清空。 -func (s *Splitter) flushBuffer() { - text := strings.TrimSpace(s.buffer.String()) - if text != "" { - s.ch <- text - } - s.buffer.Reset() -} diff --git a/backend/internal/ratelimit/bucket.go b/backend/internal/ratelimit/bucket.go new file mode 100644 index 0000000..308336f --- /dev/null +++ b/backend/internal/ratelimit/bucket.go @@ -0,0 +1,172 @@ +package ratelimit + +import ( + "context" + "sync" + "time" + + "github.com/hhs/camtalk/internal/config" +) + +// TokenBucket 内存令牌桶，适用于单实例部署。 +type TokenBucket struct { + capacity int // 桶容量 + rate float64 // 每秒填充令牌数 + tokens float64 // 当前令牌数 + lastRefill time.Time // 上次填充时间 + mu sync.Mutex +} + +// newTokenBucket 创建令牌桶。 +func newTokenBucket(capacity int, rate float64) *TokenBucket { + return &TokenBucket{ + capacity: capacity, + rate: rate, + tokens: float64(capacity), // 初始满桶 + lastRefill: time.Now(), + } +} + +// allow 尝试消耗一个令牌。 +func (b *TokenBucket) allow() (bool, time.Duration) { + b.mu.Lock() + defer b.mu.Unlock() + + now := time.Now() + elapsed := now.Sub(b.lastRefill).Seconds() + + // 补充令牌 + newTokens := elapsed * b.rate + b.tokens = min(float64(b.capacity), b.tokens+newTokens) + b.lastRefill = now + + // 尝试消耗一个令牌 + if b.tokens >= 1 { + b.tokens -= 1 + return true, 0 + } + + // 计算需要等待的时间 + if b.rate == 0 { + // rate=0 时永远无法补充令牌 + return false, 24 * time.Hour // 返回一个很大的值 + } + retryAfter := time.Duration((1-b.tokens)/b.rate*1000) * time.Millisecond + return false, retryAfter +} + +// MemoryLimiter 管理多个用户的令牌桶。 +type MemoryLimiter struct { + buckets map[string]*TokenBucket + config config.RateLimitConfig + mu sync.RWMutex + stopOnce sync.Once + done chan struct{} +} + +// NewMemoryLimiter 创建内存限流器。 +func NewMemoryLimiter(cfg config.RateLimitConfig) *MemoryLimiter { + limiter := &MemoryLimiter{ + buckets: make(map[string]*TokenBucket), + config: cfg, + done: make(chan struct{}), + } + + // 启动后台清理 goroutine + go limiter.cleanup() + + return limiter +} + +// Allow 实现 Limiter 接口。 +func (l *MemoryLimiter) Allow(ctx context.Context, key string) (bool, time.Duration) { + bucket := l.getOrCreateBucket(key) + return bucket.allow() +} + +// Stop 实现 Limiter 接口。 +func (l *MemoryLimiter) Stop() { + l.stopOnce.Do(func() { + close(l.done) + }) +} + +// getOrCreateBucket 获取或创建令牌桶。 +func (l *MemoryLimiter) getOrCreateBucket(key string) *TokenBucket { + // 先尝试读锁 + l.mu.RLock() + bucket, exists := l.buckets[key] + l.mu.RUnlock() + + if exists { + return bucket + } + + // 需要创建新桶，升级为写锁 + l.mu.Lock() + defer l.mu.Unlock() + + // 双重检查（可能其他 goroutine 已创建） + bucket, exists = l.buckets[key] + if exists { + return bucket + } + + // 根据 key 确定配置（简化版：假设 key 格式为 "userID:action"） + cfg := l.getBucketConfig(key) + bucket = newTokenBucket(cfg.Capacity, cfg.Rate) + l.buckets[key] = bucket + + return bucket +} + +// getBucketConfig 根据 key 获取桶配置。 +func (l *MemoryLimiter) getBucketConfig(key string) config.BucketConfig { + // 简化实现：从 key 后缀判断动作类型 + // 实际使用时调用方会传递正确的 key + // 默认使用 query 配置 + return l.config.Query +} + +// cleanup 定期清理不活跃的桶。 +func (l *MemoryLimiter) cleanup() { + ticker := time.NewTicker(10 * time.Minute) + defer ticker.Stop() + + for { + select { + case <-ticker.C: + l.removeInactiveBuckets() + case <-l.done: + return + } + } +} + +// removeInactiveBuckets 移除超过 10 分钟无活动的桶。 +func (l *MemoryLimiter) removeInactiveBuckets() { + l.mu.Lock() + defer l.mu.Unlock() + + now := time.Now() + for key, bucket := range l.buckets { + bucket.mu.Lock() + inactive := now.Sub(bucket.lastRefill) > 10*time.Minute + bucket.mu.Unlock() + + if inactive { + delete(l.buckets, key) + } + } +} + +// min 返回两个 float64 中的较小值。 +func min(a, b float64) float64 { + if a < b { + return a + } + return b +} + +// 编译期接口检查 +var _ Limiter = (*MemoryLimiter)(nil) diff --git a/backend/internal/ratelimit/bucket_test.go b/backend/internal/ratelimit/bucket_test.go new file mode 100644 index 0000000..ebb19e9 --- /dev/null +++ b/backend/internal/ratelimit/bucket_test.go @@ -0,0 +1,203 @@ +package ratelimit + +import ( + "context" + "sync" + "testing" + "time" + + "github.com/hhs/camtalk/internal/config" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestTokenBucket_Allow_FirstRequest(t *testing.T) { + bucket := newTokenBucket(5, 0.2) + + allowed, retryAfter := bucket.allow() + + assert.True(t, allowed) + assert.Equal(t, time.Duration(0), retryAfter) +} + +func TestTokenBucket_Allow_ConsumeUntilEmpty(t *testing.T) { + bucket := newTokenBucket(3, 0.2) + + // 连续消耗 3 个令牌 + for i := 0; i < 3; i++ { + allowed, _ := bucket.allow() + assert.True(t, allowed, "request %d should be allowed", i+1) + } + + // 第 4 个请求应被拒绝 + allowed, retryAfter := bucket.allow() + assert.False(t, allowed) + assert.Greater(t, retryAfter, time.Duration(0)) +} + +func TestTokenBucket_Allow_RetryAfterCorrect(t *testing.T) { + bucket := newTokenBucket(1, 1.0) // 每秒 1 个令牌 + + // 消耗唯一的令牌 + allowed, _ := bucket.allow() + require.True(t, allowed) + + // 立即再次请求应被拒绝 + allowed, retryAfter := bucket.allow() + assert.False(t, allowed) + // retryAfter 应约为 1 秒（允许一定误差） + assert.InDelta(t, 1000, retryAfter.Milliseconds(), 100) +} + +func TestTokenBucket_Allow_RefillAfterWait(t *testing.T) { + bucket := newTokenBucket(2, 10.0) // 每秒 10 个令牌（每 100ms 一个） + + // 消耗 2 个令牌 + bucket.allow() + bucket.allow() + + // 等待 150ms，应补充至少 1 个令牌 + time.Sleep(150 * time.Millisecond) + + allowed, _ := bucket.allow() + assert.True(t, allowed) +} + +func TestTokenBucket_Allow_CapacityLimit(t *testing.T) { + bucket := newTokenBucket(3, 1.0) + + // 等待足够长时间让桶"溢出" + time.Sleep(100 * time.Millisecond) + + // 但最多只能消耗 capacity 个令牌 + for i := 0; i < 3; i++ { + allowed, _ := bucket.allow() + assert.True(t, allowed, "request %d should be allowed", i+1) + } + + // 第 4 个应被拒绝 + allowed, _ := bucket.allow() + assert.False(t, allowed) +} + +func TestTokenBucket_Allow_ConcurrentSafe(t *testing.T) { + bucket := newTokenBucket(100, 10.0) + var wg sync.WaitGroup + successCount := 0 + var mu sync.Mutex + + // 100 个并发请求 + for i := 0; i < 100; i++ { + wg.Add(1) + go func() { + defer wg.Done() + allowed, _ := bucket.allow() + if allowed { + mu.Lock() + successCount++ + mu.Unlock() + } + }() + } + + wg.Wait() + + // 应该正好 100 个成功（桶容量为 100） + assert.Equal(t, 100, successCount) +} + +func TestTokenBucket_Allow_ZeroCapacity(t *testing.T) { + bucket := newTokenBucket(0, 1.0) + + allowed, retryAfter := bucket.allow() + assert.False(t, allowed) + assert.Greater(t, retryAfter, time.Duration(0)) +} + +func TestTokenBucket_Allow_ZeroRate(t *testing.T) { + bucket := newTokenBucket(1, 0.0) + + // 第一个通过 + allowed, _ := bucket.allow() + assert.True(t, allowed) + + // 第二个被拒绝，且 retryAfter 应为无限大（实际上会很大） + allowed, retryAfter := bucket.allow() + assert.False(t, allowed) + // rate=0 时，retryAfter 理论上无限大，实际会是一个很大的值 + assert.Greater(t, retryAfter, 1*time.Hour) +} + +func TestMemoryLimiter_Allow_DifferentKeys(t *testing.T) { + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 2, Rate: 1.0}, + } + limiter := NewMemoryLimiter(cfg) + defer limiter.Stop() + + ctx := context.Background() + + // user1 消耗 2 个令牌 + allowed, _ := limiter.Allow(ctx, "user1:query") + assert.True(t, allowed) + allowed, _ = limiter.Allow(ctx, "user1:query") + assert.True(t, allowed) + + // user1 第 3 个被拒绝 + allowed, _ = limiter.Allow(ctx, "user1:query") + assert.False(t, allowed) + + // user2 应该不受影响 + allowed, _ = limiter.Allow(ctx, "user2:query") + assert.True(t, allowed) +} + +func TestMemoryLimiter_Cleanup(t *testing.T) { + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 1, Rate: 1.0}, + } + limiter := NewMemoryLimiter(cfg) + defer limiter.Stop() + + ctx := context.Background() + + // 创建一个桶 + limiter.Allow(ctx, "user1:query") + + // 验证桶已创建 + limiter.mu.RLock() + initialCount := len(limiter.buckets) + limiter.mu.RUnlock() + assert.Equal(t, 1, initialCount) + + // 手动触发清理（模拟 10 分钟后） + limiter.mu.Lock() + for _, bucket := range limiter.buckets { + bucket.mu.Lock() + bucket.lastRefill = time.Now().Add(-11 * time.Minute) + bucket.mu.Unlock() + } + limiter.mu.Unlock() + + limiter.removeInactiveBuckets() + + // 验证桶已被清理 + limiter.mu.RLock() + finalCount := len(limiter.buckets) + limiter.mu.RUnlock() + assert.Equal(t, 0, finalCount) +} + +func TestMemoryLimiter_Stop(t *testing.T) { + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 1, Rate: 1.0}, + } + limiter := NewMemoryLimiter(cfg) + + // 多次调用 Stop 不应 panic + limiter.Stop() + limiter.Stop() +} diff --git a/backend/internal/ratelimit/limiter.go b/backend/internal/ratelimit/limiter.go new file mode 100644 index 0000000..5fa4e3c --- /dev/null +++ b/backend/internal/ratelimit/limiter.go @@ -0,0 +1,17 @@ +package ratelimit + +import ( + "context" + "time" +) + +// Limiter 速率限制器接口。 +type Limiter interface { + // Allow 判断 key 是否允许执行一次操作。 + // key 通常为 "userID:action" 格式。 + // 返回 (allowed, retryAfter)。retryAfter 表示需要等待的时间。 + Allow(ctx context.Context, key string) (bool, time.Duration) + + // Stop 停止限流器，清理资源（如后台 goroutine）。 + Stop() +} diff --git a/backend/internal/ratelimit/middleware.go b/backend/internal/ratelimit/middleware.go new file mode 100644 index 0000000..fe8708a --- /dev/null +++ b/backend/internal/ratelimit/middleware.go @@ -0,0 +1,51 @@ +package ratelimit + +import ( + "fmt" + "net/http" + + "github.com/gin-gonic/gin" + + "github.com/hhs/camtalk/internal/trace" +) + +// Middleware 返回 Gin 中间件，按 key 维度限流。 +// keyFunc 从请求中提取限流 key（如 IP、用户 ID）。 +func Middleware(limiter Limiter, keyFunc func(*gin.Context) string) gin.HandlerFunc { + return func(c *gin.Context) { + if limiter == nil { + c.Next() + return + } + + key := keyFunc(c) + if key == "" { + // key 为空时跳过限流 + c.Next() + return + } + + allowed, retryAfter := limiter.Allow(c.Request.Context(), key) + + if !allowed { + log := trace.FromContext(c.Request.Context()) + log.Warnw("rate limited", + "client_ip", c.ClientIP(), + "path", c.Request.URL.Path, + "limit_key", key, + "retry_after_sec", int(retryAfter.Seconds()+0.5)) + + // 设置 Retry-After header（秒） + c.Header("Retry-After", fmt.Sprintf("%d", int(retryAfter.Seconds()+0.5))) + + c.JSON(http.StatusTooManyRequests, gin.H{ + "code": "RATE_LIMITED", + "message": fmt.Sprintf("too many requests, retry after %s", retryAfter.Round(1)), + }) + c.Abort() + return + } + + c.Next() + } +} diff --git a/backend/internal/ratelimit/middleware_test.go b/backend/internal/ratelimit/middleware_test.go new file mode 100644 index 0000000..c9fd953 --- /dev/null +++ b/backend/internal/ratelimit/middleware_test.go @@ -0,0 +1,196 @@ +package ratelimit + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/gin-gonic/gin" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// mockLimiter 用于测试的 mock 限流器。 +type mockLimiter struct { + allowFunc func(ctx context.Context, key string) (bool, time.Duration) +} + +func (m *mockLimiter) Allow(ctx context.Context, key string) (bool, time.Duration) { + if m.allowFunc != nil { + return m.allowFunc(ctx, key) + } + return true, 0 +} + +func (m *mockLimiter) Stop() {} + +// 编译期接口检查 +var _ Limiter = (*mockLimiter)(nil) + +func TestMiddleware_Allow(t *testing.T) { + gin.SetMode(gin.TestMode) + + limiter := &mockLimiter{ + allowFunc: func(ctx context.Context, key string) (bool, time.Duration) { + return true, 0 + }, + } + + router := gin.New() + router.Use(Middleware(limiter, func(c *gin.Context) string { + return "user1:test" + })) + router.GET("/test", func(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) + }) + + req := httptest.NewRequest(http.MethodGet, "/test", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + assert.Equal(t, http.StatusOK, w.Code) + + var resp map[string]interface{} + err := json.Unmarshal(w.Body.Bytes(), &resp) + require.NoError(t, err) + assert.Equal(t, "ok", resp["status"]) +} + +func TestMiddleware_Deny(t *testing.T) { + gin.SetMode(gin.TestMode) + + limiter := &mockLimiter{ + allowFunc: func(ctx context.Context, key string) (bool, time.Duration) { + return false, 5 * time.Second + }, + } + + router := gin.New() + router.Use(Middleware(limiter, func(c *gin.Context) string { + return "user1:test" + })) + router.GET("/test", func(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) + }) + + req := httptest.NewRequest(http.MethodGet, "/test", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + // 验证返回 429 + assert.Equal(t, http.StatusTooManyRequests, w.Code) + + // 验证 Retry-After header + assert.Equal(t, "5", w.Header().Get("Retry-After")) + + // 验证响应体 + var resp map[string]interface{} + err := json.Unmarshal(w.Body.Bytes(), &resp) + require.NoError(t, err) + assert.Equal(t, "RATE_LIMITED", resp["code"]) + assert.Contains(t, resp["message"], "retry after") +} + +func TestMiddleware_NilLimiter(t *testing.T) { + gin.SetMode(gin.TestMode) + + router := gin.New() + router.Use(Middleware(nil, func(c *gin.Context) string { + return "user1:test" + })) + router.GET("/test", func(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) + }) + + req := httptest.NewRequest(http.MethodGet, "/test", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + // nil limiter 应该放行 + assert.Equal(t, http.StatusOK, w.Code) +} + +func TestMiddleware_EmptyKey(t *testing.T) { + gin.SetMode(gin.TestMode) + + limiter := &mockLimiter{ + allowFunc: func(ctx context.Context, key string) (bool, time.Duration) { + // 不应该被调用 + t.Error("Allow should not be called with empty key") + return false, 0 + }, + } + + router := gin.New() + router.Use(Middleware(limiter, func(c *gin.Context) string { + return "" // 返回空 key + })) + router.GET("/test", func(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) + }) + + req := httptest.NewRequest(http.MethodGet, "/test", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + // 空 key 应该放行 + assert.Equal(t, http.StatusOK, w.Code) +} + +func TestMiddleware_KeyFunc(t *testing.T) { + gin.SetMode(gin.TestMode) + + var capturedKey string + limiter := &mockLimiter{ + allowFunc: func(ctx context.Context, key string) (bool, time.Duration) { + capturedKey = key + return true, 0 + }, + } + + router := gin.New() + router.Use(Middleware(limiter, func(c *gin.Context) string { + // 从 query 参数提取 user_id + userID := c.Query("user_id") + return userID + ":test" + })) + router.GET("/test", func(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) + }) + + req := httptest.NewRequest(http.MethodGet, "/test?user_id=user123", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + assert.Equal(t, http.StatusOK, w.Code) + assert.Equal(t, "user123:test", capturedKey) +} + +func TestMiddleware_RetryAfterRounding(t *testing.T) { + gin.SetMode(gin.TestMode) + + limiter := &mockLimiter{ + allowFunc: func(ctx context.Context, key string) (bool, time.Duration) { + return false, 2500 * time.Millisecond // 2.5 秒 + }, + } + + router := gin.New() + router.Use(Middleware(limiter, func(c *gin.Context) string { + return "user1:test" + })) + router.GET("/test", func(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) + }) + + req := httptest.NewRequest(http.MethodGet, "/test", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + assert.Equal(t, http.StatusTooManyRequests, w.Code) + // 2.5 秒向上取整为 3 秒 + assert.Equal(t, "3", w.Header().Get("Retry-After")) +} diff --git a/backend/internal/ratelimit/redis_bucket.go b/backend/internal/ratelimit/redis_bucket.go new file mode 100644 index 0000000..6b00759 --- /dev/null +++ b/backend/internal/ratelimit/redis_bucket.go @@ -0,0 +1,132 @@ +package ratelimit + +import ( + "context" + "fmt" + "strconv" + "time" + + "github.com/hhs/camtalk/internal/config" + "github.com/hhs/camtalk/internal/trace" + "github.com/redis/go-redis/v9" +) + +// luaScript 是 Redis 令牌桶算法的 Lua 脚本。 +// 保证原子性：读取-计算-回写在一个事务中完成。 +const luaScript = ` +-- KEYS[1] = 限流 key +-- ARGV[1] = capacity（桶容量） +-- ARGV[2] = rate（每秒填充数） +-- ARGV[3] = now（当前时间戳，秒，浮点） +-- ARGV[4] = ttl（key 过期时间，秒） + +local key = KEYS[1] +local capacity = tonumber(ARGV[1]) +local rate = tonumber(ARGV[2]) +local now = tonumber(ARGV[3]) +local ttl = tonumber(ARGV[4]) + +local data = redis.call('HMGET', key, 'tokens', 'last_refill') +local tokens = tonumber(data[1]) or capacity +local last_refill = tonumber(data[2]) or now + +-- 计算新令牌 +local elapsed = math.max(0, now - last_refill) +tokens = math.min(capacity, tokens + elapsed * rate) + +local allowed = 0 +local retry_after = 0 + +if tokens >= 1 then + tokens = tokens - 1 + allowed = 1 +else + if rate == 0 then + retry_after = 86400 -- 24小时 + else + retry_after = (1 - tokens) / rate + end +end + +-- 回写状态 +redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now) +redis.call('EXPIRE', key, ttl) + +return {allowed, tostring(retry_after)} +` + +// RedisLimiter Redis 令牌桶限流器。 +type RedisLimiter struct { + client *redis.Client + config config.RateLimitConfig + script *redis.Script +} + +// NewRedisLimiter 创建 Redis 限流器。 +func NewRedisLimiter(client *redis.Client, cfg config.RateLimitConfig) *RedisLimiter { + return &RedisLimiter{ + client: client, + config: cfg, + script: redis.NewScript(luaScript), + } +} + +// Allow 实现 Limiter 接口。 +func (l *RedisLimiter) Allow(ctx context.Context, key string) (bool, time.Duration) { + log := trace.FromContext(ctx) + cfg := l.getBucketConfig(key) + + now := float64(time.Now().UnixNano()) / 1e9 // 秒，浮点 + ttl := 600 // key 过期时间 10 分钟 + + result, err := l.script.Run(ctx, l.client, []string{key}, + cfg.Capacity, cfg.Rate, now, ttl).Result() + + if err != nil { + log.Errorw("rate limit check failed", "key", key, "error", err) + // Redis 错误时降级：允许请求（fail-open 策略） + return true, 0 + } + + // 解析返回值 + vals, ok := result.([]interface{}) + if !ok || len(vals) != 2 { + return true, 0 + } + + allowed, _ := vals[0].(int64) + retryAfterStr, _ := vals[1].(string) + retryAfterSec, _ := strconv.ParseFloat(retryAfterStr, 64) + + if allowed == 1 { + return true, 0 + } + + retryAfter := time.Duration(retryAfterSec*1000) * time.Millisecond + log.Warnw("rate limit triggered", "key", key, "retry_after_sec", retryAfterSec) + return false, retryAfter +} + +// Stop 实现 Limiter 接口（Redis 不需要清理资源）。 +func (l *RedisLimiter) Stop() { + // Redis 客户端由外部管理，这里不需要操作 +} + +// getBucketConfig 根据 key 获取桶配置。 +func (l *RedisLimiter) getBucketConfig(key string) config.BucketConfig { + // 简化实现：默认使用 query 配置 + return l.config.Query +} + +// KeyPrefix 返回限流 key 的前缀。 +func KeyPrefix() string { + return "ratelimit:" +} + +// FormatKey 格式化限流 key。 +func FormatKey(userID, action string) string { + return fmt.Sprintf("%s%s:%s", KeyPrefix(), userID, action) +} + +// 编译期接口检查 +var _ Limiter = (*RedisLimiter)(nil) diff --git a/backend/internal/ratelimit/redis_bucket_test.go b/backend/internal/ratelimit/redis_bucket_test.go new file mode 100644 index 0000000..becc606 --- /dev/null +++ b/backend/internal/ratelimit/redis_bucket_test.go @@ -0,0 +1,228 @@ +package ratelimit + +import ( + "context" + "testing" + "time" + + "github.com/alicebob/miniredis/v2" + "github.com/hhs/camtalk/internal/config" + "github.com/redis/go-redis/v9" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// setupMiniRedis 创建一个内存 Redis 实例用于测试。 +func setupMiniRedis(t *testing.T) (*miniredis.Miniredis, *redis.Client) { + mr, err := miniredis.Run() + require.NoError(t, err) + + client := redis.NewClient(&redis.Options{ + Addr: mr.Addr(), + }) + + t.Cleanup(func() { + client.Close() + mr.Close() + }) + + return mr, client +} + +func TestRedisLimiter_Allow_FirstRequest(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 5, Rate: 0.2}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + allowed, retryAfter := limiter.Allow(ctx, "user1:query") + + assert.True(t, allowed) + assert.Equal(t, time.Duration(0), retryAfter) +} + +func TestRedisLimiter_Allow_ConsumeUntilEmpty(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 3, Rate: 0.2}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + key := "user1:query" + + // 连续消耗 3 个令牌 + for i := 0; i < 3; i++ { + allowed, _ := limiter.Allow(ctx, key) + assert.True(t, allowed, "request %d should be allowed", i+1) + } + + // 第 4 个请求应被拒绝 + allowed, retryAfter := limiter.Allow(ctx, key) + assert.False(t, allowed) + assert.Greater(t, retryAfter, time.Duration(0)) +} + +func TestRedisLimiter_Allow_DifferentKeys(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 2, Rate: 1.0}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + + // user1 消耗 2 个令牌 + allowed, _ := limiter.Allow(ctx, "user1:query") + assert.True(t, allowed) + allowed, _ = limiter.Allow(ctx, "user1:query") + assert.True(t, allowed) + + // user1 第 3 个被拒绝 + allowed, _ = limiter.Allow(ctx, "user1:query") + assert.False(t, allowed) + + // user2 应该不受影响 + allowed, _ = limiter.Allow(ctx, "user2:query") + assert.True(t, allowed) +} + +func TestRedisLimiter_Allow_RefillAfterWait(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 2, Rate: 10.0}, // 每秒 10 个令牌 + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + key := "user1:query" + + // 消耗 2 个令牌 + limiter.Allow(ctx, key) + limiter.Allow(ctx, key) + + // 真实等待 150ms（Lua 脚本使用系统时间） + time.Sleep(150 * time.Millisecond) + + // 应该补充了至少 1 个令牌 + allowed, _ := limiter.Allow(ctx, key) + assert.True(t, allowed) +} + +func TestRedisLimiter_Allow_CapacityLimit(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 3, Rate: 1.0}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + key := "user1:query" + + // 真实等待让桶"溢出" + time.Sleep(100 * time.Millisecond) + + // 但最多只能消耗 capacity 个令牌 + for i := 0; i < 3; i++ { + allowed, _ := limiter.Allow(ctx, key) + assert.True(t, allowed, "request %d should be allowed", i+1) + } + + // 第 4 个应被拒绝 + allowed, _ := limiter.Allow(ctx, key) + assert.False(t, allowed) +} + +func TestRedisLimiter_Allow_ZeroRate(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 1, Rate: 0.0}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + key := "user1:query" + + // 第一个通过 + allowed, _ := limiter.Allow(ctx, key) + assert.True(t, allowed) + + // 第二个被拒绝，retryAfter 应该很大 + allowed, retryAfter := limiter.Allow(ctx, key) + assert.False(t, allowed) + assert.Greater(t, retryAfter, 1*time.Hour) +} + +func TestRedisLimiter_Allow_KeyTTL(t *testing.T) { + mr, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 5, Rate: 1.0}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + key := "user1:query" + + // 第一次请求 + limiter.Allow(ctx, key) + + // 验证 key 已设置 TTL + ttl := mr.TTL(key) + assert.Greater(t, ttl, time.Duration(0)) + assert.LessOrEqual(t, ttl, 600*time.Second) +} + +func TestRedisLimiter_Allow_FailOpen(t *testing.T) { + mr, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 1, Rate: 1.0}, + } + limiter := NewRedisLimiter(client, cfg) + + ctx := context.Background() + + // 关闭 Redis 模拟故障 + mr.Close() + + // 应该 fail-open（允许请求） + allowed, retryAfter := limiter.Allow(ctx, "user1:query") + assert.True(t, allowed) + assert.Equal(t, time.Duration(0), retryAfter) +} + +func TestRedisLimiter_Stop(t *testing.T) { + _, client := setupMiniRedis(t) + + cfg := config.RateLimitConfig{ + Enabled: true, + Query: config.BucketConfig{Capacity: 1, Rate: 1.0}, + } + limiter := NewRedisLimiter(client, cfg) + + // Stop 应该不会 panic（即使多次调用） + limiter.Stop() + limiter.Stop() +} + +func TestFormatKey(t *testing.T) { + key := FormatKey("user123", "query") + assert.Equal(t, "ratelimit:user123:query", key) +} diff --git a/backend/internal/session/manager.go b/backend/internal/session/manager.go index 6a2527e..9507aa8 100644 --- a/backend/internal/session/manager.go +++ b/backend/internal/session/manager.go @@ -18,6 +18,7 @@ type ConversationSummary struct { Title string `json:"title"` LastMessage string `json:"last_message"` MessageCount int `json:"message_count"` + CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } diff --git a/backend/internal/session/memory.go b/backend/internal/session/memory.go index 8ee3f3b..4dd67ba 100644 --- a/backend/internal/session/memory.go +++ b/backend/internal/session/memory.go @@ -142,11 +142,11 @@ func (m *MemoryManager) Create(ctx context.Context, userID string, config models } m.mu.Unlock() - // Write-Through：异步写 PG + // Write-Through：异步写 PG（使用 Background context，避免 HTTP 请求结束后 context 被取消） if m.sessRepo != nil { go func() { cfgJSON, _ := json.Marshal(config) - if err := m.sessRepo.Save(ctx, store.SessionRecord{ + if err := m.sessRepo.Save(context.Background(), store.SessionRecord{ ID: id, UserID: userID, Title: models.DefaultSessionTitle, Config: cfgJSON, CreatedAt: now, UpdatedAt: now, }); err != nil { @@ -204,11 +204,11 @@ func (m *MemoryManager) UpdateConfig(ctx context.Context, sessionID string, patc cfg := entry.session.Config m.mu.Unlock() - // Write-Through：异步更新 PG + // Write-Through：异步更新 PG（使用 Background context） if m.sessRepo != nil { go func() { cfgJSON, _ := json.Marshal(cfg) - if err := m.sessRepo.UpdateConfig(ctx, sessionID, cfgJSON); err != nil { + if err := m.sessRepo.UpdateConfig(context.Background(), sessionID, cfgJSON); err != nil { logger.Log.Warnw("update session config in DB failed", "session", sessionID, "error", err) } }() @@ -233,10 +233,10 @@ func (m *MemoryManager) UpdateTitle(ctx context.Context, sessionID string, title entry.lastActive = time.Now() m.mu.Unlock() - // Write-Through：异步更新 PG + // Write-Through：异步更新 PG（使用 Background context） if m.sessRepo != nil { go func() { - if err := m.sessRepo.UpdateTitle(ctx, sessionID, title); err != nil { + if err := m.sessRepo.UpdateTitle(context.Background(), sessionID, title); err != nil { logger.Log.Warnw("update session title in DB failed", "session", sessionID, "error", err) } }() @@ -271,6 +271,7 @@ func (m *MemoryManager) ListByUser(ctx context.Context, userID string, page, siz list = append(list, ConversationSummary{ ID: rec.ID, Title: rec.Title, + CreatedAt: rec.CreatedAt, UpdatedAt: rec.UpdatedAt, }) sessionIDs = append(sessionIDs, rec.ID) @@ -320,6 +321,7 @@ func (m *MemoryManager) listByUserFromMemory(ctx context.Context, userID string, summary := ConversationSummary{ ID: entry.session.ID, Title: entry.session.Title, + CreatedAt: entry.session.CreatedAt, UpdatedAt: entry.lastActive, } summary.MessageCount = len(entry.history) @@ -394,8 +396,10 @@ func (m *MemoryManager) AppendMessage(_ context.Context, sessionID string, msg m entry.history = append(entry.history, msg) // 自动更新标题：首条 user 消息时，如果标题为默认值，自动更新为消息前 20 字符 + titleUpdated := false if msg.Role == "user" && entry.session.Title == models.DefaultSessionTitle { entry.session.Title = generateTitle(msg.Content) + titleUpdated = true } // 超过上限时裁剪，保留最新的 maxHistory 条 @@ -406,13 +410,31 @@ func (m *MemoryManager) AppendMessage(_ context.Context, sessionID string, msg m now := time.Now() entry.lastActive = now entry.session.UpdatedAt = now + + // 复制标题（释放锁后安全使用） + persistTitle := entry.session.Title m.mu.Unlock() - // Write-Through：异步写冷存储，不阻塞调用方 + // Write-Through：消息同步写入 PostgreSQL（保证调用顺序 = 插入顺序， + // 避免用户消息和 AI 消息的异步 goroutine 执行顺序不确定导致排序错乱） if m.msgRepo != nil { + if err := m.msgRepo.SaveMessage(context.Background(), sessionID, msg, 0); err != nil { + logger.Log.Warnw("persist message failed", "session", sessionID, "error", err) + } + } + + // Write-Through：异步更新会话元数据（标题 + updated_at）到 PostgreSQL + if m.sessRepo != nil { go func() { - if err := m.msgRepo.SaveMessage(context.Background(), sessionID, msg, 0); err != nil { - logger.Log.Warnw("persist message failed", "session", sessionID, "error", err) + if titleUpdated { + if err := m.sessRepo.UpdateTitle(context.Background(), sessionID, persistTitle); err != nil { + logger.Log.Warnw("persist session title failed", "session", sessionID, "error", err) + } + } else { + // 即使标题没变，也要刷新 updated_at（保证列表排序正确） + if err := m.sessRepo.Touch(context.Background(), sessionID); err != nil { + logger.Log.Warnw("touch session in DB failed", "session", sessionID, "error", err) + } } }() } @@ -540,10 +562,10 @@ func (m *MemoryManager) Destroy(ctx context.Context, sessionID string) error { delete(m.sessions, sessionID) m.mu.Unlock() - // Write-Through：异步删除 PG + // Write-Through：异步删除 PG（使用 Background context） if m.sessRepo != nil { go func() { - if err := m.sessRepo.Delete(ctx, sessionID); err != nil { + if err := m.sessRepo.Delete(context.Background(), sessionID); err != nil { logger.Log.Warnw("delete session from DB failed", "session", sessionID, "error", err) } }() diff --git a/backend/internal/session/redis.go b/backend/internal/session/redis.go index ec1dc6b..4a28bb9 100644 --- a/backend/internal/session/redis.go +++ b/backend/internal/session/redis.go @@ -10,8 +10,9 @@ import ( "github.com/google/uuid" "github.com/redis/go-redis/v9" - "github.com/hhs/camtalk/internal/logger" "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/trace" + "github.com/hhs/camtalk/internal/util" ) // RedisManager 基于 Redis 的 SessionManager 实现。 @@ -36,13 +37,23 @@ func NewRedisManager(rdb *redis.Client, ttl time.Duration, maxHistory int) *Redi return &RedisManager{rdb: rdb, ttl: ttl, maxHistory: maxHistory} } +// Ping 检查 Redis 连接是否正常。 +func (m *RedisManager) Ping(ctx context.Context) error { + return m.rdb.Ping(ctx).Err() +} + func metaKey(id string) string { return fmt.Sprintf("session:%s:meta", id) } func histKey(id string) string { return fmt.Sprintf("session:%s:history", id) } func userSessKey(id string) string { return fmt.Sprintf("user:%s:sessions", id) } // Create 创建新会话。userID 为空表示匿名会话。 func (m *RedisManager) Create(ctx context.Context, userID string, config models.SessionConfig) (string, error) { - id := uuidNew() + return m.CreateWithID(ctx, uuidNew(), userID, config) +} + +// CreateWithID 使用指定 ID 创建新会话。 +// 供 TieredManager 调用，确保 L1/L2 使用相同的 session ID。 +func (m *RedisManager) CreateWithID(ctx context.Context, id string, userID string, config models.SessionConfig) (string, error) { now := time.Now().UTC() pipe := m.rdb.Pipeline() @@ -77,7 +88,8 @@ func (m *RedisManager) Create(ctx context.Context, userID string, config models. return "", fmt.Errorf("redis create session: %w", err) } - logger.Log.Debugw("redis session created", "session", id, "user_id", userID) + log := trace.FromContext(ctx) + log.Debugw("redis session created", "session_id", id, "user_id", userID) return id, nil } @@ -86,8 +98,11 @@ const placeholderHistoryMark = "__placeholder__" // Get 获取会话。 func (m *RedisManager) Get(ctx context.Context, sessionID string) (*models.Session, error) { + log := trace.FromContext(ctx) + vals, err := m.rdb.HGetAll(ctx, metaKey(sessionID)).Result() if err != nil { + log.Errorw("redis get session failed", "session_id", sessionID, "error", err) return nil, fmt.Errorf("redis get session: %w", err) } if len(vals) == 0 { @@ -105,6 +120,7 @@ func (m *RedisManager) Get(ctx context.Context, sessionID string) (*models.Sessi sess.Config.DetailLevel = vals["config.detail_level"] sess.Config.Language = vals["config.language"] + log.Debugw("redis session retrieved", "session_id", sessionID) return sess, nil } @@ -140,7 +156,9 @@ func (m *RedisManager) UpdateConfig(ctx context.Context, sessionID string, patch // 刷新 TTL m.rdb.Expire(ctx, metaKey(sessionID), m.ttl) - logger.Log.Debugw("redis session config updated", "session", sessionID) + + log := trace.FromContext(ctx) + log.Debugw("redis session config updated", "session_id", sessionID) return nil } @@ -160,7 +178,9 @@ func (m *RedisManager) UpdateTitle(ctx context.Context, sessionID string, title } m.rdb.Expire(ctx, metaKey(sessionID), m.ttl) - logger.Log.Debugw("redis session title updated", "session", sessionID, "title", title) + + log := trace.FromContext(ctx) + log.Debugw("redis session title updated", "session_id", sessionID, "title", title) return nil } @@ -275,7 +295,11 @@ func (m *RedisManager) GetHistory(ctx context.Context, sessionID string, limit i } var msg models.Message if err := json.Unmarshal([]byte(raw), &msg); err != nil { - logger.Log.Warnw("invalid history entry", "session", sessionID, "raw", raw) + log := trace.FromContext(ctx) + log.Warnw("invalid history entry", + "session_id", sessionID, + "raw_len", len(raw), + "raw_preview", util.Truncate(raw, 100)) continue } msgs = append(msgs, msg) @@ -426,7 +450,8 @@ func (m *RedisManager) Destroy(ctx context.Context, sessionID string) error { m.rdb.SRem(ctx, userSessKey(userID), sessionID) } - logger.Log.Debugw("redis session destroyed", "session", sessionID) + log := trace.FromContext(ctx) + log.Debugw("redis session destroyed", "session_id", sessionID) return nil } diff --git a/backend/internal/session/tiered.go b/backend/internal/session/tiered.go new file mode 100644 index 0000000..9f9d3b7 --- /dev/null +++ b/backend/internal/session/tiered.go @@ -0,0 +1,361 @@ +package session + +import ( + "context" + "sync/atomic" + "time" + + "github.com/hhs/camtalk/internal/logger" + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/store" +) + +// TieredManager 三级存储 SessionManager 实现。 +// +// L1（内存）→ L2（Redis）→ L3（PostgreSQL） +// +// 读：L1 miss → L2 miss → L3，回填到 L1+L2 +// 写：L1 → L2（同步）→ L3（异步） +// 降级：Redis 不可用时，回退到 L1+L3 模式 +type TieredManager struct { + l1 *MemoryManager // L1: 内存缓存 + l2 *RedisManager // L2: Redis（可选） + sessRepo store.SessionRepository // L3: PostgreSQL 会话持久化（可选） + msgRepo store.MessageRepository // L3: PostgreSQL 消息持久化（可选） + + redisOK atomic.Bool // Redis 健康状态 + stopCh chan struct{} // 停止信号 +} + +// TieredOption TieredManager 的函数式选项。 +type TieredOption func(*TieredManager) + +// WithTieredSessionRepository 注入 L3 会话持久化仓库。 +func WithTieredSessionRepository(repo store.SessionRepository) TieredOption { + return func(m *TieredManager) { + m.sessRepo = repo + } +} + +// WithTieredMessageRepository 注入 L3 消息持久化仓库。 +func WithTieredMessageRepository(repo store.MessageRepository) TieredOption { + return func(m *TieredManager) { + m.msgRepo = repo + } +} + +// NewTieredManager 创建三级存储 SessionManager。 +// l2 为 nil 时降级为 L1+L3 模式。 +func NewTieredManager( + ttl time.Duration, + maxHistory int, + l2 *RedisManager, + opts ...TieredOption, +) *TieredManager { + m := &TieredManager{ + l2: l2, + stopCh: make(chan struct{}), + } + + for _, opt := range opts { + opt(m) + } + + // 初始化 L1（内存），注入 L3 仓库实现 Write-Through + var l1Opts []Option + if m.sessRepo != nil { + l1Opts = append(l1Opts, WithSessionRepository(m.sessRepo)) + } + if m.msgRepo != nil { + l1Opts = append(l1Opts, WithMessageRepository(m.msgRepo)) + } + m.l1 = NewMemoryManager(ttl, maxHistory, l1Opts...) + + // 初始化 Redis 健康状态 + if l2 != nil { + m.redisOK.Store(true) + go m.healthCheck() + } + + return m +} + +// healthCheck 定期检查 Redis 健康状态。 +func (m *TieredManager) healthCheck() { + ticker := time.NewTicker(30 * time.Second) + defer ticker.Stop() + + for { + select { + case <-ticker.C: + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + err := m.l2.Ping(ctx) + cancel() + + wasOK := m.redisOK.Load() + isOK := err == nil + m.redisOK.Store(isOK) + + if wasOK && !isOK { + logger.Log.Warn("Redis connection lost, degrading to L1+L3 mode") + } else if !wasOK && isOK { + logger.Log.Info("Redis connection restored, resuming L1+L2+L3 mode") + } + case <-m.stopCh: + return + } + } +} + +// isRedisOK 检查 Redis 是否可用。 +func (m *TieredManager) isRedisOK() bool { + return m.l2 != nil && m.redisOK.Load() +} + +// Stop 停止 TieredManager（清理后台 goroutine）。 +func (m *TieredManager) Stop() { + close(m.stopCh) + m.l1.Stop() +} + +// Create 创建新会话。 +// 写入：L1 → L2（同步）→ L3（异步） +func (m *TieredManager) Create(ctx context.Context, userID string, config models.SessionConfig) (string, error) { + // L1: 内存 + id, err := m.l1.Create(ctx, userID, config) + if err != nil { + return "", err + } + + // L2: Redis（同步），使用 L1 生成的 ID 保证一致性 + if m.isRedisOK() { + if _, err := m.l2.CreateWithID(ctx, id, userID, config); err != nil { + logger.Log.Warnw("Redis Create failed, continuing without L2", + "session", id, "error", err) + } + } + + // L3: PostgreSQL（异步，由 L1 的 Write-Through 处理） + + return id, nil +} + +// Get 获取会话。 +// 读取：L1 → L2（回填 L1）→ L3（回填 L1+L2） +func (m *TieredManager) Get(ctx context.Context, sessionID string) (*models.Session, error) { + // L1: 内存 + sess, err := m.l1.Get(ctx, sessionID) + if err == nil { + return sess, nil + } + if err != ErrSessionNotFound { + return nil, err + } + + // L2: Redis + if m.isRedisOK() { + sess, err = m.l2.Get(ctx, sessionID) + if err == nil { + // 回填 L1 + history, _ := m.l2.GetHistory(ctx, sessionID, 0) + m.l1.LoadSession(sess, history) + return sess, nil + } + if err != ErrSessionNotFound { + logger.Log.Warnw("Redis Get failed", + "session", sessionID, "error", err) + } + } + + // L3: PostgreSQL（由 L1 的 Cache-Aside 处理） + // L1.Get 已经实现了从 PostgreSQL 恢复的逻辑 + return nil, ErrSessionNotFound +} + +// UpdateConfig 更新会话配置。 +// 写入：L1 → L2（同步）→ L3（异步） +func (m *TieredManager) UpdateConfig(ctx context.Context, sessionID string, patch models.SessionConfigPatch) error { + // L1: 内存 + if err := m.l1.UpdateConfig(ctx, sessionID, patch); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.UpdateConfig(ctx, sessionID, patch); err != nil { + logger.Log.Warnw("Redis UpdateConfig failed", + "session", sessionID, "error", err) + } + } + + // L3: PostgreSQL（异步，由 L1 的 Write-Through 处理） + + return nil +} + +// UpdateTitle 更新会话标题。 +// 写入：L1 → L2（同步）→ L3（异步） +func (m *TieredManager) UpdateTitle(ctx context.Context, sessionID string, title string) error { + // L1: 内存 + if err := m.l1.UpdateTitle(ctx, sessionID, title); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.UpdateTitle(ctx, sessionID, title); err != nil { + logger.Log.Warnw("Redis UpdateTitle failed", + "session", sessionID, "error", err) + } + } + + // L3: PostgreSQL（异步，由 L1 的 Write-Through 处理） + + return nil +} + +// ListByUser 查询用户的会话列表。 +// 读取：L1 + L2 + L3 合并去重 +func (m *TieredManager) ListByUser(ctx context.Context, userID string, page, size int) ([]ConversationSummary, int, error) { + // 优先使用 L1（已集成 L3 回退逻辑） + return m.l1.ListByUser(ctx, userID, page, size) +} + +// GetHistory 获取对话历史。 +// 读取：L1 → L2（回填 L1）→ L3（回填 L1+L2） +func (m *TieredManager) GetHistory(ctx context.Context, sessionID string, limit int) ([]models.Message, error) { + // L1: 内存 + msgs, err := m.l1.GetHistory(ctx, sessionID, limit) + if err == nil && len(msgs) > 0 { + return msgs, nil + } + + // L2: Redis + if m.isRedisOK() { + msgs, err = m.l2.GetHistory(ctx, sessionID, limit) + if err == nil && len(msgs) > 0 { + // 回填 L1（通过 Get 触发） + m.l1.Get(ctx, sessionID) + return msgs, nil + } + } + + // L3: PostgreSQL（由 L1 的 Cache-Aside 处理） + return nil, ErrSessionNotFound +} + +// AppendMessage 追加消息。 +// 写入：L1 → L2（同步）→ L3（异步） +func (m *TieredManager) AppendMessage(ctx context.Context, sessionID string, msg models.Message) error { + // L1: 内存（Write-Through 到 L3） + if err := m.l1.AppendMessage(ctx, sessionID, msg); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.AppendMessage(ctx, sessionID, msg); err != nil { + logger.Log.Warnw("Redis AppendMessage failed", + "session", sessionID, "error", err) + } + } + + return nil +} + +// SetActiveRequest 设置当前活跃请求。 +func (m *TieredManager) SetActiveRequest(ctx context.Context, sessionID string, requestID string) error { + // L1: 内存 + if err := m.l1.SetActiveRequest(ctx, sessionID, requestID); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.SetActiveRequest(ctx, sessionID, requestID); err != nil { + logger.Log.Warnw("Redis SetActiveRequest failed", + "session", sessionID, "error", err) + } + } + + return nil +} + +// GetActiveRequestID 获取当前活跃请求 ID。 +func (m *TieredManager) GetActiveRequestID(ctx context.Context, sessionID string) (string, error) { + // L1: 内存 + id, err := m.l1.GetActiveRequestID(ctx, sessionID) + if err == nil && id != "" { + return id, nil + } + + // L2: Redis + if m.isRedisOK() { + id, err = m.l2.GetActiveRequestID(ctx, sessionID) + if err == nil && id != "" { + return id, nil + } + } + + return "", nil +} + +// ClearActiveRequest 清除当前活跃请求。 +func (m *TieredManager) ClearActiveRequest(ctx context.Context, sessionID string) error { + // L1: 内存 + if err := m.l1.ClearActiveRequest(ctx, sessionID); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.ClearActiveRequest(ctx, sessionID); err != nil { + logger.Log.Warnw("Redis ClearActiveRequest failed", + "session", sessionID, "error", err) + } + } + + return nil +} + +// Touch 刷新会话活跃时间。 +func (m *TieredManager) Touch(ctx context.Context, sessionID string) error { + // L1: 内存 + if err := m.l1.Touch(ctx, sessionID); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.Touch(ctx, sessionID); err != nil { + logger.Log.Warnw("Redis Touch failed", + "session", sessionID, "error", err) + } + } + + return nil +} + +// Destroy 销毁会话。 +// 写入：L1 → L2 → L3 +func (m *TieredManager) Destroy(ctx context.Context, sessionID string) error { + // L1: 内存（Write-Through 到 L3） + if err := m.l1.Destroy(ctx, sessionID); err != nil { + return err + } + + // L2: Redis（同步） + if m.isRedisOK() { + if err := m.l2.Destroy(ctx, sessionID); err != nil { + logger.Log.Warnw("Redis Destroy failed", + "session", sessionID, "error", err) + } + } + + return nil +} + +// ActiveCount 返回活跃会话数量。 +func (m *TieredManager) ActiveCount() int { + return m.l1.ActiveCount() +} diff --git a/backend/internal/store/cached_user.go b/backend/internal/store/cached_user.go new file mode 100644 index 0000000..43d2f58 --- /dev/null +++ b/backend/internal/store/cached_user.go @@ -0,0 +1,172 @@ +package store + +import ( + "context" + "time" + + "github.com/redis/go-redis/v9" + + "github.com/hhs/camtalk/internal/trace" +) + +// Redis key 前缀。 +const ( + refreshTokenPrefix = "auth:refresh:" // auth:refresh:{token_hash} → user_id + userRefreshPrefix = "auth:user_refresh:" // auth:user_refresh:{user_id} → Set of token_hash +) + +// CachedUserRepository 装饰器，为 UserRepository 的 refresh token 操作增加 Redis 缓存。 +// 读路径：Redis miss → DB → 回填 Redis。 +// 写路径：同步双写 Redis + DB。 +// 删路径：同步双删 Redis + DB。 +// Redis 操作失败时降级到纯 DB，不阻断主流程。 +type CachedUserRepository struct { + inner UserRepository + rdb *redis.Client + backfillTTL time.Duration // DB 回填 Redis 时使用的默认 TTL +} + +// NewCachedUserRepository 创建带 Redis 缓存的 UserRepository 装饰器。 +// backfillTTL: 从 DB 回填 Redis 时使用的 TTL（因 DB 接口不返回 expiresAt）。 +func NewCachedUserRepository(inner UserRepository, rdb *redis.Client, backfillTTL time.Duration) *CachedUserRepository { + if backfillTTL <= 0 { + backfillTTL = 24 * time.Hour + } + return &CachedUserRepository{ + inner: inner, + rdb: rdb, + backfillTTL: backfillTTL, + } +} + +// refreshTokenKey 生成 refresh token 的 Redis key。 +func refreshTokenKey(tokenHash string) string { + return refreshTokenPrefix + tokenHash +} + +// userRefreshKey 生成用户 refresh token 集合的 Redis key。 +func userRefreshKey(userID string) string { + return userRefreshPrefix + userID +} + +// --- 委托方法（不做缓存） --- + +func (r *CachedUserRepository) Create(ctx context.Context, username, passwordHash string) (string, error) { + return r.inner.Create(ctx, username, passwordHash) +} + +func (r *CachedUserRepository) FindByUsername(ctx context.Context, username string) (*User, error) { + return r.inner.FindByUsername(ctx, username) +} + +func (r *CachedUserRepository) FindByID(ctx context.Context, id string) (*User, error) { + return r.inner.FindByID(ctx, id) +} + +// --- 缓存方法 --- + +// SaveRefreshToken Write-Through：先写 DB，再写 Redis。 +func (r *CachedUserRepository) SaveRefreshToken(ctx context.Context, userID, tokenHash string, expiresAt time.Time) error { + // 先写 DB + if err := r.inner.SaveRefreshToken(ctx, userID, tokenHash, expiresAt); err != nil { + return err + } + + // 写 Redis（SET + SADD），设置 TTL 为 token 剩余有效期 + ttl := time.Until(expiresAt) + if ttl <= 0 { + return nil + } + + key := refreshTokenKey(tokenHash) + pipe := r.rdb.Pipeline() + pipe.Set(ctx, key, userID, ttl) + pipe.SAdd(ctx, userRefreshKey(userID), tokenHash) + if _, err := pipe.Exec(ctx); err != nil { + log := trace.FromContext(ctx) + log.Warnw("redis cache write failed for refresh token", "error", err) + // 降级：DB 已写入成功，Redis 失败不影响正确性 + } + return nil +} + +// FindRefreshToken Read-Through：先查 Redis，miss 时查 DB 并回填。 +func (r *CachedUserRepository) FindRefreshToken(ctx context.Context, tokenHash string) (string, error) { + key := refreshTokenKey(tokenHash) + + // 查 Redis + userID, err := r.rdb.Get(ctx, key).Result() + if err == nil { + return userID, nil + } + // redis.Nil 表示 key 不存在，其他错误记录日志后降级到 DB + if err != redis.Nil { + log := trace.FromContext(ctx) + log.Warnw("redis cache read failed for refresh token", "error", err) + } + + // 降级到 DB + userID, err = r.inner.FindRefreshToken(ctx, tokenHash) + if err != nil { + return "", err + } + + // 回填 Redis（SET + SADD），TTL 使用保守默认值 + go func() { + bgCtx := context.Background() + pipe := r.rdb.Pipeline() + pipe.Set(bgCtx, key, userID, r.backfillTTL) + pipe.SAdd(bgCtx, userRefreshKey(userID), tokenHash) + _, _ = pipe.Exec(bgCtx) + }() + + return userID, nil +} + +// DeleteRefreshToken 双删：先删 DB，再删 Redis。 +func (r *CachedUserRepository) DeleteRefreshToken(ctx context.Context, tokenHash string) error { + // 先从 Redis 获取 user_id（用于从集合中移除） + userID, _ := r.rdb.Get(ctx, refreshTokenKey(tokenHash)).Result() + + // 删 DB + if err := r.inner.DeleteRefreshToken(ctx, tokenHash); err != nil { + return err + } + + // 删 Redis + key := refreshTokenKey(tokenHash) + pipe := r.rdb.Pipeline() + pipe.Del(ctx, key) + if userID != "" { + pipe.SRem(ctx, userRefreshKey(userID), tokenHash) + } + if _, err := pipe.Exec(ctx); err != nil { + log := trace.FromContext(ctx) + log.Warnw("redis cache delete failed for refresh token", "error", err) + } + return nil +} + +// DeleteUserRefreshTokens 批量清理：先从 Redis 获取集合，逐个删缓存，再删 DB。 +func (r *CachedUserRepository) DeleteUserRefreshTokens(ctx context.Context, userID string) error { + userKey := userRefreshKey(userID) + + // 从 Redis 获取该用户所有 token hash + hashes, _ := r.rdb.SMembers(ctx, userKey).Result() + + // 批量删除 Redis 缓存 + if len(hashes) > 0 { + keys := make([]string, 0, len(hashes)+1) + for _, h := range hashes { + keys = append(keys, refreshTokenKey(h)) + } + keys = append(keys, userKey) + if err := r.rdb.Del(ctx, keys...).Err(); err != nil { + log := trace.FromContext(ctx) + log.Warnw("redis cache batch delete failed for user refresh tokens", "error", err, "user_id", userID) + } + } + + // 删 DB（无论 Redis 是否成功都执行） + return r.inner.DeleteUserRefreshTokens(ctx, userID) +} diff --git a/backend/internal/store/message_pg.go b/backend/internal/store/message_pg.go index 0bc0997..b393c9c 100644 --- a/backend/internal/store/message_pg.go +++ b/backend/internal/store/message_pg.go @@ -8,6 +8,7 @@ import ( "github.com/jackc/pgx/v5/pgxpool" "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/trace" ) // PgMessageRepository 基于 PostgreSQL 的 MessageRepository 实现。 @@ -21,14 +22,24 @@ func NewPgMessageRepository(pool *pgxpool.Pool) *PgMessageRepository { } func (r *PgMessageRepository) SaveMessage(ctx context.Context, sessionID string, msg models.Message, tokensUsed int) error { + log := trace.FromContext(ctx) + _, err := r.pool.Exec(ctx, `INSERT INTO messages (session_id, role, content, tokens_used) VALUES ($1, $2, $3, $4)`, sessionID, msg.Role, msg.Content, tokensUsed, ) - return err + if err != nil { + log.Errorw("save message failed", "session_id", sessionID, "role", msg.Role, "error", err) + return err + } + + log.Debugw("message saved", "session_id", sessionID, "role", msg.Role, "tokens_used", tokensUsed) + return nil } func (r *PgMessageRepository) GetMessages(ctx context.Context, sessionID string, limit int, beforeID int64) ([]StoredMessage, error) { + log := trace.FromContext(ctx) + if limit <= 0 { limit = 50 } @@ -56,6 +67,7 @@ func (r *PgMessageRepository) GetMessages(ctx context.Context, sessionID string, ) } if err != nil { + log.Errorw("get messages failed", "session_id", sessionID, "error", err) return nil, err } @@ -64,31 +76,39 @@ func (r *PgMessageRepository) GetMessages(ctx context.Context, sessionID string, rows[i], rows[j] = rows[j], rows[i] } + log.Debugw("messages retrieved", "session_id", sessionID, "count", len(rows)) return rows, nil } func (r *PgMessageRepository) queryMessages(ctx context.Context, query string, args ...any) ([]StoredMessage, error) { + log := trace.FromContext(ctx) + pgxRows, err := r.pool.Query(ctx, query, args...) if err != nil { + log.Errorw("query messages failed", "error", err) return nil, err } defer pgxRows.Close() - var messages []StoredMessage + messages := make([]StoredMessage, 0) for pgxRows.Next() { var m StoredMessage if err := pgxRows.Scan(&m.ID, &m.SessionID, &m.Role, &m.Content, &m.TokensUsed, &m.CreatedAt); err != nil { + log.Errorw("scan message row failed", "error", err) return nil, err } messages = append(messages, m) } if err := pgxRows.Err(); err != nil { + log.Errorw("iterate message rows failed", "error", err) return nil, err } return messages, nil } func (r *PgMessageRepository) GetLastMessage(ctx context.Context, sessionID string) (*StoredMessage, error) { + log := trace.FromContext(ctx) + var m StoredMessage err := r.pool.QueryRow(ctx, `SELECT id, session_id, role, content, tokens_used, created_at @@ -102,24 +122,34 @@ func (r *PgMessageRepository) GetLastMessage(ctx context.Context, sessionID stri return nil, ErrMessageNotFound } if err != nil { + log.Errorw("get last message failed", "session_id", sessionID, "error", err) return nil, err } + + log.Debugw("last message retrieved", "session_id", sessionID, "message_id", m.ID) return &m, nil } func (r *PgMessageRepository) GetMessageCount(ctx context.Context, sessionID string) (int, error) { + log := trace.FromContext(ctx) + var count int err := r.pool.QueryRow(ctx, `SELECT COUNT(*) FROM messages WHERE session_id = $1`, sessionID, ).Scan(&count) if err != nil { + log.Errorw("get message count failed", "session_id", sessionID, "error", err) return 0, err } + + log.Debugw("message count retrieved", "session_id", sessionID, "count", count) return count, nil } func (r *PgMessageRepository) GetSessionMessageStats(ctx context.Context, sessionIDs []string) (map[string]SessionMessageStats, error) { + log := trace.FromContext(ctx) + if len(sessionIDs) == 0 { return map[string]SessionMessageStats{}, nil } @@ -143,6 +173,7 @@ func (r *PgMessageRepository) GetSessionMessageStats(ctx context.Context, sessio sessionIDs, ) if err != nil { + log.Errorw("get session message stats failed", "session_count", len(sessionIDs), "error", err) return nil, err } defer rows.Close() @@ -152,12 +183,16 @@ func (r *PgMessageRepository) GetSessionMessageStats(ctx context.Context, sessio var sid string var stats SessionMessageStats if err := rows.Scan(&sid, &stats.MessageCount, &stats.LastMessage); err != nil { + log.Errorw("scan message stats row failed", "error", err) return nil, err } result[sid] = stats } if err := rows.Err(); err != nil { + log.Errorw("iterate message stats rows failed", "error", err) return nil, err } + + log.Debugw("session message stats retrieved", "session_count", len(sessionIDs), "result_count", len(result)) return result, nil } diff --git a/backend/internal/store/session_pg.go b/backend/internal/store/session_pg.go index 53aff40..e0ce9a0 100644 --- a/backend/internal/store/session_pg.go +++ b/backend/internal/store/session_pg.go @@ -6,6 +6,8 @@ import ( "github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5/pgxpool" + + "github.com/hhs/camtalk/internal/trace" ) // PgSessionRepository 基于 PostgreSQL 的 SessionRepository 实现。 @@ -19,6 +21,8 @@ func NewPgSessionRepository(pool *pgxpool.Pool) *PgSessionRepository { } func (r *PgSessionRepository) Save(ctx context.Context, s SessionRecord) error { + log := trace.FromContext(ctx) + _, err := r.pool.Exec(ctx, `INSERT INTO sessions (id, user_id, title, config, created_at, updated_at) VALUES ($1, $2, $3, $4, $5, $6) @@ -28,10 +32,18 @@ func (r *PgSessionRepository) Save(ctx context.Context, s SessionRecord) error { updated_at = EXCLUDED.updated_at`, s.ID, s.UserID, s.Title, s.Config, s.CreatedAt, s.UpdatedAt, ) - return err + if err != nil { + log.Errorw("save session failed", "session_id", s.ID, "error", err) + return err + } + + log.Debugw("session saved", "session_id", s.ID, "user_id", s.UserID) + return nil } func (r *PgSessionRepository) FindByID(ctx context.Context, id string) (*SessionRecord, error) { + log := trace.FromContext(ctx) + var s SessionRecord err := r.pool.QueryRow(ctx, `SELECT id, user_id, title, config, created_at, updated_at @@ -41,12 +53,17 @@ func (r *PgSessionRepository) FindByID(ctx context.Context, id string) (*Session return nil, ErrSessionNotFound } if err != nil { + log.Errorw("find session failed", "session_id", id, "error", err) return nil, err } + + log.Debugw("session found", "session_id", id) return &s, nil } func (r *PgSessionRepository) FindByUser(ctx context.Context, userID string, page, size int) ([]SessionRecord, int, error) { + log := trace.FromContext(ctx) + if page <= 0 { page = 1 } @@ -60,6 +77,7 @@ func (r *PgSessionRepository) FindByUser(ctx context.Context, userID string, pag if err := r.pool.QueryRow(ctx, `SELECT COUNT(*) FROM sessions WHERE user_id = $1`, userID, ).Scan(&total); err != nil { + log.Errorw("count user sessions failed", "user_id", userID, "error", err) return nil, 0, err } @@ -73,6 +91,7 @@ func (r *PgSessionRepository) FindByUser(ctx context.Context, userID string, pag userID, size, offset, ) if err != nil { + log.Errorw("find user sessions failed", "user_id", userID, "error", err) return nil, 0, err } defer rows.Close() @@ -81,66 +100,90 @@ func (r *PgSessionRepository) FindByUser(ctx context.Context, userID string, pag for rows.Next() { var s SessionRecord if err := rows.Scan(&s.ID, &s.UserID, &s.Title, &s.Config, &s.CreatedAt, &s.UpdatedAt); err != nil { + log.Errorw("scan session row failed", "user_id", userID, "error", err) return nil, 0, err } list = append(list, s) } if err := rows.Err(); err != nil { + log.Errorw("iterate session rows failed", "user_id", userID, "error", err) return nil, 0, err } + + log.Debugw("user sessions found", "user_id", userID, "count", len(list), "total", total) return list, total, nil } func (r *PgSessionRepository) UpdateTitle(ctx context.Context, id string, title string) error { + log := trace.FromContext(ctx) + tag, err := r.pool.Exec(ctx, `UPDATE sessions SET title = $2, updated_at = NOW() WHERE id = $1`, id, title, ) if err != nil { + log.Errorw("update session title failed", "session_id", id, "error", err) return err } if tag.RowsAffected() == 0 { return ErrSessionNotFound } + + log.Debugw("session title updated", "session_id", id) return nil } func (r *PgSessionRepository) UpdateConfig(ctx context.Context, id string, configJSON []byte) error { + log := trace.FromContext(ctx) + tag, err := r.pool.Exec(ctx, `UPDATE sessions SET config = $2, updated_at = NOW() WHERE id = $1`, id, configJSON, ) if err != nil { + log.Errorw("update session config failed", "session_id", id, "error", err) return err } if tag.RowsAffected() == 0 { return ErrSessionNotFound } + + log.Debugw("session config updated", "session_id", id) return nil } func (r *PgSessionRepository) Touch(ctx context.Context, id string) error { + log := trace.FromContext(ctx) + tag, err := r.pool.Exec(ctx, `UPDATE sessions SET updated_at = NOW() WHERE id = $1`, id, ) if err != nil { + log.Errorw("touch session failed", "session_id", id, "error", err) return err } if tag.RowsAffected() == 0 { return ErrSessionNotFound } + + log.Debugw("session touched", "session_id", id) return nil } func (r *PgSessionRepository) Delete(ctx context.Context, id string) error { + log := trace.FromContext(ctx) + tag, err := r.pool.Exec(ctx, `DELETE FROM sessions WHERE id = $1`, id, ) if err != nil { + log.Errorw("delete session failed", "session_id", id, "error", err) return err } if tag.RowsAffected() == 0 { return ErrSessionNotFound } + + log.Debugw("session deleted", "session_id", id) return nil } diff --git a/backend/internal/store/user_pg.go b/backend/internal/store/user_pg.go index fec0631..89390ed 100644 --- a/backend/internal/store/user_pg.go +++ b/backend/internal/store/user_pg.go @@ -7,6 +7,8 @@ import ( "github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5/pgxpool" + + "github.com/hhs/camtalk/internal/trace" ) // PgUserRepository 基于 PostgreSQL 的 UserRepository 实现。 @@ -20,18 +22,25 @@ func NewPgUserRepository(pool *pgxpool.Pool) *PgUserRepository { } func (r *PgUserRepository) Create(ctx context.Context, username, passwordHash string) (string, error) { + log := trace.FromContext(ctx) + var id string err := r.pool.QueryRow(ctx, `INSERT INTO users (username, password_hash) VALUES ($1, $2) RETURNING id`, username, passwordHash, ).Scan(&id) if err != nil { + log.Errorw("create user failed", "username", username, "error", err) return "", err } + + log.Debugw("user created", "user_id", id, "username", username) return id, nil } func (r *PgUserRepository) FindByUsername(ctx context.Context, username string) (*User, error) { + log := trace.FromContext(ctx) + var u User err := r.pool.QueryRow(ctx, `SELECT id, username, password_hash, created_at, updated_at FROM users WHERE username = $1`, @@ -41,12 +50,17 @@ func (r *PgUserRepository) FindByUsername(ctx context.Context, username string) return nil, ErrUserNotFound } if err != nil { + log.Errorw("find user by username failed", "username", username, "error", err) return nil, err } + + log.Debugw("user found by username", "user_id", u.ID, "username", username) return &u, nil } func (r *PgUserRepository) FindByID(ctx context.Context, id string) (*User, error) { + log := trace.FromContext(ctx) + var u User err := r.pool.QueryRow(ctx, `SELECT id, username, password_hash, created_at, updated_at FROM users WHERE id = $1`, @@ -56,20 +70,33 @@ func (r *PgUserRepository) FindByID(ctx context.Context, id string) (*User, erro return nil, ErrUserNotFound } if err != nil { + log.Errorw("find user by id failed", "user_id", id, "error", err) return nil, err } + + log.Debugw("user found by id", "user_id", id) return &u, nil } func (r *PgUserRepository) SaveRefreshToken(ctx context.Context, userID, tokenHash string, expiresAt time.Time) error { + log := trace.FromContext(ctx) + _, err := r.pool.Exec(ctx, `INSERT INTO refresh_tokens (user_id, token_hash, expires_at) VALUES ($1, $2, $3)`, userID, tokenHash, expiresAt, ) - return err + if err != nil { + log.Errorw("save refresh token failed", "user_id", userID, "error", err) + return err + } + + log.Debugw("refresh token saved", "user_id", userID) + return nil } func (r *PgUserRepository) FindRefreshToken(ctx context.Context, tokenHash string) (string, error) { + log := trace.FromContext(ctx) + var userID string err := r.pool.QueryRow(ctx, `SELECT user_id FROM refresh_tokens WHERE token_hash = $1 AND expires_at > NOW()`, @@ -79,23 +106,42 @@ func (r *PgUserRepository) FindRefreshToken(ctx context.Context, tokenHash strin return "", ErrRefreshTokenNotFound } if err != nil { + log.Errorw("find refresh token failed", "error", err) return "", err } + + log.Debugw("refresh token found", "user_id", userID) return userID, nil } func (r *PgUserRepository) DeleteRefreshToken(ctx context.Context, tokenHash string) error { + log := trace.FromContext(ctx) + _, err := r.pool.Exec(ctx, `DELETE FROM refresh_tokens WHERE token_hash = $1`, tokenHash, ) - return err + if err != nil { + log.Errorw("delete refresh token failed", "error", err) + return err + } + + log.Debugw("refresh token deleted") + return nil } func (r *PgUserRepository) DeleteUserRefreshTokens(ctx context.Context, userID string) error { + log := trace.FromContext(ctx) + _, err := r.pool.Exec(ctx, `DELETE FROM refresh_tokens WHERE user_id = $1`, userID, ) - return err + if err != nil { + log.Errorw("delete user refresh tokens failed", "user_id", userID, "error", err) + return err + } + + log.Debugw("user refresh tokens deleted", "user_id", userID) + return nil } diff --git a/backend/internal/store/user_scenario_repository.go b/backend/internal/store/user_scenario_repository.go new file mode 100644 index 0000000..a1e0eb3 --- /dev/null +++ b/backend/internal/store/user_scenario_repository.go @@ -0,0 +1,276 @@ +package store + +import ( + "context" + "fmt" + "time" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/hhs/camtalk/internal/models" + "github.com/hhs/camtalk/internal/trace" +) + +// UserScenarioRepository 用户自建情景仓储接口。 +type UserScenarioRepository interface { + Create(ctx context.Context, scenario *models.UserScenario) error + FindByID(ctx context.Context, id string) (*models.UserScenario, error) + FindByIDAndUserID(ctx context.Context, id, userID string) (*models.UserScenario, error) + FindByUserID(ctx context.Context, userID string) ([]*models.UserScenario, error) + Update(ctx context.Context, scenario *models.UserScenario) error + Delete(ctx context.Context, id string) error + CountByUserID(ctx context.Context, userID string) (int, error) +} + +// PostgresUserScenarioRepo PostgreSQL 实现。 +type PostgresUserScenarioRepo struct { + pool *pgxpool.Pool +} + +// NewPostgresUserScenarioRepo 创建 PostgreSQL 用户情景仓储。 +func NewPostgresUserScenarioRepo(pool *pgxpool.Pool) UserScenarioRepository { + return &PostgresUserScenarioRepo{pool: pool} +} + +// Create 创建用户情景。 +func (r *PostgresUserScenarioRepo) Create(ctx context.Context, scenario *models.UserScenario) error { + log := trace.FromContext(ctx) + + query := ` + INSERT INTO user_scenarios (id, user_id, name, icon, description, prompt, greeting, language, created_at, updated_at) + VALUES ($1, $2, $3, $4, NULLIF($5, ''), $6, NULLIF($7, ''), $8, $9, $10) + RETURNING id, created_at, updated_at + ` + + now := time.Now() + scenario.CreatedAt = now + scenario.UpdatedAt = now + + if scenario.ID == "" { + scenario.ID = uuid.New().String() + } + if scenario.Icon == "" { + scenario.Icon = "✨" + } + if scenario.Language == "" { + scenario.Language = "zh-CN" + } + + err := r.pool.QueryRow(ctx, query, + scenario.ID, + scenario.UserID, + scenario.Name, + scenario.Icon, + scenario.Description, + scenario.Prompt, + scenario.Greeting, + scenario.Language, + scenario.CreatedAt, + scenario.UpdatedAt, + ).Scan(&scenario.ID, &scenario.CreatedAt, &scenario.UpdatedAt) + + if err != nil { + log.Errorw("create user scenario failed", "user_id", scenario.UserID, "name", scenario.Name, "error", err) + return fmt.Errorf("create user scenario: %w", err) + } + + log.Debugw("user scenario created", "scenario_id", scenario.ID, "user_id", scenario.UserID, "name", scenario.Name) + return nil +} + +// FindByID 根据 ID 查找情景。 +func (r *PostgresUserScenarioRepo) FindByID(ctx context.Context, id string) (*models.UserScenario, error) { + log := trace.FromContext(ctx) + + query := ` + SELECT id, user_id, name, icon, description, prompt, greeting, language, created_at, updated_at + FROM user_scenarios + WHERE id = $1 + ` + + var scenario models.UserScenario + err := r.pool.QueryRow(ctx, query, id).Scan( + &scenario.ID, + &scenario.UserID, + &scenario.Name, + &scenario.Icon, + &scenario.Description, + &scenario.Prompt, + &scenario.Greeting, + &scenario.Language, + &scenario.CreatedAt, + &scenario.UpdatedAt, + ) + + if err == pgx.ErrNoRows { + return nil, fmt.Errorf("user scenario not found: %s", id) + } + if err != nil { + log.Errorw("find user scenario failed", "scenario_id", id, "error", err) + return nil, fmt.Errorf("find user scenario: %w", err) + } + + log.Debugw("user scenario found", "scenario_id", id) + return &scenario, nil +} + +// FindByIDAndUserID 根据 ID 和用户 ID 查找情景（权限校验）。 +func (r *PostgresUserScenarioRepo) FindByIDAndUserID(ctx context.Context, id, userID string) (*models.UserScenario, error) { + log := trace.FromContext(ctx) + + query := ` + SELECT id, user_id, name, icon, description, prompt, greeting, language, created_at, updated_at + FROM user_scenarios + WHERE id = $1 AND user_id = $2 + ` + + var scenario models.UserScenario + err := r.pool.QueryRow(ctx, query, id, userID).Scan( + &scenario.ID, + &scenario.UserID, + &scenario.Name, + &scenario.Icon, + &scenario.Description, + &scenario.Prompt, + &scenario.Greeting, + &scenario.Language, + &scenario.CreatedAt, + &scenario.UpdatedAt, + ) + + if err == pgx.ErrNoRows { + return nil, fmt.Errorf("user scenario not found or no permission") + } + if err != nil { + log.Errorw("find user scenario by id and user failed", "scenario_id", id, "user_id", userID, "error", err) + return nil, fmt.Errorf("find user scenario: %w", err) + } + + log.Debugw("user scenario found by id and user", "scenario_id", id, "user_id", userID) + return &scenario, nil +} + +// FindByUserID 查找用户的所有情景。 +func (r *PostgresUserScenarioRepo) FindByUserID(ctx context.Context, userID string) ([]*models.UserScenario, error) { + log := trace.FromContext(ctx) + + query := ` + SELECT id, user_id, name, icon, description, prompt, greeting, language, created_at, updated_at + FROM user_scenarios + WHERE user_id = $1 + ORDER BY created_at DESC + ` + + rows, err := r.pool.Query(ctx, query, userID) + if err != nil { + log.Errorw("find user scenarios failed", "user_id", userID, "error", err) + return nil, fmt.Errorf("find user scenarios: %w", err) + } + defer rows.Close() + + var scenarios []*models.UserScenario + for rows.Next() { + var s models.UserScenario + err := rows.Scan( + &s.ID, + &s.UserID, + &s.Name, + &s.Icon, + &s.Description, + &s.Prompt, + &s.Greeting, + &s.Language, + &s.CreatedAt, + &s.UpdatedAt, + ) + if err != nil { + log.Errorw("scan user scenario row failed", "user_id", userID, "error", err) + return nil, fmt.Errorf("scan user scenario: %w", err) + } + scenarios = append(scenarios, &s) + } + + if err = rows.Err(); err != nil { + log.Errorw("iterate user scenarios failed", "user_id", userID, "error", err) + return nil, fmt.Errorf("iterate user scenarios: %w", err) + } + + log.Debugw("user scenarios found", "user_id", userID, "count", len(scenarios)) + return scenarios, nil +} + +// Update 更新用户情景。 +func (r *PostgresUserScenarioRepo) Update(ctx context.Context, scenario *models.UserScenario) error { + log := trace.FromContext(ctx) + + query := ` + UPDATE user_scenarios + SET name = $1, icon = $2, description = $3, prompt = $4, greeting = $5, language = $6, updated_at = $7 + WHERE id = $8 AND user_id = $9 + RETURNING updated_at + ` + + scenario.UpdatedAt = time.Now() + + err := r.pool.QueryRow(ctx, query, + scenario.Name, + scenario.Icon, + scenario.Description, + scenario.Prompt, + scenario.Greeting, + scenario.Language, + scenario.UpdatedAt, + scenario.ID, + scenario.UserID, + ).Scan(&scenario.UpdatedAt) + + if err == pgx.ErrNoRows { + return fmt.Errorf("user scenario not found or no permission") + } + if err != nil { + log.Errorw("update user scenario failed", "scenario_id", scenario.ID, "user_id", scenario.UserID, "error", err) + return fmt.Errorf("update user scenario: %w", err) + } + + log.Debugw("user scenario updated", "scenario_id", scenario.ID, "user_id", scenario.UserID) + return nil +} + +// Delete 删除用户情景。 +func (r *PostgresUserScenarioRepo) Delete(ctx context.Context, id string) error { + log := trace.FromContext(ctx) + + query := `DELETE FROM user_scenarios WHERE id = $1` + + result, err := r.pool.Exec(ctx, query, id) + if err != nil { + log.Errorw("delete user scenario failed", "scenario_id", id, "error", err) + return fmt.Errorf("delete user scenario: %w", err) + } + + if result.RowsAffected() == 0 { + return fmt.Errorf("user scenario not found") + } + + log.Debugw("user scenario deleted", "scenario_id", id) + return nil +} + +// CountByUserID 统计用户的情景数量。 +func (r *PostgresUserScenarioRepo) CountByUserID(ctx context.Context, userID string) (int, error) { + log := trace.FromContext(ctx) + + query := `SELECT COUNT(*) FROM user_scenarios WHERE user_id = $1` + + var count int + err := r.pool.QueryRow(ctx, query, userID).Scan(&count) + if err != nil { + log.Errorw("count user scenarios failed", "user_id", userID, "error", err) + return 0, fmt.Errorf("count user scenarios: %w", err) + } + + log.Debugw("user scenarios counted", "user_id", userID, "count", count) + return count, nil +} diff --git a/backend/internal/trace/context.go b/backend/internal/trace/context.go new file mode 100644 index 0000000..e023a7a --- /dev/null +++ b/backend/internal/trace/context.go @@ -0,0 +1,46 @@ +package trace + +import "context" + +type traceIDKey struct{} +type requestIDKey struct{} +type sessionIDKey struct{} + +// WithTraceID 将 trace ID 注入 context（连接级/会话级标识） +func WithTraceID(ctx context.Context, traceID string) context.Context { + return context.WithValue(ctx, traceIDKey{}, traceID) +} + +// GetTraceID 从 context 提取 trace ID +func GetTraceID(ctx context.Context) string { + if v, ok := ctx.Value(traceIDKey{}).(string); ok { + return v + } + return "" +} + +// WithRequestID 将 request ID 注入 context（单次请求/查询标识） +func WithRequestID(ctx context.Context, requestID string) context.Context { + return context.WithValue(ctx, requestIDKey{}, requestID) +} + +// GetRequestID 从 context 提取 request ID +func GetRequestID(ctx context.Context) string { + if v, ok := ctx.Value(requestIDKey{}).(string); ok { + return v + } + return "" +} + +// WithSessionID 将 session ID 注入 context（会话存储标识） +func WithSessionID(ctx context.Context, sessionID string) context.Context { + return context.WithValue(ctx, sessionIDKey{}, sessionID) +} + +// GetSessionID 从 context 提取 session ID +func GetSessionID(ctx context.Context) string { + if v, ok := ctx.Value(sessionIDKey{}).(string); ok { + return v + } + return "" +} diff --git a/backend/internal/trace/eino_test.go b/backend/internal/trace/eino_test.go new file mode 100644 index 0000000..c42401c --- /dev/null +++ b/backend/internal/trace/eino_test.go @@ -0,0 +1,42 @@ +package trace_test + +import ( + "context" + "testing" + + "github.com/cloudwego/eino/compose" + "github.com/hhs/camtalk/internal/trace" +) + +func TestEinoContextPropagation(t *testing.T) { + ctx := context.Background() + testTraceID := "01J5TEST123456789" + ctx = trace.WithTraceID(ctx, testTraceID) + + var capturedTraceID string + + g := compose.NewGraph[string, string]() + g.AddLambdaNode("test_node", compose.InvokableLambda( + func(ctx context.Context, input string) (string, error) { + capturedTraceID = trace.GetTraceID(ctx) + return "ok", nil + }, + )) + g.AddEdge(compose.START, "test_node") + g.AddEdge("test_node", compose.END) + + runnable, err := g.Compile(ctx) + if err != nil { + t.Fatalf("compile failed: %v", err) + } + + _, err = runnable.Invoke(ctx, "test_input") + if err != nil { + t.Fatalf("invoke failed: %v", err) + } + + if capturedTraceID != testTraceID { + t.Errorf("trace_id lost in Eino propagation: got %q, want %q", + capturedTraceID, testTraceID) + } +} diff --git a/backend/internal/trace/gin_logger.go b/backend/internal/trace/gin_logger.go new file mode 100644 index 0000000..1125a08 --- /dev/null +++ b/backend/internal/trace/gin_logger.go @@ -0,0 +1,63 @@ +package trace + +import ( + "time" + + "github.com/gin-gonic/gin" +) + +// GinLogger 记录每个 HTTP 请求的 method/path/status/latency +func GinLogger() gin.HandlerFunc { + return func(c *gin.Context) { + start := time.Now() + path := c.Request.URL.Path + query := c.Request.URL.RawQuery + + c.Next() + + latency := time.Since(start).Milliseconds() + status := c.Writer.Status() + log := FromContext(c.Request.Context()) + + fields := []interface{}{ + "method", c.Request.Method, + "path", path, + "status", status, + "latency_ms", latency, + "client_ip", c.ClientIP(), + } + if query != "" { + fields = append(fields, "query", query) + } + if errStr := c.Errors.String(); errStr != "" { + fields = append(fields, "errors", errStr) + } + + switch { + case status >= 500: + log.Errorw("request completed", fields...) + case status >= 400: + log.Warnw("request completed", fields...) + default: + log.Infow("request completed", fields...) + } + } +} + +// GinRecovery 自定义 panic 恢复中间件，使用 zap 记录 +func GinRecovery() gin.HandlerFunc { + return func(c *gin.Context) { + defer func() { + if err := recover(); err != nil { + log := FromContext(c.Request.Context()) + log.Errorw("panic recovered", + "error", err, + "path", c.Request.URL.Path, + "method", c.Request.Method, + "client_ip", c.ClientIP()) + c.AbortWithStatus(500) + } + }() + c.Next() + } +} diff --git a/backend/internal/trace/id.go b/backend/internal/trace/id.go new file mode 100644 index 0000000..23d35f9 --- /dev/null +++ b/backend/internal/trace/id.go @@ -0,0 +1,22 @@ +package trace + +import ( + cryptorand "crypto/rand" + "sync" + "time" + + "github.com/oklog/ulid/v2" +) + +var entropyPool = sync.Pool{ + New: func() interface{} { + return ulid.Monotonic(cryptorand.Reader, 0) + }, +} + +// GenerateTraceID 生成并发安全的 ULID trace ID +func GenerateTraceID() string { + entropy := entropyPool.Get().(*ulid.MonotonicEntropy) + defer entropyPool.Put(entropy) + return ulid.MustNew(ulid.Timestamp(time.Now()), entropy).String() +} diff --git a/backend/internal/trace/logger.go b/backend/internal/trace/logger.go new file mode 100644 index 0000000..b1fc921 --- /dev/null +++ b/backend/internal/trace/logger.go @@ -0,0 +1,25 @@ +package trace + +import ( + "context" + + "github.com/hhs/camtalk/internal/logger" + "go.uber.org/zap" +) + +// FromContext 返回自动附加 trace_id/request_id/session_id 的 logger +func FromContext(ctx context.Context) *zap.SugaredLogger { + log := logger.Log + + if traceID := GetTraceID(ctx); traceID != "" { + log = log.With("trace_id", traceID) + } + if requestID := GetRequestID(ctx); requestID != "" { + log = log.With("request_id", requestID) + } + if sessionID := GetSessionID(ctx); sessionID != "" { + log = log.With("session_id", sessionID) + } + + return log +} diff --git a/backend/internal/trace/middleware.go b/backend/internal/trace/middleware.go new file mode 100644 index 0000000..0b92262 --- /dev/null +++ b/backend/internal/trace/middleware.go @@ -0,0 +1,17 @@ +package trace + +import "github.com/gin-gonic/gin" + +// TraceMiddleware 为每个 HTTP 请求生成 trace ID 并注入 context +func TraceMiddleware() gin.HandlerFunc { + return func(c *gin.Context) { + traceID := GenerateTraceID() + ctx := WithTraceID(c.Request.Context(), traceID) + ctx = WithRequestID(ctx, traceID) // REST: trace_id == request_id + + c.Request = c.Request.WithContext(ctx) + c.Header("X-Trace-ID", traceID) // 返回给客户端用于排查 + + c.Next() + } +} diff --git a/backend/internal/util/string.go b/backend/internal/util/string.go new file mode 100644 index 0000000..460be86 --- /dev/null +++ b/backend/internal/util/string.go @@ -0,0 +1,9 @@ +package util + +// Truncate 截断字符串到指定长度，超出部分用 "..." 替换 +func Truncate(s string, maxLen int) string { + if len(s) <= maxLen { + return s + } + return s[:maxLen] + "..." +} diff --git a/backend/internal/ws/handler.go b/backend/internal/ws/handler.go index cb7f25b..6b3500e 100644 --- a/backend/internal/ws/handler.go +++ b/backend/internal/ws/handler.go @@ -3,6 +3,7 @@ package ws import ( "context" "encoding/json" + "fmt" "net/http" "sync" "time" @@ -10,13 +11,16 @@ import ( "github.com/gin-gonic/gin" "github.com/gorilla/websocket" + "github.com/hhs/camtalk/internal/ai/llm" "github.com/hhs/camtalk/internal/auth" "github.com/hhs/camtalk/internal/config" "github.com/hhs/camtalk/internal/errors" - "github.com/hhs/camtalk/internal/logger" "github.com/hhs/camtalk/internal/models" "github.com/hhs/camtalk/internal/orchestrator" + "github.com/hhs/camtalk/internal/ratelimit" "github.com/hhs/camtalk/internal/session" + "github.com/hhs/camtalk/internal/store" + "github.com/hhs/camtalk/internal/trace" ) // newUpgrader 根据配置创建 WebSocket upgrader。 @@ -40,12 +44,12 @@ func newUpgrader(cfg *config.Config) websocket.Upgrader { // Client 代表一个 WebSocket 客户端连接。 type Client struct { - conn *websocket.Conn - sessionID string - sessionMgr session.Manager - orchestrator orchestrator.Orchestrator - cancelFuncs map[string]context.CancelFunc // requestID → cancel func - mu sync.Mutex + conn *websocket.Conn + sessionID string + sessionMgr session.Manager + orchestrator orchestrator.Orchestrator + cancelFuncs map[string]context.CancelFunc // requestID → cancel func + mu sync.Mutex } // SendJSON 向客户端发送 JSON 消息（公开以便 errors 包调用）。 @@ -92,21 +96,19 @@ func (w *WSClient) SendError(err models.WsError) error { } // ServeWS 处理 WebSocket 升级请求。 -func ServeWS(sessionMgr session.Manager, orch orchestrator.Orchestrator, cfg *config.Config, tokenMgr *auth.TokenManager) gin.HandlerFunc { +func ServeWS(sessionMgr session.Manager, orch orchestrator.Orchestrator, cfg *config.Config, tokenMgr *auth.TokenManager, limiter ratelimit.Limiter, scenarioRepo store.UserScenarioRepository) gin.HandlerFunc { upgrader := newUpgrader(cfg) heartbeatInterval := time.Duration(cfg.Server.HeartbeatInterval) * time.Second heartbeatTimeout := time.Duration(cfg.Server.HeartbeatTimeout) * time.Second version := cfg.App.Version - maxHistory := cfg.Session.MaxHistory - return func(c *gin.Context) { - serveWS(c, sessionMgr, orch, upgrader, heartbeatInterval, heartbeatTimeout, version, maxHistory, tokenMgr) + serveWS(c, sessionMgr, orch, upgrader, heartbeatInterval, heartbeatTimeout, version, tokenMgr, limiter, scenarioRepo) } } func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orchestrator, - upgrader websocket.Upgrader, heartbeatInterval, heartbeatTimeout time.Duration, version string, maxHistory int, tokenMgr *auth.TokenManager) { + upgrader websocket.Upgrader, heartbeatInterval, heartbeatTimeout time.Duration, version string, tokenMgr *auth.TokenManager, limiter ratelimit.Limiter, scenarioRepo store.UserScenarioRepository) { // --- JWT 认证（upgrade 前完成，失败直接返回 HTTP 错误） --- token := c.Query("token") @@ -132,9 +134,20 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche } } + // 生成连接级 trace ID（整个 WebSocket 生命周期使用） + ctx := c.Request.Context() + traceID := trace.GetTraceID(ctx) + if traceID == "" { + // 如果 REST 中间件未生成（不应发生），fallback 生成 + traceID = trace.GenerateTraceID() + ctx = trace.WithTraceID(ctx, traceID) + c.Request = c.Request.WithContext(ctx) + } + conn, err := upgrader.Upgrade(c.Writer, c.Request, nil) if err != nil { - logger.Log.Errorw("websocket upgrade failed", "error", err) + log := trace.FromContext(ctx) + log.Errorw("websocket upgrade failed", "error", err) return } defer conn.Close() @@ -143,13 +156,17 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche var sessionID string if conversationID != "" { sessionID = conversationID - logger.Log.Infow("resuming conversation", "session", sessionID, "user_id", userID) + ctx = trace.WithSessionID(ctx, sessionID) + log := trace.FromContext(ctx) + log.Infow("resuming conversation", "user_id", userID) } else { sessionID, err = sessionMgr.Create(context.Background(), userID, models.DefaultConfig()) if err != nil { - logger.Log.Errorw("create session failed", "error", err) + log := trace.FromContext(ctx) + log.Errorw("create session failed", "error", err) return } + ctx = trace.WithSessionID(ctx, sessionID) } client := &Client{ @@ -166,7 +183,8 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche SessionID: sessionID, ServerVersion: version, }) - logger.Log.Infow("client connected", "session", sessionID, "user_id", userID, "username", username) + log := trace.FromContext(ctx) + log.Infow("client connected", "user_id", userID, "username", username) // 心跳检测 lastPong := time.Now() @@ -184,7 +202,8 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche select { case <-ticker.C: if time.Since(lastPong) > heartbeatTimeout { - logger.Log.Warnw("heartbeat timeout", "session", sessionID) + log := trace.FromContext(ctx) + log.Warnw("heartbeat timeout") conn.Close() return } @@ -199,7 +218,8 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche _, message, err := conn.ReadMessage() if err != nil { if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseNormalClosure) { - logger.Log.Warnw("ws read error", "error", err) + log := trace.FromContext(ctx) + log.Warnw("ws read error", "error", err) } break } @@ -224,23 +244,36 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche errors.SendWSError(client, errors.CodeInvalidMessage, msg.RequestID, err) continue } - logger.Log.Infow("query received", "session", sessionID, "request", msg.RequestID) + + // 注入 request ID 到 context + queryCtx := trace.WithRequestID(ctx, msg.RequestID) + log := trace.FromContext(queryCtx) + log.Infow("query received", "has_image", msg.Image != "", "has_audio", msg.Audio != "") + + // 限流检查 + if limiter != nil { + key := fmt.Sprintf("%s:query", userID) + allowed, retryAfter := limiter.Allow(context.Background(), key) + if !allowed { + log.Warnw("rate limited", "user_id", userID, "retry_after", retryAfter) + errors.SendWSError(client, errors.CodeRateLimited, msg.RequestID, + fmt.Errorf("rate limited, retry after %s", retryAfter.Round(time.Second))) + continue + } + } // 刷新会话 TTL if err := client.sessionMgr.Touch(context.Background(), sessionID); err != nil { - logger.Log.Warnw("touch session failed", "session", sessionID, "error", err) + log.Warnw("touch session failed", "error", err) } // 标记活跃请求 if err := client.sessionMgr.SetActiveRequest(context.Background(), sessionID, msg.RequestID); err != nil { - logger.Log.Warnw("set active request failed", "session", sessionID, "error", err) + log.Warnw("set active request failed", "error", err) } - // 获取对话历史 - history, _ := client.sessionMgr.GetHistory(context.Background(), sessionID, maxHistory) - // 创建可取消的 context - ctx, cancel := context.WithCancel(context.Background()) + processCtx, cancel := context.WithCancel(queryCtx) client.mu.Lock() client.cancelFuncs[msg.RequestID] = cancel client.mu.Unlock() @@ -260,8 +293,9 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche _ = client.sessionMgr.ClearActiveRequest(context.Background(), sessionID) }() - if err := client.orchestrator.ProcessQuery(ctx, sessionID, msg, history, sender); err != nil { - logger.Log.Errorw("process query failed", "session", sessionID, "request", msg.RequestID, "error", err) + if err := client.orchestrator.ProcessQuery(processCtx, sessionID, msg, sender); err != nil { + log := trace.FromContext(processCtx) + log.Errorw("process query failed", "error", err) } }() @@ -282,10 +316,66 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche errors.SendWSError(client, errors.CodeInternalError, "", err) continue } - logger.Log.Infow("config updated", "session", sessionID) + + scenarioID := "" + if msg.Payload.Scenario != nil { + scenarioID = *msg.Payload.Scenario + } + log := trace.FromContext(ctx) + log.Infow("config updated", "scenario", scenarioID) + + // 如果切换了情景（非自由对话），返回首句引导 + if scenarioID != "" && scenarioID != "free_chat" { + sess, err := client.sessionMgr.Get(context.Background(), sessionID) + if err == nil && sess != nil { + // 加载用户自建情景 + var customGreetings map[string]string + if sess.UserID != "" && scenarioRepo != nil { + scenarios, err := scenarioRepo.FindByUserID(context.Background(), sess.UserID) + if err == nil && len(scenarios) > 0 { + customGreetings = make(map[string]string, len(scenarios)) + for _, s := range scenarios { + if s.Greeting != "" { + customGreetings[s.ID] = s.Greeting + } + } + } + } + + greeting := llm.GetScenarioGreeting(scenarioID, sess.Config.Language, customGreetings) + if greeting != "" { + // 发送首句作为 AI 消息 + _ = client.SendJSON(models.WsLLMChunk{ + Type: "llm_chunk", + RequestID: "scenario_greeting", + Delta: greeting, + Role: "assistant", + }) + + doneMsg := models.WsLLMDone{ + Type: "llm_done", + RequestID: "scenario_greeting", + FullText: greeting, + Model: "", + LatencyMs: 0, + } + doneMsg.TokensUsed.Prompt = 0 + doneMsg.TokensUsed.Completion = 0 + doneMsg.TokensUsed.Total = 0 + _ = client.SendJSON(doneMsg) + + // 追加首句到历史记录 + _ = client.sessionMgr.AppendMessage(context.Background(), sessionID, models.Message{ + Role: "assistant", + Content: greeting, + }) + } + } + } case "interrupt": - logger.Log.Infow("interrupt received", "session", sessionID) + log := trace.FromContext(ctx) + log.Infow("interrupt received") // 获取活跃请求 ID 并取消 reqID, _ := client.sessionMgr.GetActiveRequestID(context.Background(), sessionID) @@ -313,12 +403,14 @@ func serveWS(c *gin.Context, sessionMgr session.Manager, orch orchestrator.Orche // 取消所有活跃请求 client.mu.Lock() for reqID, cancel := range client.cancelFuncs { - logger.Log.Infow("canceling active request on disconnect", "session", sessionID, "request", reqID) + log := trace.FromContext(ctx) + log.Infow("canceling active request on disconnect", "request", reqID) cancel() } client.cancelFuncs = make(map[string]context.CancelFunc) client.mu.Unlock() // 断开连接时不销毁会话，让其自然过期（支持重连恢复） - logger.Log.Infow("client disconnected", "session", sessionID) + log = trace.FromContext(ctx) + log.Infow("client disconnected") } diff --git a/backend/internal/ws/handler_test.go b/backend/internal/ws/handler_test.go index 69f1207..72893bb 100644 --- a/backend/internal/ws/handler_test.go +++ b/backend/internal/ws/handler_test.go @@ -8,11 +8,11 @@ import ( "testing" "time" + "context" "github.com/gin-gonic/gin" "github.com/gorilla/websocket" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" - "context" "github.com/hhs/camtalk/internal/auth" "github.com/hhs/camtalk/internal/config" @@ -48,7 +48,6 @@ func (m *MockOrchestrator) ProcessQuery( ctx context.Context, sessionID string, req models.WsQuery, - history []models.Message, sender orchestrator.Sender, ) error { if m.Err != nil { @@ -149,7 +148,7 @@ func setupTestServer(t *testing.T, orch orchestrator.Orchestrator) (*httptest.Se Server: config.ServerConfig{HeartbeatInterval: 30, HeartbeatTimeout: 60}, Session: config.SessionConfig{MaxHistory: 20}, } - r.GET("/ws", ServeWS(sessionMgr, orch, cfg, tokenMgr)) + r.GET("/ws", ServeWS(sessionMgr, orch, cfg, tokenMgr, nil, nil)) srv := httptest.NewServer(r) @@ -222,9 +221,9 @@ func TestWS_QueryFullFlow(t *testing.T) { imageB64 := base64.StdEncoding.EncodeToString([]byte("fake-image-data")) mock := &MockOrchestrator{ - STTResult: "你好，世界", - LLMDeltas: []string{"你好", "，世界！"}, - TTSAudios: []string{base64.StdEncoding.EncodeToString([]byte("mp3-data-1")), base64.StdEncoding.EncodeToString([]byte("mp3-data-2"))}, + STTResult: "你好，世界", + LLMDeltas: []string{"你好", "，世界！"}, + TTSAudios: []string{base64.StdEncoding.EncodeToString([]byte("mp3-data-1")), base64.StdEncoding.EncodeToString([]byte("mp3-data-2"))}, } srv, wsURL := setupTestServer(t, mock) @@ -333,7 +332,7 @@ func TestWS_UnknownMessageType(t *testing.T) { err := conn.WriteJSON(map[string]string{"type": "unknown_type"}) require.NoError(t, err) - errMsg := readJSON(t, conn) + errMsg := readJSON(t, conn) assert.Equal(t, "error", errMsg["type"]) assert.Equal(t, "INVALID_MESSAGE", errMsg["code"]) assert.Contains(t, errMsg["message"], "unknown message type") @@ -592,7 +591,7 @@ func setupTestServerEx(t *testing.T, orch orchestrator.Orchestrator) (*httptest. Server: config.ServerConfig{HeartbeatInterval: 30, HeartbeatTimeout: 60}, Session: config.SessionConfig{MaxHistory: 20}, } - r.GET("/ws", ServeWS(sessionMgr, orch, cfg, tokenMgr)) + r.GET("/ws", ServeWS(sessionMgr, orch, cfg, tokenMgr, nil, nil)) srv := httptest.NewServer(r) return srv, tokenMgr, sessionMgr @@ -643,7 +642,7 @@ func TestWS_AuthExpiredToken(t *testing.T) { Server: config.ServerConfig{HeartbeatInterval: 30, HeartbeatTimeout: 60}, Session: config.SessionConfig{MaxHistory: 20}, } - r.GET("/ws", ServeWS(sessionMgr, &MockOrchestrator{}, cfg, tokenMgr)) + r.GET("/ws", ServeWS(sessionMgr, &MockOrchestrator{}, cfg, tokenMgr, nil, nil)) srv := httptest.NewServer(r) defer srv.Close() diff --git a/backend/migrations/004_user_scenarios.down.sql b/backend/migrations/004_user_scenarios.down.sql new file mode 100644 index 0000000..443f4ee --- /dev/null +++ b/backend/migrations/004_user_scenarios.down.sql @@ -0,0 +1,6 @@ +-- 004_user_scenarios.down.sql +-- 回滚用户自建情景表 + +DROP INDEX IF EXISTS idx_user_scenarios_created_at; +DROP INDEX IF EXISTS idx_user_scenarios_user_id; +DROP TABLE IF EXISTS user_scenarios; diff --git a/backend/migrations/004_user_scenarios.up.sql b/backend/migrations/004_user_scenarios.up.sql new file mode 100644 index 0000000..1fc8870 --- /dev/null +++ b/backend/migrations/004_user_scenarios.up.sql @@ -0,0 +1,36 @@ +-- 004_user_scenarios.up.sql +-- 用户自建情景表 + +CREATE TABLE user_scenarios ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, + name VARCHAR(50) NOT NULL, + icon VARCHAR(10) DEFAULT '✨', + description VARCHAR(100) NOT NULL, + prompt TEXT NOT NULL, + greeting VARCHAR(200), + language VARCHAR(10) DEFAULT 'zh-CN', + created_at TIMESTAMP NOT NULL DEFAULT NOW(), + updated_at TIMESTAMP NOT NULL DEFAULT NOW(), + + CONSTRAINT unique_user_scenario UNIQUE(user_id, name), + CONSTRAINT check_name_length CHECK (char_length(name) >= 2 AND char_length(name) <= 50), + CONSTRAINT check_description_length CHECK (char_length(description) >= 5 AND char_length(description) <= 100), + CONSTRAINT check_prompt_length CHECK (char_length(prompt) >= 50 AND char_length(prompt) <= 2000) +); + +-- 为用户 ID 创建索引，加速查询 +CREATE INDEX idx_user_scenarios_user_id ON user_scenarios(user_id); + +-- 为创建时间创建索引，用于排序 +CREATE INDEX idx_user_scenarios_created_at ON user_scenarios(created_at DESC); + +COMMENT ON TABLE user_scenarios IS '用户自建情景表'; +COMMENT ON COLUMN user_scenarios.id IS '情景唯一标识'; +COMMENT ON COLUMN user_scenarios.user_id IS '所属用户 ID，外键关联 users 表'; +COMMENT ON COLUMN user_scenarios.name IS '情景名称，如"创意写作导师"'; +COMMENT ON COLUMN user_scenarios.icon IS 'Emoji 图标，如"🎨"'; +COMMENT ON COLUMN user_scenarios.description IS '简短描述，显示在情景卡片上'; +COMMENT ON COLUMN user_scenarios.prompt IS '角色 System Prompt，定义 AI 行为'; +COMMENT ON COLUMN user_scenarios.greeting IS '首句引导，可选'; +COMMENT ON COLUMN user_scenarios.language IS '默认语言，如 zh-CN、en-US'; diff --git a/deploy.sh b/deploy.sh index ebc22ea..eac2c1c 100755 --- a/deploy.sh +++ b/deploy.sh @@ -4,30 +4,52 @@ set -euo pipefail PROJECT_DIR="$(cd "$(dirname "$0")" && pwd)" cd "$PROJECT_DIR" +# .env 固定路径（act_runner 容器已挂载 /opt/camtalk） +ENV_FILE="/opt/camtalk/.env" + # 颜色输出 GREEN='\033[0;32m' NC='\033[0m' info() { echo -e "${GREEN}[INFO]${NC} $*"; } +# .env 检查：首次部署时从 .env.example 复制模板，提示用户填写 +check_env() { + if [ ! -f "$ENV_FILE" ]; then + echo "==============================================" + echo " 错误: 未找到环境变量文件" + echo " 路径: $ENV_FILE" + echo " 模板参考: backend/.env.example" + echo "==============================================" + exit 1 + fi +} + +check_env + +# 所有 docker compose 命令统一使用 --env-file，用于解析 ${POSTGRES_USER} 等变量 +DC="docker compose --env-file $ENV_FILE" + cmd_build() { info "构建 Docker 镜像..." - # 启用 BuildKit 加速构建 - DOCKER_BUILDKIT=1 docker compose build --parallel + DOCKER_BUILDKIT=1 $DC build --parallel info "构建完成" } cmd_up() { info "启动服务..." - docker compose up -d + $DC up -d info "服务已启动" - info "前端: http://8.161.227.145:9000" - info "健康检查: http://8.161.227.145:9000/api/health" + PUBLIC_IP=$(curl -s --connect-timeout 3 https://ifconfig.me 2>/dev/null || \ + curl -s --connect-timeout 3 https://api.ipify.org 2>/dev/null || \ + echo "YOUR_SERVER_IP") + info "前端: http://$PUBLIC_IP:9000" + info "健康检查: http://$PUBLIC_IP:9000/api/health" } cmd_down() { info "停止服务..." - docker compose down + $DC down info "服务已停止" } @@ -38,11 +60,11 @@ cmd_restart() { } cmd_logs() { - docker compose logs -f "${@}" + $DC logs -f "${@}" } cmd_status() { - docker compose ps + $DC ps } usage() { @@ -58,6 +80,8 @@ CamTalk 部署脚本 restart 重启服务 logs 查看日志（可加服务名，如: $0 logs backend） status 查看服务状态 + +.env 路径: $ENV_FILE EOF } @@ -69,4 +93,4 @@ case "${1:-}" in logs) shift; cmd_logs "$@" ;; status) cmd_status ;; *) usage; exit 1 ;; -esac +esac \ No newline at end of file diff --git a/docker-compose.yml b/docker-compose.yml index fe4d766..f2e5ff1 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -17,25 +17,31 @@ services: context: ./backend dockerfile: Dockerfile container_name: camtalk-backend + env_file: + - /opt/camtalk/.env environment: - - APP_ENV=production - - CAMTALK_STORAGE_DRIVER=postgres - - CAMTALK_STORAGE_DSN=postgres://camtalk:camtalk123@postgres:5432/camtalk?sslmode=disable - - CAMTALK_AUTH_JWT_SECRET=78uWBBAF8XEQEotKDlrnlnd4y8i4WN3E4zXmNmC8BYQ= + # 运行环境（强制生产环境） + - APP_ENV=prod + # 三级存储配置（敏感信息通过 env_file 注入） + - CAMTALK_STORAGE_REDIS_ENABLED=${CAMTALK_STORAGE_REDIS_ENABLED:-true} + - CAMTALK_STORAGE_PERSISTENCE_ENABLED=${CAMTALK_STORAGE_PERSISTENCE_ENABLED:-true} + - CAMTALK_STORAGE_PERSISTENCE_DRIVER=${CAMTALK_STORAGE_PERSISTENCE_DRIVER:-postgres} + - CAMTALK_REDIS_ADDR=redis:6379 depends_on: postgres: condition: service_healthy + redis: + condition: service_healthy networks: - camtalk-net restart: unless-stopped postgres: - # 轩辕镜像加速，避免 Docker Hub 拉取超时 image: docker.m.daocloud.io/library/postgres:15-alpine container_name: camtalk-postgres + env_file: + - /opt/camtalk/.env environment: - POSTGRES_USER: camtalk - POSTGRES_PASSWORD: camtalk123 POSTGRES_DB: camtalk volumes: - pgdata:/var/lib/postgresql/data @@ -43,7 +49,30 @@ services: networks: - camtalk-net healthcheck: - test: ["CMD-SHELL", "pg_isready -U camtalk -d camtalk"] + test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d camtalk"] + interval: 5s + timeout: 3s + retries: 10 + restart: unless-stopped + + redis: + image: docker.m.daocloud.io/library/redis:7-alpine + container_name: camtalk-redis + env_file: + - /opt/camtalk/.env + command: > + sh -c ' + if [ -n "$$CAMTALK_REDIS_PASSWORD" ]; then + exec redis-server --appendonly yes --requirepass "$$CAMTALK_REDIS_PASSWORD" + else + exec redis-server --appendonly yes + fi' + volumes: + - redisdata:/data + networks: + - camtalk-net + healthcheck: + test: ["CMD-SHELL", "if [ -n \"$$CAMTALK_REDIS_PASSWORD\" ]; then redis-cli -a \"$$CAMTALK_REDIS_PASSWORD\" ping; else redis-cli ping; fi"] interval: 5s timeout: 3s retries: 10 @@ -51,6 +80,7 @@ services: volumes: pgdata: + redisdata: networks: camtalk-net: diff --git a/docs/01-架构设计.md b/docs/01-架构设计.md new file mode 100644 index 0000000..d23b8f4 --- /dev/null +++ b/docs/01-架构设计.md @@ -0,0 +1,369 @@ +# 架构设计 + +## 项目概述 + +CamTalk 是一款**多模态实时 AI 视觉对话助手**。用户通过摄像头和麦克风与 AI 交互，AI 理解视觉场景和语音输入后，以文字和语音形式给出自然回应。 + +核心挑战在于三个维度之间的张力： + +| 维度 | 关键问题 | +|------|---------| +| 视觉理解 | 如何准确理解摄像头画面中的人物、物体、场景？ | +| 语音交互 | 如何让对话像真人交流一样自然、低延迟？ | +| 成本控制 | 实时视频流 + LLM 推理，如何避免账单爆炸？ | + +## 系统架构 + +三层架构：**前端做轻量预处理，后端做智能编排，云端 AI 服务按需调用**。 + +```mermaid +graph TB + subgraph Browser["浏览器客户端"] + UI["UI 渲染层
React 18 + TypeScript"] + Edge["边缘预处理层
VAD / 关键帧检测"] + Media["媒体采集层
Camera / Microphone"] + end + + subgraph Gateway["Go 网关"] + WS["WebSocket Handler
连接管理 / 消息分发"] + Session["Session Manager
会话状态 / 对话历史"] + Orch["AI Orchestrator
Eino Graph 声明式编排"] + Auth["Auth 模块
JWT / bcrypt"] + REST["REST API
健康检查 / 对话管理"] + Store["Store 层
Repository 接口"] + end + + subgraph AI["云端 AI 服务"] + STT["STT
Deepgram / MiMo ASR"] + LLM["LLM
GPT-4o / 通义千问"] + TTS["TTS
OpenAI TTS / MiMo TTS"] + end + + subgraph Storage["存储层"] + Mem["Memory
进程内缓存"] + Redis["Redis
会话状态"] + PG["PostgreSQL
持久化存储"] + end + + Media --> Edge + Edge -->|"query (image+audio)"| WS + UI <-->|"WebSocket"| WS + WS --> Session + WS --> Orch + Orch --> STT + Orch --> LLM + Orch --> TTS + Session --> Store + Store --> Mem + Store --> Redis + Store --> PG + REST --> Session + WS --> Auth +``` + +> 为什么单独加一层 Go 网关，而不是让前端直连 AI API？1）API Key 安全性；2）统一的速率限制和成本管控；3）多模型路由逻辑集中在一处便于维护。 + +## 核心交互流程 + +一次完整的"用户提问 → AI 回答"流程： + +```mermaid +sequenceDiagram + participant B as 浏览器 + participant G as Go 网关（Eino Graph） + participant S as STT + participant L as LLM（ChatModel） + participant T as TTS + + B->>B: VAD 检测到语音结束 + B->>G: query {image, audio} + Note over G: EinoOrchestrator 启动 Graph.Stream() + + G->>S: STT Lambda：音频 → 文本 + S-->>G: 识别文本 + G-->>B: stt_result {text} + + G->>G: History Lambda：组装提示词 + 历史 + 多模态消息 + + G->>L: ChatModel Node：流式推理 + loop LLM 流式输出（Callback OnEndWithStreamOutput） + L-->>G: token delta + G-->>B: llm_chunk {delta} + end + + G->>G: Msg2Str + Splitter Lambda：句子切分 + G->>T: TTS Lambda：逐句合成 + T-->>G: 音频 chunk + G-->>B: tts_audio {audio} + + G->>G: Done Lambda：发送完成通知 + G-->>B: llm_done {full_text, tokens} + G-->>B: tts_audio {final: true} +``` + +**关键优化**：Eino Graph 以 Stream 模式运行，ChatModel 的 token 流通过 Callback 的 `OnEndWithStreamOutput` 实时推送到客户端（`llm_chunk`），同时 Splitter 节点将 token 流切分为句子，TTS 节点逐句合成并推送音频。LLM 文本流和 TTS 音频流**并行推送**，用户感知延迟大幅降低。 + +## 技术栈 + +### 前端 + +| 技术 | 选型 | 选择理由 | +|------|------|---------| +| 框架 | React 18 + TypeScript | 组件化开发，类型安全，生态成熟 | +| 构建 | Vite | 开发热更新快，构建产物小 | +| 实时通信 | WebSocket（原生 API） + 自封装连接管理 | 浏览器原生支持，封装心跳/重连/消息分发 | +| 语音检测 | @ricky0123/vad-web | 基于 WebRTC VAD，纯前端零延迟 | +| 媒体采集 | MediaDevices API | 浏览器原生摄像头/麦克风访问 | + +### 后端 + +| 技术 | 选型 | 选择理由 | +|------|------|---------| +| 语言 | Go | 高并发 goroutine 模型，适合长连接管理 | +| HTTP 框架 | Gin | 高性能 HTTP 路由，中间件生态成熟 | +| WebSocket | gorilla/websocket | Go 生态最成熟的 WebSocket 库 | +| 会话存储 | Memory / Redis / PostgreSQL 三级存储 | 进程内存零依赖，Redis 支持多实例，PG 持久化。TieredManager 自动降级 | +| AI 编排 | CloudWeGo Eino Graph | 声明式 DAG 编排，Stream 模式，Callback AOP | +| 持久化存储 | PostgreSQL | 对话历史、用户数据、会话元数据 | +| 配置管理 | Viper + godotenv | 支持 YAML + .env + 环境变量覆盖 | +| 日志 | Zap | 高性能结构化日志 | + +### AI 服务 + +| 能力 | 默认方案 | 备选方案 | +|------|---------|---------| +| 多模态 LLM | DashScope qwen3-vl-plus | GPT-4o 等 OpenAI 兼容模型 | +| 语音识别 STT | MiMo ASR（小米） | Deepgram | +| 语音合成 TTS | MiMo TTS（小米） | OpenAI TTS | + +> Go 网关的 AI 服务层统一封装不同服务商的调用接口，通过配置切换 provider。LLM 通过 Eino 框架的 `eino-ext/components/model/openai` 组件接入，支持任何 OpenAI 兼容接口。 + +## 后端模块 + +```mermaid +graph LR + subgraph Entry["入口层"] + Main["main.go
依赖注入 / 启动"] + end + + subgraph Transport["传输层"] + WSH["WebSocket Handler
连接管理 / 认证"] + APH["REST API Handlers
Auth / Conversation / Health"] + end + + subgraph Business["业务层"] + SM["Session Manager
会话生命周期"] + ORCH["EinoOrchestrator
Eino Graph 编排"] + AS["Auth Service
注册/登录/刷新/登出"] + end + + subgraph Eino_Layer["Eino 编排层"] + PG["PipelineGraph
7 节点 DAG"] + CB["Callback Handler
LLM token 推送"] + ST["PipelineState
跨节点状态"] + end + + subgraph AI_Layer["AI 服务层"] + STT_S["STT Service
MiMo / Deepgram"] + LLM_S["ChatModel
eino-ext OpenAI 兼容"] + TTS_S["TTS Service
MiMo / OpenAI"] + end + + subgraph Data["数据层"] + UR["UserRepository"] + MR["MessageRepository"] + SR["SessionRepository"] + end + + Main --> WSH + Main --> APH + Main --> SM + Main --> ORCH + Main --> AS + + WSH --> SM + WSH --> ORCH + APH --> SM + APH --> AS + ORCH --> PG + PG --> CB + PG --> ST + PG --> STT_S + PG --> LLM_S + PG --> TTS_S + SM --> MR + SM --> SR + AS --> UR +``` + +| 模块 | 职责 | +|------|------| +| WebSocket Handler | 管理客户端连接生命周期，JWT 认证，conversation_id 恢复，单播消息推送 | +| Session Manager | 维护用户会话状态、对话历史，三级存储架构，30 分钟 TTL | +| Eino 编排层 | 基于 CloudWeGo Eino Graph 的声明式 AI 编排，7 节点 DAG 流水线，Stream 模式调用 | +| AI Orchestrator | EinoOrchestrator 适配器，包装 Eino Graph 实现 Orchestrator 接口 | +| AI Service Layer | AI 服务抽象层，多 provider 支持（Deepgram/MiMo/OpenAI 等） | +| Auth | 用户认证与授权，JWT 双 token 轮转，bcrypt 密码哈希 | +| Store | 持久化存储层，Repository 接口与实现（内存 + PostgreSQL） | +| REST API | 健康检查、认证、对话管理端点 | +| Logger | Zap 结构化日志 | +| Models | 数据模型定义 | +| Migrations | 数据库版本化迁移 | +| Model Router | 根据请求类型选择 AI 模型（待实现） | +| Rate Limiter | 令牌桶限流，详见 [11-令牌桶限流.md](./11-令牌桶限流.md) | + +## 前端组件 + +| 组件 | 职责 | +|------|------| +| LandingPage | 未登录时的着陆页，内嵌 LoginModal 登录/注册弹窗 | +| CameraManager | 摄像头流采集 | +| MicManager | 麦克风音频采集 | +| EdgeProcessor | VAD + 关键帧检测 | +| WebSocketManager | WebSocket 连接生命周期管理 | +| ChatPanel | 消息展示、流式回复、文本输入、场景选择 | +| VideoPreview | 摄像头画面预览 | +| SessionSidebar | 左侧对话列表（搜索、重命名、删除、时间分组） | +| ConfigPanel | 右侧配置面板（主题、TTS 开关、detail level、语言、场景、登出） | +| Toast | 轻量通知提示 | + +核心 Hook：`useVisionSession()` 封装完整的视觉对话会话（摄像头、VAD、WebSocket、消息状态、认证、场景模式）。`useSessionList()` 通过 REST API 管理对话列表 CRUD。 + +### 前端会话状态模型（三态） + +前端 UI 存在三个会话状态，由 `isConnected` 和 `isCameraOn` 联合决定： + +``` +┌──────────┐ startSession() ┌──────────┐ +│ initial │ ──────────────────→ │ video │ +│ 初始态 │ │ 视频通话 │ +└──────────┘ └──────────┘ + ↑ │ + │ stopSession() stopVideo() + │ │ + │ ▼ + │ ┌──────────┐ + └──────────────────────── │ textOnly │ + │ 文字对话 │ + └──────────┘ + │ + startSession() + │ + ▼ + ┌──────────┐ + │ video │ + └──────────┘ +``` + +| 状态 | 条件 | WebSocket | 摄像头 | 消息 | 文字输入 | +|------|------|-----------|--------|------|---------| +| `initial` | `!isConnected && messages.length === 0` | 断开 | 关闭 | 空 | 可用（自动连接） | +| `video` | `isConnected && isCameraOn` | 连接 | 开启 | 有 | 可用 | +| `textOnly` | `isConnected && !isCameraOn` | 连接 | 关闭 | 保留 | 可用 | + +- **`stopVideo()`**：停止摄像头/麦克风/VAD，保持 WebSocket 连接和消息历史，用户可继续文字对话 +- **`stopSession()`**：完全断开 WebSocket、清空消息、重置状态，回到初始态 + +## 数据库设计 + +### ER 关系 + +```mermaid +erDiagram + users ||--o{ sessions : "1:N" + users ||--o{ refresh_tokens : "1:N" + sessions ||--o{ messages : "1:N" + + users { + uuid id PK + varchar username UK + varchar password_hash + timestamptz created_at + timestamptz updated_at + } + + sessions { + uuid id PK + uuid user_id FK + varchar title + jsonb config + timestamptz created_at + timestamptz updated_at + } + + messages { + bigserial id PK + uuid session_id FK + varchar role + text content + integer tokens_used + timestamptz created_at + } + + refresh_tokens { + bigserial id PK + uuid user_id FK + varchar token_hash UK + timestamptz expires_at + timestamptz created_at + } +``` + +系统采用关系型数据库存储持久化数据，包括用户账户、对话会话、消息记录和刷新令牌。数据库表定义详见 `backend/migrations/` 目录下的 SQL 迁移文件。 + +### 存储策略 + +系统采用**三级存储架构**（TieredManager）实现会话状态管理，平衡性能与可靠性： + +- **L1 Memory**：进程内缓存，提供微秒级读写性能 +- **L2 Redis**：分布式缓存层，支持多实例部署，提供毫秒级访问 +- **L3 PostgreSQL**：持久化存储层，确保数据可靠性 + +会话数据按 TTL（默认 30 分钟）在三级存储间流转，支持 Redis 故障时自动降级到 Memory + PostgreSQL 模式。配置灵活，可根据部署规模选择单级（Memory）、双级（Memory + PostgreSQL）或完整三级存储方案。 + +## 认证设计 + +系统采用 **JWT 双 token 轮转认证机制**，结合 bcrypt 密码哈希和 Refresh Token Rotation 安全策略。 + +核心机制包括：双 token 轮转（access_token 15 分钟 + refresh_token 7 天）、密码安全（bcrypt cost=10）、token 安全（SHA256 哈希存储、复用检测）、WebSocket 连接认证（基于 access_token 的 HTTP Upgrade 校验）等。认证流程、安全机制、配置要求等详细设计见 [10-鉴权体系.md](./10-鉴权体系.md)。 + +## 部署架构 + +系统采用分层部署架构，支持单实例和多实例水平扩展： + +```mermaid +graph TB + User["用户浏览器"] --> Nginx + + subgraph Nginx["Nginx 反向代理"] + Static["/ → 前端静态资源"] + API["/api/* → Go Gateway"] + WS_Proxy["/ws → Go Gateway"] + end + + subgraph Gateway_Pool["Go Gateway 实例"] + G1["Gateway-1"] + G2["Gateway-2"] + GN["Gateway-N"] + end + + Nginx --> G1 + Nginx --> G2 + Nginx --> GN + + G1 --> Redis + G2 --> Redis + GN --> Redis + + G1 --> PG_DB["PostgreSQL"] + G2 --> PG_DB + GN --> PG_DB + + G1 --> AI_Services["AI Services（外部 API）"] + G2 --> AI_Services + GN --> AI_Services +``` + +**跨域策略**：Nginx 将前端（`/`）、REST API（`/api/*`）、WebSocket（`/ws`）统一反代到同一域名，浏览器无跨域问题。 + +**开发环境**：前端 Vite :5173 通过 `server.proxy` 转发 `/ws` 和 `/api` 到后端 :8080，无需硬编码端口。 diff --git a/docs/01-项目概述.md b/docs/01-项目概述.md deleted file mode 100644 index 8a5a2c9..0000000 --- a/docs/01-项目概述.md +++ /dev/null @@ -1,28 +0,0 @@ -# 项目概述 - -## 概述 - -开发一款**多模态实时对话应用**——通过摄像头与麦克风捕获用户的视觉场景与语音输入，由 AI 理解并给出自然、流畅的回应。 - -核心挑战在于三个维度之间的张力： - -| 维度 | 关键问题 | 详见 | -|------|---------|------| -| 视觉理解 | 如何准确理解摄像头画面中的人物、物体、场景？ | `07-视觉理解.md` | -| 语音交互 | 如何让对话像真人交流一样自然、低延迟？ | `06-语音交互.md` | -| 成本控制 | 实时视频流 + LLM 推理，如何避免账单爆炸？ | `08-成本控制.md` | - -> 提升视觉精度意味着更高分辨率和更频繁的采样，但这会直接推高带宽和推理成本。架构设计需要在三者之间做好取舍。 - -## 项目目标 - -1. **用户故事规划**：明确"AI 能看、能听、能说"需要覆盖哪些场景 → `05-用户故事.md` -2. **成本控制策略**：从架构设计层面融入运营成本意识 → `08-成本控制.md` - -## 交付物 - -- 可运行的应用程序（摄像头 + 麦克风 → AI 回应） -- 设计文档，覆盖： - - 计划实现 vs 最终实现的用户故事 - - 成本控制技巧的构思 vs 实际采用的方案 - - 项目架构设计与技术选型 diff --git a/docs/02-接口文档.md b/docs/02-接口文档.md new file mode 100644 index 0000000..ce591a9 --- /dev/null +++ b/docs/02-接口文档.md @@ -0,0 +1,557 @@ +# 接口文档 + +## 概述 + +前后端通信接口契约。WebSocket 承载实时对话，REST API 支撑基础运维。 + +**设计原则**： +- WebSocket 为主：所有对话数据走 WebSocket +- REST 为辅：仅用于健康检查、认证、对话管理等低频操作 +- 接口先行：先定义契约，再填充实现——前后端可并行开发 + +## 接口全景 + +``` +浏览器 Go Gateway :8080 + WebSocket Client <--> /ws?token= （实时对话，需 JWT 认证） + HTTP Client --> GET /api/health （健康检查） + HTTP Client <--> POST /api/auth/* （注册/登录/刷新/登出） + HTTP Client <--> GET/POST/PATCH/DELETE （对话 CRUD） + /api/conversations/* + HTTP Client <--> GET /api/conversations/:id （历史消息） + /messages +``` + +--- + +## 一、WebSocket 协议 + +连接地址：`ws://localhost:8080/ws?token=&conversation_id=` + +| 参数 | 必填 | 说明 | +|------|------|------| +| `token` | 是 | JWT access_token，缺失或无效时返回 401 | +| `conversation_id` | 否 | 恢复已有对话；省略则创建新对话 | + +### 消息格式约定 + +所有 WebSocket 消息均为 JSON 文本帧，统一结构： + +```typescript +interface WsMessage { + type: string; // 消息类型，必填 + request_id?: string; // 可选，用于请求-响应关联 + timestamp?: number; // 可选，毫秒时间戳 + [key: string]: any; // 类型特定字段 +} +``` + +### 客户端 → 服务端消息 + +#### `query` — 发起一次视觉对话 + +```typescript +interface QueryMessage { + type: "query"; + request_id: string; // 客户端生成的 UUID + image: string; // Base64 编码的 JPEG 图像（不含 data: 前缀） + audio: string; // Base64 编码的音频片段（PCM 16kHz），文本输入时为空字符串 + text?: string; // 用户手动输入的文本（有值时跳过 STT，直接使用此文本） + mime_type?: string; // 音频格式，默认 "audio/pcm" +} +``` + +#### `config` — 更新会话配置 + +```typescript +interface ConfigMessage { + type: "config"; + payload: { + tts_enabled?: boolean; // 是否开启语音合成，默认 true + detail_level?: "low" | "high"; // 图像精度，默认 "low" + language?: string; // 交互语言，默认 "zh-CN" + scenario?: string; // 场景模式：free_chat / interviewer / english_teacher / debate / interpreter + }; +} +``` + +#### `interrupt` — 打断当前回复 + +```typescript +interface InterruptMessage { + type: "interrupt"; + request_id?: string; // 可选，当前实现不使用此字段，服务端始终取消当前活跃请求 +} +``` + +#### `ping` — 心跳保活 + +```typescript +interface PingMessage { + type: "ping"; +} +``` + +### 服务端 → 客户端消息 + +#### `connected` — 连接建立确认 + +```typescript +interface ConnectedMessage { + type: "connected"; + session_id: string; + conversation_id: string; + config: { + tts_enabled: boolean; + detail_level: "low" | "high"; + language: string; + scenario: string; + }; +} +``` + +#### `stt_result` — 语音识别结果 + +```typescript +interface SttResultMessage { + type: "stt_result"; + request_id: string; + text: string; // 识别出的文本 + is_final: boolean; // 当前实现始终为 true +} +``` + +#### `llm_chunk` — LLM 流式响应片段 + +```typescript +interface LlmChunkMessage { + type: "llm_chunk"; + request_id: string; + content: string; // 当前 token 片段 +} +``` + +#### `llm_done` — LLM 响应完成 + +```typescript +interface LlmDoneMessage { + type: "llm_done"; + request_id: string; + full_text: string; // 完整响应文本 +} +``` + +#### `tts_audio` — TTS 音频片段 + +音频流式推送，每个消息携带一个句子的音频数据。 + +```typescript +interface TtsAudioMessage { + type: "tts_audio"; + request_id: string; + audio: string; // Base64 编码的音频数据 + format: string; // 音频格式 + sample_rate: number; // 采样率（Hz） + sequence: number; // 句子序号，从 0 开始递增 + is_final: boolean; // 是否为最后一个句子 +} +``` + +**音频格式约束**： + +| 字段 | 值 | 说明 | +|------|-----|------| +| `format` | `"pcm"` | 线性 PCM，小端序 | +| `sample_rate` | `24000` | 24kHz 采样率 | +| 位深度 | 16-bit | 单声道 | +| 句子划分 | 按标点符号（。！？；：）分割 | 服务端按句分割 LLM 响应，并行合成 | + +#### `error` — 错误通知 + +```typescript +interface ErrorMessage { + type: "error"; + request_id?: string; // 关联的请求 ID，全局错误时为空 + code: string; // 错误码，见下文错误码表 + message: string; // 人类可读的错误描述 + details?: any; // 可选的详细错误信息 +} +``` + +#### `pong` — 心跳响应 + +```typescript +interface PongMessage { + type: "pong"; +} +``` + +### 连接管理 + +- **心跳机制**：客户端每 30 秒发送 `ping`，服务端回复 `pong`；60 秒无活动则服务端断开连接 +- **重连策略**：客户端断线后指数退避重连（1s → 2s → 4s → ... → 最大 30s） +- **并发控制**：同一连接同时只能有一个活跃的 `query` 请求；新请求到来时自动取消旧请求 + +--- + +## 二、REST API + +所有 REST 端点均使用 JSON 格式。 + +### 2.1 健康检查 + +#### `GET /api/health` + +检查服务健康状态。 + +**响应**： +```json +{ + "status": "healthy", + "timestamp": "2024-01-15T10:30:00Z", + "dependencies": { + "database": "healthy", + "redis": "healthy" + } +} +``` + +### 2.2 认证 API + +#### `POST /api/auth/register` — 用户注册 + +**请求**： +```json +{ + "username": "alice", + "email": "alice@example.com", + "password": "SecurePass123!" +} +``` + +**响应**（200 OK）： +```json +{ + "user": { + "id": "550e8400-e29b-41d4-a716-446655440000", + "username": "alice", + "email": "alice@example.com", + "created_at": "2024-01-15T10:30:00Z" + }, + "access_token": "eyJhbGc...", + "refresh_token": "eyJhbGc...", + "expires_in": 7200 +} +``` + +**错误**： +- `400 INVALID_INPUT`: 参数验证失败 +- `409 USER_EXISTS`: 用户名或邮箱已存在 + +#### `POST /api/auth/login` — 用户登录 + +**请求**： +```json +{ + "username": "alice", + "password": "SecurePass123!" +} +``` + +**响应**（200 OK）： +```json +{ + "user": { + "id": "550e8400-e29b-41d4-a716-446655440000", + "username": "alice", + "email": "alice@example.com" + }, + "access_token": "eyJhbGc...", + "refresh_token": "eyJhbGc...", + "expires_in": 7200 +} +``` + +**错误**： +- `400 INVALID_INPUT`: 参数缺失 +- `401 INVALID_CREDENTIALS`: 用户名或密码错误 + +#### `POST /api/auth/refresh` — 刷新 Access Token + +**请求头**： +``` +Authorization: Bearer +``` + +**响应**（200 OK）： +```json +{ + "access_token": "eyJhbGc...", + "refresh_token": "eyJhbGc...", + "expires_in": 7200 +} +``` + +**错误**： +- `401 INVALID_TOKEN`: Refresh Token 无效或过期 + +#### `POST /api/auth/logout` — 用户登出 + +**请求头**： +``` +Authorization: Bearer +``` + +**响应**（200 OK）： +```json +{ + "message": "Logged out successfully" +} +``` + +### 2.3 对话管理 API + +所有端点均需 JWT 认证（`Authorization: Bearer `）。 + +#### `GET /api/conversations` — 获取对话列表 + +**查询参数**： +- `page`: 页码，从 1 开始，默认 1 +- `page_size`: 每页条数，默认 20，最大 100 + +**响应**（200 OK）： +```json +{ + "conversations": [ + { + "id": "550e8400-e29b-41d4-a716-446655440000", + "title": "关于植物的对话", + "created_at": "2024-01-15T10:30:00Z", + "updated_at": "2024-01-15T11:45:00Z", + "message_count": 12 + } + ], + "total": 42, + "page": 1, + "page_size": 20 +} +``` + +#### `POST /api/conversations` — 创建新对话 + +**请求**： +```json +{ + "title": "新的对话" +} +``` + +**响应**（201 Created）： +```json +{ + "id": "550e8400-e29b-41d4-a716-446655440000", + "title": "新的对话", + "created_at": "2024-01-15T10:30:00Z", + "updated_at": "2024-01-15T10:30:00Z", + "message_count": 0 +} +``` + +#### `GET /api/conversations/:id` — 获取对话详情 + +**响应**（200 OK）： +```json +{ + "id": "550e8400-e29b-41d4-a716-446655440000", + "title": "关于植物的对话", + "created_at": "2024-01-15T10:30:00Z", + "updated_at": "2024-01-15T11:45:00Z", + "message_count": 12 +} +``` + +**错误**： +- `404 NOT_FOUND`: 对话不存在或无权访问 + +#### `PATCH /api/conversations/:id` — 更新对话 + +**请求**： +```json +{ + "title": "修改后的标题" +} +``` + +**响应**（200 OK）： +```json +{ + "id": "550e8400-e29b-41d4-a716-446655440000", + "title": "修改后的标题", + "created_at": "2024-01-15T10:30:00Z", + "updated_at": "2024-01-15T12:00:00Z", + "message_count": 12 +} +``` + +#### `DELETE /api/conversations/:id` — 删除对话 + +**响应**（204 No Content）：无响应体 + +**错误**： +- `404 NOT_FOUND`: 对话不存在或无权访问 + +#### `GET /api/conversations/:id/messages` — 获取对话消息 + +**查询参数**： +- `page`: 页码，从 1 开始，默认 1 +- `page_size`: 每页条数，默认 50，最大 100 + +**响应**（200 OK）： +```json +{ + "messages": [ + { + "id": "660e8400-e29b-41d4-a716-446655440000", + "conversation_id": "550e8400-e29b-41d4-a716-446655440000", + "role": "user", + "content": "这是什么植物？", + "image_url": "/api/images/abc123.jpg", + "created_at": "2024-01-15T10:30:00Z" + }, + { + "id": "770e8400-e29b-41d4-a716-446655440000", + "conversation_id": "550e8400-e29b-41d4-a716-446655440000", + "role": "assistant", + "content": "这是一株向日葵...", + "created_at": "2024-01-15T10:30:15Z" + } + ], + "total": 12, + "page": 1, + "page_size": 50 +} +``` + +**消息字段说明**： +- `role`: `"user"` 或 `"assistant"` +- `image_url`: 仅 `user` 消息可能包含，指向存储的图像 +- `content`: 消息文本内容 + +--- + +## 三、错误码表 + +所有错误均使用以下格式： + +```json +{ + "code": "ERROR_CODE", + "message": "Human-readable error description", + "details": {} +} +``` + +### WebSocket 错误码 + +| 错误码 | 说明 | HTTP 状态码（若适用）| +|--------|------|---------------------| +| `INVALID_MESSAGE` | 消息格式错误或缺少必填字段 | - | +| `SESSION_NOT_FOUND` | 会话不存在 | - | +| `RATE_LIMITED` | 请求频率过高 | 429 | +| `IMAGE_TOO_LARGE` | 图像超过大小限制（5MB）| - | +| `AUDIO_TOO_LARGE` | 音频超过大小限制（10MB）| - | +| `STT_ERROR` | 语音识别服务错误 | - | +| `LLM_ERROR` | LLM 服务错误 | - | +| `LLM_TIMEOUT` | LLM 响应超时（60 秒）| - | +| `TTS_ERROR` | 语音合成服务错误 | - | +| `CONCURRENT_REQUEST` | 同一连接已有进行中的请求 | - | +| `INTERNAL_ERROR` | 服务器内部错误 | 500 | + +### REST API 错误码 + +| 错误码 | 说明 | HTTP 状态码 | +|--------|------|-------------| +| `INVALID_INPUT` | 请求参数验证失败 | 400 | +| `INVALID_TOKEN` | JWT Token 无效或过期 | 401 | +| `INVALID_CREDENTIALS` | 用户名或密码错误 | 401 | +| `UNAUTHORIZED` | 未认证或认证失败 | 401 | +| `FORBIDDEN` | 无权访问资源 | 403 | +| `NOT_FOUND` | 资源不存在 | 404 | +| `USER_EXISTS` | 用户名或邮箱已存在 | 409 | +| `RATE_LIMITED` | 请求频率过高 | 429 | +| `INTERNAL_ERROR` | 服务器内部错误 | 500 | +| `SERVICE_UNAVAILABLE` | 依赖服务不可用 | 503 | + +--- + +## 四、数据模型 + +### 用户（User） + +```typescript +interface User { + id: string; // UUID + username: string; // 用户名，唯一 + email: string; // 邮箱，唯一 + created_at: string; // ISO 8601 时间戳 + updated_at: string; // ISO 8601 时间戳 +} +``` + +### 对话（Conversation） + +```typescript +interface Conversation { + id: string; // UUID + user_id: string; // 所属用户 ID + title: string; // 对话标题 + created_at: string; // ISO 8601 时间戳 + updated_at: string; // ISO 8601 时间戳 + message_count: number; // 消息数量 +} +``` + +### 消息（Message） + +```typescript +interface Message { + id: string; // UUID + conversation_id: string; // 所属对话 ID + role: "user" | "assistant"; + content: string; // 消息文本内容 + image_url?: string; // 可选，用户消息的关联图像 URL + created_at: string; // ISO 8601 时间戳 +} +``` + +### JWT Token 载荷 + +**Access Token**（有效期 120 分钟）： +```json +{ + "user_id": "550e8400-e29b-41d4-a716-446655440000", + "username": "alice", + "type": "access", + "exp": 1705318200, + "iat": 1705311000 +} +``` + +**Refresh Token**（有效期 7 天）： +```json +{ + "user_id": "550e8400-e29b-41d4-a716-446655440000", + "type": "refresh", + "exp": 1705915800, + "iat": 1705311000 +} +``` + +--- + +## 附录：版本历史 + +- **v1.0**（2024-01-15）：初始版本，定义 WebSocket 协议和 REST API +- **v1.1**（2024-01-20）：新增文本输入模式（`query.text` 字段） +- **v1.2**（2024-01-25）：新增场景模式配置（`config.scenario` 字段） +- **v2.0**（2026-06-21）：重构为纯接口契约规范，移除实现细节 diff --git a/docs/02-系统架构.md b/docs/02-系统架构.md deleted file mode 100644 index 0a2b316..0000000 --- a/docs/02-系统架构.md +++ /dev/null @@ -1,254 +0,0 @@ -# 系统架构 - -## 概述 - -三层架构：**前端做轻量预处理，后端做智能编排，云端 AI 服务按需调用**。在保证交互体验的同时控制成本。 - -## 三层架构 - -| 层级 | 职责 | 关键约束 | -|------|------|---------| -| **客户端（浏览器）** | 媒体采集、边缘预处理、UI 渲染 | 浏览器资源有限，模型需轻量 | -| **Go 网关** | 会话管理、AI 服务编排、流式管道 | 高并发、低延迟、状态管理 | -| **AI 服务** | LLM 推理、语音识别、语音合成 | 按量计费，需控制调用频率 | - -> 为什么要单独加一层 Go 网关，而不是让前端直连 AI API？1）API Key 安全性；2）统一的速率限制和成本管控；3）多模型路由逻辑集中在一处便于维护。 - -## 技术栈 - -### 前端 - -| 技术 | 选型 | 选择理由 | -|------|------|---------| -| 框架 | React 18 + TypeScript | 组件化开发，类型安全，生态成熟 | -| 构建 | Vite | 开发热更新快，构建产物小 | -| 实时通信 | WebSocket（原生 API） + 自封装连接管理 | 浏览器原生支持，封装心跳/重连/消息分发 | -| 边缘推理 | ONNX Runtime Web | 浏览器端跑轻量模型（VAD、关键帧检测） | -| 语音检测 | @ricky0123/vad-web | 基于 WebRTC VAD，纯前端零延迟 | -| 媒体采集 | MediaDevices API | 浏览器原生摄像头/麦克风访问 | - -### 后端 - -| 技术 | 选型 | 选择理由 | -|------|------|---------| -| 语言 | Go | 高并发 goroutine 模型，适合长连接管理 | -| HTTP 框架 | Gin | 高性能 HTTP 路由，中间件生态成熟 | -| WebSocket | gorilla/websocket | Go 生态最成熟的 WebSocket 库 | -| 会话存储 | Redis（规划中） / Memory（MVP 默认） | 高速 KV 存储，MVP 阶段使用进程内存，可通过配置切换到 Redis | -| 持久化存储 | PostgreSQL（规划中） | 对话历史、用量统计、用户偏好（MVP 阶段未实现） | -| 配置管理 | Viper + godotenv | 支持 YAML + .env + 环境变量覆盖，详见 `03-接口文档.md` 第六章 | -| 日志 | Zap | 高性能结构化日志 | - -### AI 服务 - -| 能力 | 主选方案 | 备选方案 | 选型考量 | -|------|---------|---------|---------| -| 多模态 LLM | GPT-4o（默认） | 通义千问等 OpenAI 兼容模型 | 通过 OpenAI 兼容接口，可灵活切换 | -| 语音识别 STT | Deepgram（默认） | MiMo ASR（小米） | 支持多 provider 切换 | -| 语音合成 TTS | OpenAI TTS（默认） | MiMo TTS（小米） | 支持多 provider 切换 | - -> 不必绑定单一厂商。Go 网关的 AI 服务层统一封装不同服务商的调用接口，通过配置切换 provider。 - -## 核心交互流程 - -一次完整的"用户提问 → AI 回答"流程： - -``` -Browser Go Gateway STT LLM TTS - | | | | | - |-- VAD 检测到语音结束 --->| | | | - | | | | | - |-- [音频+图像] -------->| | | | - | |--- 音频流 ------->| | | - | |<-- 流式文本 ------| | | - | | | | | - | |--- [图像+文本+上下文] -------->| | - | |<-- 流式回答文本 --------------| | - |<-- 推送回答文本 --------| | | | - | |--- 回答文本 ---------------------------->| - | |<-- 流式音频 --------------------------------| - |<-- 推送音频流 ----------| | | | - | | | | | - |-> 播放音频 + 渲染文字 | | | | -``` - -**关键优化**：LLM 文本流和 TTS 音频流是**并行推送**的——客户端先展示文字，同时开始播放语音，用户感知延迟大幅降低。 - -## 后端模块 - -| 模块 | 职责 | 关键实现 | -|------|------|---------| -| WebSocket Handler | 管理客户端连接生命周期，单播消息推送 | goroutine per connection | -| Session Manager | 维护用户会话状态、对话历史 | Memory（MVP 默认）/ Redis（可切换），30 分钟 TTL（详见 `03-接口文档.md` 第五章） | -| AI Orchestrator | 编排 STT→LLM→TTS 流式并行管道 | context 取消 + 超时控制 + 句子切分 | -| AI Service Layer | AI 服务抽象层（STT/LLM/TTS） | 多 provider 支持（Deepgram/MiMo/OpenAI 等） | -| REST API | 健康检查、会话管理端点 | Gin 路由 | -| Error Handler | 统一错误码定义与发送 | 错误码枚举 | -| Logger | 日志初始化封装 | Zap 结构化日志 | -| Models | 数据模型定义 | WebSocket 消息、会话、配置等 | -| Model Router | 根据请求类型选择 AI 模型（规划中） | 规则引擎 + 成本阈值 | -| Rate Limiter | 防止单用户过度消耗 API 额度（规划中） | 令牌桶算法 | - -AI Orchestrator 核心接口（`internal/orchestrator/orchestrator.go`）： - -```go -// Orchestrator AI 编排器接口。 -type Orchestrator interface { - ProcessQuery(ctx context.Context, sessionID string, req models.WsQuery, - history []models.Message, sender Sender) error -} -``` - -Pipeline 实现（`internal/orchestrator/pipeline.go`）流程： -1. Base64 解码音频/图片 -2. 调用 `stt.Recognize()` → 发送 `stt_result` -3. 调用 `llm.ChatStream()` 获取流式输出，goroutine 消费 token → 发送 `llm_chunk` + 句子切分 -4. 另一 goroutine 从句子 channel 读取 → 调用 `tts.SynthesizeStream()` → 发送 `tts_audio` -5. 流结束 → 发送 `llm_done` -6. TTS 失败静默跳过，STT/LLM 失败发送对应 error 消息 - -> **关键优化**：LLM 文本流和 TTS 音频流**并行推送**——客户端先逐 token 展示文字，同时 TTS 逐句子合成并推送音频，用户感知延迟大幅降低。详细的 AI 服务层接口和编排策略见 `03-接口文档.md` 第三、四章。 - -## 前端组件 - -| 组件 | 职责 | -|------|------| -| CameraManager | 摄像头流采集 | -| MicManager | 麦克风音频采集 | -| EdgeProcessor | VAD + 关键帧检测（Canvas 像素比较） | -| WebSocketManager | WS 连接生命周期管理 | -| ChatPanel | 消息展示 | -| VideoPreview | 摄像头画面预览 | -| ConfigPanel | 右侧抽屉式配置面板（主题、TTS 开关、detail level、语言） | -| Toast | 轻量通知提示（3 秒自动消失） | - -核心 Hook：`useVisionSession()` 封装一次完整的视觉对话会话（摄像头、VAD、WebSocket、消息状态）。 - -```typescript -function useVisionSession() { - const [messages, setMessages] = useState([]); - const wsRef = useWebSocket(`${window.location.protocol === "https:" ? "wss:" : "ws:"}//${window.location.host}/ws`); - const videoRef = useRef(null); - const { captureFrame } = useCamera(videoRef); - - const { isSpeaking } = useVAD({ - onSpeechEnd: async (audio) => { - const frame = captureFrame(); - wsRef.current?.send(JSON.stringify({ - type: "query", - image: frame.toDataURL("image/jpeg", 0.7), - audio: encodeAudio(audio) - })); - } - }); - - useEffect(() => { - wsRef.current?.on("message", (data) => { - const { text, audio } = JSON.parse(data); - setMessages(prev => [...prev, { role: "assistant", text }]); - if (audio) playAudio(audio); - }); - }, []); - - return { messages, videoRef, isSpeaking }; -} -``` - -## 存储策略（分阶段） - -| 阶段 | 存储方案 | 持久化内容 | 理由 | -|------|---------|-----------|------| -| MVP | Memory（进程内） | 无 | 快速验证核心功能，重启丢数据可接受。Redis 实现已就绪，可通过 `storage.driver` 配置切换 | -| 上线 | Redis + PostgreSQL | 对话历史、用户偏好、用量统计 | 用户需要查看历史，运营需要成本数据 | -| 规模化 | Redis + PG + 对象存储 | 图像帧、音频片段归档 | 大文件不适合存关系库 | - -冷热分离：Redis 存"热数据"（当前对话上下文，微秒级读写），PostgreSQL 存"冷数据"（历史记录）。 - -### PostgreSQL 表设计 - -```sql -CREATE TABLE sessions ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID NOT NULL, - created_at TIMESTAMPTZ DEFAULT now(), - updated_at TIMESTAMPTZ DEFAULT now() -); - -CREATE TABLE messages ( - id BIGSERIAL PRIMARY KEY, - session_id UUID REFERENCES sessions(id), - role VARCHAR(16) NOT NULL, -- "user" | "assistant" - content TEXT NOT NULL, - image_url TEXT, - tokens_used INTEGER DEFAULT 0, - created_at TIMESTAMPTZ DEFAULT now() -); - -CREATE TABLE usage_daily ( - user_id UUID NOT NULL, - date DATE NOT NULL, - llm_tokens BIGINT DEFAULT 0, - stt_seconds REAL DEFAULT 0, - tts_chars INTEGER DEFAULT 0, - estimated_cost NUMERIC(10,4) DEFAULT 0, - PRIMARY KEY (user_id, date) -); -``` - -## 部署架构 - -``` -用户浏览器 - ↓ -Nginx（同源反代 + 负载均衡） - ├── / → 前端静态资源（CDN 或本地 dist） - ├── /api/* → Go Gateway（REST API） - └── /ws → Go Gateway（WebSocket） - ├── Gateway-1 ──→ Redis - ├── Gateway-2 ──→ Redis - └── Gateway-N ──→ AI Services（外部 API） -``` - -**跨域策略**：Nginx 将前端和后端统一到同一域名下，浏览器无跨域问题。 - -### Nginx 配置 - -```nginx -server { - listen 80; - server_name camtalk.example.com; - - # 前端静态资源 - location / { - root /var/www/camtalk/dist; - try_files $uri $uri/ /index.html; - } - - # REST API 反代 - location /api/ { - proxy_pass http://127.0.0.1:8080; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - } - - # WebSocket 反代 - location /ws { - proxy_pass http://127.0.0.1:8080; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_read_timeout 86400s; # 长连接超时 24h - proxy_send_timeout 86400s; - } -} -``` - -> WebSocket 是长连接，Nginx 必须配置 `Upgrade` 和 `Connection` 头。`proxy_read_timeout` 需要覆盖心跳间隔（客户端 30s ping），否则 Nginx 会主动断开空闲连接。 - -### 开发环境 - -开发时前端（Vite :5173）和后端（Gin :8080）不同端口。前端 WebSocket 地址基于 `window.location.host` 动态构建，通过 Vite `server.proxy` 转发到后端，无需硬编码端口。 - -`vite.config.ts` 中配置了 `/ws`（WebSocket）和 `/api`（REST）的代理，目标为 `http://localhost:8080`。 diff --git a/docs/04-技术选型.md b/docs/03-技术选型.md similarity index 57% rename from docs/04-技术选型.md rename to docs/03-技术选型.md index 8875fac..f7865a4 100644 --- a/docs/04-技术选型.md +++ b/docs/03-技术选型.md @@ -4,15 +4,29 @@ 本文档记录项目中各项技术的**选型过程、替代方案对比和决策理由**。技术选型没有"绝对正确"，只有"更适合"。 -**定位**：持久化部分是拓展选型，不阻塞 MVP（MVP 用内存存储即可）。前端边缘处理部分是 MVP 阶段就需要确定的技术栈。AI 服务栈（STT/LLM/TTS）已确定默认选型，可通过配置灵活切换。 +各技术选型章节包含关键术语解释，帮助快速理解技术概念。 + +### 后端核心技术栈 + +| 名词 | 解释 | +|------|------| +| **Go (Golang)** | 高并发后端语言，Google 开发，杀手锏是 goroutine——极轻量协程，一个程序可轻松开几万个，每个只占几 KB 内存，适合管理大量 WebSocket 长连接 | +| **gorilla/websocket** | Go WebSocket 库，Go 标准库无内置 WebSocket 支持，此库是社区最成熟的选择，处理了协议握手、帧解析等底层细节 | +| **Viper** | Go 配置管理库，读取 JSON/YAML/TOML 配置，支持环境变量覆盖，方便开发/测试/生产环境用不同配置 | +| **Zap** | Go 结构化日志库，Uber 开源，输出 JSON 格式日志，方便工具搜索分析，性能远超标准库 log | ``` 技术选型 +├── AI 编排框架 +│ └── CloudWeGo Eino Graph（声明式 DAG 编排，替代手写 goroutine 管道） ├── AI 服务栈 -│ ├── STT: Deepgram（默认） / MiMo ASR -│ ├── LLM: GPT-4o（默认） / 通义千问等 OpenAI 兼容模型 -│ └── TTS: OpenAI TTS（默认） / MiMo TTS -├── 持久化层 → 数据库选型: PostgreSQL（规划中，MVP 阶段使用内存存储） +│ ├── STT: MiMo ASR（默认） / Deepgram +│ ├── LLM: DashScope qwen3-vl-plus（默认） / GPT-4o 等 OpenAI 兼容模型 +│ └── TTS: MiMo TTS（默认） / OpenAI TTS +├── 持久化层 +│ ├── 数据库: PostgreSQL（pgx/v5，手写 SQL） +│ ├── 迁移: 嵌入式 SQL 文件，自动执行 +│ └── 存储模式: 三级存储 TieredManager（L1 Memory → L2 Redis → L3 PostgreSQL） ├── 认证与用户系统 │ ├── 认证方案: JWT (HS256), access 15min + refresh 7day │ ├── JWT 库: golang-jwt/jwt/v5 @@ -20,48 +34,116 @@ │ ├── 数据库驱动: pgx/v5（手写 SQL，不用 ORM） │ └── 前端 Token 存储: localStorage └── 前端边缘处理层 - ├── 边缘推理: ONNX Runtime Web（规划中，MVP 使用 Canvas 像素比较） + ├── 关键帧检测: Canvas 像素比较（160x120 降采样） ├── 语音检测: @ricky0123/vad-web └── 媒体采集: MediaDevices API ``` --- -## 一、AI 服务栈选型 +## 一、AI 编排框架选型 + +### 关键术语 + +| 名词 | 解释 | +|------|------| +| **Eino** | 字节跳动开源的 Go AI 应用开发框架（CloudWeGo Eino），提供 Graph DAG 编排、组件抽象（ChatModel/Tool 等）、流式处理和 Callback AOP 机制 | +| **compose.Graph** | Eino 的 DAG 编排器，声明式有向无环图，节点可以是 Lambda、ChatModel、ToolsNode 等，边定义数据流向 | +| **Lambda** | Graph 中的可组合函数单元，四种模式：InvokableLambda（同步）、StreamableLambda（流式输出）、CollectableLambda（流式输入）、TransformableLambda（双向流式） | +| **StreamReader** | Eino 的流式数据抽象 `schema.StreamReader[T]`，类似 io.Reader 的语义，`Recv()` 读取一帧，`io.EOF` 表示流结束 | +| **Callback** | Eino 的 AOP 机制，类似中间件的钩子，支持节点生命周期回调（OnStart/OnEnd/OnError/OnEndWithStreamOutput） | + +> 更多 Eino 相关概念详见 [10-Eino框架与编排设计.md](10-Eino框架与编排设计.md) + +### 候选方案对比 + +| 框架 | 语言 | 特点 | CamTalk 适用性 | +|------|------|------|---------------| +| **CloudWeGo Eino** | Go | Go 原生、类型安全、流式原生、Graph DAG 编排 | ✅ 完美匹配 | +| LangChain Go | Go | 生态丰富但较重，抽象层多 | ❌ 过度抽象 | +| 自研编排 | Go | 完全可控 | ❌ 维护成本高 | + +### 选择 Eino 的理由 + +| 维度 | 手写 goroutine（旧方案） | Eino Graph（新方案） | +|------|------------------------|---------------------| +| 编排方式 | 手动 `go func()` + `sync.WaitGroup` | 声明式 DAG，类型安全 | +| 流式处理 | 自定义 `chan` 传递 | `StreamReader` + `Pipe`，自动转换 | +| 错误处理 | 各节点独立处理，不一致 | Graph 级别统一错误传播 | +| 回调/AOP | 日志散落各处 | `callbacks.Handler` 统一注入 | +| 配置灵活性 | Pipeline 创建时固定 | 每请求 `Option` 动态注入 | +| 可测试性 | 需启动 goroutine | `Graph.Invoke()` 直接测试 | +| 扩展性 | 修改 Pipeline 代码 | 添加节点 + 边，无侵入 | + +### 核心依赖 + +``` +github.com/cloudwego/eino v0.9.9 # 核心框架 +github.com/cloudwego/eino-ext/components/model/openai v0.1.13 # OpenAI 兼容 ChatModel +``` + +**核心理由**： +1. Go 原生，泛型支持，编译时类型检查 +2. 原生流式处理（`StreamReader`），适合 LLM token 级推送 +3. Graph 支持分支、并行、循环，满足当前和未来需求 +4. Callback 机制实现 AOP（日志、指标、消息推送） +5. eino-ext 提供 OpenAI ChatModel 实现，直接对接 DashScope + +> 详细的 Eino 框架使用文档见 [11-Eino框架技术文档](11-Eino框架技术文档.md)，重构方案见 [10-Eino重构方案](10-Eino重构方案.md)，实施记录见 [12-Eino重构实施记录](12-Eino重构实施记录.md)。 + +--- + +## 二、AI 服务栈选型 + +### 关键术语 + +| 名词 | 解释 | +|------|------| +| **多模态 LLM** | 能读文字又能看图片的大语言模型，如 GPT-4o（OpenAI）、Claude Sonnet（Anthropic），给照片+问题能"看懂"照片再回答 | +| **STT** | Speech-to-Text，语音转文字。流式识别延迟可低于 500ms | +| **TTS** | Text-to-Speech，文字转语音。支持流式——边生成边读，不必等全部生成完 | ### STT（语音识别） | 方案 | 延迟 | 成本 | 特点 | |------|------|------|------| -| **Deepgram**（默认） | <500ms | 按分钟计费 | 流式识别，延迟极低，WebSocket 接口 | -| **MiMo ASR**（小米） | ~1s | 按量计费 | 国产替代，兼容 OpenAI chat/completions 格式，HTTP 非流式 | +| **MiMo ASR**（默认） | ~1s | 按量计费 | 国产替代，兼容 OpenAI chat/completions 格式，HTTP 非流式 | +| **Deepgram** | <500ms | 按分钟计费 | 流式识别，延迟极低，WebSocket 接口 | | Whisper API | 1-3s | 按分钟计费 | 准确率高，支持多语言 | | FunASR | <500ms | 自部署免费 | 阿里开源，中文优化 | -当前默认使用 Deepgram nova-2，可通过 `ai.stt.provider` 配置切换到 MiMo ASR。 +当前默认使用 MiMo ASR（mimo-v2.5-asr），可通过 `ai.stt.provider` 配置切换到 Deepgram。 ### LLM（多模态大模型） | 方案 | 成本 | 特点 | |------|------|------| -| **GPT-4o**（默认） | $2.5/1M tokens | 视觉理解能力强，API 成熟，流式推理 | -| 通义千问 qwen3-vl-plus | 按量计费 | 阿里云，通过 OpenAI 兼容接口调用 | +| **DashScope qwen3-vl-plus**（默认） | 按量计费 | 阿里云，通过 OpenAI 兼容接口调用，视觉理解能力强 | +| GPT-4o | $2.5/1M tokens | OpenAI，API 成熟，流式推理 | | Claude Sonnet | $3/1M tokens | Anthropic，长上下文能力强 | -代码通过 OpenAI 兼容接口调用，可灵活切换到任何兼容服务商。配置 `ai.llm.provider`、`ai.llm.model`、`ai.llm.endpoint` 即可。 +LLM 通过 Eino 框架的 `eino-ext/components/model/openai` ChatModel 组件接入，支持任何 OpenAI 兼容接口。配置 `ai.llm.provider`、`ai.llm.model`、`ai.llm.endpoint` 即可切换。 ### TTS（语音合成） | 方案 | 成本 | 特点 | |------|------|------| -| **OpenAI TTS**（默认） | $15/1M 字符 | 音质自然，支持流式，默认模型 tts-1，语音 alloy | -| MiMo TTS（小米） | 按量计费 | 国产替代，通过配置切换 | +| **MiMo TTS**（默认） | 按量计费 | 国产替代，通过配置切换，模型 mimo-v2.5-tts | +| OpenAI TTS | $15/1M 字符 | 音质自然，支持流式，默认模型 tts-1，语音 alloy | -当前默认使用 OpenAI TTS（tts-1, alloy），可通过 `ai.tts.provider` 配置切换。 +当前默认使用 MiMo TTS（mimo-v2.5-tts），可通过 `ai.tts.provider` 配置切换到 OpenAI TTS。 --- -## 二、持久化层选型（规划中，MVP 阶段使用内存存储） +## 三、持久化层选型 + +### 关键术语 + +| 名词 | 解释 | +|------|------| +| **PostgreSQL** | 关系型数据库，支持 JSONB（JSON 二进制格式，可建索引）、窗口函数、CTE 等高级特性 | +| **Redis** | 内存 KV 数据库，数据放在内存里，读写微秒级。支持 TTL 过期自动清理 | +| **MVCC** | Multi-Version Concurrency Control，多版本并发控制，PostgreSQL 用此实现高并发读写而不阻塞 | ### 数据特征分析 @@ -146,17 +228,19 @@ ORDER BY created_at DESC LIMIT 20; ``` -### 冷热分离架构 +### 冷热分离架构（三级存储） ``` -Go Gateway - ├── 写入路径 → Redis（实时会话状态） - │ → PostgreSQL（对话历史 + 用量） - └── 读取路径 → Redis（当前上下文，快） - → PostgreSQL（历史记录，慢） +Go Gateway (TieredManager) + ├── L1: Memory（进程内缓存，微秒级） + ├── L2: Redis（分布式缓存，毫秒级） + └── L3: PostgreSQL（持久化存储，冷数据） + +读取路径：L1 → L2 → L3，逐级回源，命中后向上回填 +写入路径：L1 → L2（同步） → L3（异步） ``` -建议异步写入——实时消息先写 Redis（快），异步批量刷入 PostgreSQL（慢），不影响对话体验。 +`TieredManager` 自动管理三级存储，后台 goroutine 每 30 秒 ping Redis 健康状态，Redis 故障时自动降级为 L1+L3 模式。 ### 决策流程 @@ -173,7 +257,19 @@ Go Gateway --- -## 二、前端边缘处理层选型 +## 四、前端边缘处理层选型 + +### 关键术语 + +| 名词 | 解释 | +|------|------| +| **React 18** | 组件化 UI 框架，Facebook 开源，把页面拆成组件搭积木拼装。18 版本支持并发渲染 | +| **TypeScript** | 带类型的 JavaScript，在 JS 基础上增加类型声明，编译阶段就能发现类型错误 | +| **Vite** | 前端构建工具，利用浏览器原生 ES Module，开发时毫秒级热更新（HMR），构建产物小 | +| **WebSocket** | 浏览器与服务器的双向通道。HTTP 是"一问一答"，WebSocket 像打电话——接通后双方随时互发消息，适合实时对话场景 | +| **ONNX Runtime Web** | 浏览器端 AI 推理引擎，微软定义的通用模型格式 ONNX 的运行引擎，可在浏览器中用 WASM 加速跑轻量模型（如 VAD、关键帧检测），零延迟、不耗服务器资源 | +| **VAD** | Voice Activity Detection，语音活动检测，检测"人有没有在说话"。WebRTC 内置了高效的 VAD 算法 | +| **MediaDevices API** | 浏览器摄像头/麦克风接口，`navigator.mediaDevices.getUserMedia()` 是浏览器音视频采集的唯一标准入口，无需插件 | ### 总览 @@ -223,7 +319,16 @@ vad-web 是"够用且最轻"的平衡点——直接包装浏览器原生 WebRTC --- -## 四、认证与用户系统选型 +## 五、认证与用户系统选型 + +### 关键术语 + +| 名词 | 解释 | +|------|------| +| **JWT** | JSON Web Token，无状态 token，服务端不存 session，分布式友好 | +| **HS256** | HMAC-SHA256，JWT 对称签名算法，用同一密钥签名和验证 | +| **bcrypt** | 密码哈希算法，自适应 cost factor，抗暴力破解 | +| **pgx** | Go 生态性能最优的 PostgreSQL 驱动，原生协议实现，内置连接池 pgxpool | ### 总览 diff --git a/docs/03-接口文档.md b/docs/03-接口文档.md deleted file mode 100644 index 3f7d28f..0000000 --- a/docs/03-接口文档.md +++ /dev/null @@ -1,1622 +0,0 @@ -# 接口文档 - -## 概述 - -前后端通信接口定义。以 WebSocket 承载实时对话，REST 端点支撑基础运维。**暂不实现持久化**，但通过 Repository 接口模式为后续扩展预留接入点。 - -**设计原则**： -- WebSocket 为主：所有对话数据走 WebSocket -- REST 为辅：仅用于健康检查、会话管理等低频操作 -- 接口先行：先定义契约，再填充实现——前后端可并行开发 - -## 接口全景 - -``` -浏览器 Go Gateway :8080 - WebSocket Client <--> /ws?token= （实时对话，需 JWT 认证） - HTTP Client --> GET /api/health （健康检查） - HTTP Client <--> POST /api/auth/* （注册/登录/刷新/登出） - HTTP Client <--> GET/POST/PATCH/DELETE （对话 CRUD） - /api/conversations/* - HTTP Client <--> GET /api/conversations/:id （历史消息） - /messages - HTTP Client ~~> POST/DELETE /api/sessions （已废弃，保留兼容） -``` - ---- - -## 一、WebSocket 协议 - -连接地址：`ws://localhost:8080/ws?token=&conversation_id=` - -| 参数 | 必填 | 说明 | -|------|------|------| -| `token` | 是 | JWT access_token，缺失或无效时返回 401 | -| `conversation_id` | 否 | 恢复已有对话；省略则创建新对话 | - -> 详见"REST API → WebSocket 认证变更"章节。 - -### 消息格式约定 - -所有 WebSocket 消息均为 JSON 文本帧，统一结构： - -```typescript -interface WsMessage { - type: string; // 消息类型，必填 - request_id?: string; // 可选，用于请求-响应关联 - timestamp?: number; // 可选，毫秒时间戳 - [key: string]: any; // 类型特定字段 -} -``` - -### 客户端 → 服务端消息 - -#### `query` — 发起一次视觉对话 - -用户说完话后，客户端同时发送当前图像帧和语音片段。也支持文本输入模式（手动输入文字时跳过语音识别）： - -```typescript -interface QueryMessage { - type: "query"; - request_id: string; // 客户端生成的 UUID - image: string; // Base64 编码的 JPEG 图像（不含 data: 前缀） - audio: string; // Base64 编码的音频片段（PCM 16kHz），文本输入时为空字符串 - text?: string; // 用户手动输入的文本（有值时跳过 STT，直接使用此文本） - mime_type?: string; // 音频格式，默认 "audio/pcm" -} -``` - -> 为什么图像和音频放在同一条消息里？因为 VAD 检测到用户说完话时，需要同时捕获"此刻的画面"和"说的话"，拆成两条消息会增加时序同步的复杂度。 -> -> **文本输入模式**：当用户关闭麦克风后，可通过对话框手动输入文字。此时 `text` 字段携带用户输入，`audio` 为空字符串，服务端跳过 STT 直接使用 `text` 进行 LLM 推理。 - -#### `config` — 更新会话配置 - -```typescript -interface ConfigMessage { - type: "config"; - payload: { - tts_enabled?: boolean; // 是否开启语音合成，默认 true - detail_level?: "low" | "high"; // 图像精度，默认 "low" - language?: string; // 交互语言，默认 "zh-CN" - }; -} -``` - -#### `interrupt` — 打断当前回复 - -```typescript -interface InterruptMessage { - type: "interrupt"; - request_id?: string; // 可选，当前实现不使用此字段，服务端始终取消当前活跃请求 -} -``` - -#### `ping` — 心跳保活 - -```typescript -interface PingMessage { - type: "ping"; -} -``` - -### 服务端 → 客户端消息 - -#### `connected` — 连接建立确认 - -```typescript -interface ConnectedMessage { - type: "connected"; - session_id: string; // 服务端生成的会话 ID - conversation_id: string; // 同 session_id，便于前端统一使用 - server_version: string; // 服务端版本号，如 "0.1.0" -} -``` - -#### `stt_result` — 语音识别结果 - -```typescript -interface STTResultMessage { - type: "stt_result"; - request_id: string; - text: string; // 识别出的用户语音文本 - is_final: boolean; // 是否为最终结果 -} -``` - -#### `llm_chunk` — LLM 流式输出片段 - -```typescript -interface LLMChunkMessage { - type: "llm_chunk"; - request_id: string; - delta: string; // 本次增量文本 - role: "assistant"; -} -``` - -#### `llm_done` — LLM 输出完成 - -```typescript -interface LLMDoneMessage { - type: "llm_done"; - request_id: string; - full_text: string; // 完整回复文本 - tokens_used: { - prompt: number; - completion: number; - total: number; - }; - model: string; // 实际使用的模型名 - latency_ms: number; // 端到端延迟（毫秒） -} -``` - -#### `tts_audio` — TTS 音频流片段 - -```typescript -interface TTSAudioMessage { - type: "tts_audio"; - request_id: string; - audio: string; // Base64 编码的音频片段 - mime_type: string; // "audio/mp3" - is_last: boolean; // 当前句子的音频是否完整（每句结束时为 true） - final: boolean; // 整轮 TTS 是否结束（所有句子合成完毕后为 true） -} -``` - -**字段语义**： - -- `is_last`: 每个句子合成完毕后为 `true`，前端收到此信号即可将该句子加入播放队列。每句 TTS 音频由一次独立的 API 调用生成，对应一个 `tts_audio` 消息。 -- `final`: 所有句子合成完毕后为 `true`（此时 `audio` 为空字符串），用于前端判断本轮 TTS 已全部到齐。 - -**音频格式规范**（前端播放依赖此约定）： - -| 属性 | 值 | 说明 | -|------|------|------| -| 编码 | `audio/mp3`（MP3） | 浏览器 `