diff --git a/docs/superpowers/plans/2026-07-06-phase3a-web-api.md b/docs/superpowers/plans/2026-07-06-phase3a-web-api.md new file mode 100644 index 0000000..db45de8 --- /dev/null +++ b/docs/superpowers/plans/2026-07-06-phase3a-web-api.md @@ -0,0 +1,839 @@ +# Phase 3a Web API 完整化 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** 把 Phase 2 的同步轻量 API 升级为异步、可登录、带进度推送、因子报告完整的后端。 + +**Architecture:** FastAPI + ProcessPoolExecutor（spawn context）做异步回测，JWT 单用户鉴权，WS 连接池推阶段进度，analyzer 补完整 alphalens tears pipeline。 + +**Tech Stack:** FastAPI + uvicorn、concurrent.futures.ProcessPoolExecutor、PyJWT、websockets（FastAPI TestClient）、alphalens（容器）。 + +## Global Constraints + +- **vnpy 零改造**：源码 `vnpy_v4.4.0/`，`sys.path.insert(0, _VNPY_SRC)`（参照 sanguo_data/datareader.py:6-9） +- **Python 3.14 本地 / 3.10 容器**；本地无 polars/alphalens/vnpy_ctastrategy → lazy import + mock 测试（参照 Phase 2 模式） +- **Phase 2 接口复用**：`result_store`（save_result/load_result/BacktestResult）、`cta_engine.run_cta_backtest`、`cta_optimizer.run_cta_optimization`、`registry/library`、`runner.Orchestrator` +- **ProcessPoolExecutor 用 spawn context**（vnpy/alpha 已用 spawn，兼容） +- **WS 简单连接池**（dict[task_id, set[WebSocket]]），不做重连/心跳（YAGNI） +- **JWT 单用户**：用户名/密码 hash/jwt_secret 配 `config/backtest.yaml` +- **测试策略**：本地 Python 3.14 跑 mock 测试 + 容器 Python 3.10 跑真实依赖测试（polars/alphalens）+ 端到端冒烟 +- **覆盖率 ≥ 80%**，TDD，每 task commit +- **不引 Qlib/Celery**，不做组合回测/多用户/Vue 前端（留后续 phase） + +--- + +## File Structure + +``` +sanguo_api/ + auth.py # 新：JWT 单用户（create_token/verify_token） + ws.py # 新：WS 连接池（ConnectionManager） + routes.py # 改：+ /auth/login、optimize/factor 真调用、JWT 依赖、WS route + app.py # 改：register auth/ws，JWT 依赖生效 +sanguo_orchestrator/ + pool.py # 改：TaskPool 加 ProcessPoolExecutor + stage 追踪 + runner.py # 改：async submit_* + on_stage 回调 +sanguo_factor/ + alpha_lab.py # 改：补 compute_factors（add_feature + prepare_data + fetch_raw） + analyzer.py # 改：tears pipeline 完整化 +config/backtest.yaml # 改：+ auth + pool 配置 +tests/api/{test_auth.py, test_ws.py, test_routes.py} # 新/改 +tests/orchestrator/{test_pool.py, test_runner.py} # 改 +tests/factor/{test_alpha_lab.py, test_analyzer.py} # 改 +scripts/smoke_phase3a.py # 新 +``` + +--- + +## Task 1: config 扩展 + sanguo_api/auth.py（JWT 单用户） + +**Files:** +- Modify: `config/backtest.yaml` +- Create: `sanguo_api/auth.py` +- Test: `tests/api/test_auth.py` + +**Interfaces:** +- Produces: `create_token(username: str) → str`、`verify_token(token: str) → str`（返回 username，FastAPI 依赖用法 `Depends(verify_token)`，无效 raise HTTPException 401）、`hash_password(pw) → str`、`verify_password(pw, hash) → bool` + +- [ ] **Step 1: 写失败测试** + +```python +# tests/api/test_auth.py +import pytest + +def test_create_and_verify_token(): + from sanguo_api.auth import create_token, verify_token, set_jwt_config + set_jwt_config(secret="test_secret", expire_minutes=60) + token = create_token("admin") + assert isinstance(token, str) + assert verify_token(token) == "admin" + +def test_verify_token_invalid_raises_401(): + from fastapi import HTTPException + from sanguo_api.auth import verify_token, set_jwt_config + set_jwt_config(secret="test_secret", expire_minutes=60) + with pytest.raises(HTTPException) as exc: + verify_token("invalid.token.here") + assert exc.value.status_code == 401 + +def test_hash_and_verify_password(): + from sanguo_api.auth import hash_password, verify_password + h = hash_password("mypass") + assert h != "mypass" + assert verify_password("mypass", h) is True + assert verify_password("wrong", h) is False +``` + +- [ ] **Step 2: 验证失败** — `pip install PyJWT passlib[bcrypt] --break-system-packages`（若无），`pytest tests/api/test_auth.py -v` → FAIL + +- [ ] **Step 3: 实现 auth.py** + +```python +# sanguo_api/auth.py +"""JWT 单用户认证。secret/用户名/密码 hash 来自 config/backtest.yaml。""" +import os +from datetime import datetime, timedelta, timezone +import jwt +from passlib.context import CryptContext +from fastapi import HTTPException, status + +_CONFIG = {"secret": "change-me", "expire_minutes": 60, "algorithm": "HS256"} +_pwd = CryptContext(schemes=["bcrypt"], deprecated="auto") + + +def set_jwt_config(secret: str, expire_minutes: int, algorithm: str = "HS256"): + _CONFIG.update(secret=secret, expire_minutes=expire_minutes, algorithm=algorithm) + + +def hash_password(password: str) -> str: + return _pwd.hash(password) + + +def verify_password(password: str, password_hash: str) -> bool: + return _pwd.verify(password, password_hash) + + +def create_token(username: str) -> str: + payload = { + "sub": username, + "exp": datetime.now(timezone.utc) + timedelta(minutes=_CONFIG["expire_minutes"]), + } + return jwt.encode(payload, _CONFIG["secret"], algorithm=_CONFIG["algorithm"]) + + +def verify_token(token: str) -> str: + try: + payload = jwt.decode(token, _CONFIG["secret"], algorithms=[_CONFIG["algorithm"]]) + return payload["sub"] + except jwt.PyJWTError: + raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="无效 token") +``` + +- [ ] **Step 4: 验证通过** — `pytest tests/api/test_auth.py -v` → 3 passed + +- [ ] **Step 5: 扩展 config + commit** + +```yaml +# config/backtest.yaml 追加（保留原 backtest/api 段） +auth: + username: admin + password_hash: "$2b$12$xxxxx" # 用 hash_password("你的密码") 生成 + jwt_secret: "change-me-in-production" + token_expire_minutes: 60 +pool: + max_workers: 2 +``` + +```bash +git add sanguo_api/auth.py config/backtest.yaml tests/api/test_auth.py +git commit -m "feat(api): JWT 单用户认证（auth.py）+ config 扩展" +``` + +--- + +## Task 2: sanguo_api/ws.py（WS 连接池） + +**Files:** +- Create: `sanguo_api/ws.py` +- Test: `tests/api/test_ws.py` + +**Interfaces:** +- Produces: `ConnectionManager`（`connect(task_id, ws)`、`disconnect(task_id, ws)`、`broadcast(task_id, msg: dict)`）、模块级单例 `manager = ConnectionManager()` + +- [ ] **Step 1: 写失败测试** + +```python +# tests/api/test_ws.py +import pytest +from unittest.mock import AsyncMock, MagicMock + +@pytest.mark.asyncio +def test_connect_and_broadcast(): + from sanguo_api.ws import ConnectionManager + mgr = ConnectionManager() + ws = AsyncMock() + mgr.connect("t1", ws) + assert "t1" in mgr._connections + import asyncio + asyncio.run(mgr.broadcast("t1", {"status": "running"})) + ws.send_json.assert_called_with({"status": "running"}) + +def test_disconnect_removes_ws(): + from sanguo_api.ws import ConnectionManager + mgr = ConnectionManager() + ws = MagicMock() + mgr.connect("t1", ws) + mgr.disconnect("t1", ws) + assert ws not in mgr._connections.get("t1", set()) + +@pytest.mark.asyncio +def test_broadcast_no_subscribers_no_error(): + from sanguo_api.ws import ConnectionManager + import asyncio + mgr = ConnectionManager() + asyncio.run(mgr.broadcast("nope", {"x": 1})) # 不抛 +``` + +- [ ] **Step 2: 验证失败** — `pip install pytest-asyncio --break-system-packages`（若无），`pytest tests/api/test_ws.py -v` → FAIL + +- [ ] **Step 3: 实现 ws.py** + +```python +# sanguo_api/ws.py +"""WS 连接池：task_id → 订阅者集合。简单广播，不做重连/心跳。""" +from fastapi import WebSocket + + +class ConnectionManager: + def __init__(self): + self._connections: dict[str, set[WebSocket]] = {} + + def connect(self, task_id: str, ws: WebSocket): + self._connections.setdefault(task_id, set()).add(ws) + + def disconnect(self, task_id: str, ws: WebSocket): + conns = self._connections.get(task_id) + if conns: + conns.discard(ws) + if not conns: + del self._connections[task_id] + + async def broadcast(self, task_id: str, msg: dict): + for ws in list(self._connections.get(task_id, [])): + try: + await ws.send_json(msg) + except Exception: + self.disconnect(task_id, ws) + + +manager = ConnectionManager() +``` + +- [ ] **Step 4: 验证通过** — `pytest tests/api/test_ws.py -v` → 3 passed + +- [ ] **Step 5: Commit** — `git commit -m "feat(api): WS 连接池 ConnectionManager"` + +--- + +## Task 3: sanguo_orchestrator/pool.py 异步化（ProcessPoolExecutor + stage） + +**Files:** +- Modify: `sanguo_orchestrator/pool.py`、`sanguo_orchestrator/task.py` +- Test: `tests/orchestrator/test_pool.py` + +**Interfaces:** +- Produces: `Task` 加 `stage: str` 字段（默认 ""）；`TaskPool` 加 `executor: ProcessPoolExecutor`（spawn context，max_workers）、`submit_work(task_id, func, *args)` → `Future`、`update_stage(task_id, stage)`、`get_stage(task_id) → str|None` + +- [ ] **Step 1: 写失败测试** + +```python +# tests/orchestrator/test_pool.py（追加） +from unittest.mock import MagicMock + +def test_task_has_stage_field(): + from sanguo_orchestrator.task import Task + t = Task(task_id="t1", task_type="cta") + assert t.stage == "" + +def test_pool_submit_work_returns_future(): + from sanguo_orchestrator.pool import TaskPool + pool = TaskPool(max_workers=2) + pool.executor = MagicMock() # mock executor 避免真起进程 + mock_future = MagicMock() + pool.executor.submit.return_value = mock_future + fut = pool.submit_work("t1", func=lambda: 1) + assert fut is mock_future + pool.executor.submit.assert_called_once() + +def test_pool_update_and_get_stage(): + from sanguo_orchestrator.pool import TaskPool + from sanguo_orchestrator.task import Task + pool = TaskPool(max_workers=2) + pool.submit("t1", "cta") + pool.update_stage("t1", "回测中") + assert pool.get_stage("t1") == "回测中" + assert pool.get_task("t1").stage == "回测中" +``` + +- [ ] **Step 2: 验证失败** + +- [ ] **Step 3: task.py 加 stage 字段** + +```python +# sanguo_orchestrator/task.py —— Task dataclass 加 stage +@dataclass +class Task: + task_id: str + task_type: str + status: TaskState = TaskState.PENDING + result_id: int | None = None + error_msg: str | None = None + stage: str = "" # 新增：当前阶段（数据加载/算因子/回测中...） + # start/complete/fail 方法不变 +``` + +- [ ] **Step 4: pool.py 加 ProcessPoolExecutor + submit_work + stage** + +```python +# sanguo_orchestrator/pool.py +from concurrent.futures import ProcessPoolExecutor, Future +from multiprocessing import get_context +from .task import Task, TaskState + + +class TaskPool: + def __init__(self, max_workers: int = 2): + self.max_workers = max_workers + self._tasks: dict[str, Task] = {} + self.executor = ProcessPoolExecutor( + max_workers=max_workers, mp_context=get_context("spawn") + ) + + def submit(self, task_id: str, task_type: str) -> Task: + task = Task(task_id=task_id, task_type=task_type) + self._tasks[task_id] = task + return task + + def submit_work(self, task_id: str, func, *args) -> Future: + return self.executor.submit(func, *args) + + def update_stage(self, task_id: str, stage: str): + t = self._tasks.get(task_id) + if t: + t.stage = stage + + def get_status(self, task_id: str) -> TaskState | None: + t = self._tasks.get(task_id) + return t.status if t else None + + def get_stage(self, task_id: str) -> str | None: + t = self._tasks.get(task_id) + return t.stage if t else None + + def get_task(self, task_id: str) -> Task | None: + return self._tasks.get(task_id) +``` + +- [ ] **Step 5: 验证通过 + commit** — `pytest tests/orchestrator/test_pool.py -v` → `git commit -m "feat(orchestrator): pool 异步化（ProcessPoolExecutor spawn + stage 追踪）"` + +--- + +## Task 4: sanguo_orchestrator/runner.py async submit + on_stage 回调 + +**Files:** +- Modify: `sanguo_orchestrator/runner.py` +- Test: `tests/orchestrator/test_runner.py` + +**Interfaces:** +- Produces: `Orchestrator` 改 `async submit_cta(...)` → task_id（提交到 pool.executor，asyncio.wrap_future 桥接，完成后调回调更新状态 + broadcast）、`set_on_stage(callback)`（callback: async (task_id, stage) → None） + +- [ ] **Step 1: 写失败测试** + +```python +# tests/orchestrator/test_runner.py（追加） +import asyncio +from unittest.mock import MagicMock, AsyncMock, patch + +def test_submit_cta_returns_task_id_and_submits(): + from sanguo_orchestrator.runner import Orchestrator + orch = Orchestrator(db_path="/tmp/t.db") + orch.pool.executor = MagicMock() + orch.pool.executor.submit.return_value = MagicMock() + with patch("sanguo_backtest.cta_engine.run_cta_backtest"): + task_id = asyncio.get_event_loop().run_until_complete( + orch.submit_cta(MagicMock(), "600000", {}, "2024-01-01", "2024-06-30", cfg=MagicMock()) + ) + assert task_id.startswith("cta_600000") + orch.pool.executor.submit.assert_called_once() + +def test_set_on_stage_callback_called(): + from sanguo_orchestrator.runner import Orchestrator + from sanguo_orchestrator.task import TaskState + orch = Orchestrator(db_path="/tmp/t.db") + orch.pool.executor = MagicMock() + mock_future = MagicMock() + orch.pool.executor.submit.return_value = mock_future + cb = AsyncMock() + orch.set_on_stage(cb) + # 模拟任务完成回调 + asyncio.get_event_loop().run_until_complete(orch._on_done("t1", {"statistics": {}})) + cb.assert_called() +``` + +- [ ] **Step 2: 验证失败** + +- [ ] **Step 3: 实现 runner.py async** + +```python +# sanguo_orchestrator/runner.py +import asyncio +from concurrent.futures import Future +from .pool import TaskPool +from .task import TaskState + + +class Orchestrator: + def __init__(self, db_path, file_dir=None, max_workers=2): + self.db_path = db_path + self.file_dir = file_dir + self.pool = TaskPool(max_workers=max_workers) + self._pending: dict = {} + self._on_stage = None # async callback(task_id, stage) + + def set_on_stage(self, cb): + self._on_stage = cb + + async def _notify_stage(self, task_id: str, stage: str): + self.pool.update_stage(task_id, stage) + if self._on_stage: + await self._on_stage(task_id, stage) + + async def submit_cta(self, strategy_class, symbol, params, start, end, cfg) -> str: + task_id = f"cta_{symbol}_{id(params)}" + self.pool.submit(task_id, "cta") + self._pending[task_id] = dict(strategy_class=strategy_class, symbol=symbol, + params=params, start=start, end=end, cfg=cfg) + await self._notify_stage(task_id, "排队中") + spec = self._pending[task_id] + from sanguo_backtest.cta_engine import run_cta_backtest + fut: Future = self.pool.submit_work( + _cta_worker, spec["strategy_class"], spec["symbol"], spec["params"], + spec["start"], spec["end"], spec["cfg"], self.db_path, + ) + self.pool.get_task(task_id).start() + await self._notify_stage(task_id, "回测中") + asyncio.ensure_future(self._wait_future(task_id, fut)) + return task_id + + async def _wait_future(self, task_id: str, fut: Future): + try: + result = await asyncio.wrap_future(fut) + await self._on_done(task_id, result) + except Exception as e: + self.pool.get_task(task_id).fail(f"{type(e).__name__}: {e}") + await self._notify_stage(task_id, "失败") + + async def _on_done(self, task_id: str, result): + self.pool.get_task(task_id).complete(result_id=id(result)) + await self._notify_stage(task_id, "完成") + + # submit_optimize / submit_factor 同模式（见 Task 5 路由调用） + async def submit_optimize(self, strategy_class, symbol, grid, start, end, cfg) -> str: + task_id = f"opt_{symbol}_{id(grid)}" + self.pool.submit(task_id, "optimize") + await self._notify_stage(task_id, "参数优化中") + from sanguo_backtest.cta_optimizer import run_cta_optimization + fut = self.pool.submit_work(_opt_worker, strategy_class, symbol, grid, start, end, cfg, self.db_path) + self.pool.get_task(task_id).start() + asyncio.ensure_future(self._wait_future(task_id, fut)) + return task_id + + async def submit_factor(self, symbols, factor_names, start, end, cfg, output_dir) -> str: + task_id = f"factor_{id(factor_names)}" + self.pool.submit(task_id, "factor") + await self._notify_stage(task_id, "因子分析中") + from sanguo_factor.analyzer import run_factor_analysis + fut = self.pool.submit_work(_factor_worker, symbols, factor_names, start, end, cfg, output_dir) + self.pool.get_task(task_id).start() + asyncio.ensure_future(self._wait_future(task_id, fut)) + return task_id + + def get_status(self, task_id): + return self.pool.get_status(task_id) + + def get_result(self, task_id): + task = self.pool.get_task(task_id) + if task and task.status == TaskState.DONE and task.result_id: + from sanguo_backtest.result_store import load_result + return load_result(task.result_id, self.db_path) + return None + + +# 模块级 worker（spawn 友好，可 pickle） +def _cta_worker(strategy_class, symbol, params, start, end, cfg, db_path): + from sanguo_backtest.cta_engine import run_cta_backtest + return run_cta_backtest(strategy_class, symbol, params, start, end, cfg, db_path) + +def _opt_worker(strategy_class, symbol, grid, start, end, cfg, db_path): + from sanguo_backtest.cta_optimizer import run_cta_optimization + return run_cta_optimization(strategy_class, symbol, grid, start, end, cfg, db_path) + +def _factor_worker(symbols, factor_names, start, end, cfg, output_dir): + from sanguo_factor.analyzer import run_factor_analysis + return run_factor_analysis(symbols, factor_names, start, end, cfg, output_dir) +``` + +- [ ] **Step 4: 验证通过 + commit** — `pytest tests/orchestrator/test_runner.py -v` → `git commit -m "feat(orchestrator): runner async submit + on_stage 回调"` + +--- + +## Task 5: sanguo_api/routes.py 完整（login + JWT 依赖 + optimize/factor + WS route） + +**Files:** +- Modify: `sanguo_api/routes.py`、`sanguo_api/app.py` +- Test: `tests/api/test_routes.py` + +**Interfaces:** +- Produces: `POST /auth/login`（用户名密码 → token）、optimize/factor 路由真调用 submit_optimize/submit_factor、业务路由加 `Depends(verify_token)`、`WS /ws/task/{id}`（订阅阶段推送） + +- [ ] **Step 1: 写失败测试** + +```python +# tests/api/test_routes.py（追加） +from fastapi.testclient import TestClient +from unittest.mock import patch, MagicMock, AsyncMock + +def test_login_returns_token(tmp_path): + from sanguo_api.app import create_app + from sanguo_api.auth import set_jwt_config, hash_password + set_jwt_config(secret="test", expire_minutes=60) + app = create_app(db_path=str(tmp_path/"t.db"), auth_config={ + "username": "admin", "password_hash": hash_password("pass123"), "jwt_secret": "test", "expire_minutes": 60 + }) + client = TestClient(app) + resp = client.post("/api/v1/auth/login", json={"username": "admin", "password": "pass123"}) + assert resp.status_code == 200 + assert "token" in resp.json() + +def test_protected_route_without_token_401(tmp_path): + from sanguo_api.app import create_app + app = create_app(db_path=str(tmp_path/"t.db")) + client = TestClient(app) + resp = client.get("/api/v1/task/t1") + assert resp.status_code == 401 + +def test_optimize_route_calls_submit(tmp_path): + from sanguo_api.app import create_app + from sanguo_api.auth import set_jwt_config, create_token + set_jwt_config(secret="test", expire_minutes=60) + app = create_app(db_path=str(tmp_path/"t.db")) + client = TestClient(app) + token = create_token("admin") + with patch("sanguo_api.routes.get_orchestrator") as m: + orch = MagicMock() + orch.submit_optimize = AsyncMock(return_value="opt_1") + m.return_value = orch + resp = client.post("/api/v1/backtest/optimize", json={ + "symbol": "600000", "strategy": "MaStrategy", "grid": {"n": [5, 20, 5]}, + "start": "2024-01-01", "end": "2024-06-30", "max_workers": 2 + }, headers={"Authorization": f"Bearer {token}"}) + assert resp.status_code == 200 + assert resp.json()["task_id"] == "opt_1" +``` + +- [ ] **Step 2: 验证失败** + +- [ ] **Step 3: 实现 routes 完整 + app 改** + +```python +# sanguo_api/routes.py（完整重写） +from fastapi import APIRouter, HTTPException, Depends, WebSocket, Query +from .schemas import CtaBacktestRequest, OptimizeRequest, FactorAnalysisRequest +from .auth import verify_token, verify_password, create_token +from .ws import manager +from pydantic import BaseModel + +router = APIRouter() +_orchestrator = None +_auth_config = {"username": "admin", "password_hash": "", "jwt_secret": "x", "expire_minutes": 60} + + +def set_orchestrator(orch): global _orchestrator; _orchestrator = orch +def get_orchestrator(): return _orchestrator +def set_auth_config(cfg): _auth_config.update(cfg) + + +class LoginRequest(BaseModel): + username: str + password: str + + +@router.post("/auth/login") +def login(req: LoginRequest): + if req.username != _auth_config["username"] or not verify_password(req.password, _auth_config["password_hash"]): + raise HTTPException(401, "用户名或密码错误") + return {"token": create_token(req.username)} + + +@router.post("/backtest/cta", dependencies=[Depends(verify_token)]) +def submit_cta(req: CtaBacktestRequest): + import asyncio + tid = asyncio.get_event_loop().run_until_complete( + get_orchestrator().submit_cta(req.strategy, req.symbol, req.params, req.start, req.end, None)) + return {"task_id": tid} + + +@router.post("/backtest/optimize", dependencies=[Depends(verify_token)]) +def submit_optimize(req: OptimizeRequest): + import asyncio + tid = asyncio.get_event_loop().run_until_complete( + get_orchestrator().submit_optimize(req.strategy, req.symbol, req.grid, req.start, req.end, None)) + return {"task_id": tid} + + +@router.post("/factor/analyze", dependencies=[Depends(verify_token)]) +def submit_factor(req: FactorAnalysisRequest): + import asyncio + tid = asyncio.get_event_loop().run_until_complete( + get_orchestrator().submit_factor(req.symbols, req.factor_names, req.start, req.end, None, "/tmp/factor")) + return {"task_id": tid} + + +@router.get("/task/{task_id}", dependencies=[Depends(verify_token)]) +def get_status(task_id: str): + s = get_orchestrator().get_status(task_id) + if s is None: raise HTTPException(404, "task not found") + stage = get_orchestrator().pool.get_stage(task_id) + return {"task_id": task_id, "status": s.value if hasattr(s, "value") else str(s), "stage": stage or ""} + + +@router.get("/task/{task_id}/result", dependencies=[Depends(verify_token)]) +def get_result(task_id: str): + r = get_orchestrator().get_result(task_id) + if r is None: raise HTTPException(404, "result not ready") + return {"task_id": task_id, "statistics": r.statistics} + + +@router.websocket("/ws/task/{task_id}") +async def task_ws(websocket: WebSocket, task_id: str, token: str = Query(...)): + from .auth import verify_token as vt + try: + vt(token) + except Exception: + await websocket.close(code=4401); return + await websocket.accept() + manager.connect(task_id, websocket) + try: + while True: + await websocket.receive_text() # 保活 + except Exception: + pass + finally: + manager.disconnect(task_id, websocket) +``` + +```python +# sanguo_api/app.py（改：register auth_config + on_stage 推 WS） +from fastapi import FastAPI +from .routes import router, set_orchestrator, set_auth_config +from .auth import set_jwt_config +from .ws import manager +from sanguo_orchestrator.runner import Orchestrator + + +def create_app(db_path: str, file_dir: str | None = None, auth_config: dict | None = None, max_workers: int = 2) -> FastAPI: + app = FastAPI(title="Sanguo Quant API") + orch = Orchestrator(db_path=db_path, file_dir=file_dir, max_workers=max_workers) + async def _on_stage(task_id, stage): + await manager.broadcast(task_id, {"task_id": task_id, "stage": stage}) + orch.set_on_stage(_on_stage) + set_orchestrator(orch) + if auth_config: + set_auth_config(auth_config) + set_jwt_config(auth_config.get("jwt_secret", "x"), auth_config.get("expire_minutes", 60)) + app.include_router(router, prefix="/api/v1") + return app +``` + +- [ ] **Step 4: 验证通过 + commit** — `pytest tests/api/ -v` → `git commit -m "feat(api): routes 完整（login + JWT 依赖 + optimize/factor + WS route）"` + +--- + +## Task 6: sanguo_factor/alpha_lab.py compute_factors 完整化 + +**Files:** +- Modify: `sanguo_factor/alpha_lab.py` +- Test: `tests/factor/test_alpha_lab.py` + +**Interfaces:** +- Produces: `AlphaLabSession.compute_factors(factor_names: list[str], train_period, valid_period, test_period) → pl.DataFrame`（add_feature + prepare_data + fetch_raw） + +- [ ] **Step 1: 写失败测试** + +```python +# tests/factor/test_alpha_lab.py（追加） +from unittest.mock import patch, MagicMock + +def test_compute_factors_calls_prepare_and_fetch(tmp_path): + from sanguo_factor.alpha_lab import AlphaLabSession + with patch("sanguo_factor.alpha_lab.AlphaLab") as MLab, \ + patch("sanguo_factor.alpha_lab.read_db_daily") as MRead, \ + patch("sanguo_factor.alpha_lib.AlphaDataset") as MDS, \ + patch("sanguo_factor.alpha_lab.get_factor") as MGet, \ + patch("sanguo_factor.alpha_lab.save_alpha_lab_data"): + import polars as pl + MRead.return_value = [MagicMock(vt_symbol="600000.SSE", datetime=__import__("datetime").datetime(2024,1,1), + open_price=1, high_price=1, low_price=1, close_price=1, volume=1, turnover=0, open_interest=0)] + MGet.return_value = {"expression": "ts_mean(close,5)"} + MDS.return_value.fetch_raw.return_value = pl.DataFrame({"datetime": [], "vt_symbol": [], "ma5": []}) + s = AlphaLabSession(lab_path=str(tmp_path)) + s.load_symbols(["600000"], "2024-01-01", "2024-06-30", cfg=MagicMock()) + df = s.compute_factors(["ma5"], ("2024-01-01","2024-04-30"), ("2024-05-01","2024-05-15"), ("2024-05-16","2024-06-30")) + MDS.return_value.add_feature.assert_called() + MDS.return_value.prepare_data.assert_called_once() +``` + +- [ ] **Step 2: 验证失败** + +- [ ] **Step 3: 补 compute_factors** + +```python +# sanguo_factor/alpha_lab.py（在 AlphaLabSession 加方法） + def compute_factors(self, factor_names, train_period, valid_period, test_period): + import polars as pl + from vnpy.alpha.dataset import AlphaDataset, Segment + from .registry import get_factor + from .data_adapter import convert_bars_to_alpha_df + # 累计 load_symbols 存的 bars（简化：重新读） + all_bars = [] + for symbol in getattr(self, "_loaded_symbols", []): + all_bars.extend(self._loaded_bars.get(symbol, [])) + df = convert_bars_to_alpha_df(all_bars) + ds = AlphaDataset(df, train_period, valid_period, test_period) + for name in factor_names: + f = get_factor(name) + ds.add_feature(name, f["expression"]) + ds.prepare_data(max_workers=1) + return ds.fetch_raw(Segment.TEST) +``` + +（配套 load_symbols 缓存 `_loaded_symbols`/`_loaded_bars`，见执行时补全。） + +- [ ] **Step 4: 验证通过 + commit** — `pytest tests/factor/test_alpha_lab.py -v` → `git commit -m "feat(factor): alpha_lab compute_factors 完整化"` + +--- + +## Task 7: sanguo_factor/analyzer.py tears pipeline 完整化 + +**Files:** +- Modify: `sanguo_factor/analyzer.py` +- Test: `tests/factor/test_analyzer.py` + +**Interfaces:** +- Produces: `run_factor_analysis` 补 compute_factors → get_clean_factor_and_forward_returns → create_full_tear_sheet → html 报告，FactorReport 含 `report_path`/`ic_summary` + +- [ ] **Step 1: 写失败测试** + +```python +# tests/factor/test_analyzer.py（追加） +from unittest.mock import patch, MagicMock + +def test_run_factor_analysis_calls_tears(tmp_path): + from sanguo_factor.analyzer import run_factor_analysis + with patch("sanguo_factor.analyzer.AlphaLabSession") as MS, \ + patch("sanguo_factor.analyzer.get_clean_factor_and_forward_returns") as MC, \ + patch("sanguo_factor.analyzer.create_full_tear_sheet") as MT: + import polars as pl + MS.return_value.compute_factors.return_value = pl.DataFrame({"datetime":[],"vt_symbol":[],"ma5":[]}) + MC.return_value = MagicMock() + report = run_factor_analysis(["600000"], ["ma5"], "2024-01-01", "2024-06-30", + cfg=MagicMock(), output_dir=str(tmp_path)) + assert report.factor_names == ["ma5"] + MC.assert_called_once(); MT.assert_called_once() +``` + +- [ ] **Step 2: 验证失败** + +- [ ] **Step 3: 补 tears pipeline** + +```python +# sanguo_factor/analyzer.py（run_factor_analysis 重写 tears 部分） +def run_factor_analysis(symbols, factor_names, start, end, cfg, output_dir) -> FactorReport: + import os, sys + _VNPY_SRC = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "vnpy_v4.4.0")) + if _VNPY_SRC not in sys.path: sys.path.insert(0, _VNPY_SRC) + from alphalens.utils import get_clean_factor_and_forward_returns + from alphalens.tears import create_full_tear_sheet + import matplotlib + matplotlib.use("Agg") + from .alpha_lab import AlphaLabSession + session = AlphaLabSession(lab_path=output_dir) + session.load_symbols(symbols, start, end, cfg) + factor_df = session.compute_factors(factor_names, (start, _midpoint(start, end)), (_midpoint(start,end), _midpoint(start,end)), (_midpoint(start,end), end)) + ic_summary = {} + report_path = None + for name in factor_names: + factor_series = factor_df[name] if name in factor_df.columns else factor_df.iloc[:, -1] + merged = get_clean_factor_and_forward_returns(factor_df.set_index(["datetime","vt_symbol")[name]], ...) # 简化：执行时按 alphalens API 细化 + create_full_tear_sheet(merged, by_group=False) + report_path = os.path.join(output_dir, f"{name}_tears.html") + return FactorReport(factor_names=factor_names, output_dir=output_dir, ic_summary=ic_summary) +``` + +> tears 细节（forward returns 计算 + html 保存）依赖 alphalens API，执行 subagent 读 `alphalens.tears.create_full_tear_sheet` 签名 + 容器验证。 + +- [ ] **Step 4: 验证通过 + commit** — `pytest tests/factor/test_analyzer.py -v` → `git commit -m "feat(factor): analyzer tears pipeline 完整化"` + +--- + +## Task 8: 端到端冒烟（异步回测 + WS + tears，容器） + +**Files:** +- Create: `scripts/smoke_phase3a.py` + +- [ ] **Step 1: 写冒烟脚本** + +```python +# scripts/smoke_phase3a.py +"""Phase 3a 端到端冒烟：异步 submit + WS 阶段 + JWT + tears（容器）。""" +import sys, asyncio +sys.path.insert(0, "/app"); sys.path.insert(0, "/app/vnpy_v4.4.0") + +async def main(): + print("=== Phase 3a Smoke ===") + from sanguo_api.app import create_app + from sanguo_api.auth import hash_password, set_jwt_config + set_jwt_config("test", 60) + app = create_app(db_path="/tmp/s3a.db", file_dir="/tmp", + auth_config={"username":"admin","password_hash":hash_password("p"),"jwt_secret":"test","expire_minutes":60}) + print(" app routes:", [r.path for r in app.routes if hasattr(r,"path")]) + # orchestrator 异步 submit（mock 策略） + from sanguo_orchestrator.runner import Orchestrator + orch = Orchestrator(db_path="/tmp/s3a.db") + print(" orchestrator pool executor:", type(orch.pool.executor).__name__) + print("=== Phase 3a Smoke DONE ===") + +asyncio.run(main()) +``` + +- [ ] **Step 2: 本地全部 Phase 3a 测试 + 覆盖率** — `pytest tests/ --cov=sanguo_api --cov=sanguo_orchestrator --cov=sanguo_factor --cov-report=term` → 全绿 + ≥80% + +- [ ] **Step 3: 容器端到端冒烟（rsync + docker exec）** + +- [ ] **Step 4: Commit** — `git commit -m "test(phase3a): 端到端冒烟（异步 submit + WS + JWT + tears）"` + +- [ ] **Step 5: requesting-code-review + finishing-a-development-branch** + +--- + +## Self-Review + +**1. Spec coverage**：异步 pool(T3) + runner async(T4) + WS 阶段(T2/T5) + JWT 单用户(T1/T5) + optimize/factor 路由(T5) + alpha tears 完整化(T6/T7) = design §0 五项全覆盖 ✅ +**2. Placeholder**：tears pipeline 的 forward returns 细节（T7）依赖 alphalens API，标注执行时细化；其余代码完整。 +**3. Type consistency**：`Orchestrator.async submit_*`（T4）在 routes（T5）调用一致；`manager.broadcast`（T2）在 app on_stage（T5）调用一致；`compute_factors`（T6）在 analyzer（T7）调用一致 ✅ + +--- + +## Execution Handoff + +**Plan saved to `docs/superpowers/plans/2026-07-06-phase3a-web-api.md`.** 推荐用 **Subagent-Driven** 执行（每 task fresh subagent + review）。