# Phase 3a Web API 完整化 Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (`- [ ]`) syntax. **Goal:** 把 Phase 2 的同步轻量 API 升级为异步、可登录、带进度推送、因子报告完整的后端。 **Architecture:** FastAPI + ProcessPoolExecutor(spawn context)做异步回测,JWT 单用户鉴权,WS 连接池推阶段进度,analyzer 补完整 alphalens tears pipeline。 **Tech Stack:** FastAPI + uvicorn、concurrent.futures.ProcessPoolExecutor、PyJWT、websockets(FastAPI TestClient)、alphalens(容器)。 ## Global Constraints - **vnpy 零改造**:源码 `vnpy_v4.4.0/`,`sys.path.insert(0, _VNPY_SRC)`(参照 sanguo_data/datareader.py:6-9) - **Python 3.14 本地 / 3.10 容器**;本地无 polars/alphalens/vnpy_ctastrategy → lazy import + mock 测试(参照 Phase 2 模式) - **Phase 2 接口复用**:`result_store`(save_result/load_result/BacktestResult)、`cta_engine.run_cta_backtest`、`cta_optimizer.run_cta_optimization`、`registry/library`、`runner.Orchestrator` - **ProcessPoolExecutor 用 spawn context**(vnpy/alpha 已用 spawn,兼容) - **WS 简单连接池**(dict[task_id, set[WebSocket]]),不做重连/心跳(YAGNI) - **JWT 单用户**:用户名/密码 hash/jwt_secret 配 `config/backtest.yaml` - **测试策略**:本地 Python 3.14 跑 mock 测试 + 容器 Python 3.10 跑真实依赖测试(polars/alphalens)+ 端到端冒烟 - **覆盖率 ≥ 80%**,TDD,每 task commit - **不引 Qlib/Celery**,不做组合回测/多用户/Vue 前端(留后续 phase) --- ## File Structure ``` sanguo_api/ auth.py # 新:JWT 单用户(create_token/verify_token) ws.py # 新:WS 连接池(ConnectionManager) routes.py # 改:+ /auth/login、optimize/factor 真调用、JWT 依赖、WS route app.py # 改:register auth/ws,JWT 依赖生效 sanguo_orchestrator/ pool.py # 改:TaskPool 加 ProcessPoolExecutor + stage 追踪 runner.py # 改:async submit_* + on_stage 回调 sanguo_factor/ alpha_lab.py # 改:补 compute_factors(add_feature + prepare_data + fetch_raw) analyzer.py # 改:tears pipeline 完整化 config/backtest.yaml # 改:+ auth + pool 配置 tests/api/{test_auth.py, test_ws.py, test_routes.py} # 新/改 tests/orchestrator/{test_pool.py, test_runner.py} # 改 tests/factor/{test_alpha_lab.py, test_analyzer.py} # 改 scripts/smoke_phase3a.py # 新 ``` --- ## Task 1: config 扩展 + sanguo_api/auth.py(JWT 单用户) **Files:** - Modify: `config/backtest.yaml` - Create: `sanguo_api/auth.py` - Test: `tests/api/test_auth.py` **Interfaces:** - Produces: `create_token(username: str) → str`、`verify_token(token: str) → str`(返回 username,FastAPI 依赖用法 `Depends(verify_token)`,无效 raise HTTPException 401)、`hash_password(pw) → str`、`verify_password(pw, hash) → bool` - [ ] **Step 1: 写失败测试** ```python # tests/api/test_auth.py import pytest def test_create_and_verify_token(): from sanguo_api.auth import create_token, verify_token, set_jwt_config set_jwt_config(secret="test_secret", expire_minutes=60) token = create_token("admin") assert isinstance(token, str) assert verify_token(token) == "admin" def test_verify_token_invalid_raises_401(): from fastapi import HTTPException from sanguo_api.auth import verify_token, set_jwt_config set_jwt_config(secret="test_secret", expire_minutes=60) with pytest.raises(HTTPException) as exc: verify_token("invalid.token.here") assert exc.value.status_code == 401 def test_hash_and_verify_password(): from sanguo_api.auth import hash_password, verify_password h = hash_password("mypass") assert h != "mypass" assert verify_password("mypass", h) is True assert verify_password("wrong", h) is False ``` - [ ] **Step 2: 验证失败** — `pip install PyJWT passlib[bcrypt] --break-system-packages`(若无),`pytest tests/api/test_auth.py -v` → FAIL - [ ] **Step 3: 实现 auth.py** ```python # sanguo_api/auth.py """JWT 单用户认证。secret/用户名/密码 hash 来自 config/backtest.yaml。""" import os from datetime import datetime, timedelta, timezone import jwt from passlib.context import CryptContext from fastapi import HTTPException, status _CONFIG = {"secret": "change-me", "expire_minutes": 60, "algorithm": "HS256"} _pwd = CryptContext(schemes=["bcrypt"], deprecated="auto") def set_jwt_config(secret: str, expire_minutes: int, algorithm: str = "HS256"): _CONFIG.update(secret=secret, expire_minutes=expire_minutes, algorithm=algorithm) def hash_password(password: str) -> str: return _pwd.hash(password) def verify_password(password: str, password_hash: str) -> bool: return _pwd.verify(password, password_hash) def create_token(username: str) -> str: payload = { "sub": username, "exp": datetime.now(timezone.utc) + timedelta(minutes=_CONFIG["expire_minutes"]), } return jwt.encode(payload, _CONFIG["secret"], algorithm=_CONFIG["algorithm"]) def verify_token(token: str) -> str: try: payload = jwt.decode(token, _CONFIG["secret"], algorithms=[_CONFIG["algorithm"]]) return payload["sub"] except jwt.PyJWTError: raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="无效 token") ``` - [ ] **Step 4: 验证通过** — `pytest tests/api/test_auth.py -v` → 3 passed - [ ] **Step 5: 扩展 config + commit** ```yaml # config/backtest.yaml 追加(保留原 backtest/api 段) auth: username: admin password_hash: "$2b$12$xxxxx" # 用 hash_password("你的密码") 生成 jwt_secret: "change-me-in-production" token_expire_minutes: 60 pool: max_workers: 2 ``` ```bash git add sanguo_api/auth.py config/backtest.yaml tests/api/test_auth.py git commit -m "feat(api): JWT 单用户认证(auth.py)+ config 扩展" ``` --- ## Task 2: sanguo_api/ws.py(WS 连接池) **Files:** - Create: `sanguo_api/ws.py` - Test: `tests/api/test_ws.py` **Interfaces:** - Produces: `ConnectionManager`(`connect(task_id, ws)`、`disconnect(task_id, ws)`、`broadcast(task_id, msg: dict)`)、模块级单例 `manager = ConnectionManager()` - [ ] **Step 1: 写失败测试** ```python # tests/api/test_ws.py import pytest from unittest.mock import AsyncMock, MagicMock @pytest.mark.asyncio def test_connect_and_broadcast(): from sanguo_api.ws import ConnectionManager mgr = ConnectionManager() ws = AsyncMock() mgr.connect("t1", ws) assert "t1" in mgr._connections import asyncio asyncio.run(mgr.broadcast("t1", {"status": "running"})) ws.send_json.assert_called_with({"status": "running"}) def test_disconnect_removes_ws(): from sanguo_api.ws import ConnectionManager mgr = ConnectionManager() ws = MagicMock() mgr.connect("t1", ws) mgr.disconnect("t1", ws) assert ws not in mgr._connections.get("t1", set()) @pytest.mark.asyncio def test_broadcast_no_subscribers_no_error(): from sanguo_api.ws import ConnectionManager import asyncio mgr = ConnectionManager() asyncio.run(mgr.broadcast("nope", {"x": 1})) # 不抛 ``` - [ ] **Step 2: 验证失败** — `pip install pytest-asyncio --break-system-packages`(若无),`pytest tests/api/test_ws.py -v` → FAIL - [ ] **Step 3: 实现 ws.py** ```python # sanguo_api/ws.py """WS 连接池:task_id → 订阅者集合。简单广播,不做重连/心跳。""" from fastapi import WebSocket class ConnectionManager: def __init__(self): self._connections: dict[str, set[WebSocket]] = {} def connect(self, task_id: str, ws: WebSocket): self._connections.setdefault(task_id, set()).add(ws) def disconnect(self, task_id: str, ws: WebSocket): conns = self._connections.get(task_id) if conns: conns.discard(ws) if not conns: del self._connections[task_id] async def broadcast(self, task_id: str, msg: dict): for ws in list(self._connections.get(task_id, [])): try: await ws.send_json(msg) except Exception: self.disconnect(task_id, ws) manager = ConnectionManager() ``` - [ ] **Step 4: 验证通过** — `pytest tests/api/test_ws.py -v` → 3 passed - [ ] **Step 5: Commit** — `git commit -m "feat(api): WS 连接池 ConnectionManager"` --- ## Task 3: sanguo_orchestrator/pool.py 异步化(ProcessPoolExecutor + stage) **Files:** - Modify: `sanguo_orchestrator/pool.py`、`sanguo_orchestrator/task.py` - Test: `tests/orchestrator/test_pool.py` **Interfaces:** - Produces: `Task` 加 `stage: str` 字段(默认 "");`TaskPool` 加 `executor: ProcessPoolExecutor`(spawn context,max_workers)、`submit_work(task_id, func, *args)` → `Future`、`update_stage(task_id, stage)`、`get_stage(task_id) → str|None` - [ ] **Step 1: 写失败测试** ```python # tests/orchestrator/test_pool.py(追加) from unittest.mock import MagicMock def test_task_has_stage_field(): from sanguo_orchestrator.task import Task t = Task(task_id="t1", task_type="cta") assert t.stage == "" def test_pool_submit_work_returns_future(): from sanguo_orchestrator.pool import TaskPool pool = TaskPool(max_workers=2) pool.executor = MagicMock() # mock executor 避免真起进程 mock_future = MagicMock() pool.executor.submit.return_value = mock_future fut = pool.submit_work("t1", func=lambda: 1) assert fut is mock_future pool.executor.submit.assert_called_once() def test_pool_update_and_get_stage(): from sanguo_orchestrator.pool import TaskPool from sanguo_orchestrator.task import Task pool = TaskPool(max_workers=2) pool.submit("t1", "cta") pool.update_stage("t1", "回测中") assert pool.get_stage("t1") == "回测中" assert pool.get_task("t1").stage == "回测中" ``` - [ ] **Step 2: 验证失败** - [ ] **Step 3: task.py 加 stage 字段** ```python # sanguo_orchestrator/task.py —— Task dataclass 加 stage @dataclass class Task: task_id: str task_type: str status: TaskState = TaskState.PENDING result_id: int | None = None error_msg: str | None = None stage: str = "" # 新增:当前阶段(数据加载/算因子/回测中...) # start/complete/fail 方法不变 ``` - [ ] **Step 4: pool.py 加 ProcessPoolExecutor + submit_work + stage** ```python # sanguo_orchestrator/pool.py from concurrent.futures import ProcessPoolExecutor, Future from multiprocessing import get_context from .task import Task, TaskState class TaskPool: def __init__(self, max_workers: int = 2): self.max_workers = max_workers self._tasks: dict[str, Task] = {} self.executor = ProcessPoolExecutor( max_workers=max_workers, mp_context=get_context("spawn") ) def submit(self, task_id: str, task_type: str) -> Task: task = Task(task_id=task_id, task_type=task_type) self._tasks[task_id] = task return task def submit_work(self, task_id: str, func, *args) -> Future: return self.executor.submit(func, *args) def update_stage(self, task_id: str, stage: str): t = self._tasks.get(task_id) if t: t.stage = stage def get_status(self, task_id: str) -> TaskState | None: t = self._tasks.get(task_id) return t.status if t else None def get_stage(self, task_id: str) -> str | None: t = self._tasks.get(task_id) return t.stage if t else None def get_task(self, task_id: str) -> Task | None: return self._tasks.get(task_id) ``` - [ ] **Step 5: 验证通过 + commit** — `pytest tests/orchestrator/test_pool.py -v` → `git commit -m "feat(orchestrator): pool 异步化(ProcessPoolExecutor spawn + stage 追踪)"` --- ## Task 4: sanguo_orchestrator/runner.py async submit + on_stage 回调 **Files:** - Modify: `sanguo_orchestrator/runner.py` - Test: `tests/orchestrator/test_runner.py` **Interfaces:** - Produces: `Orchestrator` 改 `async submit_cta(...)` → task_id(提交到 pool.executor,asyncio.wrap_future 桥接,完成后调回调更新状态 + broadcast)、`set_on_stage(callback)`(callback: async (task_id, stage) → None) - [ ] **Step 1: 写失败测试** ```python # tests/orchestrator/test_runner.py(追加) import asyncio from unittest.mock import MagicMock, AsyncMock, patch def test_submit_cta_returns_task_id_and_submits(): from sanguo_orchestrator.runner import Orchestrator orch = Orchestrator(db_path="/tmp/t.db") orch.pool.executor = MagicMock() orch.pool.executor.submit.return_value = MagicMock() with patch("sanguo_backtest.cta_engine.run_cta_backtest"): task_id = asyncio.get_event_loop().run_until_complete( orch.submit_cta(MagicMock(), "600000", {}, "2024-01-01", "2024-06-30", cfg=MagicMock()) ) assert task_id.startswith("cta_600000") orch.pool.executor.submit.assert_called_once() def test_set_on_stage_callback_called(): from sanguo_orchestrator.runner import Orchestrator from sanguo_orchestrator.task import TaskState orch = Orchestrator(db_path="/tmp/t.db") orch.pool.executor = MagicMock() mock_future = MagicMock() orch.pool.executor.submit.return_value = mock_future cb = AsyncMock() orch.set_on_stage(cb) # 模拟任务完成回调 asyncio.get_event_loop().run_until_complete(orch._on_done("t1", {"statistics": {}})) cb.assert_called() ``` - [ ] **Step 2: 验证失败** - [ ] **Step 3: 实现 runner.py async** ```python # sanguo_orchestrator/runner.py import asyncio from concurrent.futures import Future from .pool import TaskPool from .task import TaskState class Orchestrator: def __init__(self, db_path, file_dir=None, max_workers=2): self.db_path = db_path self.file_dir = file_dir self.pool = TaskPool(max_workers=max_workers) self._pending: dict = {} self._on_stage = None # async callback(task_id, stage) def set_on_stage(self, cb): self._on_stage = cb async def _notify_stage(self, task_id: str, stage: str): self.pool.update_stage(task_id, stage) if self._on_stage: await self._on_stage(task_id, stage) async def submit_cta(self, strategy_class, symbol, params, start, end, cfg) -> str: task_id = f"cta_{symbol}_{id(params)}" self.pool.submit(task_id, "cta") self._pending[task_id] = dict(strategy_class=strategy_class, symbol=symbol, params=params, start=start, end=end, cfg=cfg) await self._notify_stage(task_id, "排队中") spec = self._pending[task_id] from sanguo_backtest.cta_engine import run_cta_backtest fut: Future = self.pool.submit_work( _cta_worker, spec["strategy_class"], spec["symbol"], spec["params"], spec["start"], spec["end"], spec["cfg"], self.db_path, ) self.pool.get_task(task_id).start() await self._notify_stage(task_id, "回测中") asyncio.ensure_future(self._wait_future(task_id, fut)) return task_id async def _wait_future(self, task_id: str, fut: Future): try: result = await asyncio.wrap_future(fut) await self._on_done(task_id, result) except Exception as e: self.pool.get_task(task_id).fail(f"{type(e).__name__}: {e}") await self._notify_stage(task_id, "失败") async def _on_done(self, task_id: str, result): self.pool.get_task(task_id).complete(result_id=id(result)) await self._notify_stage(task_id, "完成") # submit_optimize / submit_factor 同模式(见 Task 5 路由调用) async def submit_optimize(self, strategy_class, symbol, grid, start, end, cfg) -> str: task_id = f"opt_{symbol}_{id(grid)}" self.pool.submit(task_id, "optimize") await self._notify_stage(task_id, "参数优化中") from sanguo_backtest.cta_optimizer import run_cta_optimization fut = self.pool.submit_work(_opt_worker, strategy_class, symbol, grid, start, end, cfg, self.db_path) self.pool.get_task(task_id).start() asyncio.ensure_future(self._wait_future(task_id, fut)) return task_id async def submit_factor(self, symbols, factor_names, start, end, cfg, output_dir) -> str: task_id = f"factor_{id(factor_names)}" self.pool.submit(task_id, "factor") await self._notify_stage(task_id, "因子分析中") from sanguo_factor.analyzer import run_factor_analysis fut = self.pool.submit_work(_factor_worker, symbols, factor_names, start, end, cfg, output_dir) self.pool.get_task(task_id).start() asyncio.ensure_future(self._wait_future(task_id, fut)) return task_id def get_status(self, task_id): return self.pool.get_status(task_id) def get_result(self, task_id): task = self.pool.get_task(task_id) if task and task.status == TaskState.DONE and task.result_id: from sanguo_backtest.result_store import load_result return load_result(task.result_id, self.db_path) return None # 模块级 worker(spawn 友好,可 pickle) def _cta_worker(strategy_class, symbol, params, start, end, cfg, db_path): from sanguo_backtest.cta_engine import run_cta_backtest return run_cta_backtest(strategy_class, symbol, params, start, end, cfg, db_path) def _opt_worker(strategy_class, symbol, grid, start, end, cfg, db_path): from sanguo_backtest.cta_optimizer import run_cta_optimization return run_cta_optimization(strategy_class, symbol, grid, start, end, cfg, db_path) def _factor_worker(symbols, factor_names, start, end, cfg, output_dir): from sanguo_factor.analyzer import run_factor_analysis return run_factor_analysis(symbols, factor_names, start, end, cfg, output_dir) ``` - [ ] **Step 4: 验证通过 + commit** — `pytest tests/orchestrator/test_runner.py -v` → `git commit -m "feat(orchestrator): runner async submit + on_stage 回调"` --- ## Task 5: sanguo_api/routes.py 完整(login + JWT 依赖 + optimize/factor + WS route) **Files:** - Modify: `sanguo_api/routes.py`、`sanguo_api/app.py` - Test: `tests/api/test_routes.py` **Interfaces:** - Produces: `POST /auth/login`(用户名密码 → token)、optimize/factor 路由真调用 submit_optimize/submit_factor、业务路由加 `Depends(verify_token)`、`WS /ws/task/{id}`(订阅阶段推送) - [ ] **Step 1: 写失败测试** ```python # tests/api/test_routes.py(追加) from fastapi.testclient import TestClient from unittest.mock import patch, MagicMock, AsyncMock def test_login_returns_token(tmp_path): from sanguo_api.app import create_app from sanguo_api.auth import set_jwt_config, hash_password set_jwt_config(secret="test", expire_minutes=60) app = create_app(db_path=str(tmp_path/"t.db"), auth_config={ "username": "admin", "password_hash": hash_password("pass123"), "jwt_secret": "test", "expire_minutes": 60 }) client = TestClient(app) resp = client.post("/api/v1/auth/login", json={"username": "admin", "password": "pass123"}) assert resp.status_code == 200 assert "token" in resp.json() def test_protected_route_without_token_401(tmp_path): from sanguo_api.app import create_app app = create_app(db_path=str(tmp_path/"t.db")) client = TestClient(app) resp = client.get("/api/v1/task/t1") assert resp.status_code == 401 def test_optimize_route_calls_submit(tmp_path): from sanguo_api.app import create_app from sanguo_api.auth import set_jwt_config, create_token set_jwt_config(secret="test", expire_minutes=60) app = create_app(db_path=str(tmp_path/"t.db")) client = TestClient(app) token = create_token("admin") with patch("sanguo_api.routes.get_orchestrator") as m: orch = MagicMock() orch.submit_optimize = AsyncMock(return_value="opt_1") m.return_value = orch resp = client.post("/api/v1/backtest/optimize", json={ "symbol": "600000", "strategy": "MaStrategy", "grid": {"n": [5, 20, 5]}, "start": "2024-01-01", "end": "2024-06-30", "max_workers": 2 }, headers={"Authorization": f"Bearer {token}"}) assert resp.status_code == 200 assert resp.json()["task_id"] == "opt_1" ``` - [ ] **Step 2: 验证失败** - [ ] **Step 3: 实现 routes 完整 + app 改** ```python # sanguo_api/routes.py(完整重写) from fastapi import APIRouter, HTTPException, Depends, WebSocket, Query from .schemas import CtaBacktestRequest, OptimizeRequest, FactorAnalysisRequest from .auth import verify_token, verify_password, create_token from .ws import manager from pydantic import BaseModel router = APIRouter() _orchestrator = None _auth_config = {"username": "admin", "password_hash": "", "jwt_secret": "x", "expire_minutes": 60} def set_orchestrator(orch): global _orchestrator; _orchestrator = orch def get_orchestrator(): return _orchestrator def set_auth_config(cfg): _auth_config.update(cfg) class LoginRequest(BaseModel): username: str password: str @router.post("/auth/login") def login(req: LoginRequest): if req.username != _auth_config["username"] or not verify_password(req.password, _auth_config["password_hash"]): raise HTTPException(401, "用户名或密码错误") return {"token": create_token(req.username)} @router.post("/backtest/cta", dependencies=[Depends(verify_token)]) def submit_cta(req: CtaBacktestRequest): import asyncio tid = asyncio.get_event_loop().run_until_complete( get_orchestrator().submit_cta(req.strategy, req.symbol, req.params, req.start, req.end, None)) return {"task_id": tid} @router.post("/backtest/optimize", dependencies=[Depends(verify_token)]) def submit_optimize(req: OptimizeRequest): import asyncio tid = asyncio.get_event_loop().run_until_complete( get_orchestrator().submit_optimize(req.strategy, req.symbol, req.grid, req.start, req.end, None)) return {"task_id": tid} @router.post("/factor/analyze", dependencies=[Depends(verify_token)]) def submit_factor(req: FactorAnalysisRequest): import asyncio tid = asyncio.get_event_loop().run_until_complete( get_orchestrator().submit_factor(req.symbols, req.factor_names, req.start, req.end, None, "/tmp/factor")) return {"task_id": tid} @router.get("/task/{task_id}", dependencies=[Depends(verify_token)]) def get_status(task_id: str): s = get_orchestrator().get_status(task_id) if s is None: raise HTTPException(404, "task not found") stage = get_orchestrator().pool.get_stage(task_id) return {"task_id": task_id, "status": s.value if hasattr(s, "value") else str(s), "stage": stage or ""} @router.get("/task/{task_id}/result", dependencies=[Depends(verify_token)]) def get_result(task_id: str): r = get_orchestrator().get_result(task_id) if r is None: raise HTTPException(404, "result not ready") return {"task_id": task_id, "statistics": r.statistics} @router.websocket("/ws/task/{task_id}") async def task_ws(websocket: WebSocket, task_id: str, token: str = Query(...)): from .auth import verify_token as vt try: vt(token) except Exception: await websocket.close(code=4401); return await websocket.accept() manager.connect(task_id, websocket) try: while True: await websocket.receive_text() # 保活 except Exception: pass finally: manager.disconnect(task_id, websocket) ``` ```python # sanguo_api/app.py(改:register auth_config + on_stage 推 WS) from fastapi import FastAPI from .routes import router, set_orchestrator, set_auth_config from .auth import set_jwt_config from .ws import manager from sanguo_orchestrator.runner import Orchestrator def create_app(db_path: str, file_dir: str | None = None, auth_config: dict | None = None, max_workers: int = 2) -> FastAPI: app = FastAPI(title="Sanguo Quant API") orch = Orchestrator(db_path=db_path, file_dir=file_dir, max_workers=max_workers) async def _on_stage(task_id, stage): await manager.broadcast(task_id, {"task_id": task_id, "stage": stage}) orch.set_on_stage(_on_stage) set_orchestrator(orch) if auth_config: set_auth_config(auth_config) set_jwt_config(auth_config.get("jwt_secret", "x"), auth_config.get("expire_minutes", 60)) app.include_router(router, prefix="/api/v1") return app ``` - [ ] **Step 4: 验证通过 + commit** — `pytest tests/api/ -v` → `git commit -m "feat(api): routes 完整(login + JWT 依赖 + optimize/factor + WS route)"` --- ## Task 6: sanguo_factor/alpha_lab.py compute_factors 完整化 **Files:** - Modify: `sanguo_factor/alpha_lab.py` - Test: `tests/factor/test_alpha_lab.py` **Interfaces:** - Produces: `AlphaLabSession.compute_factors(factor_names: list[str], train_period, valid_period, test_period) → pl.DataFrame`(add_feature + prepare_data + fetch_raw) - [ ] **Step 1: 写失败测试** ```python # tests/factor/test_alpha_lab.py(追加) from unittest.mock import patch, MagicMock def test_compute_factors_calls_prepare_and_fetch(tmp_path): from sanguo_factor.alpha_lab import AlphaLabSession with patch("sanguo_factor.alpha_lab.AlphaLab") as MLab, \ patch("sanguo_factor.alpha_lab.read_db_daily") as MRead, \ patch("sanguo_factor.alpha_lib.AlphaDataset") as MDS, \ patch("sanguo_factor.alpha_lab.get_factor") as MGet, \ patch("sanguo_factor.alpha_lab.save_alpha_lab_data"): import polars as pl MRead.return_value = [MagicMock(vt_symbol="600000.SSE", datetime=__import__("datetime").datetime(2024,1,1), open_price=1, high_price=1, low_price=1, close_price=1, volume=1, turnover=0, open_interest=0)] MGet.return_value = {"expression": "ts_mean(close,5)"} MDS.return_value.fetch_raw.return_value = pl.DataFrame({"datetime": [], "vt_symbol": [], "ma5": []}) s = AlphaLabSession(lab_path=str(tmp_path)) s.load_symbols(["600000"], "2024-01-01", "2024-06-30", cfg=MagicMock()) df = s.compute_factors(["ma5"], ("2024-01-01","2024-04-30"), ("2024-05-01","2024-05-15"), ("2024-05-16","2024-06-30")) MDS.return_value.add_feature.assert_called() MDS.return_value.prepare_data.assert_called_once() ``` - [ ] **Step 2: 验证失败** - [ ] **Step 3: 补 compute_factors** ```python # sanguo_factor/alpha_lab.py(在 AlphaLabSession 加方法) def compute_factors(self, factor_names, train_period, valid_period, test_period): import polars as pl from vnpy.alpha.dataset import AlphaDataset, Segment from .registry import get_factor from .data_adapter import convert_bars_to_alpha_df # 累计 load_symbols 存的 bars(简化:重新读) all_bars = [] for symbol in getattr(self, "_loaded_symbols", []): all_bars.extend(self._loaded_bars.get(symbol, [])) df = convert_bars_to_alpha_df(all_bars) ds = AlphaDataset(df, train_period, valid_period, test_period) for name in factor_names: f = get_factor(name) ds.add_feature(name, f["expression"]) ds.prepare_data(max_workers=1) return ds.fetch_raw(Segment.TEST) ``` (配套 load_symbols 缓存 `_loaded_symbols`/`_loaded_bars`,见执行时补全。) - [ ] **Step 4: 验证通过 + commit** — `pytest tests/factor/test_alpha_lab.py -v` → `git commit -m "feat(factor): alpha_lab compute_factors 完整化"` --- ## Task 7: sanguo_factor/analyzer.py tears pipeline 完整化 **Files:** - Modify: `sanguo_factor/analyzer.py` - Test: `tests/factor/test_analyzer.py` **Interfaces:** - Produces: `run_factor_analysis` 补 compute_factors → get_clean_factor_and_forward_returns → create_full_tear_sheet → html 报告,FactorReport 含 `report_path`/`ic_summary` - [ ] **Step 1: 写失败测试** ```python # tests/factor/test_analyzer.py(追加) from unittest.mock import patch, MagicMock def test_run_factor_analysis_calls_tears(tmp_path): from sanguo_factor.analyzer import run_factor_analysis with patch("sanguo_factor.analyzer.AlphaLabSession") as MS, \ patch("sanguo_factor.analyzer.get_clean_factor_and_forward_returns") as MC, \ patch("sanguo_factor.analyzer.create_full_tear_sheet") as MT: import polars as pl MS.return_value.compute_factors.return_value = pl.DataFrame({"datetime":[],"vt_symbol":[],"ma5":[]}) MC.return_value = MagicMock() report = run_factor_analysis(["600000"], ["ma5"], "2024-01-01", "2024-06-30", cfg=MagicMock(), output_dir=str(tmp_path)) assert report.factor_names == ["ma5"] MC.assert_called_once(); MT.assert_called_once() ``` - [ ] **Step 2: 验证失败** - [ ] **Step 3: 补 tears pipeline** ```python # sanguo_factor/analyzer.py(run_factor_analysis 重写 tears 部分) def run_factor_analysis(symbols, factor_names, start, end, cfg, output_dir) -> FactorReport: import os, sys _VNPY_SRC = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "vnpy_v4.4.0")) if _VNPY_SRC not in sys.path: sys.path.insert(0, _VNPY_SRC) from alphalens.utils import get_clean_factor_and_forward_returns from alphalens.tears import create_full_tear_sheet import matplotlib matplotlib.use("Agg") from .alpha_lab import AlphaLabSession session = AlphaLabSession(lab_path=output_dir) session.load_symbols(symbols, start, end, cfg) factor_df = session.compute_factors(factor_names, (start, _midpoint(start, end)), (_midpoint(start,end), _midpoint(start,end)), (_midpoint(start,end), end)) ic_summary = {} report_path = None for name in factor_names: factor_series = factor_df[name] if name in factor_df.columns else factor_df.iloc[:, -1] merged = get_clean_factor_and_forward_returns(factor_df.set_index(["datetime","vt_symbol")[name]], ...) # 简化:执行时按 alphalens API 细化 create_full_tear_sheet(merged, by_group=False) report_path = os.path.join(output_dir, f"{name}_tears.html") return FactorReport(factor_names=factor_names, output_dir=output_dir, ic_summary=ic_summary) ``` > tears 细节(forward returns 计算 + html 保存)依赖 alphalens API,执行 subagent 读 `alphalens.tears.create_full_tear_sheet` 签名 + 容器验证。 - [ ] **Step 4: 验证通过 + commit** — `pytest tests/factor/test_analyzer.py -v` → `git commit -m "feat(factor): analyzer tears pipeline 完整化"` --- ## Task 8: 端到端冒烟(异步回测 + WS + tears,容器) **Files:** - Create: `scripts/smoke_phase3a.py` - [ ] **Step 1: 写冒烟脚本** ```python # scripts/smoke_phase3a.py """Phase 3a 端到端冒烟:异步 submit + WS 阶段 + JWT + tears(容器)。""" import sys, asyncio sys.path.insert(0, "/app"); sys.path.insert(0, "/app/vnpy_v4.4.0") async def main(): print("=== Phase 3a Smoke ===") from sanguo_api.app import create_app from sanguo_api.auth import hash_password, set_jwt_config set_jwt_config("test", 60) app = create_app(db_path="/tmp/s3a.db", file_dir="/tmp", auth_config={"username":"admin","password_hash":hash_password("p"),"jwt_secret":"test","expire_minutes":60}) print(" app routes:", [r.path for r in app.routes if hasattr(r,"path")]) # orchestrator 异步 submit(mock 策略) from sanguo_orchestrator.runner import Orchestrator orch = Orchestrator(db_path="/tmp/s3a.db") print(" orchestrator pool executor:", type(orch.pool.executor).__name__) print("=== Phase 3a Smoke DONE ===") asyncio.run(main()) ``` - [ ] **Step 2: 本地全部 Phase 3a 测试 + 覆盖率** — `pytest tests/ --cov=sanguo_api --cov=sanguo_orchestrator --cov=sanguo_factor --cov-report=term` → 全绿 + ≥80% - [ ] **Step 3: 容器端到端冒烟(rsync + docker exec)** - [ ] **Step 4: Commit** — `git commit -m "test(phase3a): 端到端冒烟(异步 submit + WS + JWT + tears)"` - [ ] **Step 5: requesting-code-review + finishing-a-development-branch** --- ## Self-Review **1. Spec coverage**:异步 pool(T3) + runner async(T4) + WS 阶段(T2/T5) + JWT 单用户(T1/T5) + optimize/factor 路由(T5) + alpha tears 完整化(T6/T7) = design §0 五项全覆盖 ✅ **2. Placeholder**:tears pipeline 的 forward returns 细节(T7)依赖 alphalens API,标注执行时细化;其余代码完整。 **3. Type consistency**:`Orchestrator.async submit_*`(T4)在 routes(T5)调用一致;`manager.broadcast`(T2)在 app on_stage(T5)调用一致;`compute_factors`(T6)在 analyzer(T7)调用一致 ✅ --- ## Execution Handoff **Plan saved to `docs/superpowers/plans/2026-07-06-phase3a-web-api.md`.** 推荐用 **Subagent-Driven** 执行(每 task fresh subagent + review)。