Files
sanguo_vnpy_v2/docs/superpowers/plans/2026-07-06-phase3a-web-api.md
T

32 KiB
Raw Blame History

Phase 3a Web API 完整化 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (- [ ]) syntax.

Goal: 把 Phase 2 的同步轻量 API 升级为异步、可登录、带进度推送、因子报告完整的后端。

Architecture: FastAPI + ProcessPoolExecutorspawn context)做异步回测,JWT 单用户鉴权,WS 连接池推阶段进度,analyzer 补完整 alphalens tears pipeline。

Tech Stack: FastAPI + uvicorn、concurrent.futures.ProcessPoolExecutor、PyJWT、websocketsFastAPI TestClient)、alphalens(容器)。

Global Constraints

  • vnpy 零改造:源码 vnpy_v4.4.0/sys.path.insert(0, _VNPY_SRC)(参照 sanguo_data/datareader.py:6-9
  • Python 3.14 本地 / 3.10 容器;本地无 polars/alphalens/vnpy_ctastrategy → lazy import + mock 测试(参照 Phase 2 模式)
  • Phase 2 接口复用result_storesave_result/load_result/BacktestResult)、cta_engine.run_cta_backtestcta_optimizer.run_cta_optimizationregistry/libraryrunner.Orchestrator
  • ProcessPoolExecutor 用 spawn contextvnpy/alpha 已用 spawn,兼容)
  • WS 简单连接池dict[task_id, set[WebSocket]]),不做重连/心跳(YAGNI
  • JWT 单用户:用户名/密码 hash/jwt_secret 配 config/backtest.yaml
  • 测试策略:本地 Python 3.14 跑 mock 测试 + 容器 Python 3.10 跑真实依赖测试(polars/alphalens+ 端到端冒烟
  • 覆盖率 ≥ 80%TDD,每 task commit
  • 不引 Qlib/Celery,不做组合回测/多用户/Vue 前端(留后续 phase)

File Structure

sanguo_api/
  auth.py          # 新:JWT 单用户(create_token/verify_token
  ws.py            # 新:WS 连接池(ConnectionManager
  routes.py        # 改:+ /auth/login、optimize/factor 真调用、JWT 依赖、WS route
  app.py           # 改:register auth/wsJWT 依赖生效
sanguo_orchestrator/
  pool.py          # 改:TaskPool 加 ProcessPoolExecutor + stage 追踪
  runner.py        # 改:async submit_* + on_stage 回调
sanguo_factor/
  alpha_lab.py     # 改:补 compute_factorsadd_feature + prepare_data + fetch_raw
  analyzer.py      # 改:tears pipeline 完整化
config/backtest.yaml  # 改:+ auth + pool 配置
tests/api/{test_auth.py, test_ws.py, test_routes.py}  # 新/改
tests/orchestrator/{test_pool.py, test_runner.py}  # 改
tests/factor/{test_alpha_lab.py, test_analyzer.py}  # 改
scripts/smoke_phase3a.py  # 新

Task 1: config 扩展 + sanguo_api/auth.pyJWT 单用户)

Files:

  • Modify: config/backtest.yaml
  • Create: sanguo_api/auth.py
  • Test: tests/api/test_auth.py

Interfaces:

  • Produces: create_token(username: str) → strverify_token(token: str) → str(返回 usernameFastAPI 依赖用法 Depends(verify_token),无效 raise HTTPException 401)、hash_password(pw) → strverify_password(pw, hash) → bool

  • Step 1: 写失败测试

# tests/api/test_auth.py
import pytest

def test_create_and_verify_token():
    from sanguo_api.auth import create_token, verify_token, set_jwt_config
    set_jwt_config(secret="test_secret", expire_minutes=60)
    token = create_token("admin")
    assert isinstance(token, str)
    assert verify_token(token) == "admin"

def test_verify_token_invalid_raises_401():
    from fastapi import HTTPException
    from sanguo_api.auth import verify_token, set_jwt_config
    set_jwt_config(secret="test_secret", expire_minutes=60)
    with pytest.raises(HTTPException) as exc:
        verify_token("invalid.token.here")
    assert exc.value.status_code == 401

def test_hash_and_verify_password():
    from sanguo_api.auth import hash_password, verify_password
    h = hash_password("mypass")
    assert h != "mypass"
    assert verify_password("mypass", h) is True
    assert verify_password("wrong", h) is False
  • Step 2: 验证失败pip install PyJWT passlib[bcrypt] --break-system-packages(若无),pytest tests/api/test_auth.py -v → FAIL

  • Step 3: 实现 auth.py

# sanguo_api/auth.py
"""JWT 单用户认证。secret/用户名/密码 hash 来自 config/backtest.yaml。"""
import os
from datetime import datetime, timedelta, timezone
import jwt
from passlib.context import CryptContext
from fastapi import HTTPException, status

_CONFIG = {"secret": "change-me", "expire_minutes": 60, "algorithm": "HS256"}
_pwd = CryptContext(schemes=["bcrypt"], deprecated="auto")


def set_jwt_config(secret: str, expire_minutes: int, algorithm: str = "HS256"):
    _CONFIG.update(secret=secret, expire_minutes=expire_minutes, algorithm=algorithm)


def hash_password(password: str) -> str:
    return _pwd.hash(password)


def verify_password(password: str, password_hash: str) -> bool:
    return _pwd.verify(password, password_hash)


def create_token(username: str) -> str:
    payload = {
        "sub": username,
        "exp": datetime.now(timezone.utc) + timedelta(minutes=_CONFIG["expire_minutes"]),
    }
    return jwt.encode(payload, _CONFIG["secret"], algorithm=_CONFIG["algorithm"])


def verify_token(token: str) -> str:
    try:
        payload = jwt.decode(token, _CONFIG["secret"], algorithms=[_CONFIG["algorithm"]])
        return payload["sub"]
    except jwt.PyJWTError:
        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="无效 token")
  • Step 4: 验证通过pytest tests/api/test_auth.py -v → 3 passed

  • Step 5: 扩展 config + commit

# config/backtest.yaml 追加(保留原 backtest/api 段)
auth:
  username: admin
  password_hash: "$2b$12$xxxxx"   # 用 hash_password("你的密码") 生成
  jwt_secret: "change-me-in-production"
  token_expire_minutes: 60
pool:
  max_workers: 2
git add sanguo_api/auth.py config/backtest.yaml tests/api/test_auth.py
git commit -m "feat(api): JWT 单用户认证(auth.py+ config 扩展"

Task 2: sanguo_api/ws.pyWS 连接池)

Files:

  • Create: sanguo_api/ws.py
  • Test: tests/api/test_ws.py

Interfaces:

  • Produces: ConnectionManagerconnect(task_id, ws)disconnect(task_id, ws)broadcast(task_id, msg: dict))、模块级单例 manager = ConnectionManager()

  • Step 1: 写失败测试

# tests/api/test_ws.py
import pytest
from unittest.mock import AsyncMock, MagicMock

@pytest.mark.asyncio
def test_connect_and_broadcast():
    from sanguo_api.ws import ConnectionManager
    mgr = ConnectionManager()
    ws = AsyncMock()
    mgr.connect("t1", ws)
    assert "t1" in mgr._connections
    import asyncio
    asyncio.run(mgr.broadcast("t1", {"status": "running"}))
    ws.send_json.assert_called_with({"status": "running"})

def test_disconnect_removes_ws():
    from sanguo_api.ws import ConnectionManager
    mgr = ConnectionManager()
    ws = MagicMock()
    mgr.connect("t1", ws)
    mgr.disconnect("t1", ws)
    assert ws not in mgr._connections.get("t1", set())

@pytest.mark.asyncio
def test_broadcast_no_subscribers_no_error():
    from sanguo_api.ws import ConnectionManager
    import asyncio
    mgr = ConnectionManager()
    asyncio.run(mgr.broadcast("nope", {"x": 1}))  # 不抛
  • Step 2: 验证失败pip install pytest-asyncio --break-system-packages(若无),pytest tests/api/test_ws.py -v → FAIL

  • Step 3: 实现 ws.py

# sanguo_api/ws.py
"""WS 连接池:task_id → 订阅者集合。简单广播,不做重连/心跳。"""
from fastapi import WebSocket


class ConnectionManager:
    def __init__(self):
        self._connections: dict[str, set[WebSocket]] = {}

    def connect(self, task_id: str, ws: WebSocket):
        self._connections.setdefault(task_id, set()).add(ws)

    def disconnect(self, task_id: str, ws: WebSocket):
        conns = self._connections.get(task_id)
        if conns:
            conns.discard(ws)
            if not conns:
                del self._connections[task_id]

    async def broadcast(self, task_id: str, msg: dict):
        for ws in list(self._connections.get(task_id, [])):
            try:
                await ws.send_json(msg)
            except Exception:
                self.disconnect(task_id, ws)


manager = ConnectionManager()
  • Step 4: 验证通过pytest tests/api/test_ws.py -v → 3 passed

  • Step 5: Commitgit commit -m "feat(api): WS 连接池 ConnectionManager"


Task 3: sanguo_orchestrator/pool.py 异步化(ProcessPoolExecutor + stage

Files:

  • Modify: sanguo_orchestrator/pool.pysanguo_orchestrator/task.py
  • Test: tests/orchestrator/test_pool.py

Interfaces:

  • Produces: Taskstage: str 字段(默认 "");TaskPoolexecutor: ProcessPoolExecutorspawn contextmax_workers)、submit_work(task_id, func, *args)Futureupdate_stage(task_id, stage)get_stage(task_id) → str|None

  • Step 1: 写失败测试

# tests/orchestrator/test_pool.py(追加)
from unittest.mock import MagicMock

def test_task_has_stage_field():
    from sanguo_orchestrator.task import Task
    t = Task(task_id="t1", task_type="cta")
    assert t.stage == ""

def test_pool_submit_work_returns_future():
    from sanguo_orchestrator.pool import TaskPool
    pool = TaskPool(max_workers=2)
    pool.executor = MagicMock()  # mock executor 避免真起进程
    mock_future = MagicMock()
    pool.executor.submit.return_value = mock_future
    fut = pool.submit_work("t1", func=lambda: 1)
    assert fut is mock_future
    pool.executor.submit.assert_called_once()

def test_pool_update_and_get_stage():
    from sanguo_orchestrator.pool import TaskPool
    from sanguo_orchestrator.task import Task
    pool = TaskPool(max_workers=2)
    pool.submit("t1", "cta")
    pool.update_stage("t1", "回测中")
    assert pool.get_stage("t1") == "回测中"
    assert pool.get_task("t1").stage == "回测中"
  • Step 2: 验证失败

  • Step 3: task.py 加 stage 字段

# sanguo_orchestrator/task.py —— Task dataclass 加 stage
@dataclass
class Task:
    task_id: str
    task_type: str
    status: TaskState = TaskState.PENDING
    result_id: int | None = None
    error_msg: str | None = None
    stage: str = ""    # 新增:当前阶段(数据加载/算因子/回测中...)
    # start/complete/fail 方法不变
  • Step 4: pool.py 加 ProcessPoolExecutor + submit_work + stage
# sanguo_orchestrator/pool.py
from concurrent.futures import ProcessPoolExecutor, Future
from multiprocessing import get_context
from .task import Task, TaskState


class TaskPool:
    def __init__(self, max_workers: int = 2):
        self.max_workers = max_workers
        self._tasks: dict[str, Task] = {}
        self.executor = ProcessPoolExecutor(
            max_workers=max_workers, mp_context=get_context("spawn")
        )

    def submit(self, task_id: str, task_type: str) -> Task:
        task = Task(task_id=task_id, task_type=task_type)
        self._tasks[task_id] = task
        return task

    def submit_work(self, task_id: str, func, *args) -> Future:
        return self.executor.submit(func, *args)

    def update_stage(self, task_id: str, stage: str):
        t = self._tasks.get(task_id)
        if t:
            t.stage = stage

    def get_status(self, task_id: str) -> TaskState | None:
        t = self._tasks.get(task_id)
        return t.status if t else None

    def get_stage(self, task_id: str) -> str | None:
        t = self._tasks.get(task_id)
        return t.stage if t else None

    def get_task(self, task_id: str) -> Task | None:
        return self._tasks.get(task_id)
  • Step 5: 验证通过 + commitpytest tests/orchestrator/test_pool.py -vgit commit -m "feat(orchestrator): pool 异步化(ProcessPoolExecutor spawn + stage 追踪)"

Task 4: sanguo_orchestrator/runner.py async submit + on_stage 回调

Files:

  • Modify: sanguo_orchestrator/runner.py
  • Test: tests/orchestrator/test_runner.py

Interfaces:

  • Produces: Orchestratorasync submit_cta(...) → task_id(提交到 pool.executorasyncio.wrap_future 桥接,完成后调回调更新状态 + broadcast)、set_on_stage(callback)callback: async (task_id, stage) → None

  • Step 1: 写失败测试

# tests/orchestrator/test_runner.py(追加)
import asyncio
from unittest.mock import MagicMock, AsyncMock, patch

def test_submit_cta_returns_task_id_and_submits():
    from sanguo_orchestrator.runner import Orchestrator
    orch = Orchestrator(db_path="/tmp/t.db")
    orch.pool.executor = MagicMock()
    orch.pool.executor.submit.return_value = MagicMock()
    with patch("sanguo_backtest.cta_engine.run_cta_backtest"):
        task_id = asyncio.get_event_loop().run_until_complete(
            orch.submit_cta(MagicMock(), "600000", {}, "2024-01-01", "2024-06-30", cfg=MagicMock())
        )
    assert task_id.startswith("cta_600000")
    orch.pool.executor.submit.assert_called_once()

def test_set_on_stage_callback_called():
    from sanguo_orchestrator.runner import Orchestrator
    from sanguo_orchestrator.task import TaskState
    orch = Orchestrator(db_path="/tmp/t.db")
    orch.pool.executor = MagicMock()
    mock_future = MagicMock()
    orch.pool.executor.submit.return_value = mock_future
    cb = AsyncMock()
    orch.set_on_stage(cb)
    # 模拟任务完成回调
    asyncio.get_event_loop().run_until_complete(orch._on_done("t1", {"statistics": {}}))
    cb.assert_called()
  • Step 2: 验证失败

  • Step 3: 实现 runner.py async

# sanguo_orchestrator/runner.py
import asyncio
from concurrent.futures import Future
from .pool import TaskPool
from .task import TaskState


class Orchestrator:
    def __init__(self, db_path, file_dir=None, max_workers=2):
        self.db_path = db_path
        self.file_dir = file_dir
        self.pool = TaskPool(max_workers=max_workers)
        self._pending: dict = {}
        self._on_stage = None  # async callback(task_id, stage)

    def set_on_stage(self, cb):
        self._on_stage = cb

    async def _notify_stage(self, task_id: str, stage: str):
        self.pool.update_stage(task_id, stage)
        if self._on_stage:
            await self._on_stage(task_id, stage)

    async def submit_cta(self, strategy_class, symbol, params, start, end, cfg) -> str:
        task_id = f"cta_{symbol}_{id(params)}"
        self.pool.submit(task_id, "cta")
        self._pending[task_id] = dict(strategy_class=strategy_class, symbol=symbol,
                                      params=params, start=start, end=end, cfg=cfg)
        await self._notify_stage(task_id, "排队中")
        spec = self._pending[task_id]
        from sanguo_backtest.cta_engine import run_cta_backtest
        fut: Future = self.pool.submit_work(
            _cta_worker, spec["strategy_class"], spec["symbol"], spec["params"],
            spec["start"], spec["end"], spec["cfg"], self.db_path,
        )
        self.pool.get_task(task_id).start()
        await self._notify_stage(task_id, "回测中")
        asyncio.ensure_future(self._wait_future(task_id, fut))
        return task_id

    async def _wait_future(self, task_id: str, fut: Future):
        try:
            result = await asyncio.wrap_future(fut)
            await self._on_done(task_id, result)
        except Exception as e:
            self.pool.get_task(task_id).fail(f"{type(e).__name__}: {e}")
            await self._notify_stage(task_id, "失败")

    async def _on_done(self, task_id: str, result):
        self.pool.get_task(task_id).complete(result_id=id(result))
        await self._notify_stage(task_id, "完成")

    # submit_optimize / submit_factor 同模式(见 Task 5 路由调用)
    async def submit_optimize(self, strategy_class, symbol, grid, start, end, cfg) -> str:
        task_id = f"opt_{symbol}_{id(grid)}"
        self.pool.submit(task_id, "optimize")
        await self._notify_stage(task_id, "参数优化中")
        from sanguo_backtest.cta_optimizer import run_cta_optimization
        fut = self.pool.submit_work(_opt_worker, strategy_class, symbol, grid, start, end, cfg, self.db_path)
        self.pool.get_task(task_id).start()
        asyncio.ensure_future(self._wait_future(task_id, fut))
        return task_id

    async def submit_factor(self, symbols, factor_names, start, end, cfg, output_dir) -> str:
        task_id = f"factor_{id(factor_names)}"
        self.pool.submit(task_id, "factor")
        await self._notify_stage(task_id, "因子分析中")
        from sanguo_factor.analyzer import run_factor_analysis
        fut = self.pool.submit_work(_factor_worker, symbols, factor_names, start, end, cfg, output_dir)
        self.pool.get_task(task_id).start()
        asyncio.ensure_future(self._wait_future(task_id, fut))
        return task_id

    def get_status(self, task_id):
        return self.pool.get_status(task_id)

    def get_result(self, task_id):
        task = self.pool.get_task(task_id)
        if task and task.status == TaskState.DONE and task.result_id:
            from sanguo_backtest.result_store import load_result
            return load_result(task.result_id, self.db_path)
        return None


# 模块级 workerspawn 友好,可 pickle
def _cta_worker(strategy_class, symbol, params, start, end, cfg, db_path):
    from sanguo_backtest.cta_engine import run_cta_backtest
    return run_cta_backtest(strategy_class, symbol, params, start, end, cfg, db_path)

def _opt_worker(strategy_class, symbol, grid, start, end, cfg, db_path):
    from sanguo_backtest.cta_optimizer import run_cta_optimization
    return run_cta_optimization(strategy_class, symbol, grid, start, end, cfg, db_path)

def _factor_worker(symbols, factor_names, start, end, cfg, output_dir):
    from sanguo_factor.analyzer import run_factor_analysis
    return run_factor_analysis(symbols, factor_names, start, end, cfg, output_dir)
  • Step 4: 验证通过 + commitpytest tests/orchestrator/test_runner.py -vgit commit -m "feat(orchestrator): runner async submit + on_stage 回调"

Task 5: sanguo_api/routes.py 完整(login + JWT 依赖 + optimize/factor + WS route

Files:

  • Modify: sanguo_api/routes.pysanguo_api/app.py
  • Test: tests/api/test_routes.py

Interfaces:

  • Produces: POST /auth/login(用户名密码 → token)、optimize/factor 路由真调用 submit_optimize/submit_factor、业务路由加 Depends(verify_token)WS /ws/task/{id}(订阅阶段推送)

  • Step 1: 写失败测试

# tests/api/test_routes.py(追加)
from fastapi.testclient import TestClient
from unittest.mock import patch, MagicMock, AsyncMock

def test_login_returns_token(tmp_path):
    from sanguo_api.app import create_app
    from sanguo_api.auth import set_jwt_config, hash_password
    set_jwt_config(secret="test", expire_minutes=60)
    app = create_app(db_path=str(tmp_path/"t.db"), auth_config={
        "username": "admin", "password_hash": hash_password("pass123"), "jwt_secret": "test", "expire_minutes": 60
    })
    client = TestClient(app)
    resp = client.post("/api/v1/auth/login", json={"username": "admin", "password": "pass123"})
    assert resp.status_code == 200
    assert "token" in resp.json()

def test_protected_route_without_token_401(tmp_path):
    from sanguo_api.app import create_app
    app = create_app(db_path=str(tmp_path/"t.db"))
    client = TestClient(app)
    resp = client.get("/api/v1/task/t1")
    assert resp.status_code == 401

def test_optimize_route_calls_submit(tmp_path):
    from sanguo_api.app import create_app
    from sanguo_api.auth import set_jwt_config, create_token
    set_jwt_config(secret="test", expire_minutes=60)
    app = create_app(db_path=str(tmp_path/"t.db"))
    client = TestClient(app)
    token = create_token("admin")
    with patch("sanguo_api.routes.get_orchestrator") as m:
        orch = MagicMock()
        orch.submit_optimize = AsyncMock(return_value="opt_1")
        m.return_value = orch
        resp = client.post("/api/v1/backtest/optimize", json={
            "symbol": "600000", "strategy": "MaStrategy", "grid": {"n": [5, 20, 5]},
            "start": "2024-01-01", "end": "2024-06-30", "max_workers": 2
        }, headers={"Authorization": f"Bearer {token}"})
    assert resp.status_code == 200
    assert resp.json()["task_id"] == "opt_1"
  • Step 2: 验证失败

  • Step 3: 实现 routes 完整 + app 改

# sanguo_api/routes.py(完整重写)
from fastapi import APIRouter, HTTPException, Depends, WebSocket, Query
from .schemas import CtaBacktestRequest, OptimizeRequest, FactorAnalysisRequest
from .auth import verify_token, verify_password, create_token
from .ws import manager
from pydantic import BaseModel

router = APIRouter()
_orchestrator = None
_auth_config = {"username": "admin", "password_hash": "", "jwt_secret": "x", "expire_minutes": 60}


def set_orchestrator(orch): global _orchestrator; _orchestrator = orch
def get_orchestrator(): return _orchestrator
def set_auth_config(cfg): _auth_config.update(cfg)


class LoginRequest(BaseModel):
    username: str
    password: str


@router.post("/auth/login")
def login(req: LoginRequest):
    if req.username != _auth_config["username"] or not verify_password(req.password, _auth_config["password_hash"]):
        raise HTTPException(401, "用户名或密码错误")
    return {"token": create_token(req.username)}


@router.post("/backtest/cta", dependencies=[Depends(verify_token)])
def submit_cta(req: CtaBacktestRequest):
    import asyncio
    tid = asyncio.get_event_loop().run_until_complete(
        get_orchestrator().submit_cta(req.strategy, req.symbol, req.params, req.start, req.end, None))
    return {"task_id": tid}


@router.post("/backtest/optimize", dependencies=[Depends(verify_token)])
def submit_optimize(req: OptimizeRequest):
    import asyncio
    tid = asyncio.get_event_loop().run_until_complete(
        get_orchestrator().submit_optimize(req.strategy, req.symbol, req.grid, req.start, req.end, None))
    return {"task_id": tid}


@router.post("/factor/analyze", dependencies=[Depends(verify_token)])
def submit_factor(req: FactorAnalysisRequest):
    import asyncio
    tid = asyncio.get_event_loop().run_until_complete(
        get_orchestrator().submit_factor(req.symbols, req.factor_names, req.start, req.end, None, "/tmp/factor"))
    return {"task_id": tid}


@router.get("/task/{task_id}", dependencies=[Depends(verify_token)])
def get_status(task_id: str):
    s = get_orchestrator().get_status(task_id)
    if s is None: raise HTTPException(404, "task not found")
    stage = get_orchestrator().pool.get_stage(task_id)
    return {"task_id": task_id, "status": s.value if hasattr(s, "value") else str(s), "stage": stage or ""}


@router.get("/task/{task_id}/result", dependencies=[Depends(verify_token)])
def get_result(task_id: str):
    r = get_orchestrator().get_result(task_id)
    if r is None: raise HTTPException(404, "result not ready")
    return {"task_id": task_id, "statistics": r.statistics}


@router.websocket("/ws/task/{task_id}")
async def task_ws(websocket: WebSocket, task_id: str, token: str = Query(...)):
    from .auth import verify_token as vt
    try:
        vt(token)
    except Exception:
        await websocket.close(code=4401); return
    await websocket.accept()
    manager.connect(task_id, websocket)
    try:
        while True:
            await websocket.receive_text()  # 保活
    except Exception:
        pass
    finally:
        manager.disconnect(task_id, websocket)
# sanguo_api/app.py(改:register auth_config + on_stage 推 WS
from fastapi import FastAPI
from .routes import router, set_orchestrator, set_auth_config
from .auth import set_jwt_config
from .ws import manager
from sanguo_orchestrator.runner import Orchestrator


def create_app(db_path: str, file_dir: str | None = None, auth_config: dict | None = None, max_workers: int = 2) -> FastAPI:
    app = FastAPI(title="Sanguo Quant API")
    orch = Orchestrator(db_path=db_path, file_dir=file_dir, max_workers=max_workers)
    async def _on_stage(task_id, stage):
        await manager.broadcast(task_id, {"task_id": task_id, "stage": stage})
    orch.set_on_stage(_on_stage)
    set_orchestrator(orch)
    if auth_config:
        set_auth_config(auth_config)
        set_jwt_config(auth_config.get("jwt_secret", "x"), auth_config.get("expire_minutes", 60))
    app.include_router(router, prefix="/api/v1")
    return app
  • Step 4: 验证通过 + commitpytest tests/api/ -vgit commit -m "feat(api): routes 完整(login + JWT 依赖 + optimize/factor + WS route"

Task 6: sanguo_factor/alpha_lab.py compute_factors 完整化

Files:

  • Modify: sanguo_factor/alpha_lab.py
  • Test: tests/factor/test_alpha_lab.py

Interfaces:

  • Produces: AlphaLabSession.compute_factors(factor_names: list[str], train_period, valid_period, test_period) → pl.DataFrameadd_feature + prepare_data + fetch_raw

  • Step 1: 写失败测试

# tests/factor/test_alpha_lab.py(追加)
from unittest.mock import patch, MagicMock

def test_compute_factors_calls_prepare_and_fetch(tmp_path):
    from sanguo_factor.alpha_lab import AlphaLabSession
    with patch("sanguo_factor.alpha_lab.AlphaLab") as MLab, \
         patch("sanguo_factor.alpha_lab.read_db_daily") as MRead, \
         patch("sanguo_factor.alpha_lib.AlphaDataset") as MDS, \
         patch("sanguo_factor.alpha_lab.get_factor") as MGet, \
         patch("sanguo_factor.alpha_lab.save_alpha_lab_data"):
        import polars as pl
        MRead.return_value = [MagicMock(vt_symbol="600000.SSE", datetime=__import__("datetime").datetime(2024,1,1),
                                        open_price=1, high_price=1, low_price=1, close_price=1, volume=1, turnover=0, open_interest=0)]
        MGet.return_value = {"expression": "ts_mean(close,5)"}
        MDS.return_value.fetch_raw.return_value = pl.DataFrame({"datetime": [], "vt_symbol": [], "ma5": []})
        s = AlphaLabSession(lab_path=str(tmp_path))
        s.load_symbols(["600000"], "2024-01-01", "2024-06-30", cfg=MagicMock())
        df = s.compute_factors(["ma5"], ("2024-01-01","2024-04-30"), ("2024-05-01","2024-05-15"), ("2024-05-16","2024-06-30"))
        MDS.return_value.add_feature.assert_called()
        MDS.return_value.prepare_data.assert_called_once()
  • Step 2: 验证失败

  • Step 3: 补 compute_factors

# sanguo_factor/alpha_lab.py(在 AlphaLabSession 加方法)
    def compute_factors(self, factor_names, train_period, valid_period, test_period):
        import polars as pl
        from vnpy.alpha.dataset import AlphaDataset, Segment
        from .registry import get_factor
        from .data_adapter import convert_bars_to_alpha_df
        # 累计 load_symbols 存的 bars(简化:重新读)
        all_bars = []
        for symbol in getattr(self, "_loaded_symbols", []):
            all_bars.extend(self._loaded_bars.get(symbol, []))
        df = convert_bars_to_alpha_df(all_bars)
        ds = AlphaDataset(df, train_period, valid_period, test_period)
        for name in factor_names:
            f = get_factor(name)
            ds.add_feature(name, f["expression"])
        ds.prepare_data(max_workers=1)
        return ds.fetch_raw(Segment.TEST)

(配套 load_symbols 缓存 _loaded_symbols/_loaded_bars,见执行时补全。)

  • Step 4: 验证通过 + commitpytest tests/factor/test_alpha_lab.py -vgit commit -m "feat(factor): alpha_lab compute_factors 完整化"

Task 7: sanguo_factor/analyzer.py tears pipeline 完整化

Files:

  • Modify: sanguo_factor/analyzer.py
  • Test: tests/factor/test_analyzer.py

Interfaces:

  • Produces: run_factor_analysis 补 compute_factors → get_clean_factor_and_forward_returns → create_full_tear_sheet → html 报告,FactorReport 含 report_path/ic_summary

  • Step 1: 写失败测试

# tests/factor/test_analyzer.py(追加)
from unittest.mock import patch, MagicMock

def test_run_factor_analysis_calls_tears(tmp_path):
    from sanguo_factor.analyzer import run_factor_analysis
    with patch("sanguo_factor.analyzer.AlphaLabSession") as MS, \
         patch("sanguo_factor.analyzer.get_clean_factor_and_forward_returns") as MC, \
         patch("sanguo_factor.analyzer.create_full_tear_sheet") as MT:
        import polars as pl
        MS.return_value.compute_factors.return_value = pl.DataFrame({"datetime":[],"vt_symbol":[],"ma5":[]})
        MC.return_value = MagicMock()
        report = run_factor_analysis(["600000"], ["ma5"], "2024-01-01", "2024-06-30",
                                      cfg=MagicMock(), output_dir=str(tmp_path))
        assert report.factor_names == ["ma5"]
        MC.assert_called_once(); MT.assert_called_once()
  • Step 2: 验证失败

  • Step 3: 补 tears pipeline

# sanguo_factor/analyzer.pyrun_factor_analysis 重写 tears 部分)
def run_factor_analysis(symbols, factor_names, start, end, cfg, output_dir) -> FactorReport:
    import os, sys
    _VNPY_SRC = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "vnpy_v4.4.0"))
    if _VNPY_SRC not in sys.path: sys.path.insert(0, _VNPY_SRC)
    from alphalens.utils import get_clean_factor_and_forward_returns
    from alphalens.tears import create_full_tear_sheet
    import matplotlib
    matplotlib.use("Agg")
    from .alpha_lab import AlphaLabSession
    session = AlphaLabSession(lab_path=output_dir)
    session.load_symbols(symbols, start, end, cfg)
    factor_df = session.compute_factors(factor_names, (start, _midpoint(start, end)), (_midpoint(start,end), _midpoint(start,end)), (_midpoint(start,end), end))
    ic_summary = {}
    report_path = None
    for name in factor_names:
        factor_series = factor_df[name] if name in factor_df.columns else factor_df.iloc[:, -1]
        merged = get_clean_factor_and_forward_returns(factor_df.set_index(["datetime","vt_symbol")[name]], ...)  # 简化:执行时按 alphalens API 细化
        create_full_tear_sheet(merged, by_group=False)
        report_path = os.path.join(output_dir, f"{name}_tears.html")
    return FactorReport(factor_names=factor_names, output_dir=output_dir, ic_summary=ic_summary)

tears 细节(forward returns 计算 + html 保存)依赖 alphalens API,执行 subagent 读 alphalens.tears.create_full_tear_sheet 签名 + 容器验证。

  • Step 4: 验证通过 + commitpytest tests/factor/test_analyzer.py -vgit commit -m "feat(factor): analyzer tears pipeline 完整化"

Task 8: 端到端冒烟(异步回测 + WS + tears,容器)

Files:

  • Create: scripts/smoke_phase3a.py

  • Step 1: 写冒烟脚本

# scripts/smoke_phase3a.py
"""Phase 3a 端到端冒烟:异步 submit + WS 阶段 + JWT + tears(容器)。"""
import sys, asyncio
sys.path.insert(0, "/app"); sys.path.insert(0, "/app/vnpy_v4.4.0")

async def main():
    print("=== Phase 3a Smoke ===")
    from sanguo_api.app import create_app
    from sanguo_api.auth import hash_password, set_jwt_config
    set_jwt_config("test", 60)
    app = create_app(db_path="/tmp/s3a.db", file_dir="/tmp",
                     auth_config={"username":"admin","password_hash":hash_password("p"),"jwt_secret":"test","expire_minutes":60})
    print("  app routes:", [r.path for r in app.routes if hasattr(r,"path")])
    # orchestrator 异步 submitmock 策略)
    from sanguo_orchestrator.runner import Orchestrator
    orch = Orchestrator(db_path="/tmp/s3a.db")
    print("  orchestrator pool executor:", type(orch.pool.executor).__name__)
    print("=== Phase 3a Smoke DONE ===")

asyncio.run(main())
  • Step 2: 本地全部 Phase 3a 测试 + 覆盖率pytest tests/ --cov=sanguo_api --cov=sanguo_orchestrator --cov=sanguo_factor --cov-report=term → 全绿 + ≥80%

  • Step 3: 容器端到端冒烟(rsync + docker exec

  • Step 4: Commitgit commit -m "test(phase3a): 端到端冒烟(异步 submit + WS + JWT + tears"

  • Step 5: requesting-code-review + finishing-a-development-branch


Self-Review

1. Spec coverage:异步 pool(T3) + runner async(T4) + WS 阶段(T2/T5) + JWT 单用户(T1/T5) + optimize/factor 路由(T5) + alpha tears 完整化(T6/T7) = design §0 五项全覆盖 2. Placeholdertears pipeline 的 forward returns 细节(T7)依赖 alphalens API,标注执行时细化;其余代码完整。 3. Type consistencyOrchestrator.async submit_*T4)在 routes(T5)调用一致;manager.broadcastT2)在 app on_stageT5)调用一致;compute_factorsT6)在 analyzerT7)调用一致


Execution Handoff

Plan saved to docs/superpowers/plans/2026-07-06-phase3a-web-api.md. 推荐用 Subagent-Driven 执行(每 task fresh subagent + review)。