sanguo_vnpy_v2/docs/archive/data/2026-07-23-local-unified-provider.md

LocalUnifiedProvider Implementation Plan (spec §6 使用层)

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 实现 spec §6 使用层 LocalUnifiedProvider——读方案A 权威数据层(dbbardata/constituent_unified/valuation_baostock),零 online,治幸存者偏差,喂 all_weather 策略。

Architecture: 新建 LocalUnifiedProvider(bullet_trade.DataProvider),内部按数据类路由方案A 权威表:日线读 dbbardata('d') raw + bs_adjust_factor 算前复权;成份股读 constituent_unified 并集(治偏差);估值读 valuation_baostock parquet + 市值读 static/valuation akshare parquet。Mac 测试用 sqlite :memory: + tmp parquet fixture,零 VPS 依赖。

Tech Stack: Python 3.10, pandas 2.3, sqlite3, pyarrow, pytest

Global Constraints(spec + 用户铁律)

• 零 online: provider 不 import baostock 调 online,纯读本地 DB/parquet(memory provider-local-data-only)。baostock 48000/天限频不波及使用层。
• surgical: 不改 LocalParquetProvider/BaostockProvider(旧链路保留,向后兼容)。
• dbbardata 不破坏: UNIQUE(symbol,exchange,interval,datetime),只读不写。
• 复权: dbbardata 存 raw,消费端按 bs_adjust_factor.foreAdjustFactor 算前复权(§14.7 最终目标,用户定不降级)。
• constituent_unified 并集模型: 表无 date 列,get_index_stocks(date) 返回 in_current∪was_removed 并集,date 参数无法精确时点过滤——治"纯当前幸存者"偏差,有轻微前视(使用说明标注)。
• 代码归一: jq_code 600519.XSHG ↔ dbbardata symbol=600519, exchange=SSE;SSE→SH, SZSE→SZ。

实测 schema(VPS 2026-07-23 probe,执行 agent 必读)

DB = C:\sanguo_vnpy_v2\data\quant_trading.db(VPS) / Mac 测试用 fixture 路径。

dbbardata('d') — 唯一行情表,raw 真实价:

列: symbol TEXT, exchange TEXT(SSE/SZSE), datetime TEXT(YYYY-MM-DD HH:MM:SS),
    interval TEXT('d'), volume REAL, turnover REAL, open_interest REAL,
    open_price REAL, high_price REAL, low_price REAL, close_price REAL
样本: 600519 10056行 2001-08-27~2026-07-22; 000005退市 8146行~2024-04-26; 510300 ETF 3439行

constituent_unified — 成份股并集(无 date!):

列: index_code TEXT(如 '000300'), code TEXT(纯6位如 '000001'), code_name TEXT,
    source TEXT('baostock'/'akshare'), in_current INT(0/1), was_removed INT(0/1)
分布: 000300=940(300当前+640被踢) 000905=1803 000016=195 000852=1000(全当前,历史不可补)
      399001=702 399005=145 399006=175 399330=150 932000=2000(全当前)

bs_adjust_factor — 复权因子:

列: code TEXT('sh.600519'), dividOperateDate TEXT(YYYY-MM-DD),
    foreAdjustFactor REAL, backAdjustFactor REAL, adjustFactor REAL
语义: foreAdjustFactor 按除权日分段,最新事件=1.0,递减往历史。qfq[t]=raw[t]*factor[date[t]]。
     600519 有 12 事件: 2020-06-24=0.856267 ... 2026-06-26=1.0

valuation_baostock/.parquet — baostock 估值(1990-2026 全年份):

列: symbol(6位), exchange(SH/SZ), date(YYYY-MM-DD), peTTM, psTTM, pcfNcfTTM, pbMRQ, turn, pctChg, isST
注: 无 market_cap/total_share 列! 市值从 static/valuation akshare 补。

static/valuation/_valuation.parquet — akshare 估值(市值/股本来源,5530 文件):

中文列(见 LocalParquetProvider._VAL_COL_MAP): 总市值→total_market_cap, 流通市值→circ_market_cap,
    总股本→total_share, PE(TTM)→pe_ttm, 市净率→pb ...

static/{balance,income,cashflow}/_.parquet — akshare 三表(balance 221列/income 170列):

通用列: SECUCODE, REPORT_DATE, REPORT_TYPE; balance 有 TOTAL_ASSETS/TOTAL_LIABILITIES/TOTAL_PARENT_EQUITY;
        income 有 BASIC_EPS/OPERATE_INCOME/PARENT_NETPROFIT/OPERATE_INCOME_YOY

---

File Structure

• Create: sanguo_portfolio/providers/local_unified_provider.py — LocalUnifiedProvider 类(~400行)
• Modify: sanguo_portfolio/providers/__init__.py — 导出 LocalUnifiedProvider
• Modify: sanguo_portfolio/runner_backtest.py — build_provider 加 unified 选项(choices + 分支)
• Create: tests/portfolio/test_local_unified_provider.py — DataProvider 契约单测(fixture: sqlite + tmp parquet)
• Create: tests/portfolio/conftest.py 追加 — local_unified_provider fixture(若需要,否则在测试文件内建)
• Create: docs/portfolio_local_unified_provider.md — 使用说明(架构/数据源/接口/复权/治偏差/Mac测试/部署)

---

Task 0: 代码转换 + DB 连接辅助 + 复权因子构造

Files:

• Create: sanguo_portfolio/providers/local_unified_provider.py(本 task 建文件骨架 + 模块级辅助函数)
• Test: tests/portfolio/test_local_unified_provider.py

Interfaces:

• Produces: jq_to_dbbardata(jq_code) -> (symbol, exchange) / dbbardata_to_jq(symbol, exchange) -> jq_code; _connect(cfg) -> sqlite3.Connection; _build_qfq_factor(code, conn, dates) -> pd.Series(factor indexed by date)

• Step 1: 写失败测试 — 代码转换

# tests/portfolio/test_local_unified_provider.py
from sanguo_portfolio.providers.local_unified_provider import (
    jq_to_dbbardata, dbbardata_to_jq, LocalUnifiedProvider,
)

def test_jq_to_dbbardata_roundtrip():
    assert jq_to_dbbardata("600519.XSHG") == ("600519", "SSE")
    assert jq_to_dbbardata("000001.XSHE") == ("000001", "SZSE")
    assert jq_to_dbbardata("600519") == ("600519", "SSE")  # 纯6位推断
    assert dbbardata_to_jq("600519", "SSE") == "600519.XSHG"
    assert dbbardata_to_jq("000001", "SZSE") == "000001.XSHE"

• Step 2: 跑测试确认 FAIL — pytest tests/portfolio/test_local_unified_provider.py::test_jq_to_dbbardata_roundtrip -v(ImportError)

• Step 3: 实现模块骨架 + 代码转换

# sanguo_portfolio/providers/local_unified_provider.py
"""LocalUnifiedProvider: 读方案A 权威数据层, 零 online, 治幸存者偏差(spec §6)。

数据源(全本地 VPS C:\\sanguo_vnpy_v2\\data\\):
- 日线: dbbardata('d') raw + bs_adjust_factor 算前复权(§14.7)
- 成份股: constituent_unified 并集(治偏差,无 date 时点)
- 估值 pe/pb/ps/pcf: valuation_baostock/<year>.parquet(baostock 权威)
- 市值/股本: static/valuation akshare parquet(baostock valuation 无市值列)
- 三表: static/{balance,income,cashflow} akshare parquet

零 online: 不 import baostock 调 online。Mac 测试用 sqlite+parquet fixture。
"""
from __future__ import annotations
import logging, os, sqlite3
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional, Union
import pandas as pd

try:
    from bullet_trade.data.providers.base import DataProvider  # type: ignore
except ImportError:
    class DataProvider:  # type: ignore[no-redef]
        name: str = "base"

logger = logging.getLogger(__name__)
_DEFAULT_DB = r"C:\sanguo_vnpy_v2\data\quant_trading.db"
_DEFAULT_DATA_DIR = r"C:\sanguo_vnpy_v2\data"

_JQ_SUFFIX_TO_EXC = {"XSHG": "SSE", "XSHE": "SZSE", "SH": "SSE", "SZ": "SZSE"}
_EXC_TO_JQ_SUFFIX = {"SSE": "XSHG", "SZSE": "XSHE"}


def jq_to_dbbardata(jq_code: str) -> tuple[str, str]:
    """600519.XSHG → ('600519', 'SSE')。纯6位按6开头=sh/0,3=sz 推断。"""
    s = (jq_code or "").strip()
    if "." not in s:
        if len(s) == 6:
            return s, ("SSE" if s.startswith("6") else "SZSE")
        return s, "SSE"
    code, suffix = s.split(".", 1)
    return code, _JQ_SUFFIX_TO_EXC.get(suffix.upper(), "SSE")


def dbbardata_to_jq(symbol: str, exchange: str) -> str:
    """('600519','SSE') → '600519.XSHG'。"""
    jq_suffix = _EXC_TO_JQ_SUFFIX.get(str(exchange).upper(), "XSHG")
    return f"{symbol}.{jq_suffix}"


# 复权因子代码转换: 600519.XSHG → 'sh.600519'(bs_adjust_factor.code 格式)
def _jq_to_bs_code(jq_code: str) -> str:
    sym, exc = jq_to_dbbardata(jq_code)
    prefix = "sh" if exc == "SSE" else "sz"
    return f"{prefix}.{sym}"

• Step 4: 跑测试确认 PASS

• Step 5: 写失败测试 — 复权因子构造

def test_build_qfq_factor(tmp_path):
    # fixture: 2 除权事件, 最新=1.0
    import sqlite3
    db = tmp_path / "t.db"
    c = sqlite3.connect(str(db))
    c.execute("CREATE TABLE bs_adjust_factor(code TEXT, dividOperateDate TEXT, foreAdjustFactor REAL, backAdjustFactor REAL, adjustFactor REAL)")
    c.executemany("INSERT INTO bs_adjust_factor VALUES(?,?,?,?,?)", [
        ("sh.600519", "2024-06-19", 0.90, 0, 0),
        ("sh.600519", "2025-06-19", 1.00, 0, 0),
    ])
    c.commit(); c.close()
    from sanguo_portfolio.providers.local_unified_provider import _build_qfq_factor
    dates = pd.to_datetime(["2023-01-01", "2024-07-01", "2025-07-01"])
    f = _build_qfq_factor("sh.600519", sqlite3.connect(str(db)), dates)
    # 2023(早于最早事件)=0.90; 2024-07(between)=0.90; 2025-07(最新后)=1.00
    assert abs(f.iloc[0] - 0.90) < 1e-6
    assert abs(f.iloc[1] - 0.90) < 1e-6
    assert abs(f.iloc[2] - 1.00) < 1e-6

• Step 6: 实现 _build_qfq_factor — asof join 逻辑(每个 date 找 ≤ 的最大 dividOperateDate 的 foreAdjustFactor;早于所有事件用最早;晚于所有用最新):

def _build_qfq_factor(bs_code: str, conn: sqlite3.Connection,
                      dates: pd.Series) -> pd.Series:
    """构造每个 date 的前复权因子(asof)。qfq[t]=raw[t]*factor[t]。"""
    rows = conn.execute(
        "SELECT dividOperateDate, foreAdjustFactor FROM bs_adjust_factor "
        "WHERE code=? ORDER BY dividOperateDate", (bs_code,)).fetchall()
    if not rows:
        return pd.Series([1.0] * len(dates), index=dates)
    ev_dates = pd.to_datetime([r[0] for r in rows])
    factors = [float(r[1]) for r in rows]
    out = []
    for d in pd.to_datetime(dates):
        # 找 <= d 的最大事件; 全部 > d 用最早(第一个); 全部 <= d 用最后一个
        mask = ev_dates <= d
        out.append(factors[mask.argmax()] if mask.any() else factors[0])
        # mask.argmax() 给第一个 True 的索引;但我们要"<= d 的最大事件"= 最后一个 True
    # 修正:取最后一个 True
    out = []
    for d in pd.to_datetime(dates):
        mask = ev_dates <= d
        idx = int(np.where(mask)[0][-1]) if mask.any() else 0
        out.append(factors[idx])
    return pd.Series(out, index=pd.to_datetime(dates))

(注意:np 需 import numpy as np。实现时简化为单次循环取最后一个 True 索引。)

• Step 7: 跑测试确认 PASS
• Step 8: Commit — feat(portfolio): LocalUnifiedProvider 代码转换+复权因子(Task0)

---

Task 1: get_price(dbbardata raw + 前复权 + panel 长表)

Files: Modify local_unified_provider.py 加 __init__ + get_price; Test 同文件。

Interfaces:

• Consumes: Task0 辅助函数 + _connect
• Produces: LocalUnifiedProvider.get_price(security, start_date, end_date, frequency, fields, skip_paused, fq, count, panel, fill_paused) -> DataFrame

策略契约(all_weather 实证):

• get_price(hold_list, end_date, freq=daily, fields=[close,high_limit], count=1, panel=False) — panel=False 长表需 time/code 列

• get_price(stocks, freq=1d, fields=[close], count=n, panel=False) — _trend_mean pivot(index=time,columns=code)

• get_price(stock, freq=1m, fq="pre", count=1, panel=False) — intraday(day 频率回测降级,1m 无数据返空)

• Step 1: 写失败测试 — get_price daily 单股 + 复权

@pytest.fixture
def unified_provider(tmp_path):
    """造小样本 sqlite + parquet fixture。"""
    db = tmp_path / "quant_trading.db"
    c = sqlite3.connect(str(db))
    c.execute("CREATE TABLE dbbardata(symbol,exchange,datetime,interval,volume,turnover,open_interest,open_price,high_price,low_price,close_price)")
    rows = [
        ("600519","SSE","2024-06-18 00:00:00","d",1000,1e6,0,1000.0,1010.0,990.0,1000.0),  # 除权前
        ("600519","SSE","2024-06-19 00:00:00","d",1000,1e6,0,900.0,910.0,890.0,900.0),    # 除权日 raw 跳水
        ("600519","SSE","2024-06-20 00:00:00","d",1000,1e6,0,910.0,920.0,900.0,910.0),
    ]
    c.executemany("INSERT INTO dbbardata VALUES(?,?,?,?,?,?,?,?,?,?,?)", rows)
    c.execute("CREATE TABLE bs_adjust_factor(code,dividOperateDate,foreAdjustFactor,backAdjustFactor,adjustFactor)")
    c.execute("INSERT INTO bs_adjust_factor VALUES('sh.600519','2024-06-19',0.9,0,0)")  # 除权日 factor
    c.commit(); c.close()
    return LocalUnifiedProvider({"db_path": str(db), "data_dir": str(tmp_path)})

def test_get_price_raw_vs_qfq(unified_provider):
    p = unified_provider
    # raw: 除权日 900 跳水
    df_raw = p.get_price("600519.XSHG", start_date="2024-06-18", end_date="2024-06-20", fq="raw")
    assert len(df_raw) == 3
    assert abs(df_raw.loc["2024-06-19", "close"] - 900.0) < 1e-6
    # qfq: 06-18 = 1000*0.9 = 900; 06-19/20 = raw(factor=0.9 当 06-19 之后? 用最新段逻辑)
    df_qfq = p.get_price("600519.XSHG", start_date="2024-06-18", end_date="2024-06-20", fq="qfq")
    assert abs(df_qfq.loc["2024-06-18", "close"] - 900.0) < 1e-6  # 1000*0.9(早于事件用最早factor)

(复权断言:06-18 早于除权日 06-19 → 用 factor 0.9 → 10000.9=900;06-19/20 ≥ 事件日 → factor 取 06-19 的 0.9 → 9000.9=810, 910*0.9=819。实现时按 _build_qfq_factor 语义校准断言。)

• Step 2: 跑测试确认 FAIL

• Step 3: 实现 __init__ + get_price

class LocalUnifiedProvider(DataProvider):  # type: ignore[misc]
    name: str = "sanguo_local_unified"
    requires_live_data: bool = False

    def __init__(self, config: Optional[Dict[str, Any]] = None) -> None:
        cfg = config or {}
        self.db_path: str = cfg.get("db_path", _DEFAULT_DB)
        self.data_dir: str = cfg.get("data_dir", _DEFAULT_DATA_DIR)
        self._conn: Optional[sqlite3.Connection] = None
        self._val_bs_cache: Dict[int, pd.DataFrame] = {}  # year -> valuation_baostock

    def _connect(self) -> sqlite3.Connection:
        if self._conn is None:
            self._conn = sqlite3.connect(self.db_path, timeout=30)
            self._conn.execute("PRAGMA busy_timeout = 30000")
        return self._conn

    def get_price(self, security, start_date=None, end_date=None, frequency="daily",
                  fields=None, skip_paused=False, fq="raw", count=None,
                  panel=True, fill_paused=True, **kwargs):
        freq = str(frequency or "").lower()
        if freq not in ("daily", "day", "1d", "d"):
            return pd.DataFrame()  # 1m/分钟 day 频率回测降级(数据层无 1m)
        secs = [security] if isinstance(security, str) else list(security or [])
        conn = self._connect()
        start_str = self._to_date_str(start_date)
        end_str = self._to_date_str(end_date) or datetime.now().strftime("%Y-%m-%d")

        frames: Dict[str, pd.DataFrame] = {}
        for jq_code in secs:
            sym, exc = jq_to_dbbardata(jq_code)
            q = "SELECT datetime, open_price, high_price, low_price, close_price, " \
                "volume, turnover FROM dbbardata WHERE symbol=? AND exchange=? " \
                "AND interval='d' AND datetime>=? AND datetime<=? ORDER BY datetime"
            df = pd.read_sql(q, conn, params=(sym, exc, start_str + " 00:00:00", end_str + " 23:59:59"))
            if df.empty:
                frames[jq_code] = df; continue
            df["datetime"] = pd.to_datetime(df["datetime"])
            df = df.set_index("datetime")
            df.index.name = None
            if count:
                df = df.tail(count)
            # 复权
            if fq in ("qfq", "pre", "前复权"):
                factor = _build_qfq_factor(_jq_to_bs_code(jq_code), conn, df.index)
                for col in ("open_price", "high_price", "low_price", "close_price"):
                    df[col] = df[col].values * factor.values
            # 策略要 close/high_limit 字段名(jq 风格)
            df = df.rename(columns={"open_price": "open", "high_price": "high",
                                    "low_price": "low", "close_price": "close"})
            # high_limit 不在 dbbardata, 留给 get_current_tick 语义;这里策略 prepare_stock_list 要 high_limit 列
            # → 缺失列返 NaN(策略 hit = close==high_limit 不会命中,降级可接受)
            if fields:
                for f in fields:
                    if f not in df.columns:
                        df[f] = float("nan")
                df = df[fields]
            frames[jq_code] = df

        if not frames or all(f.empty for f in frames.values()):
            return pd.DataFrame()
        if not panel:
            parts = []
            for jq_code, df in frames.items():
                if df.empty:
                    continue
                d = df.reset_index().rename(columns={"datetime": "time"})
                d.insert(0, "code", jq_code)
                parts.append(d)
            return pd.concat(parts, ignore_index=True) if parts else pd.DataFrame()
        if len(frames) == 1:
            return next(iter(frames.values()))
        return pd.concat(frames, axis=1)

• Step 4: 跑测试确认 PASS
• Step 5: 写失败测试 — panel=False 多股长表 + count

def test_get_price_panel_false_multi(unified_provider):
    df = unified_provider.get_price("600519.XSHG", end_date="2024-06-20", count=2, panel=False, fields=["close"])
    assert "code" in df.columns and "time" in df.columns
    assert len(df) == 2

• Step 6: 实现(Step 3 已含 panel 分支),跑 PASS
• Step 7: Commit — feat(portfolio): LocalUnifiedProvider get_price+前复权(Task1)

---

Task 2: get_index_stocks + get_constituent(constituent_unified 并集,治偏差)

Files: Modify local_unified_provider.py; Test 同文件。

Interfaces:

• Produces: get_index_stocks(index_symbol, date) -> List[str] + get_constituent(index, date) -> List[str](语义别名)

• Step 1: 写失败测试

def test_get_index_stocks_union(tmp_path):
    db = tmp_path / "t.db"; c = sqlite3.connect(str(db))
    c.execute("CREATE TABLE constituent_unified(index_code TEXT,code TEXT,code_name TEXT,source TEXT,in_current INT,was_removed INT)")
    c.executemany("INSERT INTO constituent_unified VALUES(?,?,?,?,?,?)", [
        ("000300", "600519", "贵州茅台", "baostock", 1, 0),
        ("000300", "000001", "平安银行", "baostock", 1, 0),
        ("000300", "600811", "退市股", "baostock", 0, 1),  # 被踢
    ])
    c.commit(); c.close()
    p = LocalUnifiedProvider({"db_path": str(db), "data_dir": str(tmp_path)})
    stocks = p.get_index_stocks("000300.XSHG", "2020-01-01")
    assert set(stocks) == {"600519.XSHG", "000001.XSHE", "600811.SH"}  # 并集含被踢
    # date 参数不报错(并集模型忽略)
    assert p.get_constituent("000300", None) == stocks  # 别名

• Step 2: 跑测试确认 FAIL

• Step 3: 实现 — 查 constituent_unified,index_code 匹配(去 .XXXX 后缀),返回 in_current=1 OR was_removed=1 的并集,code→jq_code:

    def get_index_stocks(self, index_symbol, date=None) -> List[str]:
        idx = index_symbol.split(".")[0] if "." in str(index_symbol) else str(index_symbol)
        conn = self._connect()
        rows = conn.execute(
            "SELECT code FROM constituent_unified WHERE index_code=? "
            "AND (in_current=1 OR was_removed=1)", (idx,)).fetchall()
        out = []
        for (code,) in rows:
            code = str(code).strip()
            if len(code) != 6:
                continue
            exc = "SSE" if code.startswith("6") else "SZSE"
            out.append(dbbardata_to_jq(code, exc))
        return out

    def get_constituent(self, index, date=None) -> List[str]:
        """spec §6 语义别名 = get_index_stocks。"""
        return self.get_index_stocks(index, date)

• Step 4: 跑测试 PASS
• Step 5: Commit — feat(portfolio): LocalUnifiedProvider 成份股并集治偏差(Task2)

---

Task 3: get_fundamentals_df(valuation_baostock + static akshare + 三表)

Files: Modify local_unified_provider.py; Test 同文件 + tmp parquet fixture。

Interfaces:

• Produces: get_fundamentals_df(stocks, date) -> DataFrame 列对齐 _FUNDAMENTAL_COLUMNS

数据源映射:

• pe_ratio/pb_ratio/ps_ratio/pcf_ratio ← valuation_baostock parquet(peTTM/pbMRQ/psTTM/pcfNcfTTM,baostock 权威)

• market_cap/circulating_market_cap ← static/valuation akshare parquet(total_market_cap/circ_market_cap,baston 无市值)

• 三表字段(eps/net_profit_margin/total_liability 等) ← static/{balance,income} akshare parquet(复用 LocalParquetProvider 读法)

• Step 1: 写失败测试 — 估值字段从 valuation_baostock

def test_get_fundamentals_valuation(tmp_path):
    # valuation_baostock/2024.parquet
    vdir = tmp_path / "valuation_baostock"; vdir.mkdir()
    pd.DataFrame({"symbol":["600519"],"exchange":["SH"],"date":["2024-09-30"],
                  "peTTM":[25.0],"psTTM":[15.0],"pcfNcfTTM":[20.0],"pbMRQ":[7.5],
                  "turn":[0.1],"pctChg":[1.0],"isST":[0]}).to_parquet(vdir/"2024.parquet")
    # static/valuation akshare(市值)
    sdir = tmp_path / "static" / "valuation"; sdir.mkdir(parents=True)
    pd.DataFrame({"数据日期":["2024-09-30"],"总市值":[2e12],"流通市值":[2e12],"总股本":[1.256e9],
                  "PE(TTM)":[25],"市净率":[7.5]}).to_parquet(sdir/"600519.SH_valuation.parquet")
    p = LocalUnifiedProvider({"db_path": str(tmp_path/"t.db"), "data_dir": str(tmp_path)})
    df = p.get_fundamentals_df(["600519.XSHG"], date="2024-09-30")
    assert abs(df.loc["600519.XSHG","pe_ratio"] - 25.0) < 1e-6   # baostock 权威
    assert abs(df.loc["600519.XSHG","pb_ratio"] - 7.5) < 1e-6
    assert abs(df.loc["600519.XSHG","market_cap"] - 2e4) < 1     # 2e12元→2e4亿

• Step 2: 跑测试确认 FAIL

• Step 3: 实现 — 读 valuation_baostock parquet(year from date)+ static/valuation akshare;合并对齐 _FUNDAMENTAL_COLUMNS(复用 LocalParquetProvider 的 _VAL_COL_MAP / to_yi / 三表读法,import 复用):

    from .local_parquet_provider import (_VAL_COL_MAP, jq_to_file_code,
        _to_float, _or_nan, _pct_to_decimal, _FUNDAMENTAL_COLUMNS)
    from ..factors.valuation import to_yi

    def get_fundamentals_df(self, stocks, date=None) -> pd.DataFrame:
        if not stocks:
            return pd.DataFrame(columns=_FUNDAMENTAL_COLUMNS)
        date_str = self._to_date_str(date) or datetime.now().strftime("%Y-%m-%d")
        rows = [self._build_fundamental_row(s, date_str) for s in stocks]
        df = pd.DataFrame(rows, columns=_FUNDAMENTAL_COLUMNS)
        if "code" in df.columns:
            df = df.set_index("code", drop=False)
        return df

    def _read_valuation_baostock(self, year: int) -> pd.DataFrame:
        if year in self._val_bs_cache:
            return self._val_bs_cache[year]
        p = os.path.join(self.data_dir, "valuation_baostock", f"{year}.parquet")
        df = pd.read_parquet(p) if os.path.exists(p) else pd.DataFrame()
        self._val_bs_cache[year] = df
        return df

    def _build_fundamental_row(self, jq_code, date_str) -> Dict[str, Any]:
        sym, exc = jq_to_dbbardata(jq_code)
        fc = jq_to_file_code(jq_code)  # 600519.SH(static akshare 文件名)
        row: Dict[str, Any] = {"code": jq_code}
        # 1. pe/pb/ps/pcf ← valuation_baostock(baostock 权威)
        year = int(date_str[:4])
        vbs = self._read_valuation_baostock(year)
        if not vbs.empty:
            sub = vbs[(vbs["symbol"].astype(str) == sym) & (vbs["date"].astype(str) <= date_str)]
            vrow = sub.iloc[-1] if not sub.empty else None
        else:
            vrow = None
        def gbs(k):
            return _to_float(vrow.get(k)) if vrow is not None else None
        row["pe_ratio"] = _or_nan(gbs("peTTM"))
        row["pb_ratio"] = _or_nan(gbs("pbMRQ"))
        row["ps_ratio"] = _or_nan(gbs("psTTM"))
        row["pcf_ratio"] = _or_nan(gbs("pcfNcfTTM"))
        # 2. 市值/股本 + 三表 ← static akshare(复用 LocalParquetProvider 读法)
        # 复用:直接实例化 LocalParquetProvider 读 static 部分,或内联读 static/valuation
        ak_val = self._read_akshare_valuation(fc, date_str)  # 返 renamed Series
        mkt = _to_float(ak_val.get("total_market_cap")) if ak_val is not None else None
        circ = _to_float(ak_val.get("circ_market_cap")) if ak_val is not None else None
        row["market_cap"] = to_yi(mkt) if mkt else float("nan")
        row["circulating_market_cap"] = to_yi(circ) if circ else float("nan")
        # 3. 三表(income/balance)— 复用 LocalParquetProvider._read_quarter + 字段提取
        # 简化:委托一个内部 LocalParquetProvider 实例读三表部分(eps/margin/liability)
        lpp = self._get_lpp_helper()
        inc = lpp._latest_row_before(lpp._read_quarter("income", fc), "REPORT_DATE", date_str)
        bal = lpp._latest_row_before(lpp._read_quarter("balance", fc), "REPORT_DATE", date_str)
        row["eps"] = _or_nan(_to_float(inc.get("BASIC_EPS")) if inc is not None else None)
        # ... net_profit_margin/total_liability/roe 等(照 LocalParquetProvider._build_fundamental_row 逻辑)
        return row

(实现时:_get_lpp_helper() 返一个复用的 LocalParquetProvider(config) 实例读 static 三表;_read_akshare_valuation 复用 LocalParquetProvider._read_valuation。DRY:不重写三表/akshare valuation 逻辑,委托 LocalParquetProvider。pe/pb 改 baostock 源覆盖 akshare 的。)

• Step 4: 跑测试 PASS
• Step 5: 写测试 — 三表字段(eps/market_cap 全 _FUNDAMENTAL_COLUMNS 有值不 NaN)
• Step 6: 实现 + PASS
• Step 7: Commit — feat(portfolio): LocalUnifiedProvider fundamentals baostock估值+akshare市值(Task3)

---

Task 4: 辅助方法(trade_days/all_securities/security_info/current_tick/split_dividend)

Files: Modify local_unified_provider.py; Test 同文件。

• Step 1-2: 写失败测试 + FAIL — get_trade_days(count=2) 返 datetime list;get_security_info 返 display_name/start_date;get_current_tick 返 close+high_limit;get_split_dividend 返 bs_adjust_factor 事件;get_all_securities 返 dbbardata distinct symbol。

• Step 3: 实现:

  • get_trade_days: 读 dbbardata 某 symbol(如 600519)distinct datetime,filter/count。
  • get_security_info: dbbardata min/max datetime → start/end_date;display_name 从 constituent_unified code_name 或 code。
  • get_current_tick: dbbardata 最近 close + valuation_baostock 最近 pctChg → high_limit=close×1.1(ST 0.05)。
  • get_split_dividend: bs_adjust_factor → events(dividOperateDate + adjustFactor)。
  • get_all_securities: dbbardata distinct symbol → DataFrame。

• Step 4: 跑测试 PASS

• Step 5: Commit — feat(portfolio): LocalUnifiedProvider 辅助方法(Task4)

---

Task 5: 接线(init 导出 + runner build_provider 加 unified)

Files: Modify sanguo_portfolio/providers/__init__.py; Modify sanguo_portfolio/runner_backtest.py。

• Step 1: __init__.py 加导出

from .local_unified_provider import LocalUnifiedProvider
__all__ = ["SanguoMiniQmtProvider", "BaostockProvider", "LocalParquetProvider", "LocalUnifiedProvider"]

• Step 2: runner_backtest build_provider 加 unified

# parse_args choices 加 "unified"; build_provider 加分支
p.add_argument("--provider", default="local", choices=["local", "baostock", "miniqmt", "unified"], ...)
# build_provider:
from .providers import LocalUnifiedProvider
if name == "unified":
    return LocalUnifiedProvider(cfg)

• Step 3: 跑 pytest tests/portfolio/ -v 全绿(回归)
• Step 4: Commit — feat(portfolio): 接线 LocalUnifiedProvider 到 runner(Task5)

---

Task 6: 使用说明 + VPS E2E 验证

Files: Create docs/portfolio_local_unified_provider.md; VPS 跑 python -m sanguo_portfolio.runner_backtest --provider unified --start 2024-01-01 --end 2024-03-31 --max-pool 20。

• Step 1: 写使用说明 docs/portfolio_local_unified_provider.md(其他 session 直用)— 含:

  • 一句话定位(读方案A权威层/零online/治偏差)
  • 数据源映射表(每接口→哪张表/parquet)
  • 接口清单(DataProvider 接口 + get_constituent)
  • 复权说明(raw存储+消费端按bs_adjust_factor算qfq;fq参数 raw/qfq)
  • 幸存者偏差说明(constituent_unified 并集模型,治纯当前偏差,有轻微前视,date 参数忽略;中证1000/2000只快照永久gap)
  • Mac 测试(fixture,零VPS依赖)
  • 部署/运行(runner --provider unified;VPS 数据依赖 dbbardata/constituent_unified/valuation_baostock/static)
  • 已知限制(high_limit 列 NaN→prepare_stock_list 涨停识别降级;1m 无数据;三表委托 LocalParquetProvider)
  • 与旧 provider 关系(LocalParquetProvider/BaostockProvider 保留,unified 是方案A 后推荐)

• Step 2: VPS E2E — rsync 代码到 VPS,跑 --provider unified --max-pool 20 小样本回测,确认:

  • get_price 读 dbbardata 出 K 线(含退市)
  • get_index_stocks 出并集成份股
  • get_fundamentals_df 出市值+pe/pb
  • 回测不崩,有选股+指标输出

• Step 3: Commit — docs(portfolio): LocalUnifiedProvider 使用说明+VPS E2E(Task6)

---

Self-Review(plan 自检)

1. Spec 覆盖: spec §6 接口(get_daily/get_constituent/get_fundamentals/...)— get_constituent 别名✓;get_price 覆盖 get_daily+get_etf_daily(都读 dbbardata,ETF 也在);get_fundamentals_df ✓;其余 §6 方法(industry/longhubang/instrument)数据层未就绪(P1),使用说明标注 NotImplementedError。✓
2. 方案A §14 一致: dbbardata 唯一行情✓;constituent_unified 治偏差✓;valuation_baostock pe/pb✓;raw+factor 复权✓;零online✓。
3. 类型一致: _build_qfq_factor(code, conn, dates) -> Series 在 Task0/Task1 调用签名一致✓。
4. 占位扫描: Task3 的 _get_lpp_helper/_read_akshare_valuation 标了"复用 LocalParquetProvider",实现 agent 须内联或委托,不留空✓。
5. 风险: get_price 的 high_limit 列缺失(NaN)→策略 prepare_stock_list 涨停识别降级,使用说明标注(Task6)✓。

Execution Handoff

Plan complete and saved to docs/superpowers/plans/2026-07-23-local-unified-provider.md.