feat(data): 方案A 数据层落地(spec §14)— DB唯一表+治幸存者偏差+权威源

spec §14 方案A 数据层迁移完成 + E2E 验证(read_db_daily: 在市/退市治偏差/ETF 全OK):
- dbbardata('d') 1826万含退市(治回测幸存者偏差, INSERT OR REPLACE staging迁移, WHERE OHLC NOT NULL+COALESCE)
- constituent_unified 7110行/9指数(300/500/50 baostock全集 + 深证4指 akshare cni union + 中证1000/2000 snapshot)
- pe/pb 不进DB -> valuation_baostock/<year>.parquet 按年宽表(2003-2026)
- 废弃 daily_baostock_full/bs_index_constituent(rename _old 保留); 旧4 schtask disabled
- 新 schtask sanguo-bs-eod 18:05(baostock个股日线+15min+拆pe/pb DAILY_LIMIT 48000) + sanguo-xt-eod 18:40(ETF/基金xtata)
- 权威源: baostock个股日线+估值+15min+复权+300/500/50 / xtata ETF+基金+当天实时 / akshare三表+事件+深证中证成份股
- 全程备份+staging+_old保留可回滚; 脚本 audit/probe/migrate/merge/cleanup/fix_config/verify/bs_eod/xt_eod/wrapper/register_schtasks
- 待办(spec §6 使用层): LocalParquetProvider 接 constituent_unified+valuation_baostock + 实时拼接
This commit is contained in:
2026-07-23 07:20:34 +08:00
parent b270faf4b9
commit c2a89d01a4
27 changed files with 2062 additions and 0 deletions
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""fix_vps_config.py — 单元2: VPS config data_paths 统一到 C:/sanguo_vnpy_v2/data (正斜杠)。
- 备份原 config -> data_platform.yaml.bak_nas (NAS 路径版)
- data_paths 改 VPS 本地 + daily_dir 统一 qfq (消除 daily/ vs qfq/ 分叉)
- 保留其他段 (yaml safe_dump, 丢注释但语义不变)
- 改完立即调 read_parquet_daily 验证 datareader 能读本地
注: rsync 同步仓库 config 会覆盖 VPS config (NAS 版)。部署 rsync 须 --exclude config,
或 VPS 用独立 config 文件。此处先改 VPS config 让回测读本地, 部署排除留作 TODO。
"""
import shutil
import sys
from pathlib import Path
import yaml
CFG = Path(r"C:\sanguo_vnpy_v2\config\data_platform.yaml")
bak = CFG.with_suffix(".yaml.bak_nas")
if not bak.exists():
shutil.copy2(CFG, bak)
print(f"backup -> {bak.name}")
else:
print(f"backup 已存在: {bak.name}")
d = yaml.safe_load(CFG.read_text(encoding="utf-8"))
print("OLD data_paths:", d.get("data_paths"))
d["data_paths"] = {
"daily_dir": "C:/sanguo_vnpy_v2/data/qfq",
"raw_dir": "C:/sanguo_vnpy_v2/data/raw",
"qfq_dir": "C:/sanguo_vnpy_v2/data/qfq",
"minute_15_dir": "C:/sanguo_vnpy_v2/data/minute_15",
"minute_15_qfq_dir": "C:/sanguo_vnpy_v2/data/minute_15",
"minute_15_raw_dir": "C:/sanguo_vnpy_v2/data/minute_15",
"vnpy_db": "C:/sanguo_vnpy_v2/data/quant_trading.db",
"stock_list": "C:/sanguo_vnpy_v2/data/stock_basic_info.csv",
}
CFG.write_text(yaml.safe_dump(d, allow_unicode=True, sort_keys=False), encoding="utf-8")
print("NEW data_paths:", d["data_paths"])
# 验证 datareader 读本地
sys.path.insert(0, r"C:\sanguo_vnpy_v2")
from sanguo_data.config import load_config
from sanguo_data.datareader import read_parquet_daily
cfg = load_config()
print("cfg daily_dir:", cfg.data_paths["daily_dir"])
print("cfg vnpy_db:", cfg.data_paths["vnpy_db"])
bars = read_parquet_daily("600519", "2026-07-01", "2026-07-22", cfg)
print("600519 bars:", len(bars),
"last:", bars[-1].datetime.date() if bars else "NONE")
print("CONFIG FIX DONE" if bars else "CONFIG FIX WARN: 0 bars")