feat(data): 方案A 数据层落地(spec §14)— DB唯一表+治幸存者偏差+权威源

spec §14 方案A 数据层迁移完成 + E2E 验证(read_db_daily: 在市/退市治偏差/ETF 全OK):
- dbbardata('d') 1826万含退市(治回测幸存者偏差, INSERT OR REPLACE staging迁移, WHERE OHLC NOT NULL+COALESCE)
- constituent_unified 7110行/9指数(300/500/50 baostock全集 + 深证4指 akshare cni union + 中证1000/2000 snapshot)
- pe/pb 不进DB -> valuation_baostock/<year>.parquet 按年宽表(2003-2026)
- 废弃 daily_baostock_full/bs_index_constituent(rename _old 保留); 旧4 schtask disabled
- 新 schtask sanguo-bs-eod 18:05(baostock个股日线+15min+拆pe/pb DAILY_LIMIT 48000) + sanguo-xt-eod 18:40(ETF/基金xtata)
- 权威源: baostock个股日线+估值+15min+复权+300/500/50 / xtata ETF+基金+当天实时 / akshare三表+事件+深证中证成份股
- 全程备份+staging+_old保留可回滚; 脚本 audit/probe/migrate/merge/cleanup/fix_config/verify/bs_eod/xt_eod/wrapper/register_schtasks
- 待办(spec §6 使用层): LocalParquetProvider 接 constituent_unified+valuation_baostock + 实时拼接
This commit is contained in:
2026-07-23 07:20:34 +08:00
parent b270faf4b9
commit c2a89d01a4
27 changed files with 2062 additions and 0 deletions
@@ -0,0 +1,23 @@
# kill_backup_verify.ps1 — 杀卡在 integrity_check 的 backup python, 快速验证 .bak 可读
$bakProc = Get-CimInstance Win32_Process -Filter "Name='python.exe'" |
Where-Object { $_.CommandLine -like '*backup_db*' }
if ($bakProc) {
Write-Output ("kill backup python pid={0}" -f $bakProc.ProcessId)
Stop-Process -Id $bakProc.ProcessId -Force -ErrorAction SilentlyContinue
Start-Sleep 3
} else { Write-Output "backup python 已不在" }
$code = @"
import sqlite3, os
BAK = r'C:\sanguo_vnpy_v2\data\quant_trading.db.bak_20260722'
print('bak size=%.2f GB' % (os.path.getsize(BAK)/1024**3))
c = sqlite3.connect(BAK, timeout=30)
tabs = [r[0] for r in c.execute("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name")]
print('tables:', tabs)
print('backtest_stats:', c.execute('SELECT COUNT(*) FROM backtest_stats').fetchone()[0])
print('bs_adjust_factor:', c.execute('SELECT COUNT(*) FROM bs_adjust_factor').fetchone()[0])
print('bs_index_constituent:', c.execute('SELECT COUNT(*) FROM bs_index_constituent').fetchone()[0])
c.close()
print('BAK READABLE OK')
"@
$code | & 'C:\Python310\python.exe' -X utf8 -