perf(data): 四热路径日期区间SARGable化——substr(datetime,1,10)对索引列套函数打不进复合索引datetime列,每股扫全量日线史取短窗——08-25晨9:30生产实锤:momentum选股1/2(RPS池)174s未达<60s验收,阶段日志精确定罪(过滤段含双seek仅7.8s);病根=PanelFetcher的30天窗SQL每只股扫~5000行日线史取~22行,3226只≈1600万行=174s量级吻合。修=裸列datetime>=start AND datetime<end+1天排他上界(与按日期前10位比较在d裸日期/15m·5m时间戳两格式下语义严格等价,含end当日全部行排除次日),区间打进复合索引第4列每股只扫窗口行。四处同病同修:PanelFetcher宽表/PriceFetcher逐只get_price/limit-status近2根90天窗(原substr版90天下界同样打不进索引)/datareader CTA回测K线。+3测试:防回潮扫描(三文件钉死禁substr谓词)+双格式边界行为(d裸日期与15m时间戳end当日含次日排)+get_price同语义;portfolio+data_platform 644绿;待VPS探针终验计时 [vps]
CI/CD / test (push) Successful in 3s
CI/CD / nas-deploy (push) Successful in 21s
CI/CD / nas-verify (push) Successful in 8s

This commit is contained in:
2026-08-25 20:54:40 +08:00
parent 841ea1536e
commit 3304ff46b2
4 changed files with 101 additions and 13 deletions
+8 -4
View File
@@ -9,7 +9,7 @@ if _VNPY_SRC not in sys.path:
sys.path.insert(0, _VNPY_SRC)
import pandas as pd
from datetime import datetime, date
from datetime import datetime, date, timedelta
from vnpy.trader.object import BarData
from vnpy.trader.constant import Exchange, Interval
from vnpy.trader.setting import SETTINGS
@@ -148,12 +148,16 @@ def read_index_daily(code: str, start, end, cfg) -> pd.DataFrame:
conn = sqlite3.connect(db_path, timeout=30)
conn.execute("PRAGMA busy_timeout = 30000")
try:
# substr(datetime,1,10) 比日期规避混合格式(有纯日期有带时间,同 provider 模式)
# SARGable 日期区间(2026-08-25 同 provider P0 治本): 裸列 datetime>=/<,
# 右端 end+1 天排他——d 裸日期与 15m/5m 时间戳混合格式下与按日期前 10 位
# 比较语义严格等价(含 end 当日全部行);substr 版打不进复合索引 datetime
# 列, 短窗读每股扫全量日线史
end_excl = (datetime.strptime(end_str, "%Y-%m-%d") + timedelta(days=1)).strftime("%Y-%m-%d")
df = pd.read_sql(
"SELECT datetime, open_price, high_price, low_price, close_price, volume "
"FROM dbbardata WHERE symbol=? AND exchange=? AND interval='d' "
"AND substr(datetime,1,10)>=? AND substr(datetime,1,10)<=? ORDER BY datetime",
conn, params=(symbol, exchange, start_str, end_str),
"AND datetime>=? AND datetime<? ORDER BY datetime",
conn, params=(symbol, exchange, start_str, end_excl),
)
finally:
conn.close()