refactor(data): 5m改同库设计——用户二次拍板2026-08-20:放弃独立库,5m直接写NAS副本主库dbbardata表(interval='5m'与镜像行d/15m唯一键正交=结构性互不干扰)——①bs_5m_eod缺省库改/volume1/stock/sanguo_vnpy_v2/data_backup/quant_trading.db(=同步目标=NAS回测provider读的库portfolio_worker.py:180),NAS就地5m回测/回放reader零改动,未来VPS扩容导interval='5m'反向merge一次迁移②同库安全性实证:同步链export不带id列+NAS侧INSERT OR IGNORE按UNIQUE去重→键永不相交双向碰不到;纯merge无文件覆盖NAS本地行不会被删;id由NAS AUTOINCREMENT自分配无跨库冲突③ensure_schema防重复索引:PRAGMA index_list/index_info识别已有同列唯一索引(NAS侧uq_dbbardata)跳过,免数GB无谓开销+双倍写放大④merge_increment补busy_timeout=60000(副本库从此有第二个写者,无它撞写锁当场locked失败,60s与5m侧对齐)⑤写锁竞争说明:双写者WAL+busy_timeout串行化,与每日增量merge(秒级)重叠单股失败次日LOOKBACK=7自愈;+2测试(缺省库契约钉死/副本已有索引不重复建);40+153绿(data_platform 155) [nas]

2026-08-20 10:10:04 +08:00
parent a6f924e950
commit 20ff70e85d
3 changed files with 77 additions and 18 deletions
@@ -31,7 +31,10 @@ def main():
    conn = sqlite3.connect(args.db)
    cur = conn.cursor()
    # 大表(1.8亿行+)merge 提速: 默认2MB cache 致 UNIQUE索引(数GB)全磁盘IO, merge>120s超时
+    # busy_timeout(2026-08-20): NAS 副本从此有第二个写者(bs_5m_eod 5分钟线),
+    # 无 busy_timeout 撞写锁会当场 "database is locked" 失败; 60s 与 5m 侧对齐。
    for _p in ("PRAGMA journal_mode=WAL", "PRAGMA synchronous=NORMAL",
+               "PRAGMA busy_timeout=60000",
               "PRAGMA cache_size=-500000", "PRAGMA temp_store=MEMORY"):
        cur.execute(_p)
    cur.execute(SCHEMA)