From dfa156a39e585a615a929d35a385b0d5fe2bf467 Mon Sep 17 00:00:00 2001 From: claude_dev Date: Sun, 6 Sep 2026 20:13:21 +0800 Subject: [PATCH] =?UTF-8?q?docs(data):=20=C2=A718.8=E8=A1=A55m=E5=88=86?= =?UTF-8?q?=E9=92=9F=E7=BA=BF=E8=A1=8C+NAS=E4=B8=A4=E4=BE=8B=E5=A4=96?= =?UTF-8?q?=E5=AD=98=E6=94=BE=E4=BF=AE=E6=AD=A3=E2=80=94=E2=80=94=E7=94=A8?= =?UTF-8?q?=E6=88=B7=E9=AA=8C=E6=94=B6=E7=82=B9=E5=90=8D=E6=89=BE=E5=9B=9E?= =?UTF-8?q?=20[nas]=E2=80=94=E2=80=94=E7=94=A8=E6=88=B7=E8=AE=B0=E5=BE=97?= =?UTF-8?q?=E3=80=8CNAS=E6=AF=94VPS=E5=A4=9A=E4=B8=80=E7=A7=8D=E6=95=B0?= =?UTF-8?q?=E6=8D=AE=3D=E5=88=86=E9=92=9F=E7=BA=BF=E3=80=8D=E5=AE=8C?= =?UTF-8?q?=E5=85=A8=E6=AD=A3=E7=A1=AE:=E2=91=A0sanguo-5m(DSM=20id=3D7,roo?= =?UTF-8?q?t,00:05)=E5=9C=A8=E8=B7=91,baostock=205min=E5=A2=9E=E9=87=8F+?= =?UTF-8?q?=E5=85=A8=E9=87=8F=E5=9B=9E=E8=A1=A52020=E8=B5=B7=E5=90=AB?= =?UTF-8?q?=E9=80=80=E5=B8=825547=E5=8F=AA,=E7=8B=AC=E7=AB=8B=E4=B8=80?= =?UTF-8?q?=E6=AC=A1=E6=80=A7=E5=AE=B9=E5=99=A8docker=20run=20--rm?= =?UTF-8?q?=E8=B7=91lock-aligned=E9=95=9C=E5=83=8F(08-22=E6=B2=BBCI?= =?UTF-8?q?=E9=87=8D=E5=90=AF=E4=B8=BB=E5=AE=B9=E5=99=A8=E6=9D=80=E5=9C=A8?= =?UTF-8?q?=E8=B7=91=E4=BB=BB=E5=8A=A1),=E8=90=BD=E7=82=B9=3DNAS=E5=89=AF?= =?UTF-8?q?=E6=9C=AC=E5=BA=93dbbardata('5m')=3D5m=E5=94=AF=E4=B8=80?= =?UTF-8?q?=E6=9D=83=E5=A8=81(VPS=E4=BE=A7'5m'=E6=98=AF=E7=A9=BA=E5=8D=A0?= =?UTF-8?q?=E4=BD=8D);=E2=91=A1minute=5Fkline/2.1G=3Dcfdaily=E6=97=B6?= =?UTF-8?q?=E4=BB=A3=E5=A4=96=E9=83=A81min/5min/15min,=E6=97=A0=E4=BB=BB?= =?UTF-8?q?=E5=8A=A1,1min=E4=B8=BA=E5=85=A8=E4=BD=93=E7=B3=BB=E5=94=AF?= =?UTF-8?q?=E4=B8=801=E5=88=86=E9=92=9F=E7=B2=92=E5=BA=A6;=E6=96=87?= =?UTF-8?q?=E6=A1=A3=E5=80=BA=E5=AE=9A=E6=80=A7:=C2=A714.2=E3=80=8C5min?= =?UTF-8?q?=E5=8D=A0=E4=BD=8D/=E4=B8=8D=E8=AE=BEschtask=E3=80=8D=E4=B8=BA0?= =?UTF-8?q?7-22=E5=BF=AB=E7=85=A7,08-20=E8=90=BD=E5=9C=B05m=E7=AE=A1?= =?UTF-8?q?=E7=BA=BF=E6=97=B6=E6=9C=AA=E5=9B=9E=E5=86=99=E8=9E=8D=E5=90=88?= =?UTF-8?q?spec(run=5F5m.sh=E5=A4=B4=E6=B3=A8=E9=87=8A+memory=E6=89=8D?= =?UTF-8?q?=E8=AE=B0=E5=BD=95),=E6=9C=AC=E8=A1=A8=E7=8E=B0=E8=A1=A5;?= =?UTF-8?q?=E5=AD=98=E6=94=BE=E4=BF=AE=E6=AD=A3=3D=E6=95=B0=E5=80=BC?= =?UTF-8?q?=E6=9D=83=E5=A8=81VPS+=E4=B8=A4=E4=BE=8B=E5=A4=96=E5=9C=A8NAS(5?= =?UTF-8?q?m+=E8=AF=AD=E6=96=99)=20[nas]?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-07-21-data-source-fusion-design.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-07-21-data-source-fusion-design.md b/docs/superpowers/specs/2026-07-21-data-source-fusion-design.md index 92d34e6..46431db 100644 --- a/docs/superpowers/specs/2026-07-21-data-source-fusion-design.md +++ b/docs/superpowers/specs/2026-07-21-data-source-fusion-design.md @@ -582,7 +582,8 @@ T3 DSM 任务注册(admin 身份,零 docker)+首周人工盯跑 → T4 backfill **一条规则,不是几套方案**:每类数据**唯一权威源+唯一权威位置**(§14.2 既定原则),**采集发起位置 跟随权威位置**——数值类(行情/估值/报表/事件面板)权威=VPS,既有任务家族不变;文本语料类(新闻/ 公告)权威=NAS(本节新增,用户三约束唯一决定:不上VPS+东财IP限流冲突+不绑docker)。两域方法论 -同款:单脚本注册表红线/慢爬防封/断点 marker/schtask 语义/容量闸门。§14.5 为 07-22 历史快照, +同款:单脚本注册表红线/慢爬防封/断点 marker/schtask 语义/容量闸门。**「数值类权威=VPS」有 +一个在跑的例外=5min,其权威在 NAS 副本库(见下表 sanguo-5m 行)**。§14.5 为 07-22 历史快照, 采集任务现势以本表为准(08-20 起 bs-eod+bs-fund 已并 sanguo-bs-daily、akshare 已拆 daily/weekly/monthly/events 三频率+ak-vintage 自检): @@ -597,12 +598,18 @@ daily/weekly/monthly/events 三频率+ak-vintage 自检): | sanguo-ak-monthly | VPS | 首周六 | 三表全量马拉松(55h 跑道) | static/ | | ak-vintage 自检 | VPS | 09:00 | 缺日/错标防线 | vintage_status.json | | NAS sync 链(DSM id=2,root) | NAS | 03:00 | VPS→NAS 副本拉取(7 域) | /volume1/stock/… | +| sanguo-5m(DSM id=7,root) | NAS | 00:05 | baostock 5min 增量+全量回补(独立一次性容器 `docker run --rm` 跑 lock-aligned 镜像,08-22 治「CI 重启主容器杀在跑任务」;窗口 2020 起,含退市 5547 只,runner=/volume1/stock/sanguo_5m) | **NAS 副本库 dbbardata('5m')=5m 唯一权威** | | **sanguo-corpus-daily(DSM,admin)** | **NAS** | **06:00** | **巨潮公告+东财新闻(本设计)** | **/volume1/stock/corpus/** | -**存放位置全景**:**VPS `data/`=数值权威**(dbbardata 26G+static/+valuation_baostock/); -**NAS `/volume1/stock/`=数值副本+大体积历史+语料唯一权威**(data_backup/quant_trading.db、 -A股数据/、minute_kline/、corpus/);Mac=开发 fixture 无生产数据。corpus 不是第三种根—— -就是既有 `/volume1/stock` 数据根下的新子目录,与 CLAUDE.md「NAS 部署目录」表一致。 +**存放位置全景**:**数值权威=VPS `data/`**(dbbardata 26G+static/+valuation_baostock/), +**两个例外在 NAS**:①**5min——权威=NAS 副本库**`/volume1/stock/sanguo_vnpy_v2/data_backup/ +quant_trading.db` 的 dbbardata('5m')(VPS 侧 '5m' 是空占位;§14.2「5min 占位/不设 schtask」 +为 07-22 历史快照,08-20 起由 DSM 00:05 任务实际落地,**当时未回写本文档=文档债,本表现补** +——用户 09-06 验收总表时点名找回);②**语料=NAS `/volume1/stock/corpus/`(本设计)**。 +NAS 其余=数值副本(sync 链产物)+大体积历史(minute_kline/ 2.1G=cfdaily 时代外部 1min/5min/ +15min,无任务在跑,其中 **1min 为全体系唯一 1 分钟粒度**;A股数据/;LLM-Models 等非本项目)。 +Mac=开发 fixture 无生产数据。corpus 不是第三种根——就是既有 `/volume1/stock` 数据根下的 +新子目录,与 CLAUDE.md「NAS 部署目录」表一致。 ## 参考(调查来源)