claude_dev
|
4803d614a4
|
docs: 迁移评估修正R1/R2——同客户端澄清(模拟账号+不勾独立交易+运行模式实盘) [nas]
CI/CD / test (push) Failing after 13m29s
CI/CD / nas-deploy (push) Has been skipped
CI/CD / nas-verify (push) Has been skipped
|
2026-09-07 13:22:33 +08:00 |
|
claude_dev
|
ce10547a4a
|
docs: miniQMT停服迁移评估(大QMT+xtquant_big_convert, deadline 0918) [nas]
CI/CD / test (push) Successful in 26s
CI/CD / nas-deploy (push) Successful in 26s
CI/CD / nas-verify (push) Successful in 9s
|
2026-09-07 13:14:26 +08:00 |
|
claude_dev
|
5ab7ce8102
|
fix(nas): run_5m 一次性容器加 --no-healthcheck,根除 unhealthy 误报 [nas]
CI/CD / test (push) Successful in 24s
CI/CD / nas-deploy (push) Successful in 10s
CI/CD / nas-verify (push) Successful in 10s
|
2026-09-07 12:33:18 +08:00 |
|
claude_dev
|
2ac4dfc810
|
fix(data): wrapper 加 --no-healthcheck(镜像 healthcheck 是 web 服务专用,爬虫容器恒 unhealthy 误报) [nas]
CI/CD / test (push) Failing after 10m37s
CI/CD / nas-deploy (push) Has been skipped
CI/CD / nas-verify (push) Has been skipped
|
2026-09-07 12:30:26 +08:00 |
|
claude_dev
|
d426619a96
|
fix(data): stats 改域内首末请求间隔(lane 总时长跨段失真) [nas]
CI/CD / test (push) Successful in 6s
CI/CD / nas-deploy (push) Successful in 7s
CI/CD / nas-verify (push) Successful in 9s
09-07 首跑 daily 收官 stats 乌龙: cninfo n=3 avg=3621s(被全文段时长摊薄)/
eastmoney 2.26s(真实1.25s); 改 DomainClient 首末请求时间戳 avg=(t_last-t_first)/(n-1),
后续 run 起直读真节奏
|
2026-09-07 12:27:46 +08:00 |
|
claude_dev
|
3b9b1e6634
|
fix(data): 账本启动对账自愈——分区有/账本无缺口永久闭合 [nas]
CI/CD / test (push) Successful in 3s
CI/CD / nas-deploy (push) Successful in 8s
CI/CD / nas-verify (push) Successful in 10s
09-07 巡检 verify FAIL 实锤: 旧代码部分段(news/fulltext/pdf)无 flush 计数器,
docker stop 的腿把内存账本丢掉(ann 差4.5k/news 差9.9w;分区数据完好 dups=0,
近窗索引兜底着去重但账本缺口30天后成理论重拉窗)。修=两 lane 启动时
_reconcile_ledgers: 近30日分区 key 对账补进账本并落盘,幂等;
新代码批量提交已让缺口收敛到单commit周期,对账再兜历史+未来任何崩溃窗
测试 55(新增对账自愈幂等钉);全套 272 绿
|
2026-09-07 07:30:55 +08:00 |
|
claude_dev
|
efcdc8d8e0
|
fix(data): 分区改多 part 追加写+批量提交,根治 day-1 O(n²) 劣化 [nas]
CI/CD / test (push) Successful in 25s
CI/CD / nas-deploy (push) Successful in 10s
CI/CD / nas-verify (push) Successful in 15s
09-07 首跑实锤: append_parquet 每 unit 整读整写当日唯一 part-0, news 段
13万+行后 5.3s/股(应1.15s)且随行数恶化,10:30 墙钟必跑不完且逐日更糟。
修(两层):
- DayBuffer: 当日新行内存缓冲→flush 落新 part-N.parquet(只写新行,
旧数据永不重写);id 去重职责收敛到调用方(账本+近窗双层,原就存在)
- Ctx 批量提交: unit_done 延后 marker 入批,每 FLUSH_EVERY unit 一次
commit=数据→账本→marker 依次落盘(崩溃=整批重拉,绝不产生
marker-done-但数据缺的洞);run_lane 段首清缓冲/finally 兜底 commit
- ③全文回补 known 检查补近窗兜底(账本 flush 崩溃窗防重)
- readers 全改 part-* glob(RecentIndex/_iter_partitions/verify_corpus)
- 测试 54:新增 多part语义/flush崩溃一致性/批量提交次序 三钉
|
2026-09-07 07:12:48 +08:00 |
|
claude_dev
|
c5db293b46
|
fix(data): vintage 测试周一翻转——快照族 hole 取最近工作日 [nas]
CI/CD / test (push) Successful in 28s
CI/CD / nas-deploy (push) Successful in 13s
CI/CD / nas-verify (push) Successful in 14s
CI#1090/1091 实锤(周一 02:46 挂、同日 01:35 绿=时间窗触发, corpus 零涉及):
hole=倒数第2工作日, 周一回看=周四, 4 日历日 > service 3 日新鲜窗 → 永不告警。
service 窗本身正确(洞在新鲜窗内每日晨检都会报), 仅测试取数翻转;改取最近
1 个工作日, 日历距周一=3/其余≤1, 恒入窗。解锁 308d493/aefa341 部署链
|
2026-09-07 03:18:29 +08:00 |
|
claude_dev
|
aefa341cc9
|
docs(data): spec§18.9 实施记录——四项偏差+可自主验收项全部实证入档 [nas]
CI/CD / test (push) Failing after 0s
CI/CD / nas-deploy (push) Has been skipped
CI/CD / nas-verify (push) Has been skipped
T1/T2/T4/T5+T6 可自主项(主容器restart免疫/docker stop断点续跑/锁让路/
容量闸门真卷演练/verify PASS)入档;四项实测倒逼偏差(szse_stock.json池/
flock跨容器锁/业绩快报标题谓词/年序新→旧);待数据累积项与运维要点留痕
|
2026-09-07 02:46:47 +08:00 |
|
claude_dev
|
308d4931e3
|
feat(data): T6 验收器 verify_corpus.py+每域请求统计日志 [nas]
CI/CD / test (push) Failing after 26s
CI/CD / nas-deploy (push) Has been skipped
CI/CD / nas-verify (push) Has been skipped
- corpus_download 收尾补 stats 行(每域 n_requests/avg_interval/[429]/[cooled])
=§18.5「日志限速合规」验收依据(平均间隔≥设定值可直读)
- verify_corpus.py 只读验收器(零写入零网络,容器内跑): 三域分区/行数/键重复/
账本一致性(ledger<uniq=FAIL)/PDF 索引↔文件/marker/depth/容量/最近日志尾部;
NAS 真数据首验 PASS(三域 dups=0, ledger=uniq, 索引=文件, 644G)
- 晨检一条命令复用(首周人工盯跑工具)
|
2026-09-07 01:31:00 +08:00 |
|
claude_dev
|
594e2efbf5
|
fix(data): fulltext 段补 --limit 预算门(三跑冒烟7分钟马拉松) [nas]
CI/CD / test (push) Successful in 23s
CI/CD / nas-deploy (push) Successful in 8s
CI/CD / nas-verify (push) Successful in 8s
_run_fulltext 漏门:new_articles 跨股去重后仍可远超 N(冒烟 2 股×100条
→~150 art @2.3s);真实运行有墙钟兜底故非事故,冒烟语义补齐+回归钉死
|
2026-09-06 23:56:05 +08:00 |
|
claude_dev
|
407fcec513
|
fix(data): wrapper rc 聚合真失败优先(1/2>3),勿让墙钟码吞失败码 [nas]
CI/CD / test (push) Successful in 26s
CI/CD / nas-deploy (push) Successful in 25s
CI/CD / nas-verify (push) Successful in 10s
|
2026-09-06 23:48:34 +08:00 |
|
claude_dev
|
2fa678671c
|
feat(data): backfill ①年序翻转新→旧(近年优先供factor) [nas]
CI/CD / test (push) Successful in 28s
CI/CD / nas-deploy (push) Successful in 2s
CI/CD / nas-verify (push) Successful in 6s
spec§18.3 字面「2000→今」与其自设验收「回补~1周后即可供factor事件回测」
矛盾(升序前8天只有2000-2002老数据,近年最后才到);翻转为年DESC同PDF④
哲学: day1 即产当年数据,~1周完成近3-4年=事件因子可用,全量完结不受影响
(§18.5「600519 年报2001起逐年可查」照过)。空年照旧合法done零成本
含 cosmetic: reset_stage 归位 docstring 之后(前次批量插入错位)
|
2026-09-06 23:47:57 +08:00 |
|
claude_dev
|
ae6e2d92d0
|
fix(data): corpus 全 fetcher 补 UA+jsonp 解析防御 [nas]
CI/CD / test (push) Successful in 20s
CI/CD / nas-deploy (push) Successful in 5s
CI/CD / nas-verify (push) Successful in 7s
NAS 冒烟二跑实锤: search-api 对无 UA 请求返 200+空 body(金丝雀当时带 UA 故
未暴露)→s.index('(') ValueError 裸崩。修: 统一 UA 常量六处 fetcher 全带;
jsonp 非 cb(...) 形态 → TransportError(带响应片段,计入域名连续错走冷却
而非裸崩);容器内 A/B 实测 no-UA='' vs withUA=正常610条
|
2026-09-06 23:44:48 +08:00 |
|
claude_dev
|
284ac472b1
|
fix(data): wrapper daily 加 --until 10:30+rc3 续走 backfill [nas]
CI/CD / test (push) Successful in 5s
CI/CD / nas-deploy (push) Successful in 25s
CI/CD / nas-verify (push) Successful in 14s
day-1 预算问题(首日 news_meta 空→recent-100 全新→全文段 ~15万 art 洪峰):
daily 无墙钟会闯夜间带违反 00:30-17:30 硬约束。daily --until 10:30=spec
4.5h 预算;rc=0|3 都续走 backfill(墙钟截断的全文残余由 ③ 账本反扫当日
兜底无洞);rc=1|2 跳过次日自愈
|
2026-09-06 23:43:00 +08:00 |
|
claude_dev
|
8ff3c23e41
|
docs: CLAUDE.md 语料库行对齐容器终版(零docker旧表述清除) [nas]
CI/CD / test (push) Successful in 26s
CI/CD / nas-deploy (push) Successful in 27s
CI/CD / nas-verify (push) Successful in 9s
|
2026-09-06 23:41:36 +08:00 |
|
claude_dev
|
89da5f71e0
|
fix(data): corpus --limit 改每段独立预算+pdf五类循环补门控 [nas]
CI/CD / test (push) Successful in 24s
CI/CD / nas-deploy (push) Successful in 0s
CI/CD / nas-verify (push) Successful in 8s
NAS 真实冒烟(--limit 2)假死 12min 事故根因(x2):
①预算共享: ann 段吃光 limit 后 news/fulltext 即刻 break(假死其一);
②pdf 五类查询循环完全无门控无计数 → 全池 6168 股 @1.3s 扫描 ~2.2h,
周日窗口 0 公告 0 日志=假死其二(docker stop 收尸, log 停在 lane start)
修: Ctx.stage_units+reset_stage()+budget_exhausted() 八段入口重置;
五类循环计数即耗预算;回归测试钉死(fivecat 调用数=预算数,非全池)
连带修正 smoke 语义=文档承诺的「每段最多 N unit」;52+269 全绿
|
2026-09-06 23:40:53 +08:00 |
|
claude_dev
|
1c3a61eb12
|
feat(data): T5 backfill ③全文 ④PDF+容量闸门 [nas]
CI/CD / test (push) Successful in 28s
CI/CD / nas-deploy (push) Successful in 2s
CI/CD / nas-verify (push) Successful in 6s
spec §18.7 T5(backfill lane 补完,四段齐):
- ③新闻全文回补: 扫 news_meta 库存→未入账 art 补正文;账本即进度(免
260万 marker 文件);404=墓碑行 content_text=None 入账不重试;文章域独立
限速 2s(RATE['article'],spec @2/s 慢车道)
- ④PDF六类回补(新→旧): ann_meta 库存标题谓词筛六类→ann_time desc;
文件在=跳过(文件即进度);404 墓碑 state/pdf_missing.json;索引复用
_append_pdf_index(daily/backfill 同管道)
- 容量闸门: 段首+每50GB 自查 _disk_free_gb,<200G 暂停 PDF 段+落
capacity_paused.json 告警(其余段不受影响,空间回升自动续)
- 测试51个(root fixture 默认 _disk_free_gb=645:曾吃 Mac 真实磁盘
8G 余量误触发闸门);全套 268 绿
|
2026-09-06 23:24:29 +08:00 |
|
claude_dev
|
10c486c64d
|
feat(data): T4 backfill lane ①ann_meta ②news_meta 落地 [nas]
CI/CD / test (push) Successful in 23s
CI/CD / nas-deploy (push) Successful in 26s
CI/CD / nas-verify (push) Successful in 6s
spec §18.7 T4(unit marker 断点+--limit 冒烟不落marker+墙钟rc3 全链复用):
- ①ann_meta 回补: unit=(stock,year) 2000→今年逐年按池序, 空年=合法done
(探测优先);今年窗口右端=today 相对;~15万 unit @1s≈3-6 跑日
- ②news_meta 回补: np-listapi unit=(stock,page) 翻到空页止;空页+page≥200
双止条件都落 news_end 哨兵(到头股零重扫);mkt前缀 6→1/其余→0
- 共用 _absorb_new(daily/backfill 同一条 id 去重+追加管道)
- 测试 46 个;修 wiring 网络泄漏:_wire_backfill_ann 曾漏锁
fetch_news_backfill_page 致 ann 测试真打 np-listapi(40s/真出网,
零网络铁律违反)——补锁后套件 159s→1.3s;全套 263 绿
|
2026-09-06 23:19:50 +08:00 |
|
claude_dev
|
e22f0e0aa8
|
feat(data): T2 corpus_download.py daily lane+断点+锁+36测全绿 [nas]
CI/CD / test (push) Successful in 26s
CI/CD / nas-deploy (push) Failing after 14m50s
CI/CD / nas-verify (push) Has been skipped
spec §18.7 T2(corpus 栈唯一新脚本,单脚本红线):
- daily 四段: 公告增量(巨潮 T-1~T 两天窗自愈)/新闻增量(search-api 带摘要,
(art_code,stock_code) 复合键=同稿多股 meta 多行)/新闻全文(昨日新 art_code,
art_code 全局一条)/当日 PDF(五类服务端码 category 码+业绩快报标题谓词——
实测 yjkb 码不存在,无效码被服务端静默忽略返回全量,故走客户端标题过滤)
- 断点: unit marker tmp+rename;--until 墙钟=完成当前 unit 后 rc=3;--limit 冒烟
绝不落 marker(5m 09-17 教训)
- 单实例锁: fcntl.flock(state/corpus.lock)——跨容器语义(bind mount 同 inode,
内核持有/进程死自动释放);5m 的 pid 活性检测在 docker run --rm 各自独立
PID namespace 下失效,corpus 换 flock(让路契约同款,锁忙 rc=3)
- id 去重双层: IdLedger 全局账本(state/ids_*.parquet int64 hash,满足 §18.5
announcement_id 全局无重)+RecentIndex 近30日分区索引(兜账本flush周期崩溃窗)
- 失败语义: 传输错 unit 不标done;真空=合法done;4xx确定性跳过不烧重试;
域名连续错×10或429冷却它域继续;rc 0完成/1致命/2限流/3墙钟或锁
- 股票池: 巨潮 szse_stock.json(实测含退市000003+北交920001,6247条)A前缀过滤,
非NAS副本库——orgId 为按股查询硬前提且退市覆盖更全(spec 偏差T6回写)
- 测试 36 个全 mock 零网络零写死日期;全套 253 绿
|
2026-09-06 23:00:42 +08:00 |
|
claude_dev
|
62a3a65419
|
feat(data): T1 语料库 wrapper run_corpus.sh+连通金丝雀全绿 [nas]
CI/CD / test (push) Successful in 25s
CI/CD / nas-deploy (push) Successful in 26s
CI/CD / nas-verify (push) Successful in 7s
spec §18.7 T1(sanguo-corpus 一次性容器,统一 5m 模式):
- repo 规范副本 scripts/nas_sync/run_corpus_standalone.sh(同 run_5m_standalone.sh
先例),部署位=/volume1/stock/corpus/run_corpus.sh(宿主文件 CI 不覆盖,scp -O 同步)
- 两改良落地:--name sanguo-corpus(杀=docker stop)+--user 1024:100(admin 属主)
- 新发现硬需求 --group-add 101:/volume1/stock Synology ACL 只授权
group:administrators,容器同 uid 无组 101 读文件 Permission denied
(POSIX rwxrwxrwx 被 NFSv4 ACL 覆盖);补组后读/写/继承全绿
- 两 lane 串行(daily 成功才跑 backfill --until 17:30),rc 聚合 daily 优先
- 金丝雀(logs/canary_t1.log):python 3.10.20+pandas/pyarrow/requests 齐备;
--user 写=uid1024 属主;四域出网 200=巨潮 query/巨潮 PDF/np-listapi/
search-api(content 摘要字段在,akshare 同款端点契约验证)
- 目录骨架 admin 属主:news_meta/news_fulltext/ann_meta/ann_pdf/state/logs
|
2026-09-06 22:40:25 +08:00 |
|
claude_dev
|
20ecd711f6
|
docs(data): spec§18三默认拍板入档+开工批准 [nas]——用户「都按你说的来吧」:①股票池=全A含退市~5547只,清单源=NAS副本库distinct symbol(本地零网络天然含退市,同dbbardata治幸存者偏差原则;退市股新闻层自然空,公告层=事件因子回测原料);②PDF类别=六类(年报/半年报/一季报/三季报/业绩预告/业绩快报≈45GB/年,权益分派增发等不进但元数据层仍全量);③时长预期知悉(T1-T3≈1-2个session工作日/ann_meta回补~1周后可供factor事件因子回测/全文PDF月级慢车道);开工已批,下一步T1=wrapper+金丝雀 [nas]
CI/CD / test (push) Successful in 7s
CI/CD / nas-deploy (push) Successful in 2s
CI/CD / nas-verify (push) Successful in 7s
|
2026-09-06 22:27:03 +08:00 |
|
claude_dev
|
fec3e8e221
|
docs(data): spec§18终版——统一到sanguo-5m一次性容器模式+断点续传互借三件 [nas]——用户终拍板「统一到5min吧」:①运行时改docker run --rm同5m全款(bash wrapper=/volume1/stock/corpus/run_corpus.sh宿主文件CI不覆盖同5m部署纪律;镜像lock-aligned版本自动跟随CI;弱依赖=daemon+tag不改名与5m同款已跑一月实证),corpus两改进=DSM任务admin跑(实测admin可docker ps)+--user admin-uid输出文件admin属主+--name sanguo-corpus运维锚点(docker stop一条命令杀);反向统一硬阻塞入档=5m写的data_backup库是03:00链root属主产物admin venv写不进;统一后全图=VPS schtask家族(数值)/NAS docker run --rm(5m+corpus对外爬取)+sync链(内部搬运)机器内一种物理下限;②断点续传对比互借(bs_5m_eod.py盘点):corpus借三件=单实例锁pid活性检测(原设计真缺口,防DSM定时与手动补跑竞写state)+退出码细分(0完成/1致命/2限流让路/3墙钟自停对齐5m语义)+state损坏幂等降级;5m反向不改造(整片推进粒度粗但回灌已完结+OR REPLACE幂等+LOOKBACK自愈,改造无未来收益),仅两个顺手项下次动run_5m.sh一并(加--name/头注释时点19:05→00:05修正);验收5/5b改docker stop+主容器restart不影响实证;T1改wrapper+金丝雀 [nas]
CI/CD / test (push) Successful in 25s
CI/CD / nas-deploy (push) Successful in 2s
CI/CD / nas-verify (push) Successful in 6s
|
2026-09-06 21:42:54 +08:00 |
|
claude_dev
|
8aa3a3d2fc
|
docs(data): spec§18补与sanguo-5m独立容器方案差异对比 [nas]——应用户问「5分独立容器和现在实现有何不同」:同治「不被主容器重启波及」但深度不同——5m=docker家族内(docker run --rm一次性容器,仍依赖daemon+镜像+tag不改名,优点=镜像环境完整+版本自动跟随主容器),corpus=零docker(按用户指令走到底);5m当年未走到底合理(bs_5m_eod依赖面=整个镜像环境含baostock,宿主3.8 venv装不齐),corpus依赖面收敛3通用包才有条件(代价=venv冻结版本漂移需手动pip,T1锁版本缓解);5m不迁(在跑稳定/迁移需重验/没坏不动),但其为存量docker依赖点,NAS docker大升级时需人工盯届时再评估 [nas]
CI/CD / test (push) Successful in 5s
CI/CD / nas-deploy (push) Successful in 26s
CI/CD / nas-verify (push) Successful in 8s
|
2026-09-06 20:19:04 +08:00 |
|
claude_dev
|
dfa156a39e
|
docs(data): §18.8补5m分钟线行+NAS两例外存放修正——用户验收点名找回 [nas]——用户记得「NAS比VPS多一种数据=分钟线」完全正确:①sanguo-5m(DSM id=7,root,00:05)在跑,baostock 5min增量+全量回补2020起含退市5547只,独立一次性容器docker run --rm跑lock-aligned镜像(08-22治CI重启主容器杀在跑任务),落点=NAS副本库dbbardata('5m')=5m唯一权威(VPS侧'5m'是空占位);②minute_kline/2.1G=cfdaily时代外部1min/5min/15min,无任务,1min为全体系唯一1分钟粒度;文档债定性:§14.2「5min占位/不设schtask」为07-22快照,08-20落地5m管线时未回写融合spec(run_5m.sh头注释+memory才记录),本表现补;存放修正=数值权威VPS+两例外在NAS(5m+语料) [nas]
CI/CD / test (push) Successful in 21s
CI/CD / nas-deploy (push) Successful in 1s
CI/CD / nas-verify (push) Successful in 7s
|
2026-09-06 20:13:21 +08:00 |
|
claude_dev
|
fcc3a2e0fa
|
docs(data): spec§18「root=和旧数据一致」前提实测推翻 [nas]——/volume1/stock属主实测=三种混合:cfdaily(A股数据/minute_kline/knowledge_base,用户记忆中的之前基本面数据恰是cfdaily非root)/root(03:00链+sanguo_5m)/admin(_dbuild/gitea/frp等);文件全rwxrwxrwx+ACL无消费方依赖属主;真正一致在方法论层(schtask语义/marker/慢爬/单脚本红线)两案同款;唯一实质差异=运维通道(admin ssh可杀可装,root全走DSM一次性任务或交互sudo自动化不可达);终判admin,用户仍拍root亦可(代价=venv引导与kill全人工) [nas]
CI/CD / test (push) Successful in 27s
CI/CD / nas-deploy (push) Successful in 24s
CI/CD / nas-verify (push) Successful in 5s
|
2026-09-06 20:09:13 +08:00 |
|
claude_dev
|
7e9319b166
|
docs(data): spec§18 root vs admin 终局论证入档 [nas]——应用户「root的数据不可修改设计更好吗」:root属主写保护=半吊子幻觉(admin不能覆写但能删,删看目录权限rwxrwxrwx,03:00链先rm再拉workaround实证),防住原地改防不住删了重建;root跑采集代价全天候=root进程admin杀不动(Synology sudo交互密码自动化做不了,本设计验收含kill-9续跑项root方案连测试都无法执行)+venv装包盯跑全要root陪跑+外网爬虫最大权限;不可改性交给正确工具=append-only协议(dt分区只追加/tmp+rename/id键控)+可再生性(源站都在断点可重拉,语料=全体系唯一可完全再生数据类),如需防篡改层验收后加DSM快照/每周tar+校验和(对任何属主生效)——结论admin落地 [nas]
CI/CD / test (push) Successful in 25s
CI/CD / nas-deploy (push) Successful in 8s
CI/CD / nas-verify (push) Successful in 8s
|
2026-09-06 20:07:17 +08:00 |
|
claude_dev
|
3d3ddb5f57
|
docs(data): 全局下载方案与存放位置总表§18.8+CLAUDE.md语料路径行 [nas]——应用户「不想搞好几套下载方案和存放位置」:不是几套,是一条规则两台机器=每类数据唯一权威源+唯一权威位置(§14.2既定原则)+采集发起位置跟随权威位置;数值类(行情/估值/报表/事件面板)权威=VPS既有8任务家族不变,文本语料类(新闻/公告)权威=NAS(用户三约束唯一决定:不上VPS+东财IP限流+不绑docker);§18.8收齐现势10行任务总表(bs-daily/xt-eod/idx-eod/ak四频率/vintage/NAS03:00链/corpus)并注明§14.5为历史快照,存放=VPS data/数值权威+NAS /volume1/stock副本+大体积历史+语料唯一权威+Mac无生产数据,corpus不是第三种根=既有数据根新子目录;CLAUDE.md关键路径加corpus行(全session可见) [nas]
CI/CD / test (push) Successful in 23s
CI/CD / nas-deploy (push) Successful in 27s
CI/CD / nas-verify (push) Successful in 9s
|
2026-09-06 20:01:39 +08:00 |
|
claude_dev
|
b90f258100
|
docs(data): spec§18存量不迁移澄清+admin权限逐项实测 [nas]——应用户问「有必要全改admin吗/admin确定有权限吗」:①存量脚本一个不迁(03:00链等root任务照旧,没坏不动;admin仅新corpus任务选型;目录/时间窗/管道三隔离零冲突)②admin权限逐项实测=/volume1/stock建目录写删全通(rwxrwxrwx+ACL admin属主)+venv --copies在volume1建成stdlib OK+pip user-site既有+CI部署路径无障碍③新坑入档:venv建/tmp会Permission denied(DSM /tmp特殊限制+会被清空)——设计本就落volume1;④设计内零root依赖,例外走人工sudo逃生门 [nas]
CI/CD / test (push) Successful in 26s
CI/CD / nas-deploy (push) Successful in 1s
CI/CD / nas-verify (push) Successful in 10s
|
2026-09-06 19:57:30 +08:00 |
|
claude_dev
|
0157094da2
|
docs(data): spec§18补admin与独立任务论证 [nas]——应用户问「为何不统一root/为何单独建定时任务」:admin三理由=03:00链root属主坑在案(admin覆盖须先rm再拉)+语料库admin侧操作密集(venv/盯跑/删分区)属主即零摩擦+最小权限出网爬虫不必root,成本为零;独立任务三理由=失败不连坐(对内行情同步vs对外慢爬马拉松,corpus卡死不拖死行情链,断路器饿死同款教训)+属主部署管道不同(sync脚本root地盘CI不覆盖vs corpus走CI的repo路径)+06:00避开03:00链跑道且00时档公告已入库全吃到 [nas]
CI/CD / test (push) Successful in 0s
CI/CD / nas-deploy (push) Successful in 3s
CI/CD / nas-verify (push) Successful in 9s
|
2026-09-06 19:52:37 +08:00 |
|
claude_dev
|
fe5d0f00a6
|
docs(data): 06:00晨窗调度依据实测固化 [nas]——应用户问「基本面数据白天被更新的可能性有多大」:深市0904全市场600条公告发布时刻实测=盘中(09-15时)3条0%+盘后15-20时46%+00时档54%(巨潮深夜批处理「半夜鸡叫」实锤);新闻当日条目盘中27%盘后晚间73%;结论=公告白天更新概率≈0且主体在零点档,T+1晨06:00拉取=前日公告100%入账,优于任何傍晚方案(17:30收工会漏19-20时18%+00时54%);新闻对日频消费零损失;spec§18.3+报告各补一段 [nas]
CI/CD / test (push) Successful in 27s
CI/CD / nas-deploy (push) Successful in 3s
CI/CD / nas-verify (push) Successful in 10s
|
2026-09-06 19:50:19 +08:00 |
|
claude_dev
|
14d3531f77
|
docs(data): 语料库spec§18修订——用户两条反馈落地 [nas]——①断点续传钦点加固:unit粒度marker+原子写+杀进程重启即从未完成unit续跑+漏日自愈(公告T-1~T两天窗/新闻recent-100天然覆盖数日缺口)+daily幂等可重跑,验收加5b条(kill -9重启续跑实证);②解绑docker(用户:「不要和docker绑定,避免docker重启就失败」):采集进程改NAS宿主机直跑,运行时=/volume1/stock/corpus/venv(--copies自举,DSM自带Python3.8.12+pip实测可用x86_64,依赖锁cp38版),容器重启/升级/停用零影响;先例=现有DSM 03:00同步链(id=2,crontab实证7条任务在跑)宿主机直跑多年零docker,本设计沿同模式改admin跑避root属主坑;验收加docker全停下照跑实证;T1改venv自举 [nas]
CI/CD / test (push) Successful in 4s
CI/CD / nas-deploy (push) Successful in 10s
CI/CD / nas-verify (push) Successful in 9s
|
2026-09-06 19:45:18 +08:00 |
|
claude_dev
|
92e63ee27c
|
docs(data): 基本面语料库NAS采集设计spec§18+三摸底实测报告 [nas]——用户拍板六项范围(除公告PDF全量):新闻标题+摘要级/全文级/历史回补+公告元数据回补2000起/每日增量/定期业绩类PDF;硬约束=存储采集全NAS不上VPS(家宽IP与VPS东财限流零冲突)+采集限于00:30-17:30+慢爬;设计=1脚本corpus_download.py+1调度入口sanguo-corpus-daily 06:00(daily+backfill两lane串行,--until 17:30墙钟自停);数据模型=id键控(dt=发现日分区,无today文件名语义,天然免疫0902/0904式错标)+append-only短窗去重;回补顺序=公告meta→新闻meta→新闻全文→PDF(新→旧,容量闸门:/volume1实测645G空闲,每50GB自查,<200G暂停升级);失败语义吸收09-01/09-05教训(确定性错跳过不烧重试不连坐);探测优先能拿多深拿多深+depth_summary记录;附三摸底报告(聚宽136字段原料层≈100%已有/巨潮2000年起130万条年/东财3年硬顶/存储大头PDF 520G年) [nas]
CI/CD / test (push) Successful in 23s
CI/CD / nas-deploy (push) Successful in 0s
CI/CD / nas-verify (push) Successful in 8s
|
2026-09-06 13:49:46 +08:00 |
|
claude_dev
|
90b37ce27f
|
feat(live): 成交回报回执日志①——09-04守恒缺口观测件 [vps]——根因件:002633/002836缺口复盘发现引擎会话当日成交视图行缺+时间字段中途退化None,QMT行dump全打查询时刻即铁证,事后无从对「哪笔什么时候以什么形态被看到」;用户钦点观测机制「再出现时根因可查」,dev契约:任何处理/匹配/去重之前打[trade-receipt] tid/order/acc/sym/vol/px/t_raw/recv八字段(②解析此格式,拼写冻结);实现=with_trade_receipt包装QmtBroker.get_trades(bullet_trade轮询制无成交推送回调,_Callback仅桥接断连,broker.get_trades是柜台行进本进程唯一咽喉,引擎同步/a窄修/b宽修全经它),与with_connect_retry同款实例级包装源码零改动;铁律=t_raw原样照记None也记/日志永不抛逐行try-except包死/get_trades原异常照抛行为不变;量控=观测层变化检测非处理层去组:同tid签名(t_raw+量+价+单号)不变不重打,退化即重打(捕获09-04形态),行照常流向消费方;③结论免改:vt_tradeid三条归因路径已承载柜台成交号(即时=引擎trades键/a窄修=trade_id/b宽修=eod:前缀),既有测试已钉(test_live_reconcile.py:123,206+test_live_instance_ledger.py:236),②按tid比回执↔live_trades即闭环;TDD 7新测先红后绿(caplog须set_level(INFO)坑)+598全绿
CI/CD / test (push) Successful in 13s
CI/CD / nas-deploy (push) Successful in 46s
CI/CD / nas-verify (push) Successful in 9s
vps-deployed
|
2026-09-06 11:14:29 +08:00 |
|
claude_dev
|
67a2e49f21
|
feat(live): 成交回报回执②回执↔账本闭环比对器——用户钦点「再出现时根因可查」:09-04实证柜台有成交/台账无行/引擎会话视图无行且时间退化,「推送到达→回调处理→落库」三跳全静默无法定位故障点;观测契约三层=①引擎侧trade-receipt指纹行(策略session实现,tag/字段名以本模块解析器为准)+②本模块parse+compare:有回执无账行=引擎处理丢(直接证据),柜台有成交无回执=柜台没推(EOD恒等式层),③账本行带柜台成交号硬键(替换会退化的时间五元组);原则宁缺勿错:tid=NONE不做模糊匹配硬凑,如实分列;ledger_only为信息项(回执上线前旧行/repair行)不判脏;TDD 8测先红(模块不存在)后绿 [vps]
CI/CD / test (push) Successful in 28s
CI/CD / nas-deploy (push) Successful in 4s
CI/CD / nas-verify (push) Successful in 12s
|
2026-09-06 10:53:24 +08:00 |
|
claude_dev
|
694a0d2238
|
fix(data): ak-weekly 断路器饿死修复=_safe_northbound_em+类型调序 [vps]——09-05 03:00 首跑 rc=2 根因:akshare stock_hsgt_individual_em 对无北向数据标的(~1117/2883,301xxx 等)在 data_json[result][pages] 抛 TypeError NoneType not subscriptable→failed→30 连败断路器→跳过后续类型,share_capital/top_holders/gdhs 全饿死(gdhs 目录至今不存在);端点健康(600519 返 1683 行,09-06 晨复测);修=①_safe_northbound_em 子串匹配 not subscriptable 返空 df 带 9 列 schema→走无北向持仓 empty 写空+done 既定语义不耗重试/断路器(同 _safe_top_10_em 手法)②weekly wrapper 调序 gdhs,share_capital,top_holders,northbound 最大最险者殿后防再连累③顺手 BOM;TDD 2 新测先红后绿+217 全绿;双脚本已 scp 生效位+Parse/py_compile 预检过;gdhs 手动补跑另行 [vps]
CI/CD / test (push) Successful in 23s
CI/CD / nas-deploy (push) Successful in 7s
CI/CD / nas-verify (push) Successful in 13s
|
2026-09-06 08:29:58 +08:00 |
|
claude_dev
|
c728f2368c
|
feat(data): hot_rank 挂载+wrapper 时间守卫 [vps]——墙测量5轮收官用户拍板方案A:emappdata 夜间开窗(22:28 实测 OK,晨午4轮全 FAIL),19:30 档失败零连带(单 unit 失败不触断路器 rc 仍 0,不写不标次夜自愈),首夜起 gap_check 即免费持续测量;三处同步=wrapper --types+PANEL_TYPES+EVENT_PANEL_TYPES(15类),零文件=零告警首日成功即成基线;时间守卫=param -Force+非 19:00-23:59 拒绝执行留痕 guard.log(治盘前手动误跑两犯:09-02 08:08/09-04 08:00,盘前拉当日=昨日态错标+marker 锁死当晚真数据);TDD 2 测翻新(白名单拒→放行+注册断言)先红后绿+806 全绿;VPS Parse 预检过+双脚本已 scp 生效位+wall_probe.py 已删 [vps]
CI/CD / test (push) Successful in 0s
CI/CD / nas-deploy (push) Successful in 5m3s
CI/CD / nas-verify (push) Successful in 11s
|
2026-09-04 18:46:46 +08:00 |
|
claude_dev
|
6adb40e7a0
|
fix(live): 恒等式逐票对账补负向幻影——09-04 收盘联检实证:002633买缺100(正向孤儿,已列)+002836卖缺300(负向幻影,漏列),旧代码unattr_positions只遍历snap_pos.items(),快照没有的票永不被遍历到,单看票单会把修账做成半拉子(只补孤儿不补幻影);修=双端键并集sorted(set(snap_pos)|set(inst_pos)),正向孤儿/负向幻影都入列(002836当日在报告里只有MV差对不上数才现形);残差拆腿同步定性:-1535=100×002633收盘15.35-300×002836收盘10.23,100%来自两笔引擎未报成交,修账后应归零;TDD 1新测先红后绿+21全过 [vps]
CI/CD / test (push) Successful in 1s
CI/CD / nas-deploy (push) Successful in 14s
CI/CD / nas-verify (push) Successful in 18s
|
2026-09-04 18:29:07 +08:00 |
|
claude_dev
|
e2902cdf1f
|
fix(data): ak_events_wrapper 修启动级语法错+加BOM——09-03 19:30首跑即死(rc=1零日志=解析期死亡):真根因=line14 (Get-Date).AddDays(-1) -Format yyyyMMdd 为非法PS语法(表达式后不能接命令参数),无BOM时GBK误读把报错行号搅到13误导视线;修=.ToString('yyyyMMdd')+前置UTF-8 BOM根治PS5.1对无BOM中文文件的GBK误读隐患;三向hash已证本地=git=VPS无部署损坏;教训=ps1改动部署前必须Parser预检(VPS已验PARSE_OK);今晚手动补跑13类+margin_sse 0902(T-1窗首验) [vps]
CI/CD / test (push) Successful in 14s
CI/CD / nas-deploy (push) Successful in 3m1s
CI/CD / nas-verify (push) Successful in 19s
|
2026-09-04 07:59:19 +08:00 |
|
claude_dev
|
029ea34aec
|
feat(live): 引擎启动期行情腿自检quote_preflight——09-03行情事故定位件:momentum 9:30六笔卖单fail-closed忽略+18/22探针零单,引擎进程内取价失败但独立探针.SH格式全通(dev反证,我先前的「服务死」误判收回:get_full_tick须.SH后缀+subscribe须xtdata.run(),两个坑全踩);需进程内直接证据区分「引擎会话坏vs服务坏」:boot时broker交易腿连接后一笔get_full_tick(510500.SH)留痕OK/EMPTY/EXC,永不抛不阻断;今晚23:05轮换后新进程即出证据,明早9:30三灯(momentum重试/small_cap调仓/9:35探针)前置诊断;3新测+591绿 [vps]
CI/CD / test (push) Successful in 1s
CI/CD / nas-deploy (push) Successful in 9s
CI/CD / nas-verify (push) Successful in 14s
|
2026-09-03 13:51:26 +08:00 |
|
claude_dev
|
37d8f85d21
|
feat(strategy): 情绪温度计四指标——涨停池三件套首轮Python化,消费provider.get_limit_pool出口(get_limit_pool/get_event_panel双端8/8已验);四口径契约与数据session 09-03对齐并逐条钉进测试:①炸板率=zbgc/(zt+zbgc)按行数,回封票(zt内break_count>0)留分母勿扣②dtgc空=0跌停合法值③history按trade_date对齐,某池某日无行按0计不丢日④连板高度=consecutive_boards之max(to_numeric吸字符串)+行业集中度用industry现成列(空串/NaN剔除);三接口=compute纯函数+daily单日快照(date=None最新落盘日)+history区间校准表;只算不判,阈值待23日真实数据校准后另定;10新测+588全绿 [vps]
CI/CD / test (push) Successful in 21s
CI/CD / nas-deploy (push) Successful in 11s
CI/CD / nas-verify (push) Successful in 14s
|
2026-09-03 08:58:49 +08:00 |
|
claude_dev
|
99355bfbea
|
docs: 活文档§17 首夜巡查修正——margin_sse T+1 结构根治/NAS 同步链扩容/回灌纪律/VPS 部署终态(同批同步,防失传) [nas]
CI/CD / test (push) Successful in 0s
CI/CD / nas-deploy (push) Successful in 6s
CI/CD / nas-verify (push) Successful in 14s
|
2026-09-02 23:09:19 +08:00 |
|
claude_dev
|
564160db11
|
fix(data): margin_sse 拆 T-1~T 两天窗——巡查实锤 SSE 融资融券 T+1 供应,当晚拉当天结构性永败 43 天(用户批准结构修) [vps]
CI/CD / test (push) Successful in 19s
CI/CD / nas-deploy (push) Successful in 5s
CI/CD / nas-verify (push) Successful in 6s
07-22 起慢性失血实锤: margin_sse 最后非空文件 20260721,之后 30 个交易日
全被旧码标成 空文件+done 锁死(每晚 19:30 拉当天,SSE 当日数据未发布→
空响应→akshare Length mismatch 崩→旧码转 empty 落盘+marker)。
已处置(同批): --force 回补 20260722~20260901 全区间 30/30 ok 59865 行;
本 commit=结构根治: wrapper 把 margin_sse 从 13 类当日批拆出,单独
--start T-1 --end T 两天窗——每晚稳定写昨日(数据已发布),当天失败不写
不标(新语义),次晚同窗自动重试补上,单晚漏跑自愈;两段 exit code 合并
任一非零即非零。0902 今晨已过 19:30,明晚首验(写 0902+0903)。
生效位=VPS scripts(手动 scp,CI 不覆盖 VPS)。
|
2026-09-02 22:47:21 +08:00 |
|
claude_dev
|
ab8f4e3782
|
fix(nas): sync_parquet_daily 扩到 A 档 13 类+gdhs 全量拉——巡查实锤 TYPES 写死旧 4 类,明晨 03:00 快照族/涨停池到不了 NAS [nas]
CI/CD / test (push) Successful in 4s
CI/CD / nas-deploy (push) Successful in 12s
CI/CD / nas-verify (push) Successful in 20s
09-02 巡查发现: sync_parquet_daily.sh TYPES 硬编码 dragon_tiger/block_trade/
margin_sse/restricted,今晚 19:30 A 档首夜产出(快照族 6 类+涨停池×3)不在拉取
清单,明晨同步后 NAS 回测侧依旧零文件(provider §16 出口在 NAS 断供)。
修: TYPES 扩到 13 类(+zt_pool×3/fund_flow×2/ths×2/xueqiu_hot/sina_sector,
7 天滚动窗逻辑不变);gdhs=per-period(季度末日期永不落 7 天窗)单独全量拉
(dir /b 枚举后逐文件 scp,≤10 文件幂等,周六 ak-weekly 产出)。
生效位=/volume1/stock(scripts/nas_sync/),CI 不覆盖,需手动部署 NAS;部署后
手动跑一轮验证=顺带把今晚首夜数据提前送 NAS(不等明晨 03:00)。
|
2026-09-02 22:31:23 +08:00 |
|
claude_dev
|
7f01c5fe9a
|
ci: 补 4966d30 环境标签(subject 尾,CI 定检 %s) [vps]
CI/CD / test (push) Successful in 23s
CI/CD / nas-deploy (push) Successful in 8s
CI/CD / nas-verify (push) Successful in 15s
4966d30 的 [vps] 误置 body 末尾,enforce-label 只查 subject 行(ci-cd.yml
git log --format=%s)→ run#1038 rerun 红在标签步(pytest 未跑)。本空 commit
使 push range(4966d30..HEAD)重查通过;nas-deploy checkout 本 commit 即含
全部变更。后续 commit 标签一律放 subject 尾(规范模板:<type>(<scope>):
<desc> [vps])。
|
2026-09-02 11:04:58 +08:00 |
|
claude_dev
|
4966d30a11
|
feat(provider): A档截面数据使用层出口——泛型底座+涨停池门面双层接口
CI/CD / test (push) Failing after 3s
CI/CD / nas-deploy (push) Has been skipped
CI/CD / nas-verify (push) Has been skipped
09-02任务①落地。§15采集的14类panel数据此前零读取出口,策略/因子session
想用只能手搓parquet路径;经业界对照(dsa get_limit_up_pool/tushare
limit_list_d U/D/Z参数化/聚宽per-domain/OpenBB TET)定稿「泛型底座+域
语义门面」:
①get_event_panel(event_type,date/start+end,trading_days_only=True):
保真底座,白名单14类统一通道(=采集注册表口径,hot_rank未挂载不入列),
返回akshare原样中文列+trade_date(文件名合成);缺文件=合法缺失→空df;
trading_days_only默认用get_trade_days滤非交易日——治快照族「周中法定
假日文件=上一交易日态复制品」的区间双计数(采集层只按周一~五落盘,
周六根本无文件,风险面=周中假日);=False按周一~五枚举(=采集口径)
②get_limit_pool(kind=zt|zbgc|dtgc,...):涨停池门面(温度计/炸板率/行业
集中度消费契约),kind三合一←tushare limit_list_d;英文标准列
code/consecutive_boards/seal_amount/break_count/industry…(键名对齐dsa);
映射表基于VPS真实parquet实测(dtgc实测含动态市盈率/封单资金/板上成交额/
连续跌停/开板次数,与文档口径有差);akshare改中文列名时本层吸收漂移,
有行缺源列→DataSchemaError fail-fast;真空日(dtgc 0跌停空文件无列)
→仍返标准列空表schema稳定
③实现=TET Fetcher(fetchers/event_panel.py双Fetcher同文件,price.py先例),
FETCHERS注册表+event_panel/limit_pool两键(未来MCP出口零成本);base.py
新增_DateRangeParams跨字段校验(date/start+end互斥二选一);LocalUnified
Provider(回测)与SanguoMiniQmtProvider(实盘,委托self._unified)双侧同款
——方法面钉死测试强制无缺口,双侧同schema支持副本对照;miniQMT/QMT/
PTrade均无此类接口(业界惯例=外部源补),实盘同读本地文件
测试:21新测试(白名单fail-fast/日期语义/保真读/假日滤/门面schema/
列漂移fail-fast/真空日空表/注册表)+全量793绿;活文档§16+§15指引行 [vps]
|
2026-09-02 10:43:11 +08:00 |
|
claude_dev
|
8c6a3f90d0
|
fix(strategy): channel_test盘外轮换改锚固定9:35——09-02 dev巡检实锤5d0a0bf副作用:随启随验「盘外不注册」×运维规则「轮换放23:05后」=每次boot都盘外→永不注册→测试通道停摆(live_18/22零任务,调度恢复又因存档模块路径vs runtime副本错配恒失败,守卫再拦回退注册,三重叠加清空);考古澄清:四固定任务自08-20 147a49d随启随验上线即有意退出live(probe_all一轮覆盖rotate/T1/swap),非本次回归;修=盘外boot改锚固定9:35(历史主调仓时刻,盘中由构造保证),盘中boot仍now+90s快验不变,执行侧守卫保留为末道防线;原「盘外不注册」测试按新规格改写为断言注册9:35;25/25+557绿 [vps]
CI/CD / test (push) Successful in 19s
CI/CD / nas-deploy (push) Successful in 3s
CI/CD / nas-verify (push) Successful in 7s
|
2026-09-02 10:25:36 +08:00 |
|
claude_dev
|
66e741b6bb
|
docs: 数据设计文档活文档化+§15 A档截面数据域增补——用户约定(09-02):唯一数据设计文档,每次架构变更同批更新(防长期失传);决策=更新原2026-07-21融合spec不新建(三层框架仍成立/引用链不断/少一份文档),头部状态从「设计草案」翻为活文档声明+最后更新日期;§15沿用「日期节追加新节覆盖旧节」惯例:15.1单脚本红线 15.2十一数据域清单表(源/集群/调度/行数/回补能力) 15.3东财墙模型(每IP每~40min窗口1发/cyq数学死/换域答案) 15.4语义与防线(panel_gap_check两档+panel契约) 15.5消费契约(炸板率口径/列名schema) [nas]
CI/CD / test (push) Successful in 2s
CI/CD / nas-deploy (push) Successful in 7s
CI/CD / nas-verify (push) Successful in 14s
|
2026-09-02 09:41:02 +08:00 |
|
claude_dev
|
46a0569b9a
|
feat(data): A档全量落地——热度三件+股东户数收尾(用户批「按此方式实施,nas验证后推vps」);①xueqiu_hot雪球5633行+sina_sector新浪行业49行对账源进ak-events 19:30(同款快照语义)②hot_rank东财人气榜fetcher/注册先行,wrapper挂载留待恢复窗测量拍板(墙未定论不带不确定单元上线防缺日告警噪音)③gdhs股东户数进ak-weekly周六窗=per-period新形状:build_gdhs_units只枚举距今>90天季度末(披露守卫,防未披露期拉空走真空+done语义被marker锁死永不重拉的09-02设计坑),单次全市场5342行,marker去重无新期零网络;vintage缺日名单+2快照族(hot_rank留位);+6测试(注册/None/df契约/gdhs守卫两形态/缺日名单)215全绿 [vps]
CI/CD / test (push) Successful in 2s
CI/CD / nas-deploy (push) Successful in 9s
CI/CD / nas-verify (push) Successful in 16s
|
2026-09-02 09:16:14 +08:00 |
|
claude_dev
|
00cfd45d1a
|
feat(data): A档第二批接入——新浪资金流×2+同花顺行业/概念榜进ak-events 19:30(用户批方案A);东财push2墙死后免费无墙替代源双机实测秒级零限流(新浪行业资金流90行带净额/领涨股+概念387行,同花顺行业90行×12列+概念375行);快照族语义显式化:端点无date参数无历史回补,19:30采集=收盘终态,节假日拉到上交易日快照接受重复由消费方按交易日历过滤;vintage缺日检查扩全panel族并分档——可回补族(zt_pool×3)近窗洞给回补指引,快照族洞即永久告警查当晚日志无回补指引,旧洞不刷告警防疲劳;+5测试(注册/None/df契约×2+快照族永久洞语义)29绿 [vps]
CI/CD / test (push) Successful in 2s
CI/CD / nas-deploy (push) Successful in 8s
CI/CD / nas-verify (push) Successful in 12s
|
2026-09-02 08:57:39 +08:00 |
|