fix(monitor): 监控班次二 P2 批——D-5 db_sample 日历不可达去 fail-open(样本可读弱检/库不可达黄)+D-6 month_bucket 损坏件 rows=None 黄(对齐 per_date)+D-8 check_disk 阈值参数化消费注册表 warn_gb/crit_gb(旧硬编码 8/20=改表不改行为校准陷阱,crit_gb 全仓无消费)+D-9 vintage/NAS ts fromisoformat 捕 TypeError+D-10 db_sample close 进 finally+AL-1 upsert 更新 severity(同日黄→红升级即时落表,旧 UPDATE 不动=次日新 id 才纠正)+purge 收口 MON-202610-04(open 黄改 last_seen 28d=活着不删,旧 first_seen 14d 硬删无声抹持续黄+断 count 升级链;resolved 统一 30d,open 红永不删)+N-1 run_monitor_nas.sh rc 先存后拼($(date) 同词先展开重置 $?=exit 恒 0,rc 隔离下 NAS 崩溃唯一现场线索)+N-2 NAS 日历豁免短期方案(last_weekday 接休市表镜像+CI 同步测钉与 monitor_registry 相等=7f9b413 镜像纪律;10-06 旧码会把 10-05 假日当昨交易日;corpus 34h 日历日节奏实证无需豁免)+P3 归因([FATAL] 级别标记不当类型归 _global,scan_signatures 注记 test-only 勿接线)+T-2 NAS red 分支(hot_rank 文件缺)测试补缺+migration_logs 保洁(data_monitor_wrapper 双班剪 14d 前旧件,13 wrapper 无限增长收口);monitoring spec §5.1/§5.2/§5.3-④/§5.4/§5.5-④/§7.6/§12 随码(C-A1 universe 异常实串/C-A2 签名表勘正补 Expecting value/C-A6 disk 阈值表述+锚点行数刷新) [vps] [nas]
This commit is contained in:
@@ -221,7 +221,7 @@ kind 语义（检查分发依据，check_entry data_monitor.py:56-232）：
|
||||
| vintage_json | `static/vintage_status.json` | checked_at≤26h + `tables.*.holes_permanent/backfillable` 挖洞数（黄） |
|
||||
| static_exists | 目录含 parquet | 存在性（黄） |
|
||||
| log_only | 无文件 | 仅过程层统计行存在且 failed=0（§5.2 ②；周班项平日无行=正常跳过） |
|
||||
| disk | — | 数据根剩余 ≥20G 黄 / <8G 红（key=`infra-disk-data`） |
|
||||
| disk | — | 数据根剩余 <8G 红 / <20G 黄 / ≥20G 绿（key=`infra-disk-data`；阈值=注册表 `warn_gb`/`crit_gb`，D-8 起被 `check_disk` 真消费——旧「≥20G 黄」为笔误，10-05 C-A6 勘正） |
|
||||
|
||||
**42 项明细**（row_band=(floor, fallback_hi) 语义见 §5.6；生产班次见 fusion §20 调度表）：
|
||||
|
||||
@@ -240,7 +240,7 @@ kind 语义（检查分发依据，check_entry data_monitor.py:56-232）：
|
||||
### §5.2 四层证据链与分类学（fusion §20.15.2 迁入，10-03 起权威在此）
|
||||
「完全与否」四层（弱→强，**以④为准**）：
|
||||
- ①任务 exit 码（类型级失败不抬码，最弱）
|
||||
- ②类型级统计行 `{"ok":N,"failed":M,"rows":R}` 解析+**失败签名分诊**（monitor_checks.py:185）——**10-05 三修**：①只扫 `ERROR`/[FATAL] 行（重试 WARNING 的 `Expecting value` 良性不再误中 source_changed）②针收紧 `418 Client Error`（裸 418 子串误中 rows=41823 进度行=anti_crawl 假阳性实证）③**签名按日志行 `[type]` 归因**（`signatures_by_type`）——他类型的反爬签名不再把本类型 failed 抬红（跨类型连坐修）：`RemoteDisconnected`=断连族红 / `NoneType`=源当日未出黄 / `418 Client Error`=反爬红 / `KeyError/TypeError`=源字段变更红（09-27 P2 起字段形状类 1 次即抛 AkShapeError 豁免断路器；**10-05 起 share_capital 次新股『公告日期』族归 empty 桶不再 failed**，见 §9.B-21）/ 断路器触发=熔断连累红（独立键 `data-akshare-breaker`，exit 2 会话，全局取）
|
||||
- ②类型级统计行 `{"ok":N,"failed":M,"rows":R}` 解析+**失败签名分诊**（`_error_lines` monitor_checks.py:190 / `signatures_by_type` :216）——**10-05 三修**：①只扫 `ERROR`/[FATAL] 行（重试 WARNING 的 `Expecting value` 良性不再误中 source_changed）②针收紧 `418 Client Error`（裸 418 子串误中 rows=41823 进度行=anti_crawl 假阳性实证）③**签名按日志行 `[type]` 归因**（`signatures_by_type`）——他类型的反爬签名不再把本类型 failed 抬红（跨类型连坐修）：`RemoteDisconnected`=断连族红 / `NoneType`=源当日未出黄 / `418 Client Error`=反爬红 / `Expecting value`=源接口形状变更红（source_changed）/ `KeyError`=源字段变更红（field_changed；09-27 P2 起字段形状类 1 次即抛 AkShapeError 豁免断路器；**10-05 起 share_capital 次新股『公告日期』族归 empty 桶不再 failed**，见 §9.B-21）/ 断路器触发=熔断连累红（独立键 `data-akshare-breaker`，exit 2 会话，全局取；**班次二 P3 补**：`[FATAL]` 是级别标记非类型——裸 FATAL 行归 `_global`，`[type] [FATAL]` 双标记行归 `[type]`；`scan_signatures` 生产路径已无调用方=仅测试消费注记）
|
||||
- ③marker 按日比对（标了≠数据对，0×0 案）
|
||||
- ④数据本体（存在+行数带宽+schema+值域）
|
||||
|
||||
@@ -248,27 +248,29 @@ kind 语义（检查分发依据，check_entry data_monitor.py:56-232）：
|
||||
|
||||
**10-05 监控三 P1（审计 D-1/2/3）**：①**D-1 归并语义**=`collect_log_layer` 跨日志统计行改 **mtime 序后到运行整行胜出**（旧 `max(rows)` 归并=早班 rows=100/failed=0 压住后到 rows=50/failed=3 → 漏报+既有 `data-X-failed` 告警被假 resolve 双失效）；②**D-2 整班崩溃网**=main 顶层兜底（裸崩溃时写 red health+`infra-monitor-crash` 告警后 exit 3）+`check_entry` 单检隔离（抛错=黄降级 `data-X-stale` 不阻断全班，对齐 strategy 侧先例）——旧形态=裸传播+wrapper 恒 0+SCHTASKS 年龄判定失明，最坏 ~22h 仅 peer 反盯；③**D-3 日历读侧归一**=`trading_days` 返回改 `[:10]` 截断+None 过滤+sorted（对齐 strategy_checks 同名版，双实现分叉收敛；dbbardata datetime 历史双格式，写侧归一不担保读侧形状）。
|
||||
|
||||
**10-05 班次二 P2 批（审计 D-5/6/8/9/10+AL-1+purge+N-1/N-2）**：①**D-5 db_sample 去 fail-open**=日历不可达（exp=None）不再无条件绿——弱检查=样本可读（对齐 per_date 存在性弱检），库也不可达=黄；②**D-6 month_bucket 损坏件黄**=`parquet_rows=None`（parquet 损坏/pyarrow 不可用）不再静默绿，对齐 per_date rows=None→黄；③**D-8 disk 阈值消费注册表**=`check_disk(path, warn_gb, crit_gb)` 参数化+disk 项传表值（旧硬编码 8/20 与注册表脱钩、crit_gb 全仓无消费=改表不改行为的校准陷阱）；④**D-9 vintage TypeError**=`fromisoformat` 捕 `(ValueError, TypeError)`（非字符串 checked_at 原会逸出到 D-2 隔离，黄走错 verdict 键；NAS 静默 ts 同型同修）；⑤**D-10 db_sample close 进 finally**=异常路径连接不泄漏；⑥**AL-1 upsert 更新 severity**=同日黄→红升级即时落表（旧 UPDATE 不动 severity=次日新 id 才纠正，前端 open_red 角标少计一晚）；⑦**purge 收口（MON-202610-04）**=open 黄改按 **last_seen 28d** 清——旧 first_seen 14d 硬删把「每日重 Sight 的持续黄」（first_seen 老/last_seen 新）无声从值班页抹掉+count 升级链断；last_seen 每次重 Sight 刷新=活着不删，僵死 28d 才清；resolved 统一 30d（resolved 黄原 14d-first_seen 通道删除）、open 红永不删；⑧**N-1**=run_monitor_nas.sh `rc=$?` 先存后拼（旧 `$(date)` 先展开把 `$?` 重置 0，exit= 恒 0=NAS 崩溃唯一现场线索失真）；⑨**N-2 NAS 日历豁免（短期方案）**=`last_weekday` 接休市表镜像（`EXCHANGE_HOLIDAYS` 镜像 monitor_registry，CI 同步测钉两表相等=7f9b413 镜像纪律）——旧只剥周末，国庆连休窗 10-06 会把 10-05 假日当「昨交易日」（当前被 hot_rank producer 日历日节奏掩住、语义仍错；收口议=⚖️-3 monitor_common）；**corpus 双 lane/xcheck 34h 无需豁免**（producer 日历日节奏=假期照采，10-01~05 分区连续实证）；⑩**migration_logs 保洁**=data_monitor_wrapper.ps1 双班顺手剪 14 天前旧件（13 wrapper 各自追加无任何清理=无限增长；无解析消费的诊断残留）；⑪T-2 补 NAS red 分支（hot_rank 昨交易日文件缺）+假日窗测试。
|
||||
|
||||
**结果层缺失分级（09-26 中秋反转案定案）**：确认日缺失=红（跨域互证=真是交易日）；未确认日缺失=黄（节假日/合并窗/源滞后三态容忍，evidence `unconfirmed_day=True`）；**不因时间升红**（f86ed10 升红规则回滚案——时间基线会在连休把全 per_date 域连坐误红）；真漏采兜底=确认集推进（复市后 exp 随 confirmed 前移→确认日红接手）+调度/过程层纵深分工。10-03 起（b24f905）已知假日直接零期望（§5.5），黄只剩「表外未知休市+源滞后」两态。
|
||||
|
||||
### §5.3 两班调度与产物（五件套）
|
||||
- **①需求**：当晚红项当晚出（23:50 终检收当日全尾部）+晨间全量补过夜项（08:05 在 bs-daily 马拉松进行中、ak-daily 19:00 已收）
|
||||
- **②两班代码零分叉**：`--shift` 仅入 health 文件名与 `health["shift"]` 字段，无行为分支——「晨班自愈回看」不是独立代码路径，是同一套全量重评在 08:05 时点自然产生（昨夜红项已自愈则本班绿→`resolve_passed` 消解；alerts 层 26h 窗仍见昨夜红回声→由 checks_overall/alerts_overall 拆分消歧）
|
||||
- **③班次表**：`sanguo-data-monitor-morn` 08:05（晨检 agent 08:10 首读 health_latest.json）/ `sanguo-data-monitor-eve` 23:50；wrapper=`data_monitor_wrapper.ps1 -Shift <s>`（清 proxy env+Set-Location 仓根+`-X utf8` 启动+日志落 `data/migration_logs/data_monitor_*.txt`+**恒 exit 0**）
|
||||
- **④坐标**：main data_monitor.py:419（10-05 D-2 后=parse+`_run_shift`+`_crash_exit` 三段壳）；write_health :391；rc `return 3 if overall=="red"` :531；测试口 `--skip-tasks`（跳过 schtasks 子进程）/`--skip-inbox`（跳过 NAS 摄取）
|
||||
- **④坐标**：main data_monitor.py:461（10-05 D-2 后=parse+`_run_shift` :482+`_crash_exit` :586 三段壳）；write_health :433；rc `return 3 if overall=="red"` :583；测试口 `--skip-tasks`（跳过 schtasks 子进程）/`--skip-inbox`（跳过 NAS 摄取）
|
||||
- **⑤坑**：schtasks 控制台 OEM/GBK 输出必须**字节捕获+手动 GBK 解码**（`-X utf8` 下 text=True 按 UTF-8 误解码→中文任务名/状态全乱→09-25 首跑 14 任务全假红，:475-482）；判活 schtask 用 `Get-ScheduledTaskInfo`（CSV /nh 无 Last Result 列）
|
||||
|
||||
### §5.4 NAS monitor v2（五件套）
|
||||
- **①需求**：NAS 侧数据面（corpus/dmsk/hot_rank/mirror/xcheck/磁盘）VPS 看不到，各看各的（用户令①）；NAS→VPS 推送通道现成、反向不通→**事件 JSON scp 到 VPS inbox、由 VPS data_monitor 摄取落库**（单写者纪律：NAS 不写远端 sqlite）
|
||||
- **②链路**：宿主 bash `run_monitor_nas.sh`（挂 `sanguo-vps-mirror` 16:00 班尾）→ `nas_monitor_check.py`（宿主 python3.8 纯 stdlib）→ `out/nas_health_{day}.json`+`nas_events_{stamp}.json` → scp（必带 `-O`，ssh 必带 `-n` 防吃 stdin）→ `C:/sanguo_vnpy_v2/data/monitor/inbox/` → VPS 两班 `ingest_inbox` 摄取（data_monitor.py:361-386：events upsert→**green_keys 回传 `resolve_passed`**[09-26 补：NAS 绿检查必须回传否则 NAS 告警永不转绿]→文件挪 `inbox/processed/`）
|
||||
- **③六族检查**（nas_monitor_check.py:120-214）：corpus 双 lane（news_meta/ann_meta 分区新鲜 ≤34h）｜dmsk 三表**季频锚**（最新 dt 分区 ≥ 最近已过披露截止季末：Q1→04-30/半年→**08-31**/三季→10-31/年报→次年 04-30，五域取 max；0928 改——旧 mtime 80h 对季频必假黄）｜hot_rank 三查（昨交易日文件 red+`.pushed` marker+当日 cron.log 无 FAIL）｜vps-mirror 完成度（≥17:00 查当日、班前查昨日 DONE fail=0）｜xcheck findings 新鲜 ≤34h｜磁盘 /volume1 水位
|
||||
- **④零事件也写**（`generated_at` 必带）→ VPS 侧静默检测：inbox 最新 ts >30h → `infra-nas-monitor-silent` 黄（data_monitor.py:488-503）
|
||||
- **⑤坑/纪律**：runner 属**宿主生效位**（`/volume1/stock/monitor_nas/`）CI 不覆盖——改 repo 副本后须 `scp -O` 手动同步（09-28 纪律）；DONE 行归属=遍历记 start 段（§9.B-1）；`cur_day=None` 防错归日（1010 P3-11）
|
||||
- **②链路**：宿主 bash `run_monitor_nas.sh`（挂 `sanguo-vps-mirror` 16:00 班尾）→ `nas_monitor_check.py`（宿主 python3.8 纯 stdlib）→ `out/nas_health_{day}.json`+`nas_events_{stamp}.json` → scp（必带 `-O`，ssh 必带 `-n` 防吃 stdin）→ `C:/sanguo_vnpy_v2/data/monitor/inbox/` → VPS 两班 `ingest_inbox` 摄取（data_monitor.py:403-428：events upsert→**green_keys 回传 `resolve_passed`**[09-26 补：NAS 绿检查必须回传否则 NAS 告警永不转绿]→文件挪 `inbox/processed/`）
|
||||
- **③六族检查**（nas_monitor_check.py:141-235）：corpus 双 lane（news_meta/ann_meta 分区新鲜 ≤34h——producer 日历日节奏=假期照采，34h 窗天然安全无需假日豁免，10-01~05 国庆分区连续实证）｜dmsk 三表**季频锚**（最新 dt 分区 ≥ 最近已过披露截止季末：Q1→04-30/半年→**08-31**/三季→10-31/年报→次年 04-30，五域取 max；0928 改——旧 mtime 80h 对季频必假黄）｜hot_rank 三查（昨交易日文件 red+`.pushed` marker+当日 cron.log 无 FAIL；**N-2 10-05 起 `last_weekday` 接休市表镜像**——旧只剥周末，连休窗把假日当昨交易日，被 producer 日历日节奏掩住；镜像表 CI 同步测钉与 monitor_registry 相等，表缺年 fail-safe 不扣）｜vps-mirror 完成度（≥17:00 查当日、班前查昨日 DONE fail=0）｜xcheck findings 新鲜 ≤34h｜磁盘 /volume1 水位
|
||||
- **④零事件也写**（`generated_at` 必带）→ VPS 侧静默检测：inbox 最新 ts >30h → `infra-nas-monitor-silent` 黄（data_monitor.py:540-555）
|
||||
- **⑤坑/纪律**：runner 属**宿主生效位**（`/volume1/stock/monitor_nas/`）CI 不覆盖——改 repo 副本后须 `scp -O` 手动同步（09-28 纪律）；DONE 行归属=遍历记 start 段（§9.B-1）；`cur_day=None` 防错归日（1010 P3-11）；**rc 先存后拼**（N-1 10-05：`$(date)` 在同 echo 词内先展开会重置 `$?`，旧版 exit= 恒 0=崩溃唯一现场线索失真）
|
||||
|
||||
### §5.5 假黄休市表 EXCHANGE_HOLIDAYS（b24f905，五件套）
|
||||
- **①需求由来**：假定集=「近 15 日工作日」把国庆/春节连休的每个工作日都当期望日→三域（valuation_baostock/dbbardata_daily/dbbardata_index）连休天天假黄（0925 中秋折腾全天首案；10-03 联合甄别确认 10-05~08 还会天天黄）→ 用户令「现在落地别等 10-04」
|
||||
- **②语义判据**：`_calendar_days(confirmed, now, lookback=15)`（data_monitor.py:402-417）——假定集只收「工作日 **且不在休市表**」；**表缺年=不扣**（fail-safe：黄照旧，绝不静默转绿——跨年不补表只丢本次收益不引入漏报）；**确认集恒保留**（数据本体是真相，表错不吞真相）；已知假日=零期望（当夜直接绿，不再走未确认黄）
|
||||
- **③参数与校准史**：2026 表 19 工作日=元旦 0101~02｜春节 0216~20+0223｜清明 0406｜五一 0501/0504/05｜端午 0619｜中秋 0925｜国庆 1001~02+1005~07（**10-08[周四]复市**——勘正全员「10-09 开盘」旧误记）；推导工法=VPS dbbardata 000300 distinct 交易日 diff 工作日（实证至 1002）+交易所 2026-09-17 公告补未来段——**只登实证/公告日，勿凭记忆添**（错登真交易日=掩盖真缺失，危险方向）
|
||||
- **④坐标**：表 monitor_registry.py:30-40；剔除逻辑 data_monitor.py:402-417（main 调用 :434-436）；CI 年更闸 `test_holiday_table_covers_current_year`（tests/data_platform/test_data_monitor.py）——当年无表即 CI 红（25063ca「禁裸墙钟」教条的**故意例外**，docstring 已注明）
|
||||
- **④坐标**：表 monitor_registry.py:30-40（NAS 镜像=nas_monitor_check.py:30-42，CI 同步测钉相等）；剔除逻辑 data_monitor.py:444-459（main 调用 :476-478）；CI 年更闸 `test_holiday_table_covers_current_year`（tests/data_platform/test_data_monitor.py）——当年无表即 CI 红（25063ca「禁裸墙钟」教条的**故意例外**，docstring 已注明）
|
||||
- **⑤坑史**：表入册后三个「未确认日容忍」测试场景日失义（0925 已知假日→期望回落 0924 直接绿）→场景日重定向双删 0924+0925；fixture 远跳未来日期会塌日志层/mtime（rc=3 假红）→远窗场景用 `_calendar_days` 单元测直钉
|
||||
|
||||
### §5.6 row-band 滚动分位带（P2-6，2026-10-01 用户拍板「监控问题统一处理」，五件套）
|
||||
@@ -344,7 +346,7 @@ kind 语义（检查分发依据，check_entry data_monitor.py:56-232）：
|
||||
|
||||
### §7.6 wrapper 退出码纪律与 rc 语义族（调度层判读必读）
|
||||
- **监控 wrapper 恒 exit 0**：红态由 alerts/health JSON 承载——防污染 schtask Last Result 判读（python 侧 3=红，wrapper 转 0）
|
||||
- **例外=xt-daily rc=1 抬码**（09-29 起）：桥板块接口死时 ETF 清单空若仍 rc=0 会静默漏采（0928 全 ETF 停 0924 实案）→ ETF 清单空=仍写北交所但 rc=1，分诊信号=last_result=1+日志「FATAL ETF/基金 universe EMPTY」
|
||||
- **例外=xt-daily rc=1 抬码**（09-29 起）：桥板块接口死时 ETF 清单空若仍 rc=0 会静默漏采（0928 全 ETF 停 0924 实案）→ ETF 清单空=仍写北交所但 rc=1，分诊信号=last_result=1+日志「FATAL ETF/基金 universe 异常(…)」（xt_eod.py:190-192 实串——**10-05 C-A1 勘正**：原记「universe EMPTY」按 spec grep 零命中会误判无信号；runbook 侧同步勘正=infra 9aa14d1）
|
||||
- **bs-daily rc=2 = graceful 自愈**：baostock 登录失败 3 重试（120s/300s backoff）后退出 2（bs_eod.py:625），次日 LOOKBACK=7 自愈；**连两日 rc=2 才升级**（10-02 假期实案：10002007 网络接收错误）
|
||||
- **0x41301=运行中 / 0x41303=从未跑 / 0x41306=被终止**：schtasks Last Result 三态非错误码，值班勿当报警（ak-weekly 28h 长跑常现 41301）
|
||||
|
||||
@@ -493,11 +495,11 @@ kind 语义（检查分发依据，check_entry data_monitor.py:56-232）：
|
||||
| strategy registry v1.3 | scripts/strategy_monitor/strategy_registry.py | 79 | 十一检 :13-59（闸门/班次/阈值全参数；weekday 键已退役 10-05）；EXCHANGE_HOLIDAYS :69-79（与 data 表 CI 强制同步） |
|
||||
| strategy checks | scripts/strategy_monitor/strategy_checks.py | 399 | CHECK_FNS :387-399；needle :22 / LOG_GLOBS :26 / ro 日历 :65-80 / 排 live 桌 :87-98 / 双轨配对+未配对清单 :112-145 / no_restart 空日历降黄 :205-211 / storm unreadable 计数 :247-252 |
|
||||
| strategy wrapper | scripts/strategy_monitor/strategy_monitor_wrapper.ps1 | 16 | 恒 exit 0 :16（env 清代理+迁移日志位） |
|
||||
| data-monitor | scripts/data_platform/data_monitor.py | 535 | check_entry :56-232 / 日志层 :237-299 / schtask 层 :304-344 / inbox 摄取 :361-386 / `_calendar_days` :402-417 / main :419-531 |
|
||||
| data-monitor | scripts/data_platform/data_monitor.py | 615 | check_entry 单检隔离 :57（分发 :69-253）/ 日志层 collect_log_layer :264 / schtask 层 :346 / inbox 摄取 :403 / `_calendar_days` :444 / main 三段壳 :461（`_run_shift` :482+`_crash_exit` :586） |
|
||||
| data registry | scripts/data_platform/monitor_registry.py | 182 | EXCHANGE_HOLIDAYS :30-40 / _PER_DATE :50-69 / REGISTRY :152-153 / SCHTASKS 19 项 :162-182 |
|
||||
| data checks | scripts/data_platform/monitor_checks.py | 228 | expected_daily :37 / expected_weekly :49 / rolling_rows_band :103-133（常量 :98-100）/ parse_schtasks_list :206 |
|
||||
| alerts_store | sanguo_data/alerts_store.py | 159 | _SCHEMA :12-32 / upsert :57 / resolve_passed :83 / purge :96 / ack :134 / summary :149 |
|
||||
| NAS monitor v2 | scripts/nas_sync/nas_monitor_check.py + run_monitor_nas.sh | 245+44 | build_checks 六族 :120-214 / main+green_keys :217-243；宿主生效位=/volume1/stock/monitor_nas/（scp -O 同步纪律） |
|
||||
| data checks | scripts/data_platform/monitor_checks.py | 270 | expected_daily :37 / expected_weekly :49 / rolling_rows_band :106-138（常量 :101-103）/ check_disk 阈值参数化 :150 / `_error_lines` :190 / scan_signatures(仅测试) :196 / signatures_by_type :216 / parse_schtasks_list :248 |
|
||||
| alerts_store | sanguo_data/alerts_store.py | 168 | _SCHEMA :12-32 / upsert(severity 跟随最新) :59 / resolve_passed :88 / purge(open 黄 last_seen 28d) :101 / ack :143 / summary :158 |
|
||||
| NAS monitor v2 | scripts/nas_sync/nas_monitor_check.py + run_monitor_nas.sh | 266+47 | EXCHANGE_HOLIDAYS 镜像 :30-42 / last_weekday 交易日版 :43 / build_checks 六族 :141-235 / main+green_keys :238；宿主生效位=/volume1/stock/monitor_nas/（scp -O 同步纪律） |
|
||||
| QMT 哨兵链 | scripts/qmt_relogin/（回库 dbf57f2） | — | sentinel 188/probe 209/gate_common 179/relogin 243 |
|
||||
|
||||
---
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
"""alerts 告警事件存储(spec §20.15.4)——监控脚本单写、sanguo_api 只读+ack、NAS 侧经 inbox 摄取.
|
||||
|
||||
事件模型: 确定性 alert_id 幂等去重(重 Sight=同 id 再 upsert → count++ 即「第 N 晚」升级);
|
||||
事件模型: 确定性 alert_id 幂等去重(重 Sight=同 id 再 upsert → count++ 即「第 N 晚」升级;
|
||||
severity 跟随最新评估——同日黄→红升级即时落表, AL-1 10-05);
|
||||
check_key(去日期维度)驱动转绿自动 resolved;人工只 ack 不 resolved。
|
||||
保留期: resolved 30 天 / 黄项 14 天, 每班 purge。
|
||||
保留期: resolved 30 天 / open 黄按 last_seen 28 天(MON-202610-04 收口 10-05:
|
||||
活着不删) / open 红不删, 每班 purge。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
@@ -55,15 +57,18 @@ def _now() -> str:
|
||||
|
||||
|
||||
def upsert_alert(conn: sqlite3.Connection, event: dict) -> str:
|
||||
"""幂等写入一条告警事件; 重 Sight(同 alert_id 仍 open/acked)→count++/last_seen 刷新."""
|
||||
"""幂等写入一条告警事件; 重 Sight(同 alert_id 仍 open/acked)→count++/
|
||||
last_seen 刷新/severity 跟随最新评估(AL-1 10-05: 同日黄→红升级不再丢,
|
||||
旧版 UPDATE 不动 severity=次日新 id 才纠正, open_red 角标少计一晚)."""
|
||||
now = _now()
|
||||
alert_id = event["alert_id"]
|
||||
cur = conn.execute(
|
||||
"UPDATE alerts SET count=count+1, last_seen=?, detail=?, evidence=?, title=? "
|
||||
"UPDATE alerts SET count=count+1, last_seen=?, detail=?, evidence=?, "
|
||||
"title=?, severity=? "
|
||||
"WHERE alert_id=? AND status IN ('open','acked')",
|
||||
(now, event.get("detail", ""),
|
||||
json.dumps(event.get("evidence", {}), ensure_ascii=False),
|
||||
event["title"], alert_id))
|
||||
event["title"], event["severity"], alert_id))
|
||||
if cur.rowcount == 0:
|
||||
exists = conn.execute(
|
||||
"SELECT 1 FROM alerts WHERE alert_id=?", (alert_id,)).fetchone()
|
||||
@@ -94,14 +99,18 @@ def resolve_passed(conn: sqlite3.Connection, passed_keys, now: str = None) -> in
|
||||
|
||||
|
||||
def purge_expired(conn: sqlite3.Connection, now: str = None) -> int:
|
||||
"""resolved 留 30 天 / 黄项留 14 天(spec §20.15.4)."""
|
||||
"""resolved 留 30 天 / open 黄按 last_seen 留 28 天(MON-202610-04 收口
|
||||
10-05: 旧 first_seen 14d 硬删把每日重 Sight 的持续黄无声从值班页抹掉+
|
||||
count 升级链断——last_seen 每次重 Sight 刷新=「活着不删」, 僵死 28d 才清;
|
||||
resolved 黄统一走 resolved_at 30d, open 红永不删)."""
|
||||
now = now or _now()
|
||||
t = datetime.fromisoformat(now)
|
||||
cut_resolved = (t - timedelta(days=30)).isoformat(timespec="seconds")
|
||||
cut_yellow = (t - timedelta(days=14)).isoformat(timespec="seconds")
|
||||
cut_open_yellow = (t - timedelta(days=28)).isoformat(timespec="seconds")
|
||||
n = conn.execute(
|
||||
"DELETE FROM alerts WHERE (status='resolved' AND resolved_at<?)"
|
||||
" OR (severity='yellow' AND first_seen<?)", (cut_resolved, cut_yellow)
|
||||
" OR (status IN ('open','acked') AND severity='yellow'"
|
||||
" AND last_seen<?)", (cut_resolved, cut_open_yellow)
|
||||
).rowcount
|
||||
conn.commit()
|
||||
return n
|
||||
|
||||
@@ -152,7 +152,8 @@ def _check_entry_impl(e, ctx):
|
||||
return done("stale", "yellow", f"<={e['max_age_days']}d", f"{age}d")
|
||||
lo, hi = e["row_band"]
|
||||
rows = chk.parquet_rows(got[1])
|
||||
if rows is not None and not (lo <= rows <= hi):
|
||||
if rows is None or not (lo <= rows <= hi):
|
||||
# D-6(10-05): 损坏件 rows=None 不再静默绿——对齐 per_date 判定
|
||||
return done("rows", "yellow", f"{lo}~{hi}", rows)
|
||||
return done("stale", "green", ym, ym)
|
||||
|
||||
@@ -183,23 +184,33 @@ def _check_entry_impl(e, ctx):
|
||||
if kind == "db_sample":
|
||||
exp = chk.expected_daily(ctx["days"], ctx["now"], e.get("ready", "22:30"),
|
||||
e.get("lag", 0))
|
||||
worst = None
|
||||
worst, readable = None, False
|
||||
for sym, exc in e["samples"]:
|
||||
got_d = None
|
||||
try:
|
||||
conn = _ro_conn(os.path.join(ctx["data_root"], "quant_trading.db"))
|
||||
row = conn.execute(
|
||||
"SELECT MAX(datetime) FROM dbbardata WHERE symbol=? AND "
|
||||
"exchange=? AND interval='d'", (sym, exc)).fetchone()
|
||||
conn.close()
|
||||
try:
|
||||
row = conn.execute(
|
||||
"SELECT MAX(datetime) FROM dbbardata WHERE symbol=? AND "
|
||||
"exchange=? AND interval='d'", (sym, exc)).fetchone()
|
||||
finally:
|
||||
conn.close() # D-10(10-05): close 进 finally, 异常路径不泄漏
|
||||
got_d = (row[0] or "").replace("-", "")
|
||||
except Exception: # noqa: BLE001 只读探测, 库坏=黄红由调用方判
|
||||
got_d = None
|
||||
if got_d:
|
||||
readable = True
|
||||
if exp and (got_d is None or got_d < exp) and worst is None:
|
||||
worst = {"symbol": f"{sym}.{exc}", "expected": exp, "actual": got_d}
|
||||
if exp and worst:
|
||||
sev = "yellow" if exp not in ctx["confirmed_days"] \
|
||||
else e.get("severity", "red")
|
||||
return done("stale", sev, exp, worst["actual"], worst)
|
||||
if exp is None:
|
||||
# D-5(10-05): 日历不可达不再无条件绿(fail-open)——弱检查=样本可读
|
||||
# (对齐 per_date 存在性弱检), 库也不可达=黄
|
||||
return done("stale", "green" if readable else "yellow", "n/a",
|
||||
"sample-readable" if readable else "db-unreadable")
|
||||
return done("stale", "green", exp, "ok")
|
||||
|
||||
if kind == "vintage_json":
|
||||
@@ -213,8 +224,8 @@ def _check_entry_impl(e, ctx):
|
||||
try:
|
||||
stale = not checked or (ctx["now"] - dt.datetime.fromisoformat(
|
||||
checked)).total_seconds() > 26 * 3600
|
||||
except ValueError:
|
||||
stale = True
|
||||
except (ValueError, TypeError):
|
||||
stale = True # D-9(10-05): 非字符串 fromisoformat 抛 TypeError 不炸
|
||||
if stale:
|
||||
return done("verdict", "yellow", "<=26h", checked)
|
||||
# 结论无顶层字段(static_vintage_check 契约): 挖洞藏在 tables.*.holes_*
|
||||
@@ -237,7 +248,10 @@ def _check_entry_impl(e, ctx):
|
||||
"exists", exists)
|
||||
|
||||
if kind == "disk":
|
||||
r = chk.check_disk(ctx["data_root"])
|
||||
# D-8(10-05): 阈值消费注册表(旧硬编码 8/20 与注册表脱钩, crit_gb 全仓
|
||||
# 无消费=改表不改行为的校准陷阱)
|
||||
r = chk.check_disk(ctx["data_root"], warn_gb=e.get("warn_gb", 20),
|
||||
crit_gb=e.get("crit_gb", 8))
|
||||
return {"key": "infra-disk-data", "status": r["status"],
|
||||
"expected": f">={e['warn_gb']}G", "actual": f"{r['free_gb']}G",
|
||||
"evidence": r}
|
||||
@@ -530,7 +544,7 @@ def _run_shift(args, db_path):
|
||||
try:
|
||||
age_h = (now - dt.datetime.fromisoformat(nas_ts)
|
||||
).total_seconds() / 3600
|
||||
except ValueError:
|
||||
except (ValueError, TypeError): # D-9(10-05): TypeError 同防
|
||||
age_h = None
|
||||
if age_h is not None and age_h > 30:
|
||||
events.append(_event(nas_silent_key, "yellow",
|
||||
|
||||
@@ -10,6 +10,12 @@ $ts = Get-Date -Format 'yyyyMMdd_HHmmss'
|
||||
$logDir = 'C:\sanguo_vnpy_v2\data\migration_logs'
|
||||
if (-not (Test-Path $logDir)) { New-Item -ItemType Directory -Path $logDir -Force | Out-Null }
|
||||
$log = Join-Path $logDir "data_monitor_$ts.txt"
|
||||
# migration_logs 保洁(10-05 infra 插单): 13 个 wrapper 各自追加、无任何清理机制
|
||||
# =无限增长; 本 wrapper 双班跑必经, 顺手剪 14 天前旧件(PS 重定向副本无解析消费,
|
||||
# 监控日志层真身=static/logs/, 诊断残留而已)。
|
||||
Get-ChildItem -Path $logDir -File -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.LastWriteTime -lt (Get-Date).AddDays(-14) } |
|
||||
Remove-Item -ErrorAction SilentlyContinue
|
||||
C:\Python310\python.exe -X utf8 C:\sanguo_vnpy_v2\scripts\data_platform\data_monitor.py --shift $Shift *>> $log
|
||||
$rc = $LASTEXITCODE
|
||||
Add-Content -Path $log -Value "data_monitor shift=$Shift rc=$rc"
|
||||
|
||||
@@ -147,13 +147,15 @@ def dir_latest_mtime(directory):
|
||||
return latest
|
||||
|
||||
|
||||
def check_disk(path):
|
||||
def check_disk(path, warn_gb=20, crit_gb=8):
|
||||
"""磁盘剩余水位; 阈值参数化(D-8 10-05: 注册表 warn_gb/crit_gb 消费——旧
|
||||
硬编码 8/20 与注册表脱钩, crit_gb 曾全仓无消费=改表不改行为的校准陷阱)."""
|
||||
target = path if os.path.exists(path) else os.path.dirname(os.path.abspath(path))
|
||||
usage = shutil.disk_usage(target)
|
||||
free_gb = usage.free / 1024 ** 3
|
||||
if free_gb < 8:
|
||||
if free_gb < crit_gb:
|
||||
status = "red"
|
||||
elif free_gb < 20:
|
||||
elif free_gb < warn_gb:
|
||||
status = "yellow"
|
||||
else:
|
||||
status = "green"
|
||||
@@ -195,7 +197,10 @@ def scan_signatures(text):
|
||||
"""10-05 三修: ①只扫 ERROR/[FATAL] 行——'Expecting value' 出现在重试
|
||||
WARNING 行(financial_abstract/000963 实证), 良性重试不再误中
|
||||
source_changed; ②'418' 针收紧为 '418 Client Error'——裸 418 子串误中
|
||||
rows=41823/ok=19418 进度行(anti_crawl 假阳性实证)。"""
|
||||
rows=41823/ok=19418 进度行(anti_crawl 假阳性实证)。
|
||||
**注(10-05 补充审计)**: 生产路径已无调用方(签名分诊走 signatures_by_type,
|
||||
data_monitor.log_layer_results/log_only_results)——本函数仅测试消费,
|
||||
勿再接线; 与 signatures_by_type 的行为差异=不按 [type] 归因。"""
|
||||
lines = _error_lines(text)
|
||||
seen, out = set(), []
|
||||
for needle, sig, severity in SIGNATURES:
|
||||
@@ -211,10 +216,15 @@ _TYPE_TAG = re.compile(r"\[(\w+)\]")
|
||||
def signatures_by_type(text):
|
||||
"""签名按日志行 [type] 归因(10-05): 'ERROR [share_capital] ...' 行的
|
||||
签名只归 share_capital——防跨类型连坐(xueqiu 的 418 把 share_capital
|
||||
的 failed 抬红实证)。断路器行归触发类型, data_monitor 侧全局取。"""
|
||||
的 failed 抬红实证)。断路器行归触发类型, data_monitor 侧全局取。
|
||||
[FATAL] 是级别标记不是类型(P3 补充审计): 裸 FATAL 行
|
||||
(akshare_static_download.py:2721 实格式)归 _global, [type] [FATAL]
|
||||
双标记行(:2160 实格式)归 [type]。"""
|
||||
out = {}
|
||||
for ln in _error_lines(text):
|
||||
m = _TYPE_TAG.search(ln)
|
||||
while m and m.group(1) == "FATAL":
|
||||
m = _TYPE_TAG.search(ln, m.end())
|
||||
t = m.group(1) if m else "_global"
|
||||
sigs = out.setdefault(t, [])
|
||||
for needle, sig, severity in SIGNATURES:
|
||||
|
||||
@@ -21,13 +21,34 @@ XCHECK_REPORTS = os.path.join(CORPUS_ROOT, "xcheck", "reports")
|
||||
OUT_DIR = "/volume1/stock/monitor_nas/out"
|
||||
DISK_WARN_GB = 100 # 初值, 首周观察校准(spec §20.15.1 磁盘水位 NAS 侧)
|
||||
|
||||
# 沪深北休市表镜像(N-2 10-05)——=monitor_registry.EXCHANGE_HOLIDAYS, CI 同步测
|
||||
# (test_holiday_table_in_sync_with_vps_registry)钉两表相等, 单边改表即红(同
|
||||
# 7f9b413 strategy 侧镜像纪律); 年更义务经 VPS 侧年更闸+同步测传导到本表。
|
||||
# 旧 last_weekday 只剥周末: 0925 中秋/国庆连休窗把假日当「昨交易日」——当前被
|
||||
# hot_rank producer 的日历日节奏掩住(files 假期照落), 语义仍错。表缺年=不扣
|
||||
# (fail-safe, 黄照旧绝不静默转绿)。
|
||||
EXCHANGE_HOLIDAYS = {
|
||||
2026: {
|
||||
"20260101", "20260102", # 元旦
|
||||
"20260216", "20260217", "20260218", "20260219", "20260220", "20260223", # 春节
|
||||
"20260406", # 清明
|
||||
"20260501", "20260504", "20260505", # 劳动节
|
||||
"20260619", # 端午
|
||||
"20260925", # 中秋
|
||||
"20261001", "20261002", "20261005", "20261006", "20261007", # 国庆(1008 复市)
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def last_weekday(before):
|
||||
"""strictly before 且为周一~五的最近一天(YYYYMMDD)——16:00 查时当日 17:35 班未采."""
|
||||
"""strictly before 且为交易日(周一~五且非休市表日)的最近一天(YYYYMMDD)
|
||||
——16:00 查时当日 17:35 班未采; N-2: 接休市表镜像, 跳过连休假日."""
|
||||
d = before - dt.timedelta(days=1)
|
||||
while d.weekday() >= 5:
|
||||
while True:
|
||||
hol = EXCHANGE_HOLIDAYS.get(d.year)
|
||||
if d.weekday() < 5 and (hol is None or d.strftime("%Y%m%d") not in hol):
|
||||
return d.strftime("%Y%m%d")
|
||||
d -= dt.timedelta(days=1)
|
||||
return d.strftime("%Y%m%d")
|
||||
|
||||
|
||||
def mtime_age_hours(path, now):
|
||||
|
||||
@@ -21,7 +21,10 @@ mkdir -p "$ROOT/out"
|
||||
{
|
||||
echo "=== $(date '+%F %T') nas monitor start ==="
|
||||
"$PY" "$ROOT/nas_monitor_check.py"
|
||||
echo "=== $(date '+%F %T') nas monitor exit=$? ==="
|
||||
rc_nas=$?
|
||||
# N-1(10-05): rc 先存后拼——旧版 $(date) 在同一 echo 词内先展开把 $? 重置
|
||||
# 为 0, exit= 恒 0; rc 隔离下这是 NAS 体检崩溃在 cron.log 的唯一现场线索
|
||||
echo "=== $(date '+%F %T') nas monitor exit=$rc_nas ==="
|
||||
# 远端 inbox 目录幂等建好; ssh 必带 -n(vps-mirror 坑: 不带会吃 stdin)
|
||||
ssh -n "${SCP_OPTS[@]}" "$VPS" \
|
||||
"powershell -Command \"New-Item -ItemType Directory -Force -Path '$VPS_INBOX' | Out-Null\"" \
|
||||
|
||||
@@ -53,6 +53,14 @@ class TestUpsert:
|
||||
rows, _ = store.list_alerts(conn)
|
||||
assert rows[0]["evidence"] == _event()["evidence"]
|
||||
|
||||
def test_resighting_updates_severity(self, conn):
|
||||
"""AL-1(10-05): 同日黄→红升级不再在 alerts 层丢失(旧 UPDATE 不更
|
||||
severity, 次日新 id 才纠正=前端 open_red 角标少计一晚)。"""
|
||||
store.upsert_alert(conn, _event(severity="yellow"))
|
||||
store.upsert_alert(conn, _event()) # 同 alert_id, severity=red
|
||||
rows, _ = store.list_alerts(conn)
|
||||
assert rows[0]["severity"] == "red" and rows[0]["count"] == 2
|
||||
|
||||
|
||||
class TestResolveAndPurge:
|
||||
def test_resolve_passed_closes_open_and_acked(self, conn):
|
||||
@@ -66,18 +74,31 @@ class TestResolveAndPurge:
|
||||
store.upsert_alert(conn, _event())
|
||||
assert store.resolve_passed(conn, ["data-other"], "2026-09-26T00:00:00") == 0
|
||||
|
||||
def test_purge_resolved_30d_and_yellow_14d(self, conn):
|
||||
store.upsert_alert(conn, _event()) # red open 留
|
||||
def test_purge_resolved_30d_open_yellow_by_last_seen(self, conn):
|
||||
"""MON-202610-04 收口(10-05 P2 批): open 黄改按 last_seen 28d 清——
|
||||
旧 first_seen 14d 硬删把「每日重 Sight 的持续黄」(first_seen 老/
|
||||
last_seen 新)无声从值班页抹掉+count 升级链断; last_seen 每次重 Sight
|
||||
刷新=活着不删, 僵死 28d 才清。resolved 统一 30d, open 红永不删。"""
|
||||
store.upsert_alert(conn, _event()) # red open: 永不 purge
|
||||
# 活着的 open 黄: first_seen 31d 前, last_seen=今日(每日重 Sight)
|
||||
store.upsert_alert(conn, _event(
|
||||
alert_id="data-static-m-stale-20260901", check_key="data-static-m-stale",
|
||||
severity="yellow"))
|
||||
conn.execute(
|
||||
"UPDATE alerts SET first_seen='2026-08-25T00:00:00'"
|
||||
" WHERE check_key='data-static-m-stale'")
|
||||
# 僵死的 open 黄: last_seen 停在 28d 窗外
|
||||
store.upsert_alert(conn, _event(
|
||||
alert_id="data-static-s-stale-20260801", check_key="data-static-s-stale",
|
||||
severity="yellow"))
|
||||
conn.execute(
|
||||
"UPDATE alerts SET first_seen='2026-08-10T00:00:00',"
|
||||
" last_seen='2026-08-20T00:00:00'"
|
||||
" WHERE check_key='data-static-s-stale'")
|
||||
# resolved 30d 前
|
||||
store.upsert_alert(conn, _event(
|
||||
alert_id="data-static-n-stale-20260901", check_key="data-static-n-stale",
|
||||
severity="red"))
|
||||
# 回拨时间位: 黄项 first_seen 拨到 14 天前, 红项拨成 30 天前已 resolved
|
||||
conn.execute(
|
||||
"UPDATE alerts SET first_seen='2026-08-01T00:00:00'"
|
||||
" WHERE check_key='data-static-m-stale'")
|
||||
conn.execute(
|
||||
"UPDATE alerts SET first_seen='2026-08-01T00:00:00',"
|
||||
" status='resolved', resolved_at='2026-08-20T00:00:00'"
|
||||
@@ -85,7 +106,8 @@ class TestResolveAndPurge:
|
||||
conn.commit()
|
||||
n = store.purge_expired(conn, "2026-09-25T00:00:00")
|
||||
keys = {r["check_key"] for r in store.list_alerts(conn)[0]}
|
||||
assert n == 2 and keys == {"data-static-xueqiu_hot-stale"}
|
||||
assert n == 2 and keys == {"data-static-xueqiu_hot-stale",
|
||||
"data-static-m-stale"}
|
||||
|
||||
|
||||
class TestAckAndSummary:
|
||||
|
||||
@@ -677,3 +677,54 @@ def test_trading_days_readside_normalization(tmp_path):
|
||||
conn.close()
|
||||
days = dm.chk.trading_days(str(db), dt.date(2026, 9, 26))
|
||||
assert days == ["2026-09-24", "2026-09-25"]
|
||||
|
||||
|
||||
# ---------- 10-05 班次二 P2 批(D-5/6/8/9) ----------
|
||||
|
||||
def test_db_sample_no_calendar_weak_check_not_fail_open(tree, tmp_path):
|
||||
"""D-5: 日历不可达(exp=None)不再无条件绿——弱检查=样本可读(对齐 per_date
|
||||
存在性弱检): 库可读且样本有数据=绿, 库不可达=黄, 不 fail-open。"""
|
||||
e = next(x for x in reg.REGISTRY if x["name"] == "dbbardata_daily")
|
||||
ctx = {"data_root": str(tree), "days": [], "now": FAKE_NOW,
|
||||
"confirmed_days": set()}
|
||||
assert dm.check_entry(e, ctx)["status"] == "green"
|
||||
(tree / "quant_trading.db").rename(tree / "quant_trading.db.bak")
|
||||
r = dm.check_entry(e, ctx)
|
||||
assert r["status"] == "yellow"
|
||||
assert r["actual"] == "db-unreadable"
|
||||
|
||||
|
||||
def test_month_bucket_corrupt_parquet_is_yellow(tree):
|
||||
"""D-6: month_bucket 最新桶损坏(parquet_rows=None)不再静默绿——
|
||||
对齐 per_date rows=None→黄(audit: 与 per_date 判定不一致)。"""
|
||||
e = next(x for x in reg.REGISTRY if x["kind"] == "month_bucket")
|
||||
p = tree / "static" / e["subdir"] / "202609_pledge_detail.parquet"
|
||||
p.write_bytes(b"PAR1garbage-truncated")
|
||||
r = dm.check_entry(e, _ctx(tree))
|
||||
assert r["key"] == "data-pledge_detail-rows" and r["status"] == "yellow"
|
||||
|
||||
|
||||
def test_disk_consumes_registry_thresholds(tmp_path, monkeypatch):
|
||||
"""D-8: check_disk 消费注册表 warn_gb/crit_gb(旧硬编码 8/20 与注册表脱钩,
|
||||
crit_gb 全仓无消费=改注册表不改变行为的校准陷阱)。"""
|
||||
from collections import namedtuple
|
||||
usage = namedtuple("usage", "total used free")
|
||||
monkeypatch.setattr(dm.chk.shutil, "disk_usage",
|
||||
lambda p: usage(100, 0, 10 * 1024 ** 3))
|
||||
e = next(x for x in reg.REGISTRY if x["name"] == "disk")
|
||||
ctx = {"data_root": str(tmp_path)}
|
||||
assert dm.check_entry(e, ctx)["status"] == "yellow" # 10G: <20 黄 >=8 不红
|
||||
monkeypatch.setitem(e, "crit_gb", 12)
|
||||
assert dm.check_entry(e, ctx)["status"] == "red" # 改表即改行为
|
||||
|
||||
|
||||
def test_vintage_checked_at_nonstring_degrades_local_yellow(tree):
|
||||
"""D-9: checked_at 非字符串(fromisoformat TypeError)被本地捕获走 verdict
|
||||
黄——旧版只捕 ValueError, TypeError 逸出靠 D-2 单检隔离兜(键变
|
||||
data-vintage-stale), 本修复让黄走对的 verdict 键。"""
|
||||
p = tree / "static" / "vintage_status.json"
|
||||
p.write_text(json.dumps({"checked_at": 12345, "tables": {}}),
|
||||
encoding="utf-8")
|
||||
e = next(x for x in reg.REGISTRY if x["name"] == "vintage")
|
||||
r = dm.check_entry(e, _ctx(tree))
|
||||
assert r["key"] == "data-vintage-verdict" and r["status"] == "yellow"
|
||||
|
||||
@@ -128,6 +128,18 @@ class TestLogParse:
|
||||
assert {d["sig"] for d in by_type["xueqiu_hot"]} == {"anti_crawl"}
|
||||
assert {d["sig"] for d in by_type["share_capital"]} == {"field_changed"}
|
||||
|
||||
def test_signatures_by_type_fatal_marker_not_a_type(self):
|
||||
"""补充审计 P3: [FATAL] 是级别标记不是类型——裸 FATAL 行
|
||||
(akshare_static_download.py:2721 实格式)归 _global 不再入幽灵
|
||||
'FATAL' 类型桶; [type] [FATAL] 双标记行(:2160 实格式)仍归 [type]。"""
|
||||
bare = "2026-10-05 12:00:00 [FATAL] 获取股票列表失败: timeout"
|
||||
out = mc.signatures_by_type(bare)
|
||||
assert "FATAL" not in out and "_global" in out
|
||||
typed = ("2026-10-05 12:00:00 [margin] [FATAL] 断路器触发: "
|
||||
"连续 30 个 unit 失败, akshare 疑似不可达")
|
||||
out = mc.signatures_by_type(typed)
|
||||
assert "margin" in out and out["margin"][0]["sig"] == "breaker"
|
||||
|
||||
|
||||
class TestSchtaskParse:
|
||||
def test_bilingual_and_na(self):
|
||||
@@ -155,3 +167,15 @@ class TestDisk:
|
||||
def test_check_disk_returns_status(self, tmp_path):
|
||||
r = mc.check_disk(str(tmp_path))
|
||||
assert r["status"] in ("green", "yellow", "red") and r["free_gb"] > 0
|
||||
|
||||
def test_check_disk_consumes_threshold_params(self, tmp_path, monkeypatch):
|
||||
"""D-8: 阈值参数化——旧硬编码 8/20 改表不改行为(校准陷阱)。"""
|
||||
from collections import namedtuple
|
||||
usage = namedtuple("usage", "total used free")
|
||||
monkeypatch.setattr(mc.shutil, "disk_usage",
|
||||
lambda p: usage(100, 0, 10 * 1024 ** 3))
|
||||
assert mc.check_disk(str(tmp_path))["status"] == "yellow" # 10G: 默认 8/20
|
||||
assert mc.check_disk(str(tmp_path), warn_gb=50,
|
||||
crit_gb=30)["status"] == "red" # 10<30 crit
|
||||
assert mc.check_disk(str(tmp_path), warn_gb=50,
|
||||
crit_gb=5)["status"] == "yellow" # 5<=10<50 warn
|
||||
|
||||
@@ -12,13 +12,34 @@ import nas_monitor_check as nmc # noqa: E402
|
||||
|
||||
@pytest.mark.parametrize("today, want", [
|
||||
(dt.date(2026, 9, 25), "20260924"), # 五→四
|
||||
(dt.date(2026, 9, 28), "20260925"), # 一→上五
|
||||
(dt.date(2026, 9, 27), "20260925"), # 日→五
|
||||
(dt.date(2026, 9, 28), "20260924"), # 一→跳过 0925 中秋休市(N-2 后=真昨交易日)
|
||||
(dt.date(2026, 9, 27), "20260924"), # 日→跳周末+0925 休市
|
||||
(dt.date(2026, 10, 6), "20260930"), # 国庆连休窗内: 期望钉在节前末交易日
|
||||
(dt.date(2026, 10, 8), "20260930"), # 复市日 16:00 查: 当日 17:35 未采
|
||||
(dt.date(2027, 1, 2), "20270101"), # 表缺年 fail-safe: 不扣(黄照旧语义)
|
||||
])
|
||||
def test_last_weekday(today, want):
|
||||
assert nmc.last_weekday(today) == want
|
||||
|
||||
|
||||
def test_holiday_table_in_sync_with_vps_registry():
|
||||
"""N-2: NAS 休市表镜像与 VPS monitor_registry 同源(CI 钉两表相等,
|
||||
单边改表即红——同 7f9b413 strategy 侧镜像纪律)。"""
|
||||
import monitor_registry as reg
|
||||
assert nmc.EXCHANGE_HOLIDAYS == reg.EXCHANGE_HOLIDAYS
|
||||
|
||||
|
||||
def test_hot_rank_file_missing_is_red(tmp_path, monkeypatch):
|
||||
"""T-2: NAS 侧唯一 red 分支(hot_rank 昨交易日文件缺)补测试覆盖——
|
||||
audit: 该分支零覆盖, rc 隔离下它是 NAS 崩溃/异常外唯一 red 源。"""
|
||||
monkeypatch.setattr(nmc, "HOTRANK_DIR", str(tmp_path))
|
||||
checks, events = nmc.build_checks(dt.datetime(2026, 9, 21, 16))
|
||||
f = [c for c in checks if c["key"] == "nas-hot_rank-file"][0]
|
||||
assert f["status"] == "red" and f["evidence"]["expected"] == "20260918"
|
||||
assert any(e["check_key"] == "nas-hot_rank-file" and e["severity"] == "red"
|
||||
for e in events)
|
||||
|
||||
|
||||
def test_mtime_age_hours(tmp_path):
|
||||
f = tmp_path / "f"
|
||||
f.touch()
|
||||
|
||||
Reference in New Issue
Block a user