fix(data): ak-weekly 断路器饿死修复=_safe_northbound_em+类型调序 [vps]——09-05 03:00 首跑 rc=2 根因:akshare stock_hsgt_individual_em 对无北向数据标的(~1117/2883,301xxx 等)在 data_json[result][pages] 抛 TypeError NoneType not subscriptable→failed→30 连败断路器→跳过后续类型,share_capital/top_holders/gdhs 全饿死(gdhs 目录至今不存在);端点健康(600519 返 1683 行,09-06 晨复测);修=①_safe_northbound_em 子串匹配 not subscriptable 返空 df 带 9 列 schema→走无北向持仓 empty 写空+done 既定语义不耗重试/断路器(同 _safe_top_10_em 手法)②weekly wrapper 调序 gdhs,share_capital,top_holders,northbound 最大最险者殿后防再连累③顺手 BOM;TDD 2 新测先红后绿+217 全绿;双脚本已 scp 生效位+Parse/py_compile 预检过;gdhs 手动补跑另行 [vps]
This commit is contained in:
@@ -1,5 +1,7 @@
|
||||
# ak_stock_wrapper.ps1 — sanguo-ak-weekly schtask wrapper (weekly 周六03:00 akshare per-stock 慢)
|
||||
# northbound+share_capital+top_holders+gdhs --force; per-stock 全量慢, 周末夜间;
|
||||
# ak_stock_wrapper.ps1 — sanguo-ak-weekly schtask wrapper (weekly 周六03:00 akshare per-stock 慢)
|
||||
# --force 全量刷新; 顺序=gdhs,share_capital,top_holders,northbound (09-06 调序:
|
||||
# 09-05 northbound 1117/2883 无数据标的连败触发断路器, 饿死后续 3 类型——
|
||||
# 最大最险的 northbound 挪最后, 即使再断路也不连累前面);
|
||||
# gdhs=股东户数 per-period 全市场单次 (09-02 入列, 披露守卫>90天季度末, marker 去重无新期零网络)
|
||||
$env:http_proxy = ''
|
||||
$env:https_proxy = ''
|
||||
@@ -9,5 +11,5 @@ $ts = Get-Date -Format 'yyyyMMdd_HHmmss'
|
||||
$logDir = 'C:\sanguo_vnpy_v2\data\migration_logs'
|
||||
if (-not (Test-Path $logDir)) { New-Item -ItemType Directory -Path $logDir -Force | Out-Null }
|
||||
$log = Join-Path $logDir "ak_stock_$ts.txt"
|
||||
C:\Python310\python.exe -X utf8 C:\sanguo_vnpy_v2\scripts\data_platform\akshare_static_download.py --types northbound,share_capital,top_holders,gdhs --force *>> $log
|
||||
C:\Python310\python.exe -X utf8 C:\sanguo_vnpy_v2\scripts\data_platform\akshare_static_download.py --types gdhs,share_capital,top_holders,northbound --force *>> $log
|
||||
exit $LASTEXITCODE
|
||||
|
||||
@@ -496,10 +496,44 @@ def fetch_valuation(symbol: str) -> Optional[pd.DataFrame]:
|
||||
return df
|
||||
|
||||
|
||||
NORTHBOUND_COLUMNS = [
|
||||
"持股日期", "当日收盘价", "当日涨跌幅", "持股数量", "持股市值",
|
||||
"持股数量占A股百分比", "今日增持股数", "今日增持资金", "今日持股市值变化",
|
||||
]
|
||||
|
||||
|
||||
def _safe_northbound_em(symbol: str) -> pd.DataFrame:
|
||||
"""akshare stock_hsgt_individual_em 包装: 容忍无北向数据标的的崩溃响应。
|
||||
|
||||
bug 根因 (实证 akshare 1.18.x stock_hsgt_em.py:1446, 2026-09-05 ak-weekly
|
||||
断路器事故): 无北向数据标的 (301xxx 等 ~1117/2883) datacenter 返
|
||||
result=None, __stock_hsgt_zh_a_em 的 data_json["result"]["pages"] 抛
|
||||
TypeError("'NoneType' object is not subscriptable") → failed → 30 连败
|
||||
断路器 rc=2, 饿死后续 share_capital/top_holders/gdhs。端点健康
|
||||
(600519 正常返 1683 行)。
|
||||
|
||||
子串匹配 "not subscriptable" (稳定签名, 同 _safe_top_10_em 手法):
|
||||
→ 返空 df (带 NORTHBOUND_COLUMNS schema), 走「无北向持仓」empty
|
||||
写空+done (脚本头部既定语义), 不消耗重试/断路器配额;
|
||||
其他 TypeError 透传给 call_ak_with_retry 走重试。
|
||||
"""
|
||||
try:
|
||||
return ak.stock_hsgt_individual_em(symbol=symbol)
|
||||
except TypeError as e:
|
||||
if "not subscriptable" in str(e):
|
||||
logger.debug(
|
||||
"stock_hsgt_individual_em(%s) → 空 (无北向数据), 返空 df", symbol)
|
||||
return pd.DataFrame(columns=NORTHBOUND_COLUMNS)
|
||||
raise
|
||||
|
||||
|
||||
def fetch_northbound(symbol: str) -> Optional[pd.DataFrame]:
|
||||
"""stock_hsgt_individual_em(symbol='600519') — 北向持股 (~1700行/股)。"""
|
||||
"""stock_hsgt_individual_em(symbol='600519') — 北向持股 (~1700行/股)。
|
||||
|
||||
通过 _safe_northbound_em 包装: 无北向数据标的 akshare 抛 NoneType
|
||||
not subscriptable, 捕获返空 df (empty 语义, 不耗断路器配额)。"""
|
||||
df, _status = call_ak_with_retry(
|
||||
ak.stock_hsgt_individual_em, f"northbound/{symbol}", symbol=symbol,
|
||||
_safe_northbound_em, f"northbound/{symbol}", symbol=symbol,
|
||||
)
|
||||
return df
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
import argparse
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
import akshare_static_download as mod # noqa: E402 (conftest 已加 sys.path)
|
||||
|
||||
@@ -205,3 +206,34 @@ class TestHotAndGdhsFamily:
|
||||
ids = [u[0] for u in units]
|
||||
assert "20250930_gdhs" in ids
|
||||
assert "20251231_gdhs" not in ids
|
||||
|
||||
|
||||
# ======================== ③ northbound 无数据崩溃安全包装 ========================
|
||||
|
||||
class TestSafeNorthbound:
|
||||
"""_safe_northbound_em (2026-09-05 ak-weekly 断路器事故修复)。
|
||||
|
||||
akshare stock_hsgt_individual_em 对无北向数据标的 (301xxx 等) 在
|
||||
data_json["result"]["pages"] 处抛 TypeError(NoneType not subscriptable)
|
||||
—— 周六实测 2883 股中 1117 连败, 末段 30 连败触发断路器 rc=2,
|
||||
饿死后续 share_capital/top_holders/gdhs (gdhs 目录至今不存在)。
|
||||
端点本身健康 (600519 正常返 1683 行, 09-06 晨复测)。
|
||||
确定性无数据 → 空 df 带 schema → 「无北向持仓」empty 写空+done
|
||||
(脚本头部既定语义), 不消耗重试/断路器配额 (同 _safe_top_10_em 手法)。
|
||||
"""
|
||||
|
||||
def test_no_data_crash_returns_empty_with_schema(self, monkeypatch):
|
||||
def boom(symbol):
|
||||
raise TypeError("'NoneType' object is not subscriptable")
|
||||
monkeypatch.setattr(mod.ak, "stock_hsgt_individual_em", boom)
|
||||
df = mod._safe_northbound_em("301638")
|
||||
assert df.empty
|
||||
assert list(df.columns) == mod.NORTHBOUND_COLUMNS
|
||||
|
||||
def test_other_typeerror_propagates(self, monkeypatch):
|
||||
"""非无数据签名的 TypeError 透传 (走 call_ak_with_retry 重试)。"""
|
||||
def boom(symbol):
|
||||
raise TypeError("connection reset by peer")
|
||||
monkeypatch.setattr(mod.ak, "stock_hsgt_individual_em", boom)
|
||||
with pytest.raises(TypeError):
|
||||
mod._safe_northbound_em("301638")
|
||||
|
||||
Reference in New Issue
Block a user