fix(nas): sync_parquet_daily 扩到 A 档 13 类+gdhs 全量拉——巡查实锤 TYPES 写死旧 4 类,明晨 03:00 快照族/涨停池到不了 NAS [nas]
09-02 巡查发现: sync_parquet_daily.sh TYPES 硬编码 dragon_tiger/block_trade/ margin_sse/restricted,今晚 19:30 A 档首夜产出(快照族 6 类+涨停池×3)不在拉取 清单,明晨同步后 NAS 回测侧依旧零文件(provider §16 出口在 NAS 断供)。 修: TYPES 扩到 13 类(+zt_pool×3/fund_flow×2/ths×2/xueqiu_hot/sina_sector, 7 天滚动窗逻辑不变);gdhs=per-period(季度末日期永不落 7 天窗)单独全量拉 (dir /b 枚举后逐文件 scp,≤10 文件幂等,周六 ak-weekly 产出)。 生效位=/volume1/stock(scripts/nas_sync/),CI 不覆盖,需手动部署 NAS;部署后 手动跑一轮验证=顺带把今晚首夜数据提前送 NAS(不等明晨 03:00)。
This commit is contained in:
@@ -7,8 +7,10 @@
|
||||
# - per-date 类文件名含日期(YYYYMMDD_type.parquet),可按日期精确拉(scp 单文件,不扫目录)。
|
||||
# - per-stock 类(valuation/三表/northbound 等)文件名无日期,走 sync_parquet.sh 周度全量。
|
||||
#
|
||||
# 覆盖:ak-events 每日产出(dragon_tiger/block_trade/margin_sse/restricted)。
|
||||
# 覆盖:ak-events 每日产出。2026-09-02 起扩到 13 类(原 4 类事件 + A 档 panel 9 类:
|
||||
# zt_pool×3 + fund_flow×2 + ths×2 + xueqiu_hot + sina_sector,见活文档 §15)。
|
||||
# 每日拉最近 7 天窗口(容错漏跑几天;节假日无文件=scp 失败=skip 正常)。
|
||||
# gdhs 为 per-period(季度末日期,永不落 7 天窗),单独全量拉(≤10 个文件,幂等)。
|
||||
set -u
|
||||
|
||||
KEY=/var/services/homes/admin/.ssh/id_ed25519_nas
|
||||
@@ -17,7 +19,7 @@ ROOT=/volume1/stock/sanguo_vnpy_v2
|
||||
DATA=$ROOT/data
|
||||
LOG=$ROOT/data_backup/parquet_daily.log
|
||||
|
||||
TYPES="dragon_tiger block_trade margin_sse restricted"
|
||||
TYPES="dragon_tiger block_trade margin_sse restricted zt_pool zt_pool_zbgc zt_pool_dtgc fund_flow_industry fund_flow_concept ths_industry ths_concept xueqiu_hot sina_sector"
|
||||
WINDOW_DAYS=7
|
||||
|
||||
TS=$(date '+%F %T')
|
||||
@@ -40,4 +42,17 @@ for type in $TYPES; do
|
||||
done
|
||||
done
|
||||
|
||||
# gdhs per-period 全量拉(季度末文件名不在滚动窗口内;周六 ak-weekly 产出,≤10 文件)
|
||||
mkdir -p "$DATA/static/gdhs"
|
||||
for f in $(ssh -i "$KEY" -o StrictHostKeyChecking=no \
|
||||
"$VPS" "dir /b C:\\sanguo_vnpy_v2\\data\\static\\gdhs\\*.parquet" 2>/dev/null); do
|
||||
if scp -q -i "$KEY" -o StrictHostKeyChecking=no \
|
||||
"$VPS:C:/sanguo_vnpy_v2/data/static/gdhs/$f" \
|
||||
"$DATA/static/gdhs/" 2>/dev/null; then
|
||||
PULLED=$((PULLED + 1))
|
||||
else
|
||||
SKIPPED=$((SKIPPED + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
echo "[$(date '+%T')] DAILY DONE pulled=$PULLED skipped=$SKIPPED (skip=节假日/未出,正常)" >> "$LOG"
|
||||
|
||||
Reference in New Issue
Block a user