fix(nas): NAS同步链补两个断更域+周日周更全量挂载——2026-08-21晚全域巡检实锤:每日链(dbbardata增量03:00/小表/events四类/valuation+financial_abstract 03:09)全绿,但①valuation_baostock(bs_eod每晚写pe/pb按年parquet)NAS断更自08-16②fundamentals_baostock(08-20新域,fund腿每晚回灌profit/dupont)NAS完全不存在,两域无任何同步器覆盖;③per-stock周更14子域(balance/cashflow/top_holders等,sync_parquet.sh 6.24G全量)从未被调度,仅08-16手动跑过一次,top_holders已漏VPS 08-16 18:18批次。修=扩展sync_valuation_daily.sh(每日实跑载体):循环外加valuation_baostock+fundamentals_baostock两域scp -r(MB级秒级,03:09时VPS写盘已静止,同款3次重试)+周日(date +%u=7)setsid脱离挂sync_parquet.sh周更全量(ak-weekly周六重写后周日兜底);sync_parquet.sh全量清单补fundamentals_baostock源+du对齐;bash -n过 [nas]
This commit is contained in:
@@ -16,7 +16,8 @@ SCP_OK=0
|
||||
for attempt in 1 2 3; do
|
||||
if scp -r -i "$KEY" -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o ServerAliveCountMax=20 \
|
||||
"$VPS:C:/sanguo_vnpy_v2/data/static" \
|
||||
"$VPS:C:/sanguo_vnpy_v2/data/valuation_baostock" ./ >> "$LOG" 2>&1; then
|
||||
"$VPS:C:/sanguo_vnpy_v2/data/valuation_baostock" \
|
||||
"$VPS:C:/sanguo_vnpy_v2/data/fundamentals_baostock" ./ >> "$LOG" 2>&1; then
|
||||
SCP_OK=1
|
||||
break
|
||||
fi
|
||||
@@ -25,5 +26,5 @@ for attempt in 1 2 3; do
|
||||
done
|
||||
echo "SCP_RC OK=$SCP_OK $(date)" >> "$LOG"
|
||||
|
||||
du -sh "$DATA/static" "$DATA/valuation_baostock" >> "$LOG" 2>&1
|
||||
du -sh "$DATA/static" "$DATA/valuation_baostock" "$DATA/fundamentals_baostock" >> "$LOG" 2>&1
|
||||
echo "PARQUET_DONE $(date)" >> "$LOG"
|
||||
|
||||
@@ -34,4 +34,29 @@ for d in $DIRS; do
|
||||
echo "[$(date '+%T')] $d $( [ "$OK" = 1 ] && echo OK || echo FAIL )" >> "$LOG"
|
||||
done
|
||||
|
||||
# 根级每日域(2026-08-21 巡检补链): valuation_baostock(bs_eod 每晚写 pe/pb 按年 parquet,
|
||||
# NAS 断更自 08-16) + fundamentals_baostock(bs_fund 腿每晚回灌 profit/dupont, 08-20 新域
|
||||
# 此前 NAS 完全没有)。两域均为 MB 级 parquet, scp -r 秒级, 03:09 时 VPS 写盘已静止。
|
||||
for d in valuation_baostock fundamentals_baostock; do
|
||||
OK=0
|
||||
for attempt in 1 2 3; do
|
||||
if scp -r -i "$KEY" -o StrictHostKeyChecking=no -o ServerAliveInterval=30 -o ServerAliveCountMax=20 \
|
||||
"$VPS:C:/sanguo_vnpy_v2/data/$d" "$DATA/" >> "$LOG" 2>&1; then
|
||||
OK=1
|
||||
break
|
||||
fi
|
||||
echo "[$(date '+%T')] $d scp attempt $attempt fail, retry..." >> "$LOG"
|
||||
sleep 5
|
||||
done
|
||||
echo "[$(date '+%T')] $d $( [ "$OK" = 1 ] && echo OK || echo FAIL )" >> "$LOG"
|
||||
done
|
||||
|
||||
# 周日顺带全量刷 per-stock 周更域(ak-weekly 周六重写 balance/cashflow/top_holders 等
|
||||
# 14 子域; sync_parquet.sh 6.24G/70min, setsid 脱离不阻塞当日链; 上次全量 2026-08-16
|
||||
# 手动, 其后 top_holders 等已滞后)。与每日增量互补: 每日链兜住快变域, 周日兜住慢变域。
|
||||
if [ "$(date +%u)" = "7" ]; then
|
||||
nohup setsid bash "$(dirname "$0")/sync_parquet.sh" </dev/null >> "$LOG" 2>&1 &
|
||||
echo "[$(date '+%T')] sunday weekly parquet full sync launched (detached)" >> "$LOG"
|
||||
fi
|
||||
|
||||
echo "[$(date '+%T')] valuation daily DONE" >> "$LOG"
|
||||
|
||||
Reference in New Issue
Block a user