我有一個在 for 回圈中并行運行的 bash 腳本,如下所示:
for (( n=0; n<${LIMIT}; n )); do
my_function $n &
my_function是一個LIMIT使用 ssh在虛擬機/節點中運行的函式。它作業正常,但我無法跟蹤執行進度。有一個進度條或任何其他可以給出完成估計的工具會很好。不幸的是,我搜索過的所有方法都不適用于這種并行情況。
謝謝
uj5u.com熱心網友回復:
下面的腳本使用廣泛打包的第 3 方工具pv(“管道查看器”)來撰寫進度條。
pv從 stdin 讀取輸入并將其寫入 stdout -- 但可以在此程序中將進度條(以及可選的 ETA 和其他統計資訊)寫入 stderr。它最常用于諸如 的情況curl | pv | tar,但也非常適合此用例。
#!/usr/bin/env bash
# version check: make sure automatic FD allocation is available
case $BASH_VERSION in
''|[1-3].*|4.[012].*) echo "ERROR: bash 4.3 required" >&2; exit 1;;
esac
my_function() { sleep $(( 2 (RANDOM % 10) )); } # replace with real function
# note "limit" is lowercase -- that's per POSIX guidelines
# see https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
limit=100 # as an example
pids=( )
# allocate a file descriptor that writes to a copy of "pv" -- "pipe viewer"
exec {pv_fd}> >(exec pv --progress --size "$limit" >/dev/null); pv_pid=$!
for (( n=0; n<limit; n )); do
# every time my_function completes, write to the copy of pv
(my_function "$n" {pv_fd}>&-; rc=$?; printf '\n' >&$pv_fd; exit "$rc") & pids[n]=$!
done
exec {pv_fd}>&- # now, close the process substitution feeding to pv
# pause here until all copies of my_function have completed
# by looping and passing each copy of wait only one PID, we can retrieve exact
# exit status for each copy.
for n in "${!pids[@]}"; do
pid=${pids[$n]}
wait "$pid" || echo "WARNING: Process $n (PID $pid) exited with status $?" >&2
done
wait "$pv_pid" # and wait for pv to exit
echo "Finished running all instances of my_function" >&2
uj5u.com熱心網友回復:
使用 GNU Parallel,您可以執行以下操作:
export -f my_function
seq 0 ${LIMIT} |
parallel -j${LIMIT} --bar my_function
轉載請註明出處,本文鏈接:https://www.uj5u.com/yidong/341826.html
