正如 Brian Goetz 所說:“TrackingExecutor 有一個不可避免的競爭條件,它可能會產生誤報:被識別為已取消但實際上已完成的任務。出現這種情況是因為執行緒池可能在任務的最后一條指令執行到當池將任務記錄為完成時。”
跟蹤執行器:
/**
* TrackingExecutor
* <p/>
* ExecutorService that keeps track of cancelled tasks after shutdown
*
* @author Brian Goetz and Tim Peierls
*/
public class TrackingExecutor extends AbstractExecutorService {
private final ExecutorService exec;
private final Set<Runnable> tasksCancelledAtShutdown =
Collections.synchronizedSet(new HashSet<Runnable>());
public TrackingExecutor(ExecutorService exec) {
this.exec = exec;
}
public void shutdown() {
exec.shutdown();
}
public List<Runnable> shutdownNow() {
return exec.shutdownNow();
}
public boolean isShutdown() {
return exec.isShutdown();
}
public boolean isTerminated() {
return exec.isTerminated();
}
public boolean awaitTermination(long timeout, TimeUnit unit)
throws InterruptedException {
return exec.awaitTermination(timeout, unit);
}
public List<Runnable> getCancelledTasks() {
if (!exec.isTerminated())
throw new IllegalStateException(/*...*/);
return new ArrayList<Runnable>(tasksCancelledAtShutdown);
}
public void execute(final Runnable runnable) {
exec.execute(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
if (isShutdown()
&& Thread.currentThread().isInterrupted())
tasksCancelledAtShutdown.add(runnable);
}
}
});
}
}
然后他創建了 Crawler,它使用TrackingExecutor:
爬蟲:
/**
* WebCrawler
* <p/>
* Using TrackingExecutorService to save unfinished tasks for later execution
*
* @author Brian Goetz and Tim Peierls
*/
public abstract class WebCrawler {
private volatile TrackingExecutor exec;
@GuardedBy("this") private final Set<URL> urlsToCrawl = new HashSet<URL>();
private final ConcurrentMap<URL, Boolean> seen = new ConcurrentHashMap<URL, Boolean>();
private static final long TIMEOUT = 500;
private static final TimeUnit UNIT = MILLISECONDS;
public WebCrawler(URL startUrl) {
urlsToCrawl.add(startUrl);
}
public synchronized void start() {
exec = new TrackingExecutor(Executors.newCachedThreadPool());
for (URL url : urlsToCrawl) submitCrawlTask(url);
urlsToCrawl.clear();
}
public synchronized void stop() throws InterruptedException {
try {
saveUncrawled(exec.shutdownNow());
if (exec.awaitTermination(TIMEOUT, UNIT))
saveUncrawled(exec.getCancelledTasks());
} finally {
exec = null;
}
}
protected abstract List<URL> processPage(URL url);
private void saveUncrawled(List<Runnable> uncrawled) {
for (Runnable task : uncrawled)
urlsToCrawl.add(((CrawlTask) task).getPage());
}
private void submitCrawlTask(URL u) {
exec.execute(new CrawlTask(u));
}
private class CrawlTask implements Runnable {
private final URL url;
CrawlTask(URL url) {
this.url = url;
}
private int count = 1;
boolean alreadyCrawled() {
return seen.putIfAbsent(url, true) != null;
}
void markUncrawled() {
seen.remove(url);
System.out.printf("marking %s uncrawled%n", url);
}
public void run() {
for (URL link : processPage(url)) {
if (Thread.currentThread().isInterrupted())
return;
submitCrawlTask(link);
}
}
public URL getPage() {
return url;
}
}
}
但我不明白 runnable.run()、exec.shutdownNow()、exec.awaitTermination(...)、exec.getCancelledTasks()、tasksCancelledAtShutdown.add(runnable)、runnable 呼叫的確切時間順序是什么完成和執行緒交錯,這會導致競爭條件。
uj5u.com熱心網友回復:
我是這樣理解的。比如TrackingExecutor在CrawlTask退出前正在關閉,這個任務也可能被記錄為 a taskCancelledAtShutdown,因為if (isShutdown() && Thread.currentThread().isInterrupted())inTrackingExecutor#execute可能是 true ,但實際上這個任務已經完成了。
private class CrawlTask implements Runnable {
public void run() {
for (URL link : processPage(url)) {
if (Thread.currentThread().isInterrupted())
return;
submitCrawlTask(link);
}
// May be here, trackingExecutor is shutting down.
// Actually this task has completed now.But this method did not exit.
}
}
public void execute(final Runnable runnable) {
exec.execute(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
// isShutdown() && Thread.currentThread().isInterrupted() may be true
if (isShutdown()
&& Thread.currentThread().isInterrupted())
tasksCancelledAtShutdown.add(runnable);
}
}
});
}
轉載請註明出處,本文鏈接:https://www.uj5u.com/houduan/371842.html
下一篇:多處理行程不運行
