HADOOP版本2.4.0 JDK 1.7.0
代碼：在map里面
FSDataOutputStream out = fs.create(tmpPath, true);
if (!ftp.download(src_file, out)) {
log.error(10014, "下載檔案 " + src_file + " 失敗，采集點："
+ collectId);
out.close();
}
報錯:
org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
at org.apache.commons.net.io.Util.copyStream(Util.java:135)
at org.apache.commons.net.ftp.FTPClient._retrieveFile(FTPClient.java:1695)
at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1669)
at com.eshore.icps.ftp.Ftp.download(Ftp.java:119)
at com.eshore.icps.collect.CollectMapper.map(CollectMapper.java:228)
at com.eshore.icps.collect.CollectMapper.run(CollectMapper.java:99)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.nio.channels.ClosedChannelException
at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at org.apache.commons.net.io.Util.copyStream(Util.java:123)
... 12 more
可以部分下載成功，比如本來要下載10個檔案，下載成功了2個，開始下載第3個的時候卡了一會就會報這個錯誤，一般 是遇到大檔案，速度在300K/S左右
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/84586.html
標籤：網絡