小弟是ambari +hdp 搭建的hadoop集群。
用sqoop 將mysql表匯入hdfs， 在測驗的時候出現這樣的問題，
bin/sqoop import --connect jdbc:mysql://huawang01:3306/testdb --username ambari --password ambari --table user --target-dir /hdfstest --num-mappers 1 --fields-terminated-by ","
Warning: /usr/hdp/3.0.1.0-187/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/05/14 17:33:33 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7.3.0.1.0-187
21/05/14 17:33:33 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/05/14 17:33:33 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
21/05/14 17:33:33 INFO tool.CodeGenTool: Beginning code generation
Fri May 14 17:33:33 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
21/05/14 17:33:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user` AS t LIMIT 1
21/05/14 17:33:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user` AS t LIMIT 1
21/05/14 17:33:34 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.0.1.0-187/hadoop-mapreduce
21/05/14 17:33:36 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-root/compile/463f32d0a76d659f9596cd40430732ef/user.java to /usr/hdp/3.0.1.0-187/sqoop/./user.java. Error: Destination '/usr/hdp/3.0.1.0-187/sqoop/./user.java' already exists
21/05/14 17:33:36 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/463f32d0a76d659f9596cd40430732ef/user.jar
21/05/14 17:33:36 WARN manager.MySQLManager: It looks like you are importing from mysql.
21/05/14 17:33:36 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
21/05/14 17:33:36 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
21/05/14 17:33:36 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
21/05/14 17:33:36 INFO mapreduce.ImportJobBase: Beginning import of user
21/05/14 17:33:37 INFO client.RMProxy: Connecting to ResourceManager at huawang01/192.168.0.43:8050
21/05/14 17:33:37 INFO client.AHSProxy: Connecting to Application History server at huawang02/192.168.0.44:10200
21/05/14 17:33:37 ERROR tool.ImportTool: Import failed: java.io.FileNotFoundException: File does not exist: hdfs://huawang01:8020/hdp/apps/3.0.1.0-187/mapreduce/mapreduce.tar.gz
at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:145)
at org.apache.hadoop.fs.AbstractFileSystem.resolvePath(AbstractFileSystem.java:488)
at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2292)
at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2288)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.resolve(FileContext.java:2294)
at org.apache.hadoop.fs.FileContext.resolvePath(FileContext.java:611)
at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:460)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:146)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:124)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:541)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:656)
at org.apache.sqoop.Sqoop.run(Sqoop.java:150)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:186)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:240)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:249)
at org.apache.sqoop.Sqoop.main(Sqoop.java:258)
這個怎么解決
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/283979.html
標籤：分布式計算/Hadoop