hadoop版本:hadoop-2.7.12.7.1
hbase版本:hbase-1.1.2
hive版本:apache-hive-2.0.0-bin
在整合了hive和hbase之后,我在hive中創建了一張關聯了hbase表的磁區表,然后在向表插入資料的時候報錯了,下面是插入陳述句和報錯,求懂的人解答。
visited_in_hive是關聯了hbase表,準備插入資料的表;hv_c_hb是hive中的另一張表(不是磁區表),里面有資料,而且欄位和visited_in_hive一樣。
插入陳述句:insert into table visited_in_hive partition (datetime='2016-03-05') select * from hv_c_hb;
報錯:WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = test2_20160308092429_26a93c7c-b945-4329-98ff-47a4067b579d
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: Must specify table name
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1106)
at org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl.checkOutputSpecs(HiveOutputFormatImpl.java:67)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:268)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:432)
at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:138)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:158)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:101)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1840)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1584)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1361)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1184)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: Must specify table name
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1128)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:1103)
... 37 more
Caused by: java.lang.IllegalArgumentException: Must specify table name
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
at org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
at org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:300)
at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:290)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1126)
... 38 more
Job Submission failed with exception 'java.io.IOException(org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: Must specify table name)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
它報的錯是沒有指定表名,但是陳述句里明明已經指定表名了啊。難道是版本問題????求高手解答。
uj5u.com熱心網友回復:
hive 1.2.1 + hbase 0.98.18 同樣遇到這個問題uj5u.com熱心網友回復:
這個問題你解決了嗎?uj5u.com熱心網友回復:
@qq_30131605 沒有解決,最后放棄,沒用這個陳述句了。uj5u.com熱心網友回復:
在定義關聯表的時候 建表陳述句中設定一下"hbase.mapred.output.outputtable" = "some_existing_table"這個引數 具體設定規則請參考https://cwiki.apache.org/confluence/display/Hive/HBaseIntegrationuj5u.com熱心網友回復:
樓上方法不行啊。。。試了不可以還是報哪個錯uj5u.com熱心網友回復:
問題已解決-- Parse a row key with 3 fixed width fields each of width 10
-- Example taken from: https://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/queries/positive/hbase_custom_key2.q
CREATE TABLE hbase_ck_4(key struct<col1:string,col2:string,col3:string>, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.table.name" = "hbase_custom2",
"hbase.mapred.output.outputtable" = "hbase_custom2",
"hbase.columns.mapping" = ":key,cf:string",
"hbase.composite.key.factory"="org.apache.hadoop.hive.hbase.SampleHBaseKeyFactory2");
uj5u.com熱心網友回復:
試了幾遍。。你在hbase上先創建表。。然后在hive創建hbase關聯的外表。。這樣就沒有問題了。。uj5u.com熱心網友回復:
是創建表順序的問題嗎?
uj5u.com熱心網友回復:
這個問題怎么解決呢?有沒有方法,求uj5u.com熱心網友回復:
請教您一個問題,hive建立磁區表與hbase映射時,磁區的欄位怎么映射Hbase的列啊?轉載請註明出處,本文鏈接:https://www.uj5u.com/qita/45841.html
