我正在Kubernetes集群上運行一個簡單的Spark作業，該作業通過Hive catologization將資料寫入HDFS。由于某種原因，我的應用程式無法運行Spark的SQL命令，出現了以下例外：
21/09/22 09:23:54 ERROR SplunkStreamListener: |exception=org.apache.spark.sql.AnalysisException
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:got exception: JAVA.io. IOException There is no primary group for UGI spark (auth: SIMPLE））；
at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:106)/span>
at org.apache.spark.sql.hive.HiveExternalCatalog.createDatabase(HiveExternalCatalog.scala:183)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.createDatabase(ExternalCatalogWithListener.scala:47)
at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createDatabase（SessionCatalog.scala:211）
at org.apache.spark.sql.execution.command.CreateDatabaseCommand.run（ddl.scala：70）
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute（commands.scala:70）
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult（commands.scala:68）
at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:194)
at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)
at org.apache.spark.sql.execution.SQLExecution$anonfun$withNewExecutionId$1.apply（SQLExecution.scala:80）
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated（SQLExecution.scala:127）
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId（SQLExecution.scala:75）
at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$withAction（Dataset.scala:3369）
at org.apache.spark.sql.Dataset.<init> (Dataset.scala:194)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
at org.apache.spark.sql.SparkSession.sql（SparkSession.scala:643）
我正在通過Thrift URL連接到Hive元存盤。docker容器以非root用戶身份運行該應用程式。我是否需要為用戶添加一些組來與元存盤同步？
uj5u.com熱心網友回復：
在設定spark背景關系之前嘗試添加這個
System.setProperty("HADOOP_USER_NAME", "root"）
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/333036.html
標籤：