我寫了一個pysaprk程式，如果不用遠程除錯，程式在遠程機器上用spark2-submit 的方式可以執行。
現在我利用VS Code的插件ptvsd 進行遠程除錯，出錯如下。
我的環境是CDH 5.14.4，spark2.4.0
請各位大佬賜教。
Exception has occurred: IndexError
pop from an empty deque
File "/opt/cloudera/parcels/SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012/lib/spark2/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
connection = self.deque.pop()
File "/opt/cloudera/parcels/SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012/lib/spark2/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 983, in send_command
connection = self._get_connection()
File "/opt/cloudera/parcels/SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012/lib/spark2/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 179, in java_import
answer = gateway_client.send_command(command)
File "/opt/cloudera/parcels/SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012/lib/spark2/python/lib/pyspark.zip/pyspark/java_gateway.py", line 124, in launch_gateway
File "/opt/cloudera/parcels/SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012/lib/spark2/python/lib/pyspark.zip/pyspark/context.py", line 307, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway(conf)
File "/opt/cloudera/parcels/SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012/lib/spark2/python/lib/pyspark.zip/pyspark/context.py", line 124, in __init__
SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
File "/home/devXu/project/HydrologicalYear/joinData.py", line 124, in stationAndLand
spark=SparkContext(appName="computeHydrolocialYear")
File "/home/devXu/project/HydrologicalYear/joinData.py", line 349, in <module>
stationAndLand()
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/25801.html
標籤：Spark