求大神解答一個問題，用python寫了一個pmt公式計算還款金額的udf：
在使用的時候發現只有將欄位轉化為string的時候才能運行成功：select transform(cast(capital as string),cast(interest as string),cast(period_num as string)) USING 'python pmt.py' AS (capital,interest,period_num,repay_amount) from dw_tmp.calculate_repay_amount;
如果不轉化capital、interest是浮點數，period_num是整型時，總是報錯，這是為什么啊？
報錯內容如下：
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 2.0 failed 4 times, most recent failure: Lost task 1.3 in stage 2.0 (TID 10, data14, executor 3): ExecutorLostFailure (executor 3 exited caused by one of the running tasks) Reason: Container marked as failed: container_1555125275242_853949_01_000004 on host: data14. Exit status: 50. Diagnostics: Exception from container-launch.
Container id: container_1555125275242_853949_01_000004
Exit code: 50
Stack trace: ExitCodeException exitCode=50:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 50
Driver stacktrace:
SQLState: null
ErrorCode: 0
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/32416.html
標籤：Spark