專案場景：

因最近公司考慮接入實時數倉專案，選型一些專案作為flink任務管理的工具，偶然間發現gitee上年度gvp專案streamx,https://gitee.com/streamxhub/streamx，感覺挺符合我們的使用場景，話不多說下載下來開測，

streamx版本：1.2.1

hadoop版本：3.1.1

linux版本：centos7

flink版本：1.13.5

問題描述：

在除錯flinksql功能時遇到一個困擾多天的問題

本以為是檔案權限的問題，于是修改hdfs上此檔案權限為777，結果還是報錯，

繼續分析查看了下yarn提交時加載的classpath

Classpath: :flink-1.13/streamx-flink-shims_flink-1.13-1.2.1.jar:lib/commons-cli-1.3.1.jar:lib/doris-flink-1.0-SNAPSHOT.jar:lib/flink-csv-1.13.5.jar:lib/flink-json-1.13.5.jar:lib/flink-shaded-hadoop-3-uber-3.1.1.7.2.1.0-327-9.0.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.11-1.13.5.jar:lib/flink-table_2.11-1.13.5.jar:lib/log4j-1.2-api-2.16.0.jar:lib/log4j-api-2.16.0.jar:lib/log4j-core-2.16.0.jar:lib/log4j-slf4j-impl-2.16.0.jar:streamx-flink-sqlclient-1.2.1.jar:lib/flink-dist_2.11-1.13.5.jar:flink-conf.yaml::/usr/hdp/3.1.5.0-152/hadoop/conf:/usr/hdp/3.1.5.0-152/hadoop/azure-data-lake-store-sdk-2.3.3.jar:/usr/hdp/3.1.5.0-152/hadoop/azure-keyvault-core-1.0.0.jar:/usr/hdp/3.1.5.0-152/hadoop/azure-storage-7.0.0.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-annotations-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-annotations.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-auth-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-auth.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-azure-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-azure-datalake-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-azure-datalake.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-azure.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-common-3.1.1.3.1.5.0-152-tests.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-common-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-common-tests.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-common.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-kms-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-kms.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-nfs-3.1.1.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/hadoop-nfs.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/ranger-plugin-classloader-1.2.0.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/ranger-yarn-plugin-shim-1.2.0.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/ranger-hdfs-plugin-shim-1.2.0.3.1.5.0-152.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/json-smart-2.3.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/accessors-smart-1.2.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jersey-core-1.19.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/xz-1.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/animal-sniffer-annotations-1.17.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/netty-3.10.5.Final.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/asm-5.0.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/nimbus-jose-jwt-4.41.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/avro-1.7.7.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jsp-api-2.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/checker-qual-2.8.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jsr305-3.0.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-beanutils-1.9.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/slf4j-log4j12-1.7.25.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-cli-1.2.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jsr311-api-1.1.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-codec-1.11.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jcip-annotations-1.0-1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-collections-3.2.2.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jul-to-slf4j-1.7.25.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-compress-1.4.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jersey-json-1.19.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-configuration2-2.1.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/paranamer-2.3.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-io-2.5.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-admin-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-lang-2.6.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-client-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-lang3-3.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-common-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-logging-1.1.3.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-core-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-math3-3.1.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/protobuf-java-2.5.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/commons-net-3.6.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-crypto-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/curator-client-2.12.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jersey-server-1.19.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/curator-framework-2.12.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-identity-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/curator-recipes-2.12.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jersey-servlet-1.19.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/error_prone_annotations-2.3.2.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-server-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/failureaccess-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/re2j-1.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/gson-2.2.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/slf4j-api-1.7.25.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/guava-28.0-jre.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-servlet-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/htrace-core4-4.1.0-incubating.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-simplekdc-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/httpclient-4.5.2.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/snappy-java-1.0.5.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/httpcore-4.4.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerb-util-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/j2objc-annotations-1.3.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jettison-1.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-annotations-2.10.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerby-asn1-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-core-2.10.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerby-config-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerby-pkix-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-databind-2.10.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerby-util-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-util-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/kerby-xdr-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jackson-xc-1.9.13.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/log4j-1.2.17.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/javax.servlet-api-3.1.0.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/token-provider-1.0.1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jaxb-api-2.2.11.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/metrics-core-3.2.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-http-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-webapp-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-io-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-xml-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-security-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jsch-0.1.54.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/jetty-server-9.3.24.v20180605.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/stax2-api-3.1.4.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/woodstox-core-5.0.3.jar:/usr/hdp/3.1.5.0-152/hadoop/lib/zookeeper-3.4.6.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-3.1.1.3.1.5.0-152-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-client-3.1.1.3.1.5.0-152-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-client-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-client-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-client.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-httpfs-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-httpfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-native-client-3.1.1.3.1.5.0-152-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-native-client-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-native-client-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-native-client.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-rbf-3.1.1.3.1.5.0-152-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-rbf-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-rbf-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-rbf.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-server-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/accessors-smart-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-json-1.19.jar:/usr/hdp/current/hadoop-hdfs-client/lib/animal-sniffer-annotations-1.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-5.0.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/avro-1.7.7.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-servlet-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/checker-qual-2.8.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-beanutils-1.9.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/re2j-1.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr311-api-1.1.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.11.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.19.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-admin-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-servlet-1.19.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-configuration2-2.1.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-client-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/snappy-java-1.0.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-common-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-core-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang3-3.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-crypto-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-identity-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/stax2-api-3.1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-net-3.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-server-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/curator-client-2.12.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-http-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/curator-framework-2.12.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-simplekdc-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/curator-recipes-2.12.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/error_prone_annotations-2.3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerb-util-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/failureaccess-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/token-provider-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/woodstox-core-5.0.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-28.0-jre.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-io-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core4-4.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerby-asn1-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/httpclient-4.5.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/httpcore-4.4.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerby-config-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/j2objc-annotations-1.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-security-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-annotations-2.10.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerby-pkix-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-2.10.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerby-util-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/kerby-xdr-1.0.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-databind-2.10.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/json-simple-1.1.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.10.5.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/javax.servlet-api-3.1.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jaxb-api-2.2.11.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.52.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/nimbus-jose-jwt-4.41.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jcip-annotations-1.0-1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.7.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.19.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-ajax-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/json-smart-2.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-webapp-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-xml-9.3.24.v20180605.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsch-0.1.54.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.6.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/hdp/current/hadoop-hdfs-client/lib/zookeeper-3.4.6.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-services-api-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-router-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-router.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-services-api.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-pluginstorage-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-pluginstorage.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-services-core-3.1.1.3.1.5.0-152.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-services-core.jar:/usr/hdp/current/hadoop-yarn-client/lib/HikariCP-java7-2.4.12.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/bcpkix-jdk15on-1.60.jar:/usr/hdp/current/hadoop-yarn-client/lib/bcprov-jdk15on-1.60.jar:/usr/hdp/current/hadoop-yarn-client/lib/dnsjava-2.1.7.jar:/usr/hdp/current/hadoop-yarn-client/lib/ehcache-3.3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.50.jar:/usr/hdp/current/hadoop-yarn-client/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-4.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-4.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-base-2.10.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-json-provider-2.10.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-module-jaxb-annotations-2.10.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jakarta.activation-api-1.2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jakarta.xml.bind-api-2.3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-util-1.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.19.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.19.jar:/usr/hdp/current/hadoop-yarn-client/lib/json-io-2.5.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/metrics-core-3.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/mssql-jdbc-6.2.1.jre7.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/snakeyaml-1.16.jar:/usr/hdp/current/hadoop-yarn-client/lib/swagger-annotations-1.5.4.jar

里面也包含了plugins/streamx-flink-sqlclient-1.2.1.jar，那為什么會報找不到檔案呢？

沒辦法只能上手除錯了，于是根據streamx的官方檔案搭建起了本地環境開發環境 | StreamX

開始debug之旅，在原始碼中找到了flinksql型別的任務提交邏輯，

1.先獲取本地家目錄下plugins里有沒有streamx-flink-sqlclient*.jar

2.然后判斷運行模式，yarn模式下獲取遠程的對應的streamx-flink-sqlclient*.jar

3.最后將其封裝到SubmitRequest 物件中，根據模式不同匹配對應的提交模式

4.找到yarn對應的提交模式代碼，其中又往任務中添加了一些依賴包其中包含了plugins目錄下的所有jar包

呃！！！到此還是一臉懵逼，這沒問題啊！該加的依賴，該掃描的包都已經提交上去了，

于是我又下載了1.2.0版本的繼續測驗，1.2.0的居然跑成功了，本以為是因為代碼改動導致的，于是我用對比工具對比了下兩個版本sqlclient的變化，結果幾乎一模一樣！哎這就苦惱了，

再次分析

拿出成功和失敗的日志做對比

失敗日志

成功日志

嗯好像有新發現1.2.0的包能成功是因為ln鏈接位置不一樣，但是為什么不一樣呢？

于是我將1.2.1版本的sqlclient包重命名為1.2.0繼續測驗，結果成功了！！！

改名就能成功，不改名就失敗，這是什么原理，難不成這名字還有要求？

接下來的日子深深的陷入了迷茫，一度懷疑是不是yarn有問題，于是一個大膽的想法浮現，遠程debug yarn原始碼，Hadoop 如何使用IntelliJ IDEA進行遠程除錯代碼debug方式請參考這篇博客，

將yarn3.1.1版本原始碼匯入idea并配置remote jvm debug

找到日志中關鍵日志的列印

全域搜索關鍵字Setting up job resources定位到資源加載的位置ContainerExecutor.java，開啟flinksql任務等待debug

嗯？為什么資源里存在兩個streamx-flink-sqlclient-1.2.1.jar，繼續往下debug

這段代碼就有意思了，大概意思就是回圈遍歷resources，將資源放入一個新的map中，map的key為jar的名字，value為jar的路徑，debug到resourceEntry為stream-flink-sqlclient-*.jar時，value值里有兩個path，一個是plugins/streamx-flink-sqlclient-*.jar一個是streamx-flink-sqlclient-*.jar，因為回圈list放入map，所以后面的將前一個path覆寫掉了，到此恍然大悟 ，但是為什么list里的值順序不同呢？繼續往前查看resources的加載

map的key為jar的path ，因為hashmap是無序的，streamx-flink-sqlclient-1.2.1.jar就排在了plugins/streamx-flink-sqlclient-1.2.1.jar的前面，于是我寫了個demo測驗了一下名字的排序

果然終于找到了為什么改個名就好使的問題了，1.2.0最后加載的是streamx-flink-sqlclient-1.2.0.jar這個包而1.2.1最后加載的是 streamx-flink-sqlclient-1.2.1.jar，

于是我又匯總了一下最近debug的結論，

1.yarn啟動時會將hdfs:///streamx/plugins/*加載到classpath

2.yarn啟動時加載resources時如果有兩個同名jar會取最后一個

3.yarn同名jar的存放是由hashmap控制的是無序的

4.flink提交yarn任務時的命令為

./bin/flink run -t yarn-per-job -d \
-yD yarn.provided.lib.dirs=hdfs://myhdfs/flink/lib,hdfs://myhdfs/flink/plugins
-Djobmanager.memory.process.size=1024m \
-Dtaskmanager.memory.process.size=1024m \
-Dtaskmanager.numberOfTaskSlots=1 \
-Dparallelism.default=1 \
-Dyarn.application.name="MyFlinkApp" \
-Dyarn.application.queue=default ./examples/table/StreamSQLExample.jar

5.streamx在提交時的yarn.provided.lib.dirs包含了plugins/*

到此問題終于定位了，flinksql在跑任務時指定了flinkUserJar，后面加載yarn classpath時把plugins下的streamx-flink-sqlclient-1.2.1.jar又加載了一遍，所以yarn的resources里有兩個streamx-flink-sqlclient-1.2.1.jar的path，因為hashmap是無序的，所以隨機替換掉了一個path，

于是我修改了streamx原始碼邏輯將streamx-flink-sqlclient.jar從plugins中挪走，打包重新測驗，嗯不出意外一把過，再次debug驗證這次yarn resource只有一個streamx-flink-sql-clinet-1.2.1.jar，成功解決掉這個bug，