spark 第三方jar包

spark 第三方jar包

./spark-submit –jars $LIBJARS –class test.MyApp –master local myApp.jar

在~/.bashrc 加上

for i in `ls /path/to/your/third/party/*.jar`do LIBJARS=$i,$LIBJARSdoneexport LIBJARS=$LIBJARS

​​ClassNotFoundException的错，可以在spark-env.sh里SPARK_CLASSPATH=your external jars

实际跑的时候会有

SPARK_CLASSPATH was detected (set to ‘/usr/local/cloudwave-ha/cloudwave/lib/cloudwave-jdbc.jar’). This is deprecated in Spark 1.0+.

Please instead use: - ./spark-submit with –driver-class-path to augment the driver classpath - spark.executor.extraClassPath to augment the executor classpath

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。