Spark installation problem: ERROR SparkContext: Error initializing SparkContext. java.lang.reflect.InvocationTargetException

created at 10-25-2021 views: 373

Problem background:

  • Java version: 1.8.0.291
  • Spark version: spark-3.2.0-bin-hadoop3.2
  • Windows operating system 64-bit
  • Spark initial installation

Problem Description:

After configuring the Java environment and Spark path, when executing Spark-shell in cmd, the following error occurs

ERROR SparkContext: Error initializing SparkContext.
java.lang.reflect.InvocationTargetException
.....
Caused by: java.net.URISyntaxException: Illegal character in path at index 32: spark://LAPTOP-US4D0J27:64591/C:\classes
        at java.net.URI$Parser.fail(URI.java:2845)
        at java.net.URI$Parser.checkChars(URI.java:3018)
        at java.net.URI$Parser.parseHierarchical(URI.java:3102)
        at java.net.URI$Parser.parse(URI.java:3050)
        at java.net.URI.<init>(URI.java:588)
        at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)
        ... 70 more

problem solved

Judging from the bug printed on the console, it seems that there are illegal characters in the java file executed at startup, which caused the startup to fail. After repeated searching, I could not find which java file he started with an error.
Try to lower the version and start successfully after using spark-3.1.2-bin-hadoop3.2
It may be that the configuration of the higher version has changed, and the understanding is not deep, 0.0

created at:10-25-2021
edited at: 10-25-2021: