AnsweredAssumed Answered

Spark-submit: connection to port 7077 refused (but port is listening)

Question asked by selinali on Mar 7, 2018
Latest reply on Apr 10, 2018 by Harikrishnan Cheneperth Kunhumveettil

When I submit java application to local, things going well:

./spark-submit --class "com.mycompany.simpleapp.SimpleApp" --master local[4] /home/mapr/SimpleApp-1.0-SNAPSHOT.jar

 

But when I tried to submit it to Spark standalone cluster:

./spark-submit --class "com.mycompany.simpleapp.SimpleApp" --master spark://10.XXX.XXX.XX:7077 /home/mapr/SimpleApp-1.0-SNAPSHOT.jar

 

I got exception:

Failed to connect to master 10.XXX.XXX.XX:7077
org.apache.spark.SparkException: Exception thrown in awaitResult
at org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)

...

 

Other information:

jps -lm | grep -i spark
19016 org.apache.spark.deploy.history.HistoryServer
3476 org.apache.spark.deploy.master.Master

 

lsof -i -P | grep 3476
java 3476 mapr 374u IPv6 1312018 0t0 TCP *:8080 (LISTEN)
java 3476 mapr 434u IPv6 1314942 0t0 TCP localhost:7077 (LISTEN)
java 3476 mapr 449u IPv6 1312023 0t0 TCP localhost:6066 (LISTEN)

 

telnet 10.XXX.XXX.XX:7077
Trying 10.XXX.XXX.XX...
telnet: connect to address 10.XXX.XXX.XX: Connection refused

 

Why connection failed while the port 7077 is listening? I tried other port and other port can be connected. Why port 7077 cannot?

 

Can anyone tell how to deal with this? Thanks.

Outcomes