AnsweredAssumed Answered

Spark-submit: connection to port 7077 refused (but port is listening)

Question asked by selinali on Mar 7, 2018
Latest reply on Apr 10, 2018 by Harikrishnan Cheneperth Kunhumveettil

When I submit java application to local, things going well:

./spark-submit --class "com.mycompany.simpleapp.SimpleApp" --master local[4] /home/mapr/SimpleApp-1.0-SNAPSHOT.jar


But when I tried to submit it to Spark standalone cluster:

./spark-submit --class "com.mycompany.simpleapp.SimpleApp" --master spark://10.XXX.XXX.XX:7077 /home/mapr/SimpleApp-1.0-SNAPSHOT.jar


I got exception:

Failed to connect to master 10.XXX.XXX.XX:7077
org.apache.spark.SparkException: Exception thrown in awaitResult
at org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)



Other information:

jps -lm | grep -i spark
19016 org.apache.spark.deploy.history.HistoryServer
3476 org.apache.spark.deploy.master.Master


lsof -i -P | grep 3476
java 3476 mapr 374u IPv6 1312018 0t0 TCP *:8080 (LISTEN)
java 3476 mapr 434u IPv6 1314942 0t0 TCP localhost:7077 (LISTEN)
java 3476 mapr 449u IPv6 1312023 0t0 TCP localhost:6066 (LISTEN)


telnet 10.XXX.XXX.XX:7077
Trying 10.XXX.XXX.XX...
telnet: connect to address 10.XXX.XXX.XX: Connection refused


Why connection failed while the port 7077 is listening? I tried other port and other port can be connected. Why port 7077 cannot?


Can anyone tell how to deal with this? Thanks.