AnsweredAssumed Answered

Retrying connect to server: 0.0.0.0/0.0.0.0:8032

Question asked by karthikSpark on Dec 27, 2016
Latest reply on Jan 17, 2017 by maprcommunity

Hi all,

Over view:

1.Trying to write in to hdfs(Hadoop 2.7.0-mapr-1607) of node A by using spark streaming (1.6.1 v, Hadoop 2.7.0-mapr-1607) from node B.

2. All the ports are open, yarn-site.xml is configured well with RM essentials.

3.Running as yarn cluster manager, master = yarn-client.using sbt to build the project

4.provided hive metastore to the hive context just in case.

5.I'm using centos. 

 Here is my code snippet,

 1 val messages = KafkaUtils.createStream[String, String,StringDecoder,StringDecoder](ssc, kafkaConf, topicMaps, StorageLevel.MEMORY_ONLY_SER)
2 messages.foreachRDD(rdd=>
{
3 val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
4 //sqlContext.setConf("hive.metastore.uris","thrift://metastore server:9083")
5 import sqlContext.implicits._

6 val dataframe =sqlContext.read.json(rdd.map(_._2))
7 val myDF =dataframe.toDF()

8 val tablepath = Map("path" -> "maprfs:///path_of/table/in/hdfs")

9 myDF.write.format("parquet").options(tablepath).mode(org.apache.spark.sql.SaveMode.Append).saveAsTable("table_name")

})

But after doing sbt run,

I'm getting this ,

> run
[info] Running org.receiver.HiveGenerator
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mapr/.ivy2/cache/org.slf4j/slf4j-log4j12 /jars/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mapr/.ivy2/cache/org.apache.avro/avro-to ols/jars/avro-tools-1.8.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/12/28 00:42:51 INFO SparkContext: Running Spark version 1.6.1
16/12/28 00:42:51 WARN NativeCodeLoader: Unable to load native-hadoop library fo r your platform... using builtin-java classes where applicable
16/12/28 00:42:51 INFO SecurityManager: Changing view acls to: mapr
16/12/28 00:42:51 INFO SecurityManager: Changing modify acls to: mapr
16/12/28 00:42:51 INFO SecurityManager: SecurityManager: authentication disabled ; ui acls disabled; users with view permissions: Set(mapr); users with modify pe rmissions: Set(mapr)
16/12/28 00:42:52 INFO Utils: Successfully started service 'sparkDriver' on port 54553.
16/12/28 00:42:53 INFO Slf4jLogger: Slf4jLogger started
16/12/28 00:42:53 INFO Remoting: Starting remoting
16/12/28 00:42:53 INFO Remoting: Remoting started; listening on addresses :[akka .tcp://sparkDriverActorSystem@######:45504]
16/12/28 00:42:53 INFO Utils: Successfully started service 'sparkDriverActorSyst em' on port 45504.
16/12/28 00:42:53 INFO SparkEnv: Registering MapOutputTracker
16/12/28 00:42:53 INFO SparkEnv: Registering BlockManagerMaster
16/12/28 00:42:53 INFO DiskBlockManager: Created local directory at /tmp/blockmg r-9c553788-2359-46d6-91a1-1a18ae339f2d
16/12/28 00:42:53 INFO MemoryStore: MemoryStore started with capacity 495.0 MB
16/12/28 00:42:53 INFO SparkEnv: Registering OutputCommitCoordinator
16/12/28 00:42:53 INFO Utils: Successfully started service 'SparkUI' on port 404 0.
16/12/28 00:42:53 INFO SparkUI: Started SparkUI at http://######:4040
16/12/28 00:42:54 INFO RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
16/12/28 00:42:55 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
16/12/28 00:42:56 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
16/12/28 00:42:57 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
16/12/28 00:42:58 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

My yarn is somehow connecting to the local RM, but not to actual RM in my yarn-site.xml.

My admin confirmed there is no issue in configurations, now its up to me.

Any help will be appreciated.

Outcomes