AnsweredAssumed Answered

Example of anyone installing CDAP for mapr6.0?

Question asked by reedv on May 21, 2018
Latest reply on Jun 1, 2018 by reedv

Interested in testing the MapR partner software Cask CDAP on our cluster, but getting stuck at the section covering system specific configurations (Installation for MapR). Following the instructions from here as much as able, and starting the services (Installation for MapR), unable to see anything in the web UI and it is most likely because I'm not correctly configuring the system (ie. the /etc/cdap/conf/cdap-site.xml file (mainly because I don't actually understand what most of the fields they are referring to are, some fields are either not discussed in the docs or I don't understand the meaning of their descriptions (Appendix: cdap-site.xml, cdap-default.xml) or the docs provide little instruction on them aside from 'configure this field for your system')). If anyone has experience using CDAP on their MapR cluster (6.0.0), would appreciate pointing out what I'm doing wrong here or anything else to note about installation that you think is not made totally clear in their documentation.

 

My current cdap-site.xml provided below for reference (note most of the fields are left as the default / example values).

 

<configuration>

  <!-- General Configuration -->

  <property>
    <name>hdfs.namespace</name>
    <value>/${root.namespace}</value>
    <description>
      Root directory for HDFS files written by CDAP
    </description>
  </property>
 
  <property>
    <name>hdfs.user</name>
    <value>cdap</value>
    <description>
      User name for accessing HDFS
    </description>
  </property>

  <property>
    <name>root.namespace</name>
    <value>cdap</value>
    <description>
      Root for this CDAP instance; used as the parent (or root) node for
      ZooKeeper, as the directory under which all CDAP data and metadata is
      stored in HDFS, and as the prefix for all HBase tables created by
      CDAP; must be composed of alphanumeric characters
    </description>
  </property>

  <property>
    <name>zookeeper.quorum</name>
    <value>mapr001.co.local:2181,mapr002.co.local:2181,mapr003.co.local:2181/${root.namespace}</value>
    <description>
      ZooKeeper quorum string; specifies the ZooKeeper host:port; substitute the quorum
      (FQDN1:2181,FQDN2:2181,...) for the components shown here
    </description>
  </property>


  <!-- Applications Configuration -->

  <property>
    <name>master.services.bind.address</name>
    <value>0.0.0.0</value>
    <description>
      Bind address for app fabric service and dataset service
    </description>
  </property>


  <!-- Datasets Configuration -->

  <property>
    <name>data.tx.bind.address</name>
    <value>0.0.0.0</value>
    <description>
      Transaction service bind address
    </description>
  </property>


  <!-- Kafka Server Configuration -->

  <property>
    <name>kafka.default.replication.factor</name>
    <value>1</value>
    <description>
      CDAP Kafka service replication factor; used to replicate Kafka
      messages across multiple machines to prevent data loss in the event of
      a hardware failure. The recommended setting is to run at least two
      CDAP Kafka servers. If you are running two CDAP Kafka servers, set
      this value to 2; otherwise, set it to the maximum number of tolerated
      machine failures plus one (assuming you have that number of machines).
    </description>
  </property>
 
  <property>
    <name>kafka.server.log.dirs</name>
    <value>/tmp/kafka-logs</value>
    <description>
      Comma-separated list of CDAP Kafka service log storage directories
    </description>
  </property>

  <property>
    <name>kafka.seed.brokers</name>
    <value>mapr001.co.local:9092,mapr002.co.local:9092,mapr003.co.local:9092,mapr004.co.local:9092,mapr005.co.local:9092,mapr006.co.local:9092</value>
    <description>
      Comma-separated list of CDAP Kafka service brokers; for distributed CDAP,
      replace with list of FQDN:port brokers
    </description>
  </property>


  <!-- Metrics Configuration -->
 
  <property>
    <name>metrics.query.bind.address</name>
    <value>0.0.0.0</value>
    <description>
      Metrics Query service bind address
    </description>
  </property>


  <!-- Router Configuration -->

  <property>
    <name>router.bind.address</name>
    <value>0.0.0.0</value>
    <description>
      CDAP Router service bind address
    </description>
  </property>
 
  <property>
    <name>router.bind.port</name>
    <value>11015</value>
    <description>
      CDAP Router service bind port
    </description>
  </property>

  <!-- Filled in, but don't actually know what this is -->
  <property>
    <name>router.server.address</name>
    <value>mapr07.co.local</value>
    <description>
      CDAP Router service address to which CDAP UI connects
    </description>
  </property>

  <property>
    <name>router.server.port</name>
    <value>${router.bind.port}</value>
    <description>
      CDAP Router service port to which CDAP UI connects
    </description>
  </property>


  <!-- UI Configuration -->
 
  <property>
    <name>dashboard.bind.port</name>
    <value>11011</value>
    <description>
      CDAP UI bind port
    </description>
  </property>

</configuration>

Outcomes