I am looking for some clarification on the (Apache Spark) documentation about aggregated logging with Yarn + Spark and how it works in a MapR environment
"You need to have both the Spark history server and the MapReduce history server running and configure
yarn-site.xmlproperly. The log URL on the Spark history server UI will redirect you to the MapReduce history server to show the aggregated logs."
I can't find any documentation about if/how this would work in a MapR Environment beyond a question in the forums (Monitoring MapR Spark Activity) where bmohanam refers the the poster to the same quote above
It looks like the requester ended up viewing completed MR jobs at the JobHistory Server url (http://jobhistoryserver:19888/jobhistory) and Completed Spark jobs at the Spark History Server url (http://sparkhistoryserver:18080/)
The quote from Apache Spark documentation seems to indicate that it would be possible to have a "Single Pane" to see all completed jobs both MR and Spark in one place. Is this possible/supported in a MapR Hadoop Environment?
Am I interpreting the Apache documentation correctly or is it just wishful thinking?
Thank you for any insight you can provide.