We are running 5.2 with MapR Spark (1.6.1) installed using YARN. I am able to run jobs and view them within the Resource Manager page until they complete. The History Server "Job History" does not show Spark jobs, only Hadoop jobs. Even while a Spark job is running, the Spark Master web page shows nothing:
- Alive Workers: 0
- Cores in use: 0 Total, 0 Used
- Memory in use: 0.0 B Total, 0.0 B Used
- Applications: 0 Running, 0 Completed
- Drivers: 0 Running, 0 Completed
- Status: ALIVE
- How can I find which nodes have MapR-spark installed and running? (It is not under services)
- How can I see which nodes are being used for a job and what their resource levels are, etc.?
- How can I tell what Spark workers are known to the master?
Thanks for any guidance.