Spark Troubleshooting guide: Profiling Spark: How to collect jstack for a known executor or driver process

Document created by hdevanath Employee on Jun 19, 2017
Version 1Show Document
  • View in full screen mode

One useful utility to troubleshoot Java / JVM application-related issues is jstack. A jstack thread dump is a list of all the Java threads that are currently active in a Java Virtual Machine (JVM).

 

 

 The jstack command-line utility attaches to the specified process or core file and prints the stack traces of all threads that are attached to the virtual machine, including Java threads and VM internal threads, and optionally native stack frames. The utility also performs deadlock detection.

Case 1) Collect jstack using the PID of your Spark driver/executor Java process
 

jstack <PID>

 
Case 2) If the jstack pid command does not respond because of a hung process, then the -F option can be used (on Oracle Solaris and Linux operating systems only) to force a stack dump, as shown in the example below for any known driver / executor process PID
 

jstack -F <PID>

 
For troubleshooting situations needing MapR Tech Support's assistance, we recommend you run these diagnostics and submit the logs along with a Tech Support case. This typically expedites a value-added initial response from the Support. 
 

Attachments

    Outcomes