I have a secure cluster (using PAM) and I have several ecosystem components (yarn, hive with impersonation/authentication, drill, spark) running fine on the cluster. I can login to the cluster, generate a maprlogin ticket and run any of these services.
I was hoping that securing the cluster would also secure HttpFS. However, passing the "user.name" in the query string of the HTTPS request, any user can pretend to be anyone else and is able to communicate with the file system.
I then tried to conifigure HttpFS authentication using PAM. I followed the instructions in the given link (PAM Authentication for HttpFS ) but to no avail. When I run the command "curl -i -- user https://........", I'm prompted for a password but after entering the password, I get the error "HTTP/ 1.1 401 unauthorized error".
Here are my questions:
1. Can I configure HttpFS authentication using PAM and without using Kerberos? Are there any resources apart from the above link to help me with the configuration?
2. I'm having a similar problem with Web UI - node manager, resource manager and Hue(With Hue the problem is with connecting to MaprFS and resourcemanager). All of these components have stopped working after securing the cluster. I have read references to Kerberos SPNEGO authentication to secure these components but was hoping to avoid Kerberos as much as possible. Is this feasible or do I need Kerberos to make these components work in a secure cluster?
Thanks for your help