AnsweredAssumed Answered

Failed login on MCS after rolling upgrade to 6.0

Question asked by twalberg on Dec 5, 2017
Latest reply on Jan 18, 2018 by maprcommunity

I'm in the middle of a rolling upgrade from 5.2.x to 6.0, and at this point have updated one of the two webserver nodes in the cluster. Before I proceed with the other, I wanted to switch to the new web MCS so that I still have web access while I finish the upgrade. The apiserver service is running, and when I point my browser to the URL, I get the login page. However, when I enter username and password and hit "Log In", logging in fails with "Cannot read property 'UserProperties' of undefined." The same results occur for the "mapr" login and my own personal login.

 

From the CLI, I can see the users are defined and configured correctly:

 

# maprcli acl show -type cluster | egrep 'mapr|twalberg|Allowed'
Allowed actions Principal
[login, ss, cv, a, fc] User mapr
[login, ss, cv, a, fc] User twalberg

 

I can log into the node using either account as well, so authentication is definitely working outside the web interface.

 

Here's the associated messages from apiserver.log for a login attempt:

 

2017-12-05 13:13:47,103 INFO RequestFilter GET /
2017-12-05 13:13:47,103 INFO RequestFilter POST /login
2017-12-05 13:13:47,356 INFO RequestFilter GET /files/var/mapr/cluster/admin/.twalberg.mcs.json
2017-12-05 13:13:47,358 INFO MapRFileSystem User mapr impersonates user twalberg
2017-12-05 13:13:47,361 WARN HttpChannelState java.lang.ClassCastException: com.hazelcast.web.HazelcastHttpSession cannot be cast to org.eclipse.jetty.server.session.SessionHandler$SessionIf while invoking onComplete listener org.eclipse.jetty.server.session.SessionHandler$SessionAsyncListener@d0bf4b8
2017-12-05 13:13:47,369 INFO RequestFilter POST /rest/dashboard/info
2017-12-05 13:13:47,409 INFO RequestFilter POST /rest/alarm/group/listGroup
2017-12-05 13:13:47,412 ERROR CLDBRpcCommonUtils No data returned in RPC: CLDB Ips: 10.244.29.224-, Port: 7222. Continue searching for correct CLDB
2017-12-05 13:13:47,412 INFO CLDBRpcCommonUtils Bad CLDB credentials removed: CLDB Ips: 10.244.29.224-, Port: 7222
2017-12-05 13:13:47,412 ERROR CLDBRpcCommonUtils No data returned in RPC: CLDB Ips: 10.244.29.221-, Port: 7222. Continue searching for correct CLDB
2017-12-05 13:13:47,412 INFO CLDBRpcCommonUtils Bad CLDB credentials removed: CLDB Ips: 10.244.29.221-, Port: 7222
2017-12-05 13:13:47,412 ERROR AlarmCommands RPC Request to list alarms failed. No data returned
2017-12-05 13:13:47,430 INFO RequestFilter POST /rest/acl/userperms
2017-12-05 13:13:47,432 ERROR CLDBRpcCommonUtils No data returned in RPC: CLDB Ips: 10.244.29.224-, Port: 7222. Continue searching for correct CLDB
2017-12-05 13:13:47,432 INFO CLDBRpcCommonUtils Bad CLDB credentials removed: CLDB Ips: 10.244.29.224-, Port: 7222
2017-12-05 13:13:47,433 ERROR CLDBRpcCommonUtils No data returned in RPC: CLDB Ips: 10.244.29.221-, Port: 7222. Continue searching for correct CLDB
2017-12-05 13:13:47,433 INFO CLDBRpcCommonUtils Bad CLDB credentials removed: CLDB Ips: 10.244.29.221-, Port: 7222
2017-12-05 13:13:47,441 INFO RequestFilter POST /rest/acl/userperms
2017-12-05 13:13:47,443 ERROR CLDBRpcCommonUtils No data returned in RPC: CLDB Ips: 10.244.29.224-, Port: 7222. Continue searching for correct CLDB
2017-12-05 13:13:47,443 INFO CLDBRpcCommonUtils Bad CLDB credentials removed: CLDB Ips: 10.244.29.224-, Port: 7222
2017-12-05 13:13:47,443 ERROR CLDBRpcCommonUtils No data returned in RPC: CLDB Ips: 10.244.29.221-, Port: 7222. Continue searching for correct CLDB
2017-12-05 13:13:47,443 INFO CLDBRpcCommonUtils Bad CLDB credentials removed: CLDB Ips: 10.244.29.221-, Port: 7222
2017-12-05 13:13:47,452 INFO RequestFilter POST /rest/userconfig/load
2017-12-05 13:13:47,462 INFO RequestFilter POST /rest/license/list
2017-12-05 13:13:47,475 INFO RequestFilter POST /rest/license/apps
2017-12-05 13:13:47,487 INFO RequestFilter POST /rest/config/load
2017-12-05 13:13:47,504 INFO session Session node0fwz7y1v3rk9h1j0ihlscjlsia26 already being invalidated

 

The CLDB messages look curious, but looking at the cluster in the old 5.2.x web interface, everything is healthy, including this node. I wonder if maybe I need to fully complete the upgrade, including the CLDB nodes, before the new web interface becomes functional?

 

The credentials in use are all held in LDAP - not local accounts. This hasn't been an issue in the past, but maybe something changed there, as well...

 

I have re-run configure.sh on this node, so, in theory, that should not be the issue.

Outcomes