AnsweredAssumed Answered

NFS Service not coming up after upgrade

Question asked by anirudh on Mar 27, 2014
Latest reply on May 14, 2014 by mbehamin
    I have upgraded from 2.0.1 to 3.1.0. After upgrade NFS Server is not coming up. The error logs are as
    
    nfsserver.logs
    --------------
    2014-03-28 10:13:32,2033 INFO nfsserver[22641] fs/nfsd/main.cc:482 ***** NFS server starting: pid=22641, mapr-version: 3.1.0.23553EMC *****
    2014-03-28 10:13:32,2034 INFO nfsserver[22641] fs/nfsd/main.cc:496 ******* NFS server MAPR_HOME=/opt/mapr, NFS_PORT=2049, NFS_MGMT_PORT=9998, NFSMON_PORT=9997
    2014-03-28 10:13:32,2117 INFO nfsserver[22641] fs/nfsd/nfsserver.cc:928 0.0.0.0[0] running the cmd /opt/mapr/server/maprexecute pmapset set 100003 3 6 2049, ret 30464
    2014-03-28 10:13:32,2128 INFO nfsserver[22641] fs/nfsd/nfsserver.cc:971 0.0.0.0[0] Use32BitFileId is 1
    2014-03-28 10:13:32,2129 INFO nfsserver[22641] fs/nfsd/nfsserver.cc:984 0.0.0.0[0] AutoRefreshExportsTimeInterval is 0
    2014-03-28 10:13:32,2129 ERROR nfsserver[22641] fs/nfsd/main.cc:66 0.0.0.0[0] Error registering NFS program
    
    maprexecute.log
    ---------------
    2014-03-28 10:06:14:ERROR:18857: failed to setgid (0) cmd: pmapset
    2014-03-28 10:13:11:INFO:22364: maprexecute pmapset by uid 2147483632 gid 2147483632
    Cmd Line: /opt/mapr/server/maprexecute pmapset set 100003 3 6 2049
    2014-03-28 10:13:11:ERROR:22364: failed to setgid (0) cmd: pmapset
    2014-03-28 10:13:22:INFO:22560: maprexecute pmapset by uid 2147483632 gid 2147483632
    Cmd Line: /opt/mapr/server/maprexecute pmapset set 100003 3 6 2049
    2014-03-28 10:13:22:ERROR:22560: failed to setgid (0) cmd: pmapset
    2014-03-28 10:13:32:INFO:22645: maprexecute pmapset by uid 2147483632 gid 2147483632
    Cmd Line: /opt/mapr/server/maprexecute pmapset set 100003 3 6 2049
    2014-03-28 10:13:32:ERROR:22645: failed to setgid (0) cmd: pmapset
    
    ll /opt/mapr/server
    -------------------
    -rwxr-xr-x. 1 root root       109 Jan 14 22:53 adjustoom
    -rwxr-xr-x. 1 root root   5623472 Jan 14 22:59 checkdataloss
    -rwxr-xr-x. 1 root root      1943 Jan 14 22:53 cleanup-maprconf
    -rwxr-xr-x. 1 root root      3263 Jan 14 22:53 clusterconf.sh
    -rwxr-xr-x. 1 root root     13762 Jan 14 22:53 clusterinstall.sh
    -rwxr-xr-x. 1 root root     18952 Jan 14 22:53 clusterinstallv2.sh
    -rwxr-xr-x. 1 root root      1524 Jan 14 22:53 clustersetup.sh
    -rwxr-xr-x. 1 root root      1955 Jan 14 22:53 collectTaskDiagnostics.sh
    -rwxr-xr-x. 1 root root      2727 Jan 14 22:53 config-mapr-user.sh
    -rwxr-xr-x. 1 root root     17418 Jan 14 22:53 configure-common.sh
    -rwxr-xr-x. 1 root root     69726 Jan 14 22:53 configure.sh
    -rwxr-xr-x. 1 root root      4564 Jan 14 22:53 createJTVolume.sh
    -rwxr-xr-x. 1 root root     11099 Jan 14 22:53 createsystemvolumes.sh
    -rwxr-xr-x. 1 root root     20312 Jan 14 22:53 createTTVolume.sh
    -rwxr-xr-x. 1 root root      2466 Jan 14 22:53 create-volumes.sh
    -rwxr-xr-x. 1 root root      9809 Jan 14 22:53 discoverRawDisks
    -rwxr-xr-x. 1 root root       722 Jan 14 22:53 diskadd.sh
    -rwxr-xr-x. 1 root root      1271 Jan 14 22:53 diskcommon.sh
    -rwxr-xr-x. 1 root root       541 Jan 14 22:53 diskdev.sh
    -rwxr-xr-x. 1 root root     14823 Jan 14 22:53 disklist.sh
    -rwxr-xr-x. 1 root root     21737 Jan 14 22:53 diskremove
    -rwxr-xr-x. 1 root root      4455 Jan 14 22:53 diskremove.sh
    -rwxr-xr-x. 1 root root     35683 Jan 14 22:53 disksetup
    drwxr-xr-x. 2 root root      4096 Mar 27 10:37 filters
    -rwxr-xr-x. 1 root root      1135 Jan 14 22:53 format.sh
    -rwxr-xr-x. 1 root root   6564544 Jan 14 22:59 fsck
    -rwxr-xr-x. 1 root root 157699652 Jan 14 22:53 fsck-phase6
    -rwxr-xr-x. 1 root root    164480 Jan 14 22:59 gtracedump
    -rwxr-xr-x. 1 root root      6138 Jan 14 22:53 handle_disk_failure.sh
    -rwxr-xr-x. 1 root root    351256 Jan 14 22:59 hoststats
    -rwxr-xr-x. 1 root root      6236 Jan 14 22:53 initscripts-common.sh
    -rwxr-xr-x. 1 root root  10480064 Jan 14 22:59 logdump
    -rwsr-x---. 1 root mapr      4207 Jan 14 22:53 manageSSLKeys.sh
    -rwsr-x---. 1 root mapr    128672 Jan 14 22:59 maprexecute
    -rwxr-xr-x. 1 root root     63877 Jan 14 22:53 maprinstall
    -rwxr-xr-x. 1 root root 164269920 Jan 14 22:53 mfs
    -rwxr-xr-x. 1 root root   5818480 Jan 14 22:59 mirrorclient
    -rwxr-xr-x. 1 root root   5765712 Jan 14 22:59 mrconfig
    -rwxr-xr-x. 1 root root   6273344 Jan 14 22:59 mrdisk
    -rwxr-xr-x. 1 root root      7592 Jan 14 22:59 mruuidgen
    -rwxr-xr-x. 1 root root     20242 Jan 14 22:53 nfsmon_if_script.pl
    -rwxr-xr-x. 1 root root  53892142 Jan 14 22:53 nfsserver
    -rwxr-xr-x. 1 root root     12139 Jan 14 22:53 nodeinstallv2.sh
    -rwxr-xr-x. 1 root root      1655 Jan 14 22:53 parse_instance_info.py
    drwxr-xr-x. 2 root root      4096 Mar 27 10:37 permissions
    -rwxr-xr-x. 1 root root     15366 Jan 14 22:53 pmapset
    -rwxr-xr-x. 1 root root      3744 Jan 14 22:53 prerequisitecheck.sh
    -rwxr-xr-x. 1 root root      4269 Jan 14 22:53 pullcentralconfig
    -rwxr-xr-x. 1 root root      1698 Jan 14 22:53 remove-volumes.sh
    drwxr-xr-x. 2 root root      4096 Mar 27 10:37 roles-controller
    -rwxr-xr-x. 1 root root    120480 Jan 14 22:59 sanitize-core
    -rwxr-xr-x. 1 root root      3067 Jan 14 22:53 scripts-common.sh
    -rwxr-xr-x. 1 root root      3162 Jan 14 22:53 startcluster.sh
    -rwxr-xr-x. 1 root root      2777 Jan 14 22:53 startclusterv2.sh
    -rwxr-xr-x. 1 root root      2365 Jan 14 22:53 stopcluster.sh
    -rwxr-xr-x. 1 root root      1349 Jan 14 22:53 stopclusterv2.sh
    drwxr-xr-x. 3 root root      4096 Mar 27 10:37 test
    -rwxr-xr-x. 1 root root  48960935 Jan 14 22:53 testcrypto
    drwxr-xr-x. 2 root root      4096 Mar 27 10:37 test_emr
    -rwxr-xr-x. 1 root root      1106 Jan 14 22:53 timedcmd.sh
    drwxr-xr-x. 2 root root      4096 Mar 27 10:37 tools
    -rwxr-xr-x. 1 root root      1698 Jan 14 22:53 upgrade
    -rwxr-xr-x. 1 root root      3738 Jan 14 22:53 upgrade2maprexecute
    -rwxr-xr-x. 1 root root      2706 Jan 14 22:53 upgrade2mapruser.sh
    
    
    permissions to mapr in sudoer
    ------------------------------
    mapr    ALL= (root)     NOPASSWD:       /sbin/ip
    mapr    ALL= (root)     NOPASSWD:       /bin/mount
    mapr    ALL= (root)     NOPASSWD:       /bin/umount
    mapr    ALL= (root)     NOPASSWD:       /sbin/ifconfig
    mapr    ALL= (root)     NOPASSWD:       /usr/bin/arping
    mapr    ALL= (root)     NOPASSWD:       /opt/mapr/server/pmapset
    mapr    ALL= (root)     NOPASSWD:       /opt/mapr/server/mrdisk
    mapr    ALL= (root)     NOPASSWD:       /bin/chgrp
    mapr    ALL= (root)     NOPASSWD:       /bin/chmod
    mapr    ALL= (root)     NOPASSWD:       /usr/bin/renice
    mapr    ALL= (root)     NOPASSWD:       /usr/sbin/dmidecode
    mapr    ALL= (root)     NOPASSWD:       /sbin/hdparm
    mapr    ALL= (root)     NOPASSWD:       /usr/bin/sdparm
    mapr    ALL= (root)     NOPASSWD:       /opt/mapr/server/suexec
    
    Other Info
    ----------------------------------------
    rpm -qa | grep rpcbind
    rpcbind-0.2.0-11.el6.x86_64
    
    service rpcbind status
    rpcbind (pid  2479) is running...
    
    service nfs status
    rpc.svcgssd is stopped
    rpc.mountd is stopped
    nfsd is stopped

    - Any more debugging steps?
    - I am able to start the NFS if I start as root using '#/etc/init.d/mapr-nfsserver start"
    - That makes it problem with permissions I guess. I have already ran /opt/mapr/server/upgrade2maprexecute'
    - Is there any more files that needs fixing ?


    Thanks,
    Anirudh

Outcomes