AnsweredAssumed Answered

Issues with Mounting NFS

Question asked by anthony.kalinde on Apr 28, 2015
Latest reply on May 3, 2015 by nabeel
Hi,

I have read a lot of posts about NFS mounting issues and I really didn't want to ask another one. I should be able to figure it out but I have been struggling for a couple of hours now so...

The goal is to test a rather simple setup, for data dump and  some exploration with Apache Drill and some scripts in spark. 

My setup:

    Mapr-version: 4.0.2
    1 Master node: m3.xlarge
    1 Core node : m3.xlarge
    License: M3 community edition
    Standard vpc (no special networking other than AWS password-less ssh)
    Didn't bootstrap spark or drill (wanted to learn more about the ecosystem as I go)

What I did:

    Ssh into the cluster - no issues
    Installed Spark - ran some samples - no issues
    Installed Drill - yet to play with it but no issues
    Setup NFS and mount it onto a linux machine(not mounting on the cluster) - issues

Where I am struggling with:

    Actually mounting the cluster onto a machine o.o, it times out.

What I understand about mounting NFS having spent the better part of the afternoon on the Mapr pages:
stock Linux NFS service must be disabled and it is

    ps -ef| grep nfs
    root      7466 25059  0 15:30 pts/0    00:00:00 grep nfs
    root     12326     2  0 08:24 ?        00:00:00 [nfsiod]
    hadoop   22166     1  0 15:08 ?        00:00:00 /opt/mapr/server/nfsserver
    hadoop   22167 22166  0 15:08 ?        00:00:00 /opt/mapr/server/nfsserver
    
    service nfs status
    rpc.svcgssd is stopped
    rpc.mountd is stopped
    nfsd is stopped

nlockmgr must be disabled and I believe it is
 

    rpcinfo -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  52213  status
        100024    1   tcp  36643  status
        100003    3   tcp   2049  nfs
        100005    3   tcp   2049  mountd
        100005    1   tcp   2049  mountd
        100005    3   udp   2049  mountd
        100005    1   udp   2049  mountd

rpcbind  must be running
  

      # ps ax | grep rpcbind
        11203 ?        Ss     0:00 rpcbind
        12515 pts/0    S+     0:00 grep rpcbind

mapr-nfs present and running on one node and mcs shows 1 nfs gateway

    # service mapr-nfsserver status
    nfsserver (pid 22167 22166) is running...

Primary group of user listed in mapr.deamon.user is mapr.deamon.group, and I think it is...

  

      mapr.daemon.user=hadoop
        mapr.daemon.group=hadoop
        mapr.daemon.runuser.warden=1

At this point is where things get a little unclear
I have my.cluster.com and I want to mount the it via NFS to a Linux machine running ubuntu. According the docs I can mount the cluster on a linux, mac or windows machine.

What I don't understand about actually mounting the cluster:
Are there any other steps to take on the cluster? 

    What I tried on the cluster:
     setup the mount point and the export in  /opt/mapr/conf/exports
    commented out:
    
         /mapr (rw)
    
    and created a new entry pointing directly to my cluster under /mapr
    
         /mapr/clusterName (rw)

Showmount looks ok, except for the two stars. Why does the cluster export have two stars?

    showmount -e localhost
    Export list for localhost:
    /mapr                        *
    /mapr/My.cluster.com *,*

What I tried on the client:

    Various mount options, soft, hard, explicit port and protocol to use tcp, (allow all incoming on AWS security group) explicit version 3 nfs, checked for nlockmgr.
    Everything times out, rpcinfo -p servername and any of the mount options just time out. Verbose mode didn't really give any clues.

NFS log after last restart:

   

     2015-04-28 15:08:11,4762 INFO nfsserver[22167] fs/nfsd/main.cc:532 ***** NFS server starting: pid=22167, mapr-version: 4.0.2.29870.GA *****
        2015-04-28 15:08:11,4764 INFO nfsserver[22167] fs/nfsd/main.cc:546 ******* NFS server MAPR_HOME=/opt/mapr, NFS_PORT=2049, NFS_MGMT_PORT=9998, NFSMON_PORT=9997
        2015-04-28 15:08:11,4907 INFO nfsserver[22167] fs/nfsd/mount.cc:2150 Export info: /mapr (rw)
        2015-04-28 15:08:11,4912 INFO nfsserver[22167] fs/nfsd/mount.cc:1784 CLDB info: x.x.x.x:7222
        2015-04-28 15:08:11,4913 INFO nfsserver[22167] fs/nfsd/mount.cc:2150 Export info: /mapr/My.cluster.com (rw)
        2015-04-28 15:08:11,4913 INFO nfsserver[22167] fs/nfsd/mount.cc:1784 CLDB info: x.x.x.x:7222
        2015-04-28 15:08:11,5320 INFO nfsserver[22167] fs/nfsd/nfsha.cc:476 hostname: ip-x.x.x.x.us-west-2.compute.internal, hostid: 0x6eb2a4b26a7c37b7
        2015-04-28 15:08:11,5325 INFO nfsserver[22167] fs/nfsd/requesthandle.cc:479 using default NFS_HEAPSIZE: 512 MB
        2015-04-28 15:08:11,8138 INFO nfsserver[22167] fs/nfsd/main.cc:643 NFS server started ... pid=22167, uid=498

Only thing I can think of now is that there is some confusion on my part on AWS EMR internal and external address, for example internal host names is that cluster uses, but the client is trying to access the public name which nat should handle I imagine.

Any help on where to look would be amazing.

Many many thanks in advance!

























Outcomes