AnsweredAssumed Answered

hadoop mini cluster not working on mapr 4.0.1

Question asked by tongqing_qiu on Jan 17, 2015
Latest reply on Jan 19, 2015 by tongqing_qiu
I tried to use MiniDFSCluster to do some testing. It shows me the following error and cluster is not started.


    15/01/17 08:51:52 WARN datanode.DataNode: The reported NameNode version is too low to communicate with this DataNode. NameNode version: '1.0.3-mapr-1408-SNAPSHOT' Minimum NameNode version: '2.1.0-beta'
    15/01/17 08:51:52 FATAL datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:50661. Exiting.
    org.apache.hadoop.hdfs.server.common.IncorrectVersionException: The reported NameNode version is too low to communicate with this DataNode. NameNode version: '1.0.3-mapr-1408-SNAPSHOT' Minimum NameNode version: '2.1.0-beta'
     at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.checkNNVersion(BPServiceActor.java:196)
     at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.retrieveNamespaceInfo(BPServiceActor.java:183)
     at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:215)
     at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:816)
     at java.lang.Thread.run(Thread.java:745)
    
    
    15/01/17 08:52:02 WARN hdfs.MiniDFSCluster: Waiting for the Mini HDFS Cluster to start...
    Exception in thread "main" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
     at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:418)
     at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:391)
     at MiniClusterTest.main(MiniClusterTest.java:23)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
    Caused by: java.lang.reflect.InvocationTargetException
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
     at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:409)
     ... 7 more
    Caused by: java.io.IOException: Timed out waiting for Mini HDFS Cluster to start
     at org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1150)
     at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:791)
     at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:438)
     ... 12 more

Here is the sample code and maven dependencies

    
    Configuration conf = new Configuration();
     dfsCluster = new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
    DistributedFileSystem fileSystem = dfsCluster.getFileSystem();
    

    <dependencies>
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-core</artifactId>
                <version>2.4.1-mapr-1408</version>
            </dependency>
        
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-minicluster</artifactId>
                <version>2.4.1-mapr-1408</version>
                <scope>test</scope>
            </dependency>
        </dependencies>


Outcomes