AnsweredAssumed Answered

How do I pass in Scan to Spark Job to read from MaprDb

Question asked by vamdiya1 on Mar 10, 2017
Latest reply on Apr 10, 2017 by cmcdonald

Hi,

 

I am looking to read from Maprdb from Spark application using the newHadoopRDD API. The current constraint is to read data using a columnValueFilter. I have tried in to pass in a scan object shown below, but filter doesn't seem to work. Any help is appreciated on this.

   

        byte[][] b = { "r:test".getBytes() };

        Scan s = new Scan()
        s.setFilter(singleColumnValueFilter);

        Job job = Job.getInstance(conf)

//Adding the scan along with columns.

        TableInputFormat.addColumns(s, b);

 

//Read from table using TableInputFormat class
        RDD<Tuple2<ImmutableBytesWritable, Result>> dataRDD = sc
                .newAPIHadoopRDD(job.getConfiguration(),
                        TableInputFormat.class, ImmutableBytesWritable.class,
                        Result.class);

 

 

Is this functionality supported by Maprdb?

Outcomes