AnsweredAssumed Answered

Query MapR Stream Topic Json Messages using Apache Drill?

Question asked by Lilia on May 11, 2017
Latest reply on May 12, 2017 by maprcommunity

Dear team,

 

Is there a functional possibility to query mapr stream topic directly with apache drill without the need to create a mapr dbjson table and a drill table/view on top of it?

I have tried the following but doesn't work:

MapR Cluster version 5.2

Drill version:1.8

Drill Storage Plugin for dfs

 "type": "file",
  "enabled": true,
  "connection": "hdfs://mapr-fs directory/",
  "config": null,
  "workspaces": {
    "root": {
      "location": "/",
      "writable": false,
      "defaultInputFormat": "csv"
    },
    "tmp": {
      "location": "/OGG/streams",
      "writable": true,
      "defaultInputFormat": null
    }
  },
.....
    "json": {
      "type": "json",
      "extensions": [
        "json"
      ]
.....
}

Mapr Stream topic messages are emitted:'/OGG/streams:test'

Apache drill create table statement:  CREATE TABLE dfs.tmp.`customer_topic` (first_name,last_name,age) AS SELECT columns[0]first_name,columns[1]last_name,columns[2]age FROM dfs.`/mapr-fs directory/OGG/streams:test`;
Error: Table 'dfs./mapr-fs directory/OGG/streams' not found.

Outcomes