AnsweredAssumed Answered

M7 with OpenTSDB 2.0 Scanner Error

Question asked by mkrysiak on Nov 15, 2014
Latest reply on Nov 17, 2014 by aditya
Has anyone had success with MapR 4.0.1/M7 with OpenTSDB 2.0.X?  I made the changes in https://github.com/mkrysiak/opentsdb/commit/a2fd25dcd39ce24abe9f4044489ee9e018bcf528, and successfully built OpenTSDB, but I randomly encounter Scanner errors when running queries.

12:35:10.732 INFO  [GraphHandler.logInfo] - [id: 0x66234dd9, /192.168.5.18:53663 => /192.42.52.109:4242] Query failed (stack trace coming): TsdbQuery  (start_time=1415813640, end_time=1415900098, metric=[0, 0, 56] (filer.quota.ifs.used), tags={}, rate=false, aggregator=sum, group_bys=())
12:35:10.736 ERROR [HttpQuery.logError] - [id: 0x66234dd9, /192.168.5.18:53663 => /192.42.52.109:4242] Internal Server Error on /q?start=2014/11/12-12:34:00&end=2014/11/13-12:34:58&m=sum:filer.quota.ifs.used&o=&yrange=%5B0:%5D&wxh=1416x668&json
net.opentsdb.core.IllegalDataException: HBase returned a row that doesn't match our scanner (Scanner(table="tsdb", start_key=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], stop_key="\x00\x008Td\xF9\xD4", columns={"t"}, populate_blockcache=true, max_num_rows=128, max_num_kvs=4096, region=none, filter=null, scanner_id=0x0EAD000CC000DEAD), mresultScanner=com.mapr.fs.MapRResultScanner@29a32eb9)! [KeyValue(key=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],family="t", qualifier=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], value=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], timestamp=1415891595201)] does not start with [0, 0, 56]
        at net.opentsdb.core.TsdbQuery$1ScannerCB.call(TsdbQuery.java:399) ~[tsdb-2.0.0.jar:]
        at net.opentsdb.core.TsdbQuery$1ScannerCB.call(TsdbQuery.java:353) ~[tsdb-2.0.0.jar:]
        at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257) ~[suasync-1.4.0.jar:fe17b98]
        at com.stumbleupon.async.Deferred.callback(Deferred.java:1005) ~[suasync-1.4.0.jar:fe17b98]
        at org.hbase.async.HBaseRpc.callback(HBaseRpc.java:538) ~[asynchbase-1.5.0-mapr-1408.jar:na]
        at org.hbase.async.MapRThreadPool$ScanRpcRunnable.run(MapRThreadPool.java:429) ~[asynchbase-1.5.0-mapr-1408.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_65]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_65]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]

Outcomes