AnsweredAssumed Answered

Problem with MapR 6.0.1 Streams connectivity (node.js)

Question asked by rbukarev on Jul 24, 2018
Latest reply on Jul 29, 2018 by rbukarev

So I have MapR 6-something with Kafka 1.0.1, and I have a topic with numbers that I want to display on a live dashboard. The "live dashboard" part is for demo only, hence can be quick&dirty, which is why I picked Node.js with kafka-node package to consume data from the topic. I mostly stole the code from the Github example.

 

It should be noted that I can display the contents of my topic using a sample Kafka's console client:

/opt/mapr/kafka/kafka-1.0.1/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic /mystream:events

 

 

[UPDATE1]

So after I realised there are no brokers on 9092, I switched to connection via Zookeeper (port 5181). Getting a new error in the node.js console now, and I'm not sure whether it's a node.js or Zookeeper issue. Here's a relevant bit from Zookeeper's log:

2018-07-26 01:08:48,344 [myid:0] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:5181:NIOServerCnxn@349] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x64d37869a10a84, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:222)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:748)
2018-07-26 01:08:48,344 [myid:0] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:5181:NIOServerCnxn@1001] - Closed socket connection for client /10.x.x.x:38462 which had sessionid 0x64d37869a10a84

 

I made sure there was data being posted to the topic, so the stream wasn't empty. Is there anything else I can do to narrow down the problem to either ZK or my code? (Meanwhile I'm gonna try it on Hortonworks to see if the code itself is OK).

 

[UPDATE2]

So I tried the code in Hortonworks and it works perfectly. Could be something with zookeeper settings in MapR?

 

[========Obsolete part================]

However, when I run my node.js script, this is the error I get -- connection to a kafka broker refused:

{ Error: connect ECONNREFUSED 127.0.0.1:9092
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1163:14)
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 9092 }
Error: connect ECONNREFUSED 127.0.0.1:9092
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1163:14)

 

Then I tried to check if there's connectivity at all (as in, 'port open'):

bash$ sudo nc -zv localhost 9092
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connection to ::1 failed: Connection refused.
Ncat: Trying next address...
Ncat: Connection refused.

 

If nc can't see a port open, how can console consumer? Am I doing something completely wrong here? Could the authentication be related?

[======== End of obsolete part================]

Outcomes