AnsweredAssumed Answered

How to interpret output of mapr streams python api example (or what is going wrong with the documentation example not working)?

Question asked by reedv on Dec 26, 2017
Latest reply on Jan 16, 2018 by maprcommunity

 Going through the example provided in the docs for using the mapr streams api for python (Developing a MapR-ES Python Application), when finally

the consumer.py program script simply hangs (yet looking at the source coded indicates that there should be data printed to standard output). Adding some print statements to the code shows that the consumer is polling the stream and finding it to be empty. My question is where did the data go? Or if my question is wrong-headed due to a misunderstanding of how streams work, how am I supposed to be seeing the output of the consumer that the documentation example seems to indicate is being published to the stream via the provided producer script.

 

It seems like the consumer.py is not actually consuming anything. Looking at the stream, I see

[mapr@mapr001 src]$ maprcli stream topic info -path /my_stream -topic mytopic -json
{
"timestamp":1514328853556,
"timeofday":"2017-12-26 12:54:13.556 GMT-1000 PM",
"status":"OK",
"total":3,
"data":[
{
"partitionid":0,
"physicalsize":0,
"logicalsize":0,
"maxoffset":-1,
"minoffsetacrossconsumers":0,
"mintimestamp":"1969-12-31T02:00:00.000-1000 PM",
"maxtimestamp":"1969-12-31T02:00:00.000-1000 PM",
"mintimestampacrossconsumers":"1969-12-31T02:00:00.000-1000 PM",
"fid":"2081.32.262542",
"master":"mapr003.mycluster.local:5660",
"servers":"mapr001.mycluster.local:5660, mapr003.mycluster.local:5660, mapr004.mycluster.local:5660"
},
{
"partitionid":1,
"physicalsize":0,
"logicalsize":0,
"maxoffset":-1,
"minoffsetacrossconsumers":0,
"mintimestamp":"1969-12-31T02:00:00.000-1000 PM",
"maxtimestamp":"1969-12-31T02:00:00.000-1000 PM",
"mintimestampacrossconsumers":"1969-12-31T02:00:00.000-1000 PM",
"fid":"2081.32.262542",
"master":"mapr003.mycluster.local:5660",
"servers":"mapr001.mycluster.local:5660, mapr003.mycluster.local:5660, mapr004.mycluster.local:5660"
},
{
"partitionid":2,
"physicalsize":0,
"logicalsize":0,
"maxoffset":-1,
"minoffsetacrossconsumers":0,
"mintimestamp":"1969-12-31T02:00:00.000-1000 PM",
"maxtimestamp":"1969-12-31T02:00:00.000-1000 PM",
"mintimestampacrossconsumers":"1969-12-31T02:00:00.000-1000 PM",
"fid":"2081.32.262542",
"master":"mapr003.mycluster.local:5660",
"servers":"mapr001.mycluster.local:5660, mapr003.mycluster.local:5660, mapr004.mycluster.local:5660"
}
]
}

which makes me think the stream topic is empty (since the physical sizes are all zero and all of the mintimestamps are at the beginning of epoch time), so what could have happened?

Thanks.

Outcomes