AnsweredAssumed Answered

Problems when writing Tweets to MapR-FS from Java application.

Question asked by lu4nm3 on Jun 24, 2014
Latest reply on Apr 27, 2015 by snayeem
Hello,

Currently, I have a MapR M7 setup on Amazon's EMR cloud that I'm using to test out the system. I made an application to write tweets to MapR using much of the same code found here: http://doc.mapr.com/display/MapR2/Accessing+MapR-FS+in+Java+Applications

In my application, the Twitter stream saves all of the tweets into a queue as strings. I then have 5 threads that pull out the tweet strings, convert them to byte arrays, and write them to MapR using the FSDataOutputStream exactly like in the example above. That's pretty much it.

While some tweets do seem to be getting written to the .w file, not all of my tweets are actually saving. For example, out of 100 tweets that I tried to write, only 34 of them were written. And out of those 34 I noticed that parts of the tweets were cut off on every line when I looked at the .w file.

I'm thinking that this might have something to do with some of the options that I'm giving to the FSDataOutputStream which are the ones found on the link above but I can't be sure. Does anyone have any suggestions as to what might be going on here?

Outcomes