I'm heavily using MapR streams (~10 billion messages a day), and I have a 4 day TTL set on the stream.
Multiple times now, I have filled up the volume the stream is on.
When looking in MCS at the volume page, I can see that it is in fact full. When I go to "MapR Tables" and view the stream there, I see that it is taking a huge amount of space compared to what is expected after 4 days.
Here's the interesting part... As I'm viewing the stream in MCS, I see the stream size start to fall down gradually like it's being compacted. I just observed one go from 7TB to 800GB gradually and have observed others fall on multiple occasions now. So, it seems like even though I'm constantly reading/writing to the topics in the stream, the stream is not being "cleaned" (for lack of a better word) until I view it in MCS.
Is this normal behavior in MapR? Am I supposed to be calling some sort of "cleanse" or "purge" operation regularly that it doesn't do itself? What does MCS implicitly do when you view a stream/table?