Tiered Storage within a Node

Idea created by MichaelSegel on Nov 29, 2017
    Active
    Score40
    • cathy
    • jcr
    • ryanvictory
    • amuthan
    • MichaelSegel
    • Wibe2017
    • mandoskippy
    • JFMartin

    With respect to storage tiering,  MapR currently only allows you to segregate machines in to groups where all disks/devices on a node must be homogeneous. 

     

    So if I wanted to offer 'faster storage' to a cluster, then all of the devices on a specific node must be of the same type.

    That means adding a machine or set of machines that only have SSDs or NVMe drives where the rest of the cluster uses spinning rust.

     

    Instead it should be an option to add a separate tier of storage to an existing node where each node in the cluster could support HDs (spinning rust), SSDs and NVMe each as separate tiers in different volumes and then pool the volumes of like storage across the cluster.

     

    The current solution will force segmentation of the cluster nodes which can lead to data locality issues.

    Note: Its possible that data locality is a moot point when considering the storage/compute model, however having the option to add a tier of storage to an existing cluster would be extremely valuable.

     

    I'm not sure why current MapR topology won't allow you to create storage groups of tiered storage within a single node, but that's what I'm being told.

     

    IMHO this shouldn't be that hard to implement and would be extremely valuable when considering using mixed sources to store the underlying data and then potential indexes on a faster medium....