How To Extend The MapR Sandbox VM's Storage Space

Document created by cwarman Employee on Jun 16, 2016Last modified by Karen Whipple on Aug 5, 2016
Version 7Show Document
  • View in full screen mode

Note:  This document is intended for users running a MapR Sandbox on VMware.  For instruction how to how to achieve the same objective with Oracle’s VirtualBox see the article How to Resize an Oracle VirtualBox Disk.

 

The root volume shipped with the MapR Sandbox for Hadoop is basically just large enough to run the VM.  If you try to install any additional components you'll typically receive a "disk full" error message, such as this one (from the yum utility):

Transaction Check Error:
  installing package mapr-drill.noarch.rpm  needs 265MB on the / filesystem
Error Summary
-------------
Disk Requirements:
  At least 265MB more space needed on the / filesystem.

 

You can observe the free space available using the df -h command, like so:

# df -h
Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/vg_maprdemo-lv_root    8.4G  8.0G     0 100%  /
tmpfs                              2.9G     0  2.9G   0%  /dev/shm
/dev/sda1                          477M   41M  411M  10%  /boot
localhost:/mapr                    100G     0  100G   0%  /mapr
localhost:/mapr/demo.mapr.com/user  15G  5.2G  9.8G  35%  /user

 

The highlighted area above shows the root volume on my MapR Sandbox.  It's full, so I can't install anything else.  These instructions illustrate how to extend this volume so that you'll have the space needed for installing additional components.

 

Two notes before you begin:

  • I'm using the 5.1 Sandbox for VMware on a Mac OSX laptop.  Don't worry if you're running a different virtualization environment (such as VirtualBox) or OS (such as Windows or Linux) - Pretty much everything presented below takes place within the Sandbox itself - the only place where you'll need to make an adjustment for your particular environment is where you tell your virtualization environment to re-size the Sandbox VM's disk, and I've flagged that accordingly.
  • My instructions are based on these instructions, and assume that you are connected to the MapR Sandbox via ssh as user mapr (the default password for that user is mapr, in case you weren't aware).

 

If all goes well, you should be done with this in roughly 10-15 minutes.  Let's get started!

 

  • Once you've ssh'd into your Sandbox as mapr you'll need to change to root:

# su -
Password: mapr

 

  • Now display the existing volume groups:

# vgdisplay -v
Using volume group(s) on command line.
Finding all volume groups.
--- Volume group ---
VG Name               vg_maprdemo 
<< This is the volume group that will be extended
                                     
Note this name as it will be used later
System ID 
Format                lvm2
Metadata Areas        1
Metadata Sequence No  3
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                2
Open LV               2
Max PV                0
Cur PV                1
Act PV                1
VG Size               9.41 GiB
PE Size               4.00 MiB
Total PE              2409
Alloc PE / Size       2409 / 9.41 GiB
Free  PE / Size       0 / 0
VG UUID               aiIP2i-Xqri-fff9-8U9x-FQEK-eEdX-lDvkDb

--- Logical volume ---
LV Path                /dev/vg_maprdemo/lv_root 
<< This is the logical volume that will be extended
                                                   
Note this name as it will be used later
LV Name                lv_root
VG Name                vg_maprdemo
LV UUID                IdEpIt-E1LH-sdhe-4jxB-tAMI-4FEZ-ak5fFL
LV Write Access        read/write
LV Creation host, time maprdemo.local, 2016-03-07 09:50:09 -0800
LV Status              available
# open                 1
LV Size                8.61 GiB
Current LE             2205
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:0

--- Logical volume ---
LV Path                /dev/vg_maprdemo/lv_swap
LV Name                lv_swap
VG Name                vg_maprdemo
LV UUID                xw5iRo-IuzQ-S1kZ-mmc1-RHEt-wTBJ-sdWALC
LV Write Access        read/write
LV Creation host, time maprdemo.local, 2016-03-07 09:50:10 -0800
LV Status              available
# open                 1
LV Size                816.00 MiB
Current LE             204
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:1

--- Physical volumes ---
PV Name               /dev/sda2 
<< This is the corresponding physical volume
                                   
(although it's actually virtualized by VMware)
PV UUID               q9VVmn-ciGG-sZrJ-ygqo-mdNE-IYPu-w2pqZd
PV Status             allocatable
Total PE / Free PE    2409 / 0

  • Display the existing partition table:

# fdisk -l
Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sda: 16.1 GB, 16106127360 bytes 
<< This is the corresponding physical disk
                                             
(again, it's actually virtualized by VMware)
255 heads, 63 sectors/track, 1958 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006f230

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64        1293     9870336   8e  Linux LVM

Disk /dev/mapper/vg_maprdemo-lv_root: 9248 MB, 9248440320 bytes
255 heads, 63 sectors/track, 1124 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_maprdemo-lv_swap: 855 MB, 855638016 bytes
255 heads, 63 sectors/track, 104 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

  • Shut down the VM:

# shutdown -h now

  • Locate the virtualized hard disk that corresponds to the physical disk identified above - it will be located in the virtual machine's settings area, and its size should be somewhere in the neighborhood of 16 GB (as indicated in the highlighted area directly above).  Change the disk size as desired - The screenshot you see below is where I changed mine from 15GB to 25GB in VMware on my Mac laptop:

Expand.png

This is where you'll need to do whatever is appropriate for your particular virtualization environment (such as VirtualBox) or OS (such as Windows or Linux), as I noted earlier.

  • Restart the VM, then re-connect to the MapR Sandbox via ssh as user mapr and change to root:

# su -
Password: mapr

  • Display the updated partition table:

# fdisk -l
Disk /dev/sda: 26.8 GB, 26843545600 bytes  
<< The physical disk is larger now - Note that its position
                                              
in the list has changed (this can be ignored)
255 heads, 63 sectors/track, 3263 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006f230

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64        1293     9870336   8e  Linux LVM

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_maprdemo-lv_root: 9248 MB, 9248440320 bytes
255 heads, 63 sectors/track, 1124 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_maprdemo-lv_swap: 855 MB, 855638016 bytes
255 heads, 63 sectors/track, 104 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

  • Now create a new primary partition - note that the highlighted device should match the newly enlarged physical disk indicated above:

# fdisk /dev/sda

    • Press p to print the partition table to identify the number of partitions. By default, there are 2 - sda1 and sda2:

Disk /dev/sda: 26.8 GB, 26843545600 bytes
255 heads, 63 sectors/track, 3263 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x0006f230
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64        1293     9870336   8e  Linux LVM

    • Press n to create a new primary partition.
    • Press p for primary.
    • Press 3 for the partition number (since 1 and 2 have been used, as indicated by the printed partition table above)
    • Press Enter two times to take the default first and last cylinder values (mine were 1293 and 3263, respectively).
    • Press t to change the system's partition ID.
    • Press 3 to select the newly-created partition.
    • Type 8e to change the Hex Code of the partition to Linux LVM.
    • Press w to write the changes to the partition table.  This will exit the fdisk utility and return back to the shell command prompt.
  • Reboot the VM:

# reboot

  • Re-connect to the MapR Sandbox via ssh as user mapr and change to root:

# su -
Password: mapr

  • Display the updated partition table to verify that the new partition was created with an 8e type:

# fdisk -l
Disk /dev/sda: 26.8 GB, 26843545600 bytes
255 heads, 63 sectors/track, 3263 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006f230

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              64        1293     9870336   8e  Linux LVM
/dev/sda3            1293        3263    15826687+  8e  Linux LVM

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_maprdemo-lv_root: 9248 MB, 9248440320 bytes
255 heads, 63 sectors/track, 1124 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/vg_maprdemo-lv_swap: 855 MB, 855638016 bytes
255 heads, 63 sectors/track, 104 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

  • Convert the new partition to a physical volume - note that the device name should match that of the newly-created partition:

# pvcreate /dev/sda3
  Physical volume "/dev/sda3" successfully created

  • Extend the volume group using the newly-created physical volume - Note that the volume group name should match that which was noted at the first step of these instructions (where we ran vgdisplay):

# vgextend vg_maprdemo /dev/sda3
  Volume group "vg_maprdemo" successfully extended

  • Determine the free space now available to the Volume Group:

# vgdisplay vg_maprdemo | grep "Free"
  Free  PE / Size       3863 / 15.09 GiB

  • Extend the logical volume using the free space in GB determined immediately above - Note that the full number should be specified, including any decimals, and that the logical volume name should match that which was noted at the first step of these instructions (where we ran vgdisplay):

# lvextend -L+15.09G /dev/vg_maprdemo/lv_root
  Rounding size to boundary between physical extents: 16.00 MiB
  Size of logical volume vg_maprdemo/lv_root changed from 8.61 GiB (2205 extents)
     to 23.70 GiB (6067 extents).
  Logical volume lv_root successfully resized

  • Now expand the ext3 filesystem online - inside of the Logical Volume - using the same logical volume name again:

# resize2fs /dev/vg_maprdemo/lv_root
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/vg_maprdemo/lv_root is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/vg_maprdemo/lv_root to 6212608 (4k) blocks.
The filesystem on /dev/vg_maprdemo/lv_root is now 6212608 blocks long.

  • Finally, verify that the / filesystem has the new space available:

# df -h
Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/vg_maprdemo-lv_root     24G  7.9G   15G  36%  / 
<< Much better!
tmpfs                              2.9G     0  2.9G   0%  /dev/shm
/dev/sda1                          477M   41M  411M  10%  /boot
localhost:/mapr                    100G     0  100G   0%  /mapr
localhost:/mapr/demo.mapr.com/user  15G  5.2G  9.8G  35%  /user

 

At this point it should now be possible to install additional components onto the MapR Sandbox.  Enjoy!

7 people found this helpful

Attachments

    Outcomes