Working with Linux volumes on vSphere

Modern Linux distributions running on VMware vSphere are capable for dynamic storage management, it is possible to create new file systems or extend existing without need to stop services and reboot servers. Here are some examples how to that, these methods work at least on Red Hat and CentOS 5.x distributions.

Things to consider with Linux volumes

If your intention is to use all of your new disk for single file system I recommend that you do not use partitioning, use a plain disk as LVM physical volume instead, it simplifies managing volumes a lot. You could also chose not to use LVM and format new disk as is with ext3 but I do not recommend that, using LVM has many benefits which I might cover on some later articles.

If you need to use partitioning you should be aware of following caveats

  1. To avoid performance penalty from file system misalignment you need to match beginning of new partition with underlying storage device using fdisk expert commands
  2. Using partitions makes extending file systems much more complicated compared to using plain disks instead

If you fail to align file system properly with your storage, you can lose up to 15% of your available IOPS.

Attach new disks without reboot

After new virtual disk have been added to virtual machine you need to rescan new disks in Linux. Scan new disks with command below, replace “host0″ with SCSI controller ID you added disks to

use dmesg command to check what new devices have been added, watch for “Attached scsi disk” message

Now create LVM Physical Volume, Volume Group, Logical Volume and file system on your new disk

Or if you know that you don’t need LVM you can simply format plain disk with

Your new volume is ready for mounting.

Resizing existing disks

After virtual disk have been resized you need to rescan changes in Linux. You must know SCSI ID of disk changed, replace 0:0:1:0 in example below with correct SCSI ID

use dmesg to check that resize is successful, watch for “capacity change” message

if LVM is not used resizing file system can be done with command

Other ways proceed to extend LVM volume

Extending LVM volume and file system, easy way with no partitions

Rescan LVM physical volume size change with pvresize, replace sdb with correct device

Extend LVM logical volume to new full size of volume group

Now you can resize mounted ext3 file system

And you are done

Extending LVM volume and file system, hard way with partitions in use

You cannot resize LVM physical device on partition without reboot, when using single disk your only option is to create new partition on space created by extending disk and use partprobe command to reload partition table without need to reboot. You could also create new disk and extend Volume Group to that.

Create a new partition to existing disk

Run partprobe to reload partition table

Create new LVM physical volume to new partition

Add new physical volume to existing Volume Group

Extend Logical Volume up to maximum size of Volume Group

Finally you can resize mounted ext3 file system

Done. See how much work there is when using partitions? There is absolutely no point at all to create partition table for standard ext3 file system when intention is to use whole disk for single file system.

7 comments to Working with Linux volumes on vSphere

  • Juan

    Great article Tomi. I am using Ubuntu 8.40 kernel 2.6.24-24 on an ESX server VM.

    I have an existing volume group sitting on a virtual disk that I want to grow, I expand the vmdk disk without any problems, but when it comes to detect the disk change on the Linux Vm I have problems.

    I rescan the scsi bus as stated on the tutorial but I don’t get the capacity change message on my logs. I can see that the bus is rescanned but fdisk keeps showing the old size (10G).

    Here are the kernel messages:

    Dec 10 10:31:57 vmdk-grow-test kernel: [70197.862014] ata2: EH complete
    Dec 10 10:32:16 vmdk-grow-test kernel: [70217.037848] sd 2:0:1:0: [sdb] 31457280 512-byte hardware sectors (16106 MB)
    Dec 10 10:32:16 vmdk-grow-test kernel: [70217.037899] sd 2:0:1:0: [sdb] Write Protect is off
    Dec 10 10:32:16 vmdk-grow-test kernel: [70217.037902] sd 2:0:1:0: [sdb] Mode Sense: 03 00 00 00
    Dec 10 10:32:16 vmdk-grow-test kernel: [70217.037930] sd 2:0:1:0: [sdb] Cache data unavailable
    Dec 10 10:32:16 vmdk-grow-test kernel: [70217.037932] sd 2:0:1:0: [sdb] Assuming drive cache: write through

    If I reboot the VM the changes are applied, but I don’t want to do that unless it is completely required.

    Any help is appreciated.

  • Chris

    Great article!
    In testing I found that you can resize the disk without knowing the SCSI ID:

    echo 1 > /sys/block/sdb/device/rescan

    Makes the process even easier.

  • John Beckmann

    You can actually extend a partition by simply deleting it and recreating it using the new size:-

    # fdisk /dev/sdb
    Command (m for help): d
    Selected partition 1

    Command (m for help):n
    Command action
    e extended
    p primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-46705, default 1):
    Using default value 3134
    Last cylinder, +cylinders or +size{K,M,G} (1-46705, default 46705):
    Using default value 46705

    Command (m for help): t
    Selected partition 1
    Hex code (type L to list codes): 8e
    Changed system type of partition 1 to 8e (Linux LVM)

    Command (m for help): w
    The partition table has been altered!

    Calling ioctl() to re-read partition table.

    WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
    The kernel still uses the old table.
    The new table will be used at the next reboot.
    Syncing disks.

    Run partprobe to reload partition table
    # partprobe /dev/sdb

    Rescan LVM physical volume size change with pvresize, replace sdb with correct device

    # pvresize /dev/sdb

    Extend LVM logical volume to new full size of volume group

    # lvextend -l+100%FREE /dev/VolGroup01/LogVol00

    Now you can resize mounted ext3 file system

    # resize2fs /dev/VolGroup01/LogVol00

    And you are done

  • [...] This post was mentioned on Twitter by VMware Planet V12n, Mikael Sennerholm. Mikael Sennerholm said: Working with Linux volumes on vSphere, bloggpost om volymhantering på Linux i vmware, verkar funka på ubuntu 10.04. http://lnkd.in/NuR9jj [...]

  • […] on what and how LVM is.  I certainly recommend checking out Tomi’s post on LVM here as I’m probably just going to regurgitate a little of what he wrote. […]

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">