vReality

EMC Unified Storage – what’s new in FLARE 30

By Tomi Hakala, on July 10th, 2010 | Leave a comment

Those EMC customers who missed most EMC World 2010 announcements, like me, will find included video very exciting, it shows new storage efficiency features, and new middle range storage management GUI, coming to CLARiiON CX4 in FLARE 30. New FLARE will be released some time during this summer.

Features shown in video are

New CLARiiON and Celerra management GUI called Unisphere, based on Adobe Flex technology
Sub-LUN tiering – “VMware DRS” equivalency for storage medium, will migrate data based on usage between Flash, FC and SATA
Flash cache – AKA FAST cache, up to 2 terabytes(!) of 2nd level read/write cache
Block level compression – told to save 30% to 50% on average on your storage consumption

It is not told directly in this video but it looks like thin pools will become an standard feature on CX4, at the moment is is sold as an add-on option.

These and new vStorage API integration features make me feel that this is a good time to be EMC customer.

VMware Data Recovery 1.2 Linux file level restore client

By Tomi Hakala, on July 2nd, 2010 | Leave a comment

Latest VMware Data Recovery release 1.2 brought file level restore capability (aka FLR) for Linux VMs. Windows FLR client has been officially supported since 1.1, see my previous blog post about it. I did a quick review of using Linux FLR on CentOS Linux.

VMware Data Recovery 1.2 documentation says that Linux FLR client can be used in following Linux distributions, including both 32-bit and 64-bit versions

Red Hat Enterprise Linux (RHEL) 5.4/CentOS 5.4
Red Hat 4.8/CentOS 4.8
Ubuntu 8.04
Ubuntu 8.10
Ubuntu 9.04

I have successfully tested it on 64-bit CentOS 5.5, and because so many versions of Ubuntu are listed I’d guess that FLR client works on any recent Debian releases also, just make sure it has support for FUSE 2.5 or later. If you have custom kernel make sure you have all FUSE dependencies compiled in. Note that even though your Linux distribution may be 64-bit version, 32-bit version of FUSE is required. Note the absence of any SuSE or Novell SLES distrubtions from tested and supported list, not that FLR client won’t work on them though, I am sure it will.

Before you begin you should know that FLR client needs to access VDR appliance TCP port 22024, if your VM and VDR appliance are in different networks make sure that any possible firewalls between them allow this. You also need root privileges on Linux VM to be able to execute FLR client.

To install Linux FLR client you need to mount VMware Data Recovery appliance ISO to your Linux VM, then cd to LinuxFLR directory on that ISO and untar VMwareRestoreClient.tgz in to a directory you want to install it in, I decided to install FLR client in /opt/vdr.

Like previously brought up Linux FLR client uses Linux FUSE API for accessing the restore point, for FUSE to work on CentOS Linux you need to have “fuse” and “fuse-libs” packages installed.

On CentOS Linux with access to yum repository, required FUSE packages are installed as simply as

# yum -y install fuse fuse-libs

Linux FLR client does not come along with SELinux so you need to either disable it or configure it in a way so it will allow FLR client to function. Disabling SELinux will have impact on Linux VM security and may violate your company security policy so you have to fully understand what you are doing if you chose to disable it, because of this I am not showing you an example how to disable SELinux.

Make sure that you have at least one restore point available (one backup made), then execute FLR client as root with “/path/to/VdrFileRestore -a ip-or-name-of-vdr-appliance“, note capital V on FLR executable.

VdrFileRestore will show you all available restore points, and query which restore point you wish to access

Type in restore point index and press enter

Once restore point is mounted you need to open second session to this Linux host for accessing backup file set. After logged in you can see that there is now new file systems mounted to your system, restore point is mounted under root-user home directory into directory matching backup date and timestamps.

Because restore point is mounted to root-user home directory no-one without root privileges cannot access backup file set, which is a good security measure.

For a system with multiple file systems you should see “Mount1, Mount2, Mount3 ..” etc directories for each mount point. This example system is a CentOS default install with /boot and / file systems, /boot is mounted to Mount1, / is mounted to Mount2. CD to mount directory which matches the file system you need to restore files from

Copy files you wish to restore with normal Linux commands and at the end remember to change directory out of backup file set directory. Note that even though backup file set is mounted with read/write access any file changes or deletes from it will not change or remove files from actual VDR backup storage, file changes are simply visible in run-time restore point mount.

After you have copied all files you need, and changed working directory out of backup file set, you may bring up VdrFileRestore session window and close restore point with “unmount” command

That is simply how you can restore invidual files from VDR backup store.

You can embed FLR client into Linux templates but you should always use matching versions of FLR client and VDR appliance.

Using NexentaStor ZFS storage appliance with vSphere

By Tomi Hakala, on June 29th, 2010 | 3 comments

NexentaStor is an storage appliance based on OpenSolaris kernel and its advanced ZFS file system. Latest free Community Edition comes with goodies like online deduplication, transparent compression and snapshots to name few. NexentaStor can share storage over NFS, CIFS, iSCSI, FTP, RSYNC, WebDAV and even FC. On this article I am concentrating using NexentaStor NFS shares with VMware vSphere.

NexentaStor is available as commercially supported enterprise version and as community supported free to use Community Edition. Community Edition is limited to 12 terabytes of used storage, it is also lacking some optional modules available on enterprise version.

I downloaded NexentaStor Community Edition 3.0.3 and installed it on PC I built of some left over components I had at hand. My finished storage appliance consist of

AMD Athlon 4850e dual core CPU
Asus M3A78 Pro motherboard
4 GB DDR2 RAM
1 x 80GB Seagate 7200.10 ATA disk as boot drive
5 x 500GB Seagate 7200.11 SATA disks as ZFS pool
Intel X25-M G2 as flash cache
Intel PRO/1000 PT Quad Port Ethernet adapter

Above system fits to NexentaStor hardware recommendations like a glove, 64-bit CPU, 4GB RAM, dedicated spindles for ZFS pool and dedicated boot drive.

Although Community Edition is free, it requires registration key which is unique for each hardware appliance is installed on. You get registration key in email by submitting machine signature to Nexenta, machine signature is generated and presented during appliance installation. I got mine registration key in email within a minute of submitting signature, which was nice.

NexentaStor recognized all of my storage system components correctly and installation was finished 15 minutes later. Once registration was active I was queried of management IP and WebGUI port settings, those set I launched web browser and pointed it to my new storage appliance.

Configuration wizard

First step at WebGUI is to set attributes such as host and domain name, NTP server and localization.

Second step is network configuration,

including iSCSI target settings.

In third and fourth step you create your ZFS volume, you may also configure any available SSD drives as secondary read cache (L2ARC), or as ZFS intent log device (ZIL) which is used for writes

SSD as secondary cache

Employing fast SSDs as secondary disk cache is a really neat feature. SSDs are much larger in sizes and cheaper per gigabyte than RAM and they still provide incredible amount of IOPS, for example even consumer grade SSD Intel X25-M is rated up to 35,000 IOPS on 4k reads! You need approximately 175 15kRPM spindles to deliver that amount of IOPS!

Folders

Now that you have your ZFS volume created, you are asked to create folders used as network shares, I created single folder named “nfs”. You can also set ZFS attributes such as deduplication and compression

NFS share settings

Once you save and review your configuration you are finished with configuration wizard. Now select “Data Management” from top menu, and in drop down list select “Shares”

If you did not yet enable NFS sharing you may do it now, then click “Configure” in “NFS Server” box on left side of screen

Set Server and Client Version to 3, it’s the version vSphere is using and it is recommended by Nexenta to limit NFS version to 3 when using with vSphere, click Save

Go back to Shares view and click folder link

In this view you see mount point in which folder is available to NFS client, make note of it

If you scroll down the view there is options like deduplication, checksum, compression and access time. Checksum is ZFS feature for detecting data corruption, it works only if you are using RAID-Z or ZFS mirroring, if you use hardware RAID you should disable this since it has small CPU overhead. You might disable access time as well, it is not needed with vSphere datastores and it has small disk IO overhead. You can enable or disable deduplication and compression any time you like, new setting will affect only new data written to ZFS.

Mounting NFS share to vSphere

I assume that you have necessary VMkernel port set at ESX so I won’t go into details with that. Open vSphere Client and mount NAS datastore, folder path is mountpoint you saw at folder settings view

And you have NexentaStor NFS share mounted to vSphere

NexentaStor and ESX 4.0 both support jumbo frames so you can improve NFS performance by configuring MTU up to 9000k or what is maximum supported by your NICs and/or switches.

Performance

I configured my ZFS volume as striped RAID-0 (since this is purely experimental, no valuable data stored in here), I have ZIL enabled and stored at ZFS pool, Intel X25-M SSD as L2ARC (flash cache) device. Compression and deduplication are off.

Raw throughput of NexentaStor NAS is quite good, below is an results of large file write with 8k blocks within CentOS Linux VM

# dd if=/dev/zero of=/sdb/test bs=8k count=838860
838860+0 records in
838860+0 records out
6871941120 bytes (6.9 GB) copied, 58.3875 seconds, 118 MB/s

and same file read in 8k blocks

# dd if=/sdb/test of=/dev/null bs=8k
838860+0 records in
838860+0 records out
6871941120 bytes (6.9 GB) copied, 79.2254 seconds, 86.7 MB/s

By looking read performance there might be something wrong with my L2ARC setup as reads are slower than writes, running iostat in storage appliance also reveals that L2ARC device is not read that much. Write performance could be improved a quite lot (to point of GigE saturation at least) by assigning second SSD drive as ZIL device, or I could just disable ZIL altogether but lose on data integrity. Well, I think I’ll investigate that on some another day.