Dealing With Peak Storage I/O

A peak in storage I/O occurs when an application suddenly has a spike of I/O requests to and from the storage device. Prior to virtualization we sized servers that applications ran on and the storage infrastructure that those servers used specifically for those peak times. This means that most of the time those servers sat idle. Thanks to virtualization, we can't size hosts to handle the load if all its VMs peaked.Performance tuning and dealing with peaks in storage I/O demands has now become a top concern for storage administrators. In fact in a recent survey conducted by Storage Switzerland over 50% of the respondents listed performance tuning and troubleshooting as the biggest storage challenge caused by server virtualization. It is also the primary reason often given for virtualization stall because the organization struggles with how to maintain performance service levels with the business units.

Peak storage I/O can come from an increase in batch processing like at the end of a quarter or it can come from a sudden spike in the number of online users. The impact in a virtual environment may be that the application may never be able to service the peak load because the resources are not there. If it does it may mean that other virtual machines on the physical host become starved for resources and their performance suffers severely. That's the challenge with virtualization, everything impacts everything. In either case the result is not good and applications become so slow that they actually feel like they have stopped to the user. This is just as bad as an application crash so avoiding that situation is critical.

One solution is to build, as we discussed in our recent webinar, "Stopping The Storage Roadblock To Server Virtualization", a much faster storage infrastructure. Storage networks built on 10GbE are affordable and deliver a significant performance boost to the environment especially those that do not have to deal with IP overhead. The other option is to understand your environment and make better use of the current resources. The reality is you will probably need to build both a faster network and now how to fine tune that network.

Monitoring the virtual environment requires real-time or near real-time information to be able to assess how resources like storage I/O are being consumed. It also means seeing that consumption at both the virtual machine level and the physical host level. You need to know which virtual machines are chewing up resources and you need to know which physical hosts have resources available if you decide to move virtual machines around to balance out the load.

When a peak storage I/O load occurs, if you can identify the virtual machine causing the problem via a monitoring tool you have several choices. One choice is to move the other VMs on the physical host to other physical hosts. This frees up most of the resources of the host for that particular task. You could also move the VM causing the I/O peak to a host that has plenty of free resources. In extreme situations you may want to actually move the peak VM out of the virtual environment. As we discussed in our article "Virtual To Physical Machine Conversion To Mitigate Risk" some migration tools are enabling those capabilities and it is a good one to have on your IT utility belt.

The good news is that server virtualization provides the flexibility to deal with peak storage I/O loads. The critical component however is knowing which of those options is going to best help you get through the peak load. The only way to know that is through the use of a monitoring tool that can give you the analytics to make the right decision.

Track us on Twitter: http://twitter.com/storageswiss

Subscribe to our RSS feed.

George Crump is lead analyst of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. Find Storage Switzerland's disclosure statement here.