Out Of Band Data Movers

Another form of data mover is the out-of-band data mover. Unlike Global Namespaces or agent-based data movers, these data movers crawl selected servers when doing their analysis. As they access each file, they analyze it to see if meets any criteria that you might have set for data movement. Since this crawl is merely a scan of the file system, as long as the out-of-band data mover supports the type of network file system that your servers use, then it will work. Not surprisingly, most of the players in the space support both NFS (for Unix systems) and CIFS (for Windows systems). Companies that offer this type of solution include Enigma Data Solutions with its Smartmove product and Arkivio's Autostor (recently acquired by Rocket Software).There are two distinct advantages of this crawl technique. First, support for the platform is already built in. There is no need to support each individual OS since CIFS and NFS provides most of the support you will ever need. As a result, this also includes support of products that don't support agents -- network-attached storage products, for example, also are handled well by out-of-band data mover solutions. The second advantage is its out-of-band implementation. It removes the need to install an agent on every server to be monitored. Again, for a NAS that does not have a global file system built in, this is critical, and also for many IT professionals that don't want to deploy agents on every platform. It is also easier to implement -- and if all you want to do is simple data movement, it may be the ideal way to go.

Out-of-band data movement is not without its challenges. First, depending on the solution you choose and your environment, it can be very time consuming to crawl the entire environment, especially if there are millions of files to examine. Some of the solutions keep a list of previously scanned files and don't need to examine them again if nothing has changed since the last file system scan. Regardless, be prepared for scan times of hours and maybe days in some cases, especially on the first scan. As a result, these scans are scheduled events that need to be integrated into all the other tasks that you have going on during "idle" times. When a file qualifies for a certain type of action -- move an old file, for example -- that action does not occur until the next time the out-of-band data mover performs its scan.

The final area of concern is how the out-of-band data movers maps its way back to files that it has moved. It does this by leaving a stub file behind. A stub file is a small version of the file that points to where the new location of the original file is. Since they don't have a local agent, out-of-band data movers need to leverage what the operating system provides to be able to create this link. For Windows files, they will leave a shortcut, for Unix systems a symbolic link. These files need to be managed and accounted for. Your backup application has to be set to ignore them and your users taught not to delete them.

Out-of-band data movers provide a sense of simplicity to the data management task. In a future entry, we will compare global file systems, agent-based data movers, and out-of-band data movers to give some guidance on selecting the right technique for your data center.

George Crump is founder of Storage Switzerland, an analyst firm focused on the virtualization and storage marketplaces. It provides strategic consulting and analysis to storage users, suppliers, and integrators. An industry veteran of more than 25 years, Crump has held engineering and sales positions at various IT industry manufacturers and integrators. Prior to Storage Switzerland, he was CTO at one of the nation's largest integrators.