Netapp and Singularization

Murat Songür
January 13, 2012

What is singularization? Can you explain the main logic of singularization?

In real, singularization is the entire technology for storing similar data as a single copy. We can talk about some different singularization technology and scenario.

In case of replica files, only one copy file is kept as an original by singularization technology; the simplest singularization method is establishing a logical link for each repeating file. Best sample for this is user scenario storing corporate introductory file having 2 MB size in personal area. Instead of using 20 MB area for 10 users in total, almost 90% saving can be achieved with 2 MB original file and links pointing that file.

In addition to this, tailor-made singularization scenarios can be added into the solutions. We keep different component separately for the files having very tiny differences between them.  When a user changes name and surname on the first slide of the introductory file; instead of creating a second copy; it only creates a difference component and stores the file via logical relation between the original file. Significant storage area saving can be succeeded by this scenario.

These two singularization technologies can be applied both at application level or file system. As we go below levels, we mention block-based singularization technologies.  Here, singularization is audited at repeating data block level in the data that is transferred to storage system directly. Logical block links are created for repeating blocks. There are only a few privileged manufacturers that can apply this method in their storage systems. It is also a technique which is widely preferred in backup systems as well.

What is the importance of singularization for the companies? Why is it needed?

Singularization enables more efficient investment utilization in storage areas in each scenario. First of all, singularization reduces storage area size demand so it decreases storage capacity investment.  Backup area reduces in parallel to this, backup time decreases.

Data that is transferred between storage systems and bandwidth to which data is transferred reduce. They are other behind-the-scene advantages which brings significant benefits to the companies.

Singularization is more important in the environments where virtualization is used. Operating system virtual discs used in virtual server or desktop systems are mainly composed of repeating components such as same operating system, same office package.  You can accommodate these virtual machines by using very low area if you use singularization. In addition, since storage systems consider this data as frequently read data, they can hold them in their cache memories.  Virtual machines can transfer to server or client system without tiring rotating components of disc systems. Backup of these systems becomes very easy. Actually, you backup one main virtual disc and differences of other virtual discs with this main copy. Therefore, requested capacity and backup time reduce significantly.

What is the difference of singularization from other backup technologies?

We consider singularization is a storage technology used in backup not as backup technology. Singularization technology brings a smart approach for instantaneous storage and backup issues. It can solve your lots of problems by constantly increasing sources. Main point here is that data you own and data blocks forming this data is managed in a smarter way.

Can you explain your solutions that NetApp provides?

Today, singularization comes in all NetApp storage systems as a standard feature without any extra charge. NetApp storage systems can perform block-based singularization independent from access protocol. Of course, this is an optional feature.  Since disc controllers form a great load to control the data, if not properly used, singularization may reduce performance. Therefore, NetApp runs this operation during the hours defined by the user and most probably when the system produces less work.

In addition, in order to backup NetApp storage systems, NetApp SnapProtect and SnapManager backup software running on NetApp storage systems use singularization technologies to obtain all backup benefits of singularization.