Content Aware De-Duplication

Network De-Duplication

Network de-duplication in all WAN optimization solutions works on the same principle: content is stored on disk at both ends of the network and when similar content is seen again, messages are sent to the peer device to replay the content locally rather than re-transmitting the data across the WAN. This results in significant bandwidth savings. The performance of different solutions depends on how well they organize the data on disk and how fast they are able to locate and recreate the previously seen content. Performance is also affected not only by how large the disk is but how large the usable disk is.

aCelera Content Aware De-Duplication

The aCelera de-duplication technology goes much further than any other vendor. With Content Aware De-duplication as a data stream is processed by the aCelera instead of just segmenting and building a history of the stream, the aCelera is able to distinguish the protocol used to transfer the content. The aCelera is able to strip off not only the TCP/IP encapsulation but also the application protocol encapsulation e.g. HTTP headers. The result is that the history built on disk is just the content and is very clean.

This has two benefits. Firstly, the disk is not being filled up with one time protocol encapsulation that will never be matched in the future, this means that the disk is being used more effectively resulting in better long term performance. Secondly, as the content is being written very cleanly on disk it gets better matching, in fact a single token can represent the entire content as opposed to the match being interrupted by the non-matching metadata. The result is the better data reduction and faster data transfers and as the data is cleanly written on disk, matching is better when the same content is transferred using different protocols.

For WAN optimization solutions one of the lesser known challenges is that of long-term performance degradation. Without Content Aware De-duplication, the history disk fills up quickly with non-important protocol metadata resulting a in a performance drop off.