Huawei’s up to 90:1 compression card

Huawei has announced a hardware compression card with an up to 90:1 reduction ratio. It sounds too good to be true. How has it done that?

The card is part of its all-flash, OceanProtect Backup Storage systems and uses “HZU algorithms for deep compression, delivering up to 90:1 data reduction ratio”, which is claimed to be 20 percent higher than the leading alternative. It also has, Huawei says, the industry’s only effective capacity guarantee, “eliminating the need for reduction ratio evaluation.”

The deduction ratio involves a combination of multi-layer, inline and variable-length deduplication with feature-based compression, and byte-level compaction applied to backup workloads. These typically have high data redundancy, especially ones such as daily full backups of virtual machines. A month’s worth of such backups can have an extremely high amount of duplicated data.

There is a family of HZU (HZBC) compression algorithms and they involve, according to Huawei’s 2022 Annual Report (page 67), a “fast nonlinear transformation and lightweight context prediction method.” Huawei says “This algorithm can generate results superior to those of the Lempel-Ziv (LZ) compression paradigm and boost the compression ratio by about 30 percent.”

We understand there is a 4-step data reduction pipeline:

• Preprocessing to clean data,
• Multi-layer, inline, variable-length deduplication,
• HZBC compression,
• Byte-level compaction.

Huawei has patented the dedupe and compression algorithms involved. The technique involves selecting compression algorithms most suited to a backup and the data types involved, and the actual achieved ratio will clearly depend upon the application type and backup policy.

The compression card is said to offload the backup system’s main CPU by up to 22 percent. As the OceanProtect systems use all-flash media, and not less expensive disk, then the more effective the data reduction ratio the better. Huawei is using QLC storage media with an adaptive SLC zone for hot data and overall faster data recovery. 

The existing OceanProtect systems are the X3000, X6000 and E8000. Two new ones have been announced; the X8100 and X9100. The earlier systems, with previous generations of compression technology, achieved an up to 72:1 deduction ratio. The new systems are up to 50 percent faster.

Prospective customers will need to evaluate the OceanProtect system with this compression card against their specific backup data sets. 

Bootnote

The OceanProtect X8100 has ransomware protection with 99.99 percent accuracy. This uses Huawei technology and not Index Engines’ CyberSense software.

The acronym HZBC could refer to Huawei Zip-Based Compression but this not an official Huawei definition.

A table listing OceanStor Protect product parameters is included and the new X8100 and X9100 data will be added when it's available.