High Performance Storage System

HPSS Logo
Incremental Scalability
Based on storage needs and deployment schedules, HPSS scales incrementally by adding computer, network and storage resources. A single HPSS namespace can scale from petabytes of data to exabytes of data, from millions of files to billions of files, and from a few file-creates per second to thousands of file-creates per second.
About HPSS   :    HPSS Big Picture - metadata storage



RHEL Platform   |   Metadata Storage   |   Disk and Tape Storage   |   User Interfaces
HPSS File Operations Per Second

Every HPSS file operation requires one or more Db2 database transactions (e.g. an HPSS file create with data requires four Db2 metadata transactions). HPSS services help with planning, provisioning, and configuring Db2 to meet your transaction requirements.

HPSS Backed by Db2

Metadata describing the files in HPSS are managed by Db2 relational databases. Each relational database has tables and indexes, transaction log files, archived transaction log files, and database backup files. All Db2 transaction log files and the Db2 database backup files are often called the Db2 recovery objects.

Mandatory HPSS Redundancy

All Db2 database components are protected by RAID storage. The Db2 transaction log files are further protected by configuring Db2 log mirroring to use a separate RAID-device. Additionally, a second copy of Db2 log archives are stored on a separate RAID-device. Typically, only one copy of the Db2 database backup files are kept on RAID storage. All Db2 recovery objects are further protected using your existing site disaster recovery plan and stored on remote-disk or remote-tape. The site backup serves as the second copy of the Db2 database backup files.

Optional HPSS Metadata Redundancy

When Db2 metadata must survive a complete chassis failure, a second local server is provisioned with the RAID-device for the second copy of Db2 recovery objects discussed earlier. The separate chassis can be mounted in a separate rack to protect the HPSS metadata from a rack failure.

Db2 log shipping is used to protect HPSS metadata from a wider range of failures, including a site failure, and to help reduce downtime caused by these failures. Db2 log shipping is used to replicate HPSS Db2 metadata spanning geography.

HPSS Db2 Capacity

Here is an example of Db2 storage for 1 billion HPSS files in a single chassis metadata configuration:
  1. High IOPS storage (at lease two RAID devices)
    1. 1 TB RAID: Db2 transaction log files and half of the Db2 tables and indexes
    2. 1 TB RAID: Db2 transaction log mirror files and half of the Db2 tables and indexes
  2. High bandwidth storage (at lease two RAID devices)
    1. 4 TB RAID: Db2 archived transaction log files and half of the Db2 database backup files
    2. 4 TB RAID: Db2 archived transaction log morror files and half of the Db2 database backup files
  3. Integrating Db2 recovery files into site backup
    1. Db2 recovery files are typically sent to tape using backup software (e.g. Bacula)
    2. 8 TB stores about seven days of Db2 recovery files
SSD vs. HDD for metadata storage

The solid state disks (SSD) are cost-effective for the high IOPS (Db2 tables, indexes and logs), high bandwidth (for faster backup/restore times), and low bit-error-rate (high integrity) requirements of HPSS Db2. The spinning hard disk drive (HDD) has been reduced to a capacity-play for semi-active data, but may still be economical for larger Db2 recovery object requirements.

< Home

Come meet with us!
HPSS @ MSST 2018
The 34th International Conference on Massive Storage Systems and Technology will be in Santa Clara, California from May 14th through May 18th, 2018 - Learn More. Please contact us if you would like to meet with the IBM business and technical leaders of HPSS at Santa Clara University.

HPSS @ ISC18
The 2018 international conference for high performance computing, networking, and storage will be in Frankfurt, Germany from June 24th through 28th, 2018 - Learn More. Come visit the HPSS folks at the IBM booth and contact us if you would like to meet with the IBM business and technical leaders of HPSS in Frankfurt.

2018 HUF
The 2018 HPSS User Forum will be hosted by the UK Met Office in Exeter, United Kingdom from October 15th through October 18th, 2018. Please contact us if you would like to meet with the HPSS Collaboration leaders, HPSS Collaboration developers, architects, system engineers, support staff, project managers, and many of the HPSS customers.

The 2018 HUF information and registration web site is coming soon.

HPSS @ SC18
The 2018 international conference for high performance computing, networking, storage and analysis will be in Dallas, Texas from November 11th through 15th, 2018 - Learn More. Come visit the HPSS folks at the IBM booth and contact us if you would like to meet with the IBM business and technical leaders of HPSS in Dallas.
What's New?
HPSS Meltdown and Spectre Vulnerability Statement - Learn More.

HPSS Vendor Partnership Grows - HPSS begins Quantum Scalar i6000 tape library testing in 2018. Other HPSS tape vendor partners include IBM, Oracle, and Spectra Logic.

Swift On HPSS - Leverage OpenStack Swift to provide an object interface to data in HPSS. Directories of files and containers of objects can be accessed and shared across ALL interfaces with this OpenStack Swift Object Server implementation - Contact Us for more information, or Download Now.

Capacity Leader - ECMWF (European Center for Medium-Range Weather Forecasts) has a single HPSS namespace with 360 PB spanning 328 million files.

File-Count Leader - LLNL (Lawrence Livermore National Laboratory) has a single HPSS namespace with 41 PB spanning 1.084 billion files.

RAIT - Oak Ridge National Laboratory cut redundant tape cost-estimates by 75% with 4+P HPSS RAIT (tape stripe with rotating parity) and enjoy large file tape transfers beyond 1 GB/s.
Home    |    About HPSS    |    Services    |    Contact us
Copyright 2017, IBM Corporation. All Rights Reserved.