High Performance Storage System

Incremental Scalability
Based on storage needs and deployment schedules, HPSS scales incrementally by adding computer, network and storage resources. A single HPSS namespace can scale from petabytes of data to exabytes of data, from millions of files to billions of files, and from a few file-creates per second to thousands of file-creates per second.
About HPSS   :    Publicly disclosed HPSS deployments   :   Table   or    Map

These are the publicly disclosed HPSS deployments that have accumulated a petabyte or more of data. Each entry represents a single instance of HPSS; the capacity numbers for a single HPSS namespace.

Sites - 2Q2021 Petabytes M Files Since
(ECMWF) European Centre for Medium-Range Weather Forecasts 567.48 399.91 2002
(UKMO) United Kingdom Met Office 451.11 876.34 2009
(SSC) Shared Services Canada 300.35 31.28 2017
(Meteo-France) Meteo France - French Weather and Climate 266.52 717.87 2015
(NOAA-RD) National Oceanic and Atmospheric Administration Research & Development 240.32 108.80 2002
(LBNL-User) Lawrence Berkeley National Laboratory - User 231.54 236.22 1998
(BNL) Brookhaven National Laboratory 212.14 224.72 1998
(MPCDF) Max Planck Computing and Data Facility 198.29 343.50 2011
(ORNL) Oak Ridge National Laboratory 148.71 500.10 1997
(DKRZ) Deutsches Klimarechenzentrum 135.51 24.24 2009
(CEA TERA) Commissariat a l`Energie Atomique - Tera Project 129.41 29.77 1999
(LANL-Secure) Los Alamos National Laboratory - Secure 101.51 1016.55 1997
(IU) Indiana University 101.20 247.58 1999
(LLNL-Secure) Lawrence Livermore National Laboratory - Secure 97.79 1399.09 1998
(ANL) Argonne National Laboratory 96.83 778.46 2008
(CEA TGCC) Commissariat a l`Energie Atomique - Très Grand Centre de calcul 95.72 30.48 2010
(IN2P3) Institut National de Physique Nucleaire et de Physique des Particules 94.79 97.60 1999
(NCAR) National Center for Atmospheric Research 74.49 253.29 2009
(DWD) Deutscher Wetterdienst 69.63 38.79 2011
(LLNL-Open) Lawrence Livermore National Laboratory - Open 65.04 1540.44 1997
(JAXA) Japan Aerospace Exploration Agency 60.44 260.44 2009
(SLAC) Stanford Linear Accelerator Center 54.11 15.03 1999
(Purdue) Purdue University 53.51 417.59 2011
(HLRS) High Performance Computing Center Stuttgart 46.46 11.88 2008
(KEK) High Energy Accelerator Research Organization 40.70 88.33 2001
(KIT) Karlsruhe Institute of Technology 33.61 320.00 2015
(LBNL-Backup) Lawrence Berkeley National Laboratory - Backup 32.98 20.32 1997
(SciNet) SciNet HPC Consortium 32.91 48.46 2011
(PNNL) Pacific Northwest National Laboratory 25.53 144.19 2010
(CNES) Centre National d'Etudes Spatiales 22.06 24.12 2015
(NOAA-C Asheville) National Oceanic and Atmospheric Administration - Comprehensive Large Array-data Stewardship System - Asheville 17.60 794.53 2014
(ASDC) NASA Atmospheric Science Data Center 15.79 53.91 2018
(NOAA-C Boulder) National Oceanic and Atmospheric Administration - Comprehensive Large Array-data Stewardship System - Boulder 15.56 706.94 2014
(LaRC) NASA Langley Research Center 14.47 52.03 1998
(LANL-Open) Los Alamos National Laboratory - Open 9.65 176.75 1997
(IITM) Indian Institute of Tropical Meteorology 9.39 71.06 2020
(NCEI) National Center for Environmental Information 7.56 113.51 1998
(SNL-Secure) Sandia National Laboratory - Secure 6.48 86.46 1996
(SNL-Open) Sandia National Laboratory - Open 5.89 141.42 1996
(NCMRWF) India's National Centre for Medium Range Weather Forecasting 5.62 158.53 2020
(UTAS) University of Tasmania Research Data Management System 0.23 11.52 2020

< Home

Come meet with us!
COVID-19 has disrupted the 2021 HPSS User Forum (HUF) and the Karlsruhe Institute of Technology (KIT) in Karlsruhe, Germany is no longer hosting the event. The 2021 HUF will be hosted online for six days spread across three weeks in October 2021 with no admission cost. This will be a great opportunity to hear from HPSS users, collaboration developers, testers, support folks and leadership (from IBM and DOE Labs) - Learn More. Please contact us if you are not a customer but would like to attend.

The 2021 international conference for high performance computing, networking, storage and analysis will be in St. Louis, MO from November 15th through 18th, 2021 - Learn More. As we do each year, we are scheduling and meeting with customers via IBM Single Client Briefings. Please contact your local IBM client executive or contact us to schedule a HPSS Single Client Briefing to meet with the IBM business and technical leaders of HPSS.

HPSS @ STS 2022
The 4th Annual Storage Technology Showcase is in the planning stage, but HPSS expects to support the event in March of 2022. Check out their web site - Learn More. We expect an update in early fall 2021.

HPSS @ MSST 2022
The 37th International Conference on Massive Storage Systems and Technology will be in Santa Clara, California in May of 2022 - Learn More. Please contact us if you would like to meet with the IBM business and technical leaders of HPSS at Santa Clara University.

What's New?
HPSS 9.2 Release - HPSS 9.2 was released on May 11th, 2021 and introduces eight new features and numerous minor updates.

Atos Press Release - Atos boosts Météo-France’s data storage capacity to over 1 exabyte in 2025 to improve numerical modeling and climate predictions. Want to read more?

HUF 2020 - The HPSS User Forum was hosted virtually at no cost in October 2020.

HPSS 9.1 Release - HPSS 9.1 was released on September 24th, 2020 and introduces a few new features.

HPSS 8.3 Release - HPSS 8.3 was released on March 31st, 2020 and introduces one new feature and many minor changes.

HPSS 8.2 Release - HPSS 8.2 was released on December 6th, 2019 and introduces a few new features.

New Globus DSI - Version 2.9 of the HPSS DSI is now available from the GitHub release page. It provides the capability to resume interrupted Globus transfers.

Lots Of Data - In November 2019 IBM/HPSS delivered a system to a customer in Canada and demonstrated a sustained tape ingest rate of 11,574 MB/sec (1 PB/day peak tape ingest) while simultaneously demonstrating a sustained tape recall rate of 8,832 MB/sec (791 TB/day peak tape recall). HPSS pushed four 13-frame IBM TS4500 tape libraries (scheduled to house over 500 PB of tape media) to 2,168 mounts/hour.

HPSS 8.1 Release - HPSS 8.1 was released on October 1st, 2019 and introduces a few new features.

July 2019 - Argonne Team Breaks Record for Globus Data Movement from the Summit supercomputer at Oak Ridge National Laboratory to HPSS tape.

Capacity Leader - ECMWF (European Center for Medium-Range Weather Forecasts) has a single HPSS namespace with over 567 PB spanning over 399 million files.

File-Count Leader - LLNL (Lawrence Livermore National Laboratory) has a single HPSS namespace with over 65 PB spanning 1.540 billion files.

Explosive data growth - HPSS Collaboration leadership from Lawrence Berkeley National Laboratory's National Energy Research Scientific Computing Center (NERSC) helped author the "NERSC Storage 2020" report, and NERSC trusts HPSS to meet their immediate and long term data storage challenges.

Older News - Want to read more?
  • LLNL Logo
  • LLNL Logo
  • NERSC Logo
  • ORNL Logo
  • SANDIA Logo
  • IBM Logo
  • ANL Logo
  • BNL Logo
  • CEA Logo
  • DKRZ Logo
  • ECMWF Logo
  • HLRS Logo
  • IN2P3 Logo
  • IU Logo
  • JAXA Logo
  • KEK Logo
  • NASA LaRC Logo
  • NASA ASDC Logo
  • UCAR Logo
  • NOAA NCDC Logo
  • NOAA Logo
  • NCEP Logo
  • PNNL Logo
  • SLAC Logo
  • MetOffice Logo
  • SciNet Logo
  • SSC Logo
  • UTAS Logo
Home    |    About HPSS    |    Services    |    Contact us
Copyright 1992 - 2021, HPSS Collaboration. All Rights Reserved.