High Performance Storage System

Purge on migrate flag

HPSS now supports setting a flag on a bitfile which informs HPSS that the file should not be kept on disk after migration. This can be used for files which an application or user knows will not be needed again soon. When a file is written with this flag set, the file will immediately be purged off disk after it has been migrated to tape. When a file is staged with this flag enabled, it will behave normally, according to the purge policy. The purge on migrate flag can be set using the HPSS C API, the HPSS Python client, and scrub. See the Programmer's Reference for more information.

Low overhead read interface (LORI)

lori is a new tool which behaves similarly to quaid and shares many of the same options including organizing requests by tape, callouts upon read completion, filtering, and more. However, while quaid stages files up to the top level of the hierarchy, lori uses PIO to read data out of HPSS with minimal overhead and into the selected output directory. See the lori man page for more details.

In the future, lori will take advantage of upcoming changes to enable mass recall with read recovery.

Discover new control paths in the SCSI PVR during runtime

The SCSI PVR can now re-scan the SCSI Media Changer devices it has access to looking for new control paths to the tape library. This is triggered by doing a “Reinit” of the SCSI PVR server. Note that it will detect new devices, but will not remove old devices unless they are not in use by the SCSI PVR. Old devices which may have been in use but no longer exist will be marked as DOWN and will not be used by the SCSI PVR.

Optional ASAN-enabled versions of HPSS Servers

RPMs are now provided for ASAN-enabled versions of HPSS servers. These special binaries are otherwise the same as standard binaries, but have support compiled in to improve memory issue detection. These binaries can be used to spot certain kinds of memory use errors that generate crashes or memory leaks. The information from these binaries can reduce speed to resolution for these defect categories. One major advantage over prior tools is that an ASAN-enabled server may be modestly slower (20-30%) than the typical server, whereas prior tools could be multiple times slower.

Disable Direct IO

Historically, the HPSS Mover only performed direct IO to disk devices. The HPSS administrator may disbale direct IO for faster disk transfer performance. Use HPSS end-to-end data integrity to identify silent data corruption.

Simplify AWS tape storage gateway support

Deliver script to create hpss-compatible tape volumes in AWS tape storage gateweays.

< Home