HPSS Logo
HPSS Module for GridFTP: Administrator's Guide
Table of Contents

1. Introduction
2. Prerequisites
3. Downloading
4. Installing
5. Running
6. Configuration
7. Known Issues
8. Other

This is a standard Globus package that will build 3 GridFTP server DSI modules for HPSS 6.2. The first two modules are required if you wish to use the HPSS PIO API for high performance transfers from the HPSS movers. The third module, uses the traditional HPSS Read/Write Client API calls.

  1. hpss_control is the frontend DSI for a single or multistripe GridFTP configuration.
  2. hpss_data is the backend(s) DSI for a single or multistripe GridFTP configuration. Both frontend and backend GridFTP processes are required for striped operation. This is the preferred configuration.
  3. hpss_local provides HPSS access via a single GridFTP process. This gives poor performance compared to frontend/backend striped configuration, but is acceptable for light use.

Before proceeding, make sure to go through the following:

This package requires a working GridFTP server, from at least version 4.0.1 of the toolkit. A threaded build of the server is required. To build a threaded gridftp server, run

make globus_gridftp_server-thr

from the source installer.

The HPSS DSI modules for gridftp can be found here: globus_gridftp_server_hpss-0.2.tar.gz

From this package:

gpt-build [CONFIGOPTS_GPTMACRO="[options]"] \
globus_gridftp_server_hpss-x.x.tar.gz gcc32dbgpthr

From a CVS checkout, from this dir, run:

./bootstrap
gpt-build [CONFIGOPTS_GPTMACRO="[options]"] gcc32dbgpthr

Options are:

--with-hpssdir=
[hpssdir]
/opt/hpss is the default HPSS dir, use this to specify a different path.
--with-hpss
-auth-libs
enable linking of libhpssunixauth.so and libhpsskrb5auth.so
This is should be used if those libs exist in [hpssdir]/lib/.
--without-rpath If this option is set, [hpssdir] needs to be added to the environment library path (LIBPATH, LD_LIBRARY_PATH, etc, depending on platform) before the server is run.

Note: the flavor here (gcc32dbgpthr) must match the flavor of the server. It is usually gcc32dbgpthr on Linux and vendorcc32dbgpthr on other platforms. The command 'gpt-query globus_gridftp_server' can be run to verify which flavor is installed.

In order to use the module, the server must be started with the -dsi command line option:

globus-gridftp-server -dsi hpss_local -p 5000 -debug

will start a server listening on port 5000 using the hpss_local module.

hpss_control and hpss_data also require other options to specify frontend/backend:

globus-gridftp-server -dsi hpss_control -debug \
-p 5000 -r localhost:5001,localhost:5002

globus-gridftp-server -dsi hpss_data -p 5001 -dn -debug
globus-gridftp-server -dsi hpss_data -p 5002 -dn -debug

will start a frontend listening on port 5000 that will connect to 2 backends on ports 5001 and 5002, using the striped hpss modules.

Here are xinetd confs and daemon mode commands for running as a user. The user (globus in this case, but it can be any non-root user) needs read permission on the HPSS keytab, as well as ownership of /home/globus/.globus/*.

This is assuming both processes on the same machine. Change '-r localhost' to the appropriate hostname.

service gridftp_frontend (/etc/services as 2811)
{
instances = 250
socket_type = stream
wait = no
user = globus
env += GLOBUS_LOCATION=[GL]
env += GRIDMAP=/etc/grid-security/grid-mapfile
env += X509_USER_CERT=/home/globus/.globus/hostcert.pem
env += X509_USER_KEY=/home/globus/.globus/hostkey.pem
server = [GL]/sbin/globus-gridftp-server
server_args = -i -r localhost:2812 \
-auth-level 4 -dsi hpss_control
log_on_success += DURATION
log_on_failure += ATTEMPT, HOST
nice = 10
disable = no
}

service gridftp_backend1 (/etc/services as 2812)
{
instances = 250
socket_type = stream
wait = no
user = globus
env += GLOBUS_LOCATION=[GL]
env += GRIDMAP=/etc/grid-security/grid-mapfile
env += X509_USER_CERT=/home/globus/.globus/hostcert.pem
env += X509_USER_KEY=/home/globus/.globus/hostkey.pem
server = [GL]/sbin/globus-gridftp-server
server_args = -i -dn -auth-level 4 -dsi hpss_data
log_on_success += DURATION
log_on_failure += ATTEMPT, HOST
nice = 10
disable = no
}

or as a daemon:

export GLOBUS_LOCATION=[GL]
export GRIDMAP=/etc/grid-security/grid-mapfile
export X509_USER_CERT=/home/globus/.globus/hostcert.pem
export X509_USER_KEY=/home/globus/.globus/hostkey.pem
$GLOBUS_LOCATION/sbin/globus-gridftp-server -S -p 2811 \
-r localhost:2812 -auth-level 4 -dsi hpss_control
$GLOBUS_LOCATION/sbin/globus-gridftp-server -S -p 2812 \
-dn -auth-level 4 -dsi hpss_data

A configuration file is requried to specify security parameters. By default, this file is expected to be at /var/hpss/etc/gridftp-hpss.conf. If you need to point to a file in a different location, specify the path with the -dsi option to globus-gridftp-server: -dsi hpss_control:/path/to/config.file

The format of this file, and required parameters:

login_name "hpss_login@host.domain"
realm_name "kerb_realm.host.domain"
keytab "/var/hpss/etc/hpss.keytab"

The keytab must be readable by the user that GridFTP runs as (typically "globus").

The following are optional:

unixauth [0|1] #to enable HPSS UNIX authentication (kerberos is the default)
noauth [0|1] #disable additional authentication calls -- the process will run
#with whatever permissions running user has.

  • There are problems involving mismatched GridFTP block sizes in striped mode.
  • Writes to a multiple backend coniguration will likely fail.
  • unixauth currently just does the same thing as noauth.
  • Memory may become corrupted if a transfer is aborted, this shouldn't pose a problem except in
    -debug mode when the process does not fork for each new connection.
  • Transfers from the same GridFTP-HPSS server to itself may fail.

For a general understanding of the interface that these modules implement, see: http://viewcvs.globus.org/viewcvs.cgi/*checkout*/gridftp/server/src/dsi_bones/README.txt and

http://viewcvs.globus.org/viewcvs.cgi/*checkout*/gridftp/server/src/globus_gridftp_server.h (or $GLOBUS_LOCATION/include/globus_gridftp_server.h).

Additionally, there are notes and comments in each module's source.

< Home

What's New?
HPSS @ ISC17 - ISC17 is the 2017 International Supercomputing Conference for high performance computing, networking, storage and analysis. ISC17 will be in Frankfurt, Germany, from June 18th through 22nd.

IBM single client briefing details found HERE.

Schedule a briefing by sending an email to Fabienne.Wegener@de.ibm.com

2017 HUF - The 2017 HPSS User Forum will be hosted by the high energy accelerator research organization Kō Enerugī Kasoki Kenkyū Kikō, known as KEK, in Tsukuba, Japan from October 16th through October 20th, 2017 - Learn More

HPSS @ SC17 - SC17 is the 2017 international conference for high performance computing, networking, storage and analysis. SC17 will be in Denver, Colorado from November 14th through 17th.

Come visit the HPSS folks at the IBM booth and schedule an HPSS briefing at the IBM Executive Briefing Center - URL coming soon!

Swift On HPSS - Leverage OpenStack Swift to provide an object interface to data in HPSS. Directories of files and containers of objects can be accessed and shared across ALL interfaces with this OpenStack Swift Object Server implementation - Contact Us for more information, or Download Now.

Capacity Leader - ECMWF (European Center for Medium-Range Weather Forecasts) has a single HPSS namespace with 269 PB spanning 282 million files.

File-Count Leader - LLNL (Lawrence Livermore National Laboratory) has a single HPSS namespace with 64 PB spanning 966 million files.

RAIT - Oak Ridge National Laboratory cut redundant tape cost-estimates by 75% with 4+P HPSS RAIT (tape stripe with rotating parity) and enjoy large file tape transfers beyond 1 GB/s.
 
Home    |    About HPSS    |    Services    |    Contact us
Copyright 2015, IBM Corporation. All Rights Reserved.