You can transferred data TO and FROM the HPC Cluster using one of two methods:

  • Fast Method

  • Slow Method ( possibly slow )

Fast Method:

When you need to transfer large amount of data to & from HPC (moving data outside of the HPC cluster) you will want to use an HPC ionode (I/O node).

HPC has 9 dedicated and exclusive ionodes each with it’s own GigE network interface going directly to the building router.

Note By using an I/O node you will have a full dedicated GigE Ethernet connection and your data transfer will run that much faster.

On HPC, you request an ionode with:

$ qrsh -q ionode

You then start your sftp, scp, rsync, wget, filezilla, etc, file transfer. You can connect to any host on campus or to any host outside the campus (the internet) using an HPC ionode. You cannot however connect to an HPC ionode from outside of HPC.

Warning HPC ionodes block all incoming network traffic for security reasons. From an ionode you can establish outgoing connections, but not the other way around.

If you like a GUI interface, you may be interested in using filezilla. Once you get an ionode, you start filezilla with:

$ filezilla

A window should now open on your work-station display (assuming you have X display configured correctly) and you then connect to the server you want to transfer files to/from and then begin your data transfer.

Slow Method:

The slow or possibly slower method of transferring data to & from HPC is to use hpc.oit.uci.edu for your file transfer.

The reason this method may be slower is that hpc.oit.uci.edu is the login entry point to the Cluster and HPC can easily have 150 on-line users at any given time, so you are competing with 150 other users network traffic for your data transfer.

You can use hpc.oit.uci.edu from inside the campus but you cannot use hpc.oit.uci.edu from outside the campus. The access table below illustrates what is possible.

Access Table:

The following table illustrates what is allowed or not depending on where the transfer initiates:

FROM TO Allowed?

UCI Campus

HPC

Yes

Outside Campus

HPC

NO

HPC

UCI Campus

Yes

HPC

Outside Campus

Yes

UCI Campus systems are systems on the UCI network (.uci.edu) and that have a Fully Qualified Domain Name FQDN.

Transfer Examples:

Here are some examples of transferring data to/from HPC (examples assumes that your desktop and/or other hosts have scp & rsync available).

From your Desktop on Campus to HPC:

To transfer data FROM your UCI Desktop system TO HPC:

To send my-file located on your desktop system to your HPC home directory (/data/users/$USER):

my-desktop> scp my-file  hpc.oit.uci.edu:/data/users/YourUCInetID

To send my-file located on your desktop system to your HPC work-space 2 located at /w2/$USERID:

my-desktop> scp my-file  hpc.oit.uci.edu:/w2/YourUCInetID

To send my-file located on your desktop system to your HPC BIO directory located at /bio/$USERID:

my-desktop> scp my-file  hpc.oit.uci.edu:/bio/YourUCInetID

To send an entire directory my-directory located on your desktop system to your HPC BIO directory locate at /bio/$USERID:

my-desktop> rsync -av my-directory  hpc.oit.uci.edu:/bio/YourUCInetID

To send an entire directory my-directory located on your desktop system to your HPC SOM directory locate at /som/$USERID:

my-desktop> rsync -av my-directory  hpc.oit.uci.edu:/som/YourUCInetID

To receive hpc-file located on your HPC BIO directory at /bio/$USERID/hpc-file to your desktop:

my-desktop> scp hpc.oit.uci.edu:/bio/YourUCInetID/hpc-file  .

To receive hpc-directory located on your HPC BIO directory at /bio/$USERID/hpc-directory to your desktop:

my-desktop> rsync -av hpc.oit.uci.edu:/bio/YourUCInetID/hpc-directory .

From HPC to your Desktop / Internet:

To transfer data FROM HPC TO your Desktop / Internet host:

(the following assumes that the receiving host, that being your Desktop system or the Internet host allows scp & rsync connections)

To send hpc-file located on HPC to a host located on campus or outside the campus:

hpc> scp hpc-file  other-host:/path

To send hpc-directory located on HPC to a host located on campus or outside the campus:

hpc> rsync -av hpc-directory  other-host:/path

To receive my-file located on your Desktop or internet host to your HPC BIO location at /bio/$USERID:

hpc> scp your-host.uci.edu:/path/my-file  /bio/YourUCInet

To receive my-directory located on your Desktop or internet host to your HPC BIO location at /bio/$USERID:

hpc> rsync -av your-host.uci.edu:/path/my-directory  /bio/YourUCInet

To send or receive using an HPC ionode (I/O node) with a dedicated GigE connection:

hpc> qrsh -q ionode

And now you can start your scp, rsync, etc file transfer to the other host.

HPC Public WWW data:

Please see: