Massive Data Transfer Between Supercomputers And Grid Clusters Using Scp Controlled By Fts

Data transfers between Supercomputing centers and Grid sites may be complicated since, in general, Supercomputing centers are not completely incorporated in the Grid environment. In particular, the Port d'Informacio Cientifica (PIG) had to transfer data of cosmological simulations from the Barcelona Supercomputing Center (BSC) to Grid storage resources in order to post-process those data at PIG. The data connection is limited to the Secure Shell (SSH) protocol using the scp command. Besides a limitof bandwidth, we had to deal with a lack of automation and reliability of the transfers. PIG managed to control those scp transfers through gLite's File Transfer Service (FTS). Therefore, the discs containing the data at the BSC are mounted via SSH to a local Storage Resource Management (SRM) server at PIG. Once established the access to the data through the SRM server, the data can be transfered to any other SRM server, including the storage system at PIG, using grid protocols. Although we cannot improve the bandwidth, we reach a level of automation and reliability comparable to transfers from the Tier-0 to Tier-1 sites of the Large Hadron Collider (LHC) experiments.