Bandwidth Challenge 06: End-to-End Achievement

The National Center for Data Mining at UIC has won the HPC Bandwidth

Challenge at SC06 in Tampa, FL, sponsored by Qwest. Nine institutions participated in the

competition. NCDM won by sustaining a data transfer rate of 8Gb/s over a 10Gb/s link,

with a peak rate of 9.18Gb/s during the competition window. NCDM uses its own open source

software products, UDT and SECTOR, to transfer large datasets efficiently at high speeds

on optical networks.

This year the Bandwidth Challenge focused on a specific facet of networking:

End-to-End achievement. Competitors were asked to fully utilize one 10 Gig path,

end-to-end, disk-to-disk, from SC06 in Tampa back to their home institution, using

the actual production network back home. Participants were required to realize, demonstrate and

publish all the configuration, troubleshooting, tuning and policies used.

The SC06 show floor was connected with the major US research networks, specifically:

Abilene, ESnet, NLR PacketNet, NLR FrameNet, and HOPI. The US research networks provided

transit for the international networks with which they peer.

NCDM transferred Sloan Digital Sky Survey Data (SDSS) between the SC06 show floor in Tampa, Fl

and its lab in 4223 SEL at the University of Illinois in Chicago. It used SECTOR, the newly

developed distributed data space management system. SECTOR transparently manages the file

locating and data moving, while the NCDM developed UDT software is used for the actual

data transfer. The data transfer was disk to disk over one 10Gb/s shared routed link between

SC06 and UIC, via StarLight.

Bandwidth Challeng Competitors:


National Center for Data Mining (NCDM) at UIC, Northwestern Univ., Johns Hopkins Univ., "Transporting Sloan Digital Sky Survey Data Using


Honorable Mention

1) CalTech, CERN, Univ. of Florida, Univ. of Michigan, "High Speed Data Gathering, Distribution

and Analysis for Physics Discoveries at the Large Hadron Collider"

2) Indiana Univ., Pittsburgh SuperComputing Center, Oak Ridge National Laboratory, "All in a

Day's Work: Advancing Data Intensive Research with the Data Capacitor"

Additional teams:

1) Japanese Aerospace Exploration Agency

2) Pacific Northwest National Laboratory

3) Purdue University

4) Internet2, Univ. of Washington

5) Texas A&M Univ. , Univ. of Delaware

6) Univ. of Tokyo

For more information on the HPC Challenge see

Copyright 2016 The Board of Trustees
of the University of
Helping Women Faculty Advance
Funded by NSF