Close

Presentation

Student Technical Paper
Technical Paper
:
Seamless Utilization of Heterogenous XSede Resources to Accelerate Processing of a High Value Functional Neuroimaging Dataset
Event Type
Student Technical Paper
Technical Paper
Application Tags
Applications
HPC Applications
Technical Paper Tags
Technical Paper
Student Technical Paper
TimeWednesday, July 2510:30am - 10:45am
DescriptionWe describe the technical effort used to process magnetoencephalographic (MEG) recordings from the CamCAN Lifespan Neuroimaging Dataset , Cambridge (UK) Centre for Ageing and Neuroscience. The dataset includes demographic and screening measures, a high-resolution MRI scan of the brain, and MEG recordings from 619 neurologically normal individuals, ages 18-87, during eyes closed rest (560 sec), a sensorimotor task (540 sec), and passive listening/viewing (140 sec).

Primary processing utilized the referee consensus solver deployed on the Open Science Grid (OSG). The pool of computing elements used by the solver was boosted using XSede allocations on Bridges (Pittsburgh Supercomputing Center) and Comet (San Diego Supercomputing Center). Computing elements were added directly to the OSG’s HTcondor pool. This required no changes to the solver and only minor changes to the workflow management software which was fully operational in two days.

HTcondor’s glidein mechanism is used to add the booster computing elements to the HTcondor pool. A script was deployed on each machine which queues Glidein’s only when there are idle computing elements. Each Glidein’s lifetime is limited to two hours, insuring that these jobs do not significantly delay other users of these important supercomputing resources. This opportunistic use of Bridges and Comet delivered more than a 5-fold increase in the number of core hours available for this effort. Processing of the resting recordings is completed. That utilized 15,000,000 core hours and required 7 months. Processing of the task recordings is ongoing with 253 completed to date, 8Dec2017.