SCHEDULE: NOV 12-18, 2011

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

You can also create your personal schedule on the SC11 app (Boopsie) on your smartphone. Simply select a session you want to attend and "add" it to your plan. Continue in this manner until you have created your own personal schedule. All your events will appear under "My Event Planner" on your smartphone.

On the Duality of Data-intensive File System Design: Reconciling HDFS and PVFS

SESSION: MapReduce and Network QoS

EVENT TYPE: Paper

TIME: 2:00PM - 2:30PM

AUTHOR(S):Wittawat Tantisiriroj, Swapnil Patil, Garth Gibson, Seung Son, Samuel Lang, Robert Ross

ROOM:TCC 303

ABSTRACT:
Data-intensive applications fall into two computing styles: Internet services (cloud computing) or high-performance computing (HPC). In both categories, the underlying file system is a key component in scalable application performance. In this paper, we explore the similarities and differences between PVFS, a parallel file system used in HPC at large scale, and HDFS, the primary storage system used in cloud computing with Hadoop. We integrate PVFS into Hadoop and compare its performance to HDFS using a set of data-intensive computing benchmarks. We study how HDFS-specific optimizations can be matched using PVFS and how consistency, durability, and persistence tradeoffs made by these file systems affect application performance. We show how to embed multiple replicas into a PVFS file, including a mapping with a complete copy local to the writing client, to emulate HDFS's file layout policies. We also highlight implementation issues with HDFS's dependence on disk bandwidth and benefits from pipelined replication.

Chair/Author Details:

Wittawat Tantisiriroj - Carnegie Mellon University

Swapnil Patil - Carnegie Mellon University

Garth Gibson - Carnegie Mellon University

Seung Son - Argonne National Laboratory

Samuel Lang - Argonne National Laboratory

Robert Ross - Argonne National Laboratory

  Click here to download .ics calendar file

  Click here to download .vcs calendar file

  Click here to add event to your Google Calendar

The full paper can be found in the ACM Digital Library