The vesta parallel file system

Peter F. Corbett, Dror G. Feitelson

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

7 Scopus citations

Abstract

The Vesta parallel file system is designed to provide parallel file access to application programs running on multicomputers with parallel I/O subsystems. Vesta uses a new abstraction of files: a file is not a sequence of bytes, but rather it can be partitioned into multiple disjoint sequences that are accessed in parallel. The partitioning - which can also be changed dynamically - reduces the need for synchronization and coordination during the access. Some control over the layout of data is also provided, so the layout can be matched with the anticipated access patterns. The system is fully implemented, and forms the basis for the AIX Parallel I/O File System on the IBM SP2. The implementation does not compromise scalability or parallelism. In fact, all data accesses are done directly to the I/O node that contains the requested data, without any indirection or access to shared metadata. Disk mapping and caching functions are confined to each I/O node, so there is no need to keep data coherent across nodes. Performance measurements show good scalability with increased resources. Moreover, different access patterns are shown to achieve similar performance.

Original languageAmerican English
Title of host publicationHigh Performance Mass Storage and Parallel I/O
Subtitle of host publicationTechnologies and Applications
PublisherWiley-IEEE Press
Pages285-308
Number of pages24
ISBN (Electronic)9780470544839
ISBN (Print)0471208094, 9780471208099
DOIs
StatePublished - 1 Jan 2001

Bibliographical note

Publisher Copyright:
© 2002 by the Institute of Electrical and Electronics Engineers, Inc. All rights reserved.

Keywords

  • Computers
  • Distributed databases
  • File servers
  • File systems
  • Parallel processing
  • Scalability
  • Semantics

Fingerprint

Dive into the research topics of 'The vesta parallel file system'. Together they form a unique fingerprint.

Cite this