[Next] [Previous] [Top]

An Overview of PVM

Parallel Virtual Machine

PVM

• PVM is a software package that permits a heterogeneous collection of serial, parallel and vector computers which are connected to a network to appear as one large computing resource.
  • PVM combines the power of a number of computers

  • PVM may be used to create a virtual computer from multiple supercomputers enabling the solution of previously unsolvable "grand challenge" problems

  • PVM may be used as an educational tool to create a parallel computer from low cost workstations

Key features

• Easily obtainable public domain package

• Easy to install

• Easy to configure

• Many different virtual machines may co-exist on the same hardware

• Program development using a widely adopted message passing library

• Supports C and Fortran

• Installation only requires a few Mb of disk space

• Simple migration path to MPI

PVM history

• PVM project began in 1989 at Oak Ridge National Laboratory - prototype PVM 1.0

• Version 2.0 was written at the University of Tennessee and released in March 1991

• PVM 2.1 -> 2.4 evolved as a response to user feedback

• Version 3 was completed in February 1993

• The current version is 3.3 and subsequent releases are expected

Introduction

2 major developments have assisted parallel processing acceptance

• massively parallel processors (MPPs)-turning now to parallel processing to get even more computational power.

• the widespread use of distributed computing - process whereby a set of computers connected by a network are used collectively to solve a single large problem

• common to both is the notion of message-passing - in all parallel processing data must be exchanged between co-operating tasks.

PVM

• Parallel Virtual Machine (PVM) uses the message passing model to allow programmers to exploit distributed computing across a wide variety of computer types, including MPPs.

• Heterogeneous Network Computing When a programmer wishes to exploit a collection of networked computers, they may have to contend with several different types of heterogeneity:
  • architecture

  • data format

  • computational speed

  • machine load and

  • network load.

Distributed computing

Advantages

• Using existing hardware, keeps costs low

• Performance can be optimised by assigning each individual task to the most appropriate architecture

• Exploitation of the heterogeneous nature of a computation ie provides access to different types of processors for those parts of an application that can only run on a certain platform

• Virtual computer resources can grow in stages and take advantage of the latest computational and network technologies

• Program development can be enhanced by using a familiar environment ie editors,
  compilers, debuggers that are available on individual machines

• Individual computers and workstations are usually stable and substantial expertise in their use should be available

• User-level or program-level fault tolerance can be implemented with little effort either in the application or in the underlying operating system

• Facilitates collaborative work

PVM overview

• PVM provides a unified framework within which parallel programs can be developed efficiently using existing hardware.

• PVM transparently handles all message routing, data conversion, and task scheduling across a network of incompatible computer architectures.

• The programming interface is straightforward allowing simple program structures to be implemented in an intuitive manner.

• The user writes his application as a collection of co-operating tasks which access PVM resources through a library of standard interface routines.

Underlying principles

• User-configured host pool:
  • tasks execute on a set of machines selected by the user for a given run of the PVM program

  • the host pool may be altered by adding/deleting machines during operation

• Translucent access to hardware:
  • programs may view the hardware environment as an attributeless collection of virtual processing elements or

  • may choose to exploit the capabilities of specific machines by placing certain tasks on certain machines.

• Process-based computation:
  • A task is the unit of parallelism in PVM - it is an independent sequential thread of control that alternates between communication and computation.

• Explicit message-passing model:
  • Tasks co-operate by explicitly sending and receiving messages to and from one another.

  • Message size is limited only by the amount of available memory.

• Heterogeneity support:
  • PVM supports heterogeneity in terms of machines, networks and applications.

  • Messages can also contain one or more data types to be exchanged between machines having different data representations.

• Multiprocessor support:
  • PVM uses the native message-passing facilities on multi-processors to take advantage of the underlying hardware.

Terms

• Functional Parallelism - an application can be parallelised along its functions so that each task performs a different function eg input, problem setup, solution, output and display.

• Data Parallelism - all the tasks are the same but each one only knows and solves a part of the data. This is also known as SPMD(single-program multiple data).

PVM programming paradigm

• A user writes one or more sequential programs in C, C++ or Fortran 77 containing embedded calls to the PVM library.

• Each program corresponds to a task making up the application.

• The programs are compiled for each architecture in the host pool and the resulting object files are placed at a location accessible from machines in the host pool

• An application is executed when the user starts one copy of the application ie the `master' or `initiating' task, by hand from a machine within the host pool.

• The `master` process subsequently starts other PVM tasks, eventually there are a number of active tasks to compute and communicate to solve the problem.

• Tasks may interact through explicit message-passing using the system assigned opaque TID to identify each other.

• Finally once the tasks are finished they and the `master' task disassociate themselves from PVM by exiting from the PVM system.

Parallel models

• The "crowd" computing model - consists of a collection of closely related processes, typically executing the same code, performing calculations on different portions of the workload and usually involving the periodic exchange of intermediate results eg
  • master-slave (or host-mode) model has a separate `control' program ie the master which is responsible for spawning, initialization, collection and display of results. The slave programs perform the actual computations on the workload allocated either by the master or by themselves

  • node-to-node model where multiple instances of a single program execute, with one process (typically the one initiated manually) taking over the noncomputational responsibilities as well as contributing to the calculation itself.

• A `tree' computation model - processes are spawned (usually dynamically as the computation grows) in a tree-like manner establishing a tree-like parent-child relationship eg branch-and-bound algorithms, alpha-beta search, and recursive `divide-and-conquer' algorithms.

• A `hybrid' or combination model- possesses an arbitrary spawning structure in that at any point in the application execution the process relationship may resemble an arbitrary and changing graph.

Workload allocation

Data decomposition

• assumes that the overall problem involves applying computational operations or transformations on one or more data structures and that these data structures may be divided and operated upon.

• Eg if vectors of N elements and P processors are available N/P elements are assigned to each process.

• Partitioning can be done either `statically' where each processor knows at the beginning its workload or `dynamically' where a control or master process assigns portions of the workload to processor as and when they become free.

Function decomposition

• divides the work based on different operations or functions.

• Eg the three stages of a typical program execution - input, processing and output or results could take the form of three separate and distinct programs each one being dedicated to one of the stages. Parallelism is obtained by concurrently executing the three programs and establishing a `pipeline' between them.

• A more realistic form of function decomposition is to partition the workload according to functions within the computational phase eg the different sub-algorithms which typically exist in application computations.

Heterogeneity

Supported at 3 levels

• Application: tasks may be placed on processors to which they are most suited

• Machine: Computers with different data formats, different architectures(serial or parallel), and different operating systems

• Network: A virtual machine may span different types of networks such as FDDI, ethernet, ATM

Portable

• PVM is available across a wide range of Unix based computers

• PVM version 3 may be ported to non-Unix machines

• PVM will soon be ported to VMS

• Versions of PVM are available on machines such as CCM-5, CS-2 and iPSC/860, Cray T3D