[SOLVED] Microsoft PowerPoint Chapter 6 Programming Using the Message Passing Interface

Microsoft PowerPoint Chapter 6 Programming Using the Message Passing Interface

Introduction to
Parallel Computing

Irene Moulitsas
Programming using the
Message-Passing Paradigm

MPI Background
MPI : Message Passing Interface
Began in Supercomputing 92

Vendors
IBM, Intel, Cray

Library writers
PVM

Application specialists
National Laboratories, Universities

Why MPI ?
One of the oldest libraries
Wide-spread adoption. Portable.
Minimal requirements on the underlying
hardware
Explicit parallelization

Intellectually demanding
Achieves high performance
Scales to large number of processors

MPI Programming Structure
Asynchronous

Hard to reason
Non-deterministic behavior

Loosely synchronous
Synchronize to perform interactions
Easier to reason

SPMD
Single Program Multiple Data

MPI Features
Communicator Information
Point to Point communication
Collective Communication
Topology Support
Error Handling

Six Golden MPI Functions
MPI is 125 functions
MPI has 6 most used functions

MPI Functions: Initialization

Must be called by all processes
MPI_SUCCESS

mpi.h

MPI Functions: Communicator

MPI_Comm
MPI_COMM_WORLD

Hello World !

Hello World ! (correct)

MPI Functions: Send, Recv

source
MPI_ANY_SOURCE

MPI_Status
MPI_SOURCE
MPI_TAG
MPI_ERROR

MPI Functions: Datatypes

Send/Receive Examples

Blocking Non-Buffered
Communication

Send/Receive Examples

Blocking Buffered Communication

Send/Receive Examples

MPI Functions: SendRecv

MPI Functions: ISend, IRecv

Non-blocking
MPI_Request

MPI Functions: Test, Wait

MPI_Test tests if operation finished.
MPI_Wait blocks until operation is finished.

Non-Blocking Non-Buffered
Communication

Example

Example

Example

MPI Functions: Synchronization

Collective Communications
One-to-All Broadcast
All-to-One Reduction
All-to-All Broadcast & Reduction
All-Reduce & Prefix-Sum
Scatter and Gather
All-to-All Personalized

MPI Functions: Broadcast

MPI Functions: Scatter & Gather

MPI Functions: All Gather

MPI Functions: All-to-All Personalized

MPI Functions: Reduction

MPI Functions: Operations

MPI Functions: All-reduce
Same as MPI_Reduce, but all processes receive
the result of MPI_Op operation.

MPI Functions: Prefix Scan

MPI Names

MPI Functions: Topology

Performance Evaluation
Elapsed (wall-clock) time

Matrix/Vector Multiply