Microsoft PowerPoint – Chapter 6 – Programming Using the Message Passing Interface
Introduction to
Parallel Computing
Irene Moulitsas
Programming using the
Message-Passing Paradigm
MPI Background
MPI : Message Passing Interface
Began in Supercomputing ’92
Vendors
IBM, Intel, Cray
Library writers
PVM
Application specialists
National Laboratories, Universities
Why MPI ?
One of the oldest libraries
Wide-spread adoption. Portable.
Minimal requirements on the underlying
hardware
Explicit parallelization
Intellectually demanding
Achieves high performance
Scales to large number of processors
MPI Programming Structure
Asynchronous
Hard to reason
Non-deterministic behavior
Loosely synchronous
Synchronize to perform interactions
Easier to reason
SPMD
Single Program Multiple Data
MPI Features
Communicator Information
Point to Point communication
Collective Communication
Topology Support
Error Handling
Six Golden MPI Functions
MPI is 125 functions
MPI has 6 most used functions
MPI Functions: Initialization
Must be called by all processes
MPI_SUCCESS
“mpi.h”
MPI Functions: Communicator
MPI_Comm
MPI_COMM_WORLD
Hello World !
Hello World ! (correct)
MPI Functions: Send, Recv
source
MPI_ANY_SOURCE
MPI_Status
MPI_SOURCE
MPI_TAG
MPI_ERROR
MPI Functions: Datatypes
Send/Receive Examples
Blocking Non-Buffered
Communication
Send/Receive Examples
Blocking Buffered Communication
Send/Receive Examples
MPI Functions: SendRecv
MPI Functions: ISend, IRecv
Non-blocking
MPI_Request
MPI Functions: Test, Wait
MPI_Test tests if operation finished.
MPI_Wait blocks until operation is finished.
Non-Blocking Non-Buffered
Communication
Example
Example
Example
MPI Functions: Synchronization
Collective Communications
One-to-All Broadcast
All-to-One Reduction
All-to-All Broadcast & Reduction
All-Reduce & Prefix-Sum
Scatter and Gather
All-to-All Personalized
MPI Functions: Broadcast
MPI Functions: Scatter & Gather
MPI Functions: All Gather
MPI Functions: All-to-All Personalized
MPI Functions: Reduction
MPI Functions: Operations
MPI Functions: All-reduce
Same as MPI_Reduce, but all processes receive
the result of MPI_Op operation.
MPI Functions: Prefix Scan
MPI Names
MPI Functions: Topology
Performance Evaluation
Elapsed (wall-clock) time
Matrix/Vector Multiply
Reviews
There are no reviews yet.