Syllabus
Parallel Algorithms and Processing
Spring2003

William D. Shoaff

Syllabus Information

• Course Overview
  • Goals
  • Course Structure
  • Catalog Description
  
  
• Course Materials
  • Textbook:
  • Additional References
  • Lecture Notes
  • Sample Programs
  • Interactive Quizzes
  
  
• Class Information
  • Instructor
  • Office Hours
  • Class Schedule
  • Class URL
  • Class Forum
  
  
• External Learning Material
• Bibliography
• Assignments and Grading
  • Beowulf Programming Projects
  • Programming Presentations
  • Joint Work
  • Grading
  
  
• Important dates
• Class Evaluation

Course Overview

Students will learn SPMD (single program multiple data) programming using the MPI (Message Passing Interface) on Florida Tech's 48-node Beowulf cluster. Design, coding, performance analysis, debugging and other aspects of parallel algorithm development will be covered.

Goals

Students will be expected to:

• understand Flynn's taxonomy for parallel architectures and other hardware issues;
• understand various parallel programming models;
• understand performance measurements for parallel programs;
• analyze parallel algorithms to predict performance;
• design parallel algorithms and implement parallel codes;
• implement parallel algorithms using MPI.

Course Structure

Part I Basic Parallel Programming

Students will learn to write, compile, execute, and collect statistics on runs of elementary parallel programs. MPI will be used to write code. Examples will be given in C, but FORTRAN 77 examples are also available. The Portland Group compilers are available and are optimized for compiling parallel programs on the Beowulf computer. GNU compiler are also available.

Various MPI interprocessor communication functions (protocols) will be studied. Some examples are MPI_Send, MPI_Recv, MPI_Reduce, MPI_Broadcast, MPI_Sum, MPI_Gather, MPI_Scatter, MPI_Allgather, and MPI_Allreduce. Sample codes to be studied include:

• Parallel computation of a dot (inner) product of two vectors [Pacheco(1997)], Section 5.5.
• The trapezoidal rule for numerical integration [Pacheco(1997)], (Section 4.2)
• Parallel matrix-vector multiplication $Ax$ [Pacheco(1997)], (Section 5.8)
• Fox's algorithm for the parallel product of two matrices $AB$ [Pacheco(1997)], Section 7.8.

Part II Analysis of Parallel Algorithms

A general discussion of parallel computers architecture. Performance Analysis: communication time, computation time, speed-up measures, scalability, load-Balancing, and Amdahl's Law. Debugging techniques and debuggers.

Reference material on these subjects can be found in [Pacheco(1997)] Chapters 2, 9, and 11 and [Wilkinson and Allen(1999)] chapters 1 and 2.

Part III Serious Applications

Some implementations of significant parallel codes for specific purposes in (i) numerical linear algebra (Gaussian Elimination, QR factorizations) (ii) sorting algorithms (iii) image processing algorithms (iv) searching and optimization.

Reference material on these subjects can be found in chapters 10, and 14 of [Pacheco(1997)] and chapter 9, 10, 11, and 12 of [Wilkinson and Allen(1999)].

Catalog Description

CSE 4082 Parallel and Real-Time Algorithms (3 credits) The course introduces models of computation for parallel systems. Algorithms for parallel sorting, searching, and matrix operations, and numerical computations are presented and analyzed. Real-time constraints and real-time scheduling algorithms are also discussed. (Prerequisite: CSE 4081.)

CSE 5250 Parallel Processing (3 credits) Architectures for parallel computers and parallel algorithms for computational problems are investigated. Performance evaluation metrics for the performance of parallel processing are discussed. (Prerequisites: CSE 5100 and 5220.)

Course Materials

Textbook:

Pacheco(1997)

Peter S. Pacheco.
Parallel Programming with MPI.
Morgan Kaufmann, San Francisco, CA, 1997.
ISBN: 1-55860-339-5.

Additional References

Snir et al.(2000)

M. Snir, S. Otto, S. Huss-Lederman, D. Walker, and J. Dongarra.
MPI-The Complete Reference, volume 1.
MIT Press, 2000.

Wilkinson and Allen(1999)

B. Wilkinson and M. Allen.
Parallel Programming Techniques and Apllications Using Networked Workstations and Parallel Computers.
Prentice-Hall, 1999.

Lecture Notes

A collection of class lecture notes is available.

Sample Programs

An annotated collection of programs has been developed for the class. Pacheco's examples in C and FORTRAN, are also available.

Interactive Quizzes

A collection of interactive quizzes. (will be) available for you to assess your understanding of material presented in the class.

Class Information

Instructor

William D. Shoaff, 252 Olin Engineering Building (407) 674-8066, wds@cs.fit.edu

Office Hours

See http://www.cs.fit.edu/wds/classes/office.html or by electronic mail wds@cs.fit.edu, or by appointment.

Class Schedule

The class meet each Tuesday and Thursday from 3:30 p.m. to 4:45 p.m. in Q-18.

Class URL

Online information for this class can be found at the URL

http://www.cs.fit.edu/wds/classes/parallel

Class Forum

A mailing list for questions, answers, and discussions is available. To join the list, start at

http://lyris.fit.edu:81/cgi-bin/lyris.pl

and follow the link from Computer Sciences to cse4082.

External Learning Material

Bibliography

natexlab url

Pacheco(1997)

Peter S. Pacheco.
Parallel Programming with MPI.
Morgan Kaufmann, San Francisco, CA, 1997.
ISBN: 1-55860-339-5.

Snir et al.(2000)Snir, Otto, Huss-Lederman, Walker, and Dongarra

M. Snir, S. Otto, S. Huss-Lederman, D. Walker, and J. Dongarra.
MPI-The Complete Reference, volume 1.
MIT Press, 2000.

Wilkinson and Allen(1999)

B. Wilkinson and M. Allen.
Parallel Programming Techniques and Apllications Using Networked Workstations and Parallel Computers.
Prentice-Hall, 1999.

Assignments and Grading

Beowulf Programming Projects

Programming assignments are to be turned in using the department's submit server.

Programming Presentations

As part of the course, students will periodically and randomly demonstrate programs that they have written.

Joint Work

Projects are provided to exercise your skills so you can master parallel programming applications. These exercises can often best be accomplished by working with others. You may pair with one other person to develop the projects, but the pair must following the extreme programming paradigm for how two people develop programs as a pair.

Interactive quizzes are provided for topics in the course. Although they are graded automatically, no scores are recorded. Use these to assess your knowledge of parallel programming. Absolutely no collaboration is allowed on the final exam.

Plagiarism and other anti-intellectual behavior will not be tolerated. Refer to the department code of honor and the university policy on academic dishonesty.

Grading

Grades are based on scores obtained on programming projects, presentations, and the final examination.

Four programming project and presentations: (400 points)

Midterm exam (100 points)

Final exam (100 points)

Letter grades will be assigned based on percentage of total points.

A=90 - 100%     B=80 - 89%     C=70 - 79% D= 60 - 69% F=0 - 59%

Important dates

• University administrative events
  • January 8, 2003: classes begin
  • January 14, 2003: Last day to register and add a class
  • January 21, 2003: Last day to drop a class without receiving a 'W'
  • February 19-23,2003: Homecoming
  • February 28, 2003: Last day to drop a class with W
  • April 26, 2003: Last day of classes
  • February 3, 2003: Registration for Summer 2003 begins
  • March 31, 2003: Registration for Fall 2003 begins
  • Aprir 28 - May 3, 2003: Final exams
  • May 3, 2003: Commencement
• Class assignments and exams
  • Project1: Due before January 24, 2003
  • Project 2: Due before February 12, 2003
  • Project 3: Due April 17, 2003
  • Project 4: Due before Z
  • Final Exam: Friday, May 2, 3:30 p.m.-5:30 p.m.
• Holidays
  • March 3-7: Spring break
  • Other Holidays (not officially observed by Florida Tech):
    • Monday, January 20, 2003: Martin Luther King Day
    • Saturday, February 1, 2003: Chinese New Year (Gui-Wei)
    • Sunday, February 2, 2003: Groundhog Day
    • Friday, February 14, 2003: Valentine's Day
    • Monday, February 17, 2003: President's Day
    • Wednesday, March 5, 2003: Islamic New Year 1424
    • Wednesday, March 5, 2003: Ash Wednesday
    • Monday, March 17, 2003: St. Patrick's Day
    • Thursday, March 20, 2003: Vernal Equinox 8:00pm (EST)
    • Tuesday, April 1, 2003: April Fools' Day
    • Sunday, April 6, 2003: Daylight Savings Time Begins 2:00am (EST)
    • Thursday, April 17, 2003: Passover
    • Friday, April 18, 2003: Good Friday
    • Sunday, April 20, 2003: Easter Sunday
    • Sunday, May 11, 2003: Mother's Day

Class Evaluation

Review this class anonymously at TeacherReviews.com.

Copyright © 2002 Florida Tech, this server is currently maintained by the Department of Computer Sciences. Please send your questions, comments and suggestions to www@cs.fit.edu.