CS4402/CS9635b -- Distributed and Parallel Systems

Course Description

The efficient usage of parallel and distributed systems (multi-processors and computer networks) is nowadays an essential task for computer scientists.

This course studies the fundamental aspects of parallel systems and aims at providing an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modelling and analysis, experimenting and measuring, application to scientific computing.

Course topics may include but are not limited to: hierarchical memory, cache complexity, multi-core architectures, fork-join parallelism, scheduling, scalability, GPU computing, data parallelism, pipelining, message passing (MPI), parallel and distributed data-structures, and applications of parallel and distributed computing.

Follow this link for various resources (software tools and tutorials, hardware documentation, conferences, other HPC course web sites, etc.) regarding this course and HPC in general.

Prerequisites for undergraduate students

• Students must be fluent in C or C++; they must also be familiar with UNIX software tools (shell scripts, makefiles, debuggers).

Instructor

Lecture Notes and Textbook

Notes of each lecture will be available on the course website, approximately one or two days after the oral presentation. There is no textbook.

Course and OWL Websites

The course web site is accessible here For CS4402 and CS9635, the OWL web sites are respectively here and there.

Please check the site often for updates on lecture notes and errata. Also be aware that the course website is not a substitute for actual classroom attendance!

Lecture Topics

The list of topics will be something on the order of:

1. Overview of parallel computing
2. Hierarchical memories, cache complexity
3. The Cilk concurrency platform
4. Multithreading parallelism and performance
5. Scheduling and Synchronizing
6. Parallelism overheads
7. Parallel Random-Access Machines.
8. Many-core programming (GPGPUs)
9. Code optimization techniques for multi-cores and GPGPUs
10. High-Performance Computing with CUDA
11. Pipelining
12. Multiprocessed parallelism, message passing (MPI)
13. Other concurrency platforms (Open MP, Array Building Blocks, Threading Building Blocks)

Class Schedule

Lectures: 3 hours (Tuesdays 15:30 - 16:20 in MC 320 and Wednesdays 16:30 - 18:20 in MC-320.

Each student is expected to attend the lectures. In particular, quizzes (short written tests) may take place without notice.

Student Evaluation

• Evaluation methods are quizzes, assignments and projects.
• Assignments and projects constitute 1/3 and 1/3 of the course mark, respectively. There is no midterm examination and no final examination. However, there will be at least three quizzes. Quizzes constitute the last 1/3 of the course mark.
• Assignments consist of theoretical and programming exercises. CS 4402 projects can be theoretical work (research article presentations), practical (programming, experimentation and analysis) or a combination of both. CS 9635 projects are more advanced and must deal with current research topics, see details below.
• A quiz is a series of short and simple exercises similar to the examples of the course notes, whereas assignments consist of more advanced exercises.
• In order to successfully complete the course, a student must achieve at least 50% on assignments, 50% on projects and 50% on quizzes. This is to prevent a serious lack of effort in either area. Thus, a student cannot get 100% on assignments and projects and barely pass the quizzes.
• A CS4402 project topic is chosen by the student from a list of topics proposed by the instructor. Project topics will be posted by February 26 and each student must choose a project topic by March 14 (at the very latest). The projects will be presented in class by the students during the last week of classes. Each presentation will consist of a 15 minute talk followed by questions for 10 minutes. A detailed project report will also be required and due by April 6.
• For CS9635, there is no Assignment Two, while the Project is of much larger scale and start during the Assignment Two period. CS9635 projects must deal with current research topics (in the scope of this course's contents) and can be related to the student thesis (in fact, this is recommended). The topic of each CS9635 project must be discussed individually with the instructor during the Assignment Two period. This implies a literature review to be done by the student and presented to the instructor during a face-to-face meeting. By March 14, (preferably earlier) the objectives of each CS9635 project must be well-defined by the student and approved by the instructor. Together with the literature review, this will give the mark for Assignment Two. Projects must then be implemented and will be presented in class during the last two weeks of class, as for CS4402. With this definition of the CS9635 project and CS9635 Assignment Two, the mark allotment is the same as for CS4402. That is, assignments, projects and quizzes constitute 1/3, 1/3 and 1/3 of the CS9635 mark, respectively.

Assignment/Project/Quiz Schedule

All dates are tentative and currently subject to change, although it is doubtful by any significant amount.

If for any reason the schedule given above cannot be adhered to, the assignment, project and quiz marks will be pro-rated. For instance, if an assignment has to be cancelled for any reason, the remaining assignment weight will be prorated to add up to 1/3.)

Every effort will be made to have assignments, projects and quizzes marked and handed back within 3 weeks of the hand-in date, preferably sooner.

Quizzes

Quizzes may be held without being announced in advance.

Quizzes will be closed book.

No electronic devices may be in the possession of students during quizzes

Assignments

Assignments will be due on the (tentative) dates listed above.

Extensions will be granted only by the course instructor. If you have serious medical or compassionate grounds for an extension, you should take supporting documentation to the office of the Dean of your faculty, who will contact the instructor.

The submission of assignments is electronic using OWL.

Any changes, updates, and clarifications to assignments will also be posted on the website. It is your responsibility to monitor these pages closely.

Every effort will be made to have homework assignments and quizzes marked and handed back within 2 weeks of the hand-in date. If we are unable to comply with our intended return dates, revised dates will be posted on the course website.

When a student submits an assignment for evaluation, this student automatically certifies that the material she/he has handed in is exclusively her/his own work.

It is your responsibility to keep up-to-date backups of assignment disk files in case of system crashes or inadvertently erased files. Retain disk copies of all material handed in, as well as the actual graded assignment, to guard against the possibility of lost assignments or errors in recording marks. It is not safe to discard these materials until you are satisfied that your final mark for the course has been computed properly.

Lecture attendance

Academic Accommodation for Medical Illness

If you are unable to meet a course requirement (assignment due date) due to illness or other serious circumstances, you must provide valid medical or supporting documentation to the Academic Counselling Office of your home faculty as soon as possible. This rule does not apply for quizzes; see below.

If you are a science student, the Academic Counselling Office of the Faculty of Science is located in WSC 191, and can be contacted at 519-661-3040 or scibmsac@uwo.ca. See their Their web site.

A student requiring academic accommodation due to illness must use the Student Medical certificate when visiting an off-campus medical facility.

For further information, please consult the university's medical illness policy. and the university's policy on academic accommodation for student with disabilities.

There are no make-up quizzes. If your faculty's Academic Counselling Office has approved your circumstances, the value of the missed test or quiz will be reallocated.

Accessibility Statement

Please contact the course instructor if you require material in an alternate format or if you require any other arrangements to make this course more accessible to you. You may also wish to contact Services for Students with Disabilities (SSD) at 661-2111 x 82147 for any specific question regarding an accommodation.

Support Services

Learning-skills counsellors at the Student Development Centre. are ready to help you improve your learning skills. They offer presentations on strategies for improving time management, multiple-choice exam preparation/writing, textbook reading, and more. Individual support is offered throughout the Winter/Winter terms in the drop-in Learning Help Centre, and year-round through individual counselling.

The website for Registrarial Services.

Ethical Conduct

Scholastic offences are taken seriously and students are directed to read the appropriate policy, specifically, the definition of what constitutes a Scholastic Offence, at the following web site.

All assignments are individual assignments. You may discuss approaches to problems among yourselves. However, the actual details of the work (assignment coding, answers to concept questions, etc.) must be your individual effort. Assignments that are judged to be the result of academic dishonesty will, for the student's first offence. You are responsible for reading and respecting the Computer Science Department's policy on Rules of Ethical Conduct and Scholastic Offenses.

All required papers may be subject to submission for textual similarity review to the commercial plagiarism detection software under license to the University for the detection of plagiarism. All papers submitted for such checking will be included as source documents in the reference database for the purpose of detecting plagiarism of papers subsequently submitted to the system. Use of the service is subject to the licensing agreement, currently between The University of Western Ontario and Turnitin.com.

Computer-marked multiple-choice tests and/or exams may be subject to submission for similarity review by software that will check for unusual coincidences in answer patterns that may indicate cheating.

Computing Facilities

Each student will be given an account on the Computer Science Department senior undergraduate computing facility, GAUL. In accepting the GAUL account, a student agrees to abide by the department's Rules of Ethical Conduct

Note: After-hours access to certain Computer Science lab rooms is by student card. If a student card is lost, a replacement card will no longer open these lab rooms, and the student must bring the new card to a member of the Systems Group in Middlesex College Room 346.

Email Contact

The instructor will occasionally need to send email messages to the whole class, or to students individually. Email will be sent to your GAUL email address. You must make sure that you read your email on GAUL on a frequent and regular basis, or have it forwarded to an alternative email address if you prefer to read it there.

However, you should note that email at ITS (your UWO account) and other email providers such as hotmail.com or yahoo.com may have quotas or limits on the amount of space they can use. If you let your email accumulate there, your mailbox may fill up and you may lose important email from the instructor. Losing email that you have forwarded to an alternative email address is not an excuse for not knowing about the information that was sent.

Students can ask questions via email, however if there are any large, somewhat complicated issues, it is recommended to discuss them during office hours. Moreover, you MUST use your UWO account or your GAUL account in order to write to the instructor. (Emails from non-academic accounts will be automatically ignored.) In addition, you must include "CS4402" in the subject line.