HPC 101: Introduction to High Performance Computing

This repository provides an introduction to the concepts of parallel programming using Python.

Note

This project is under active development.

Contents

• Prerequisite
• Learning Outcomes
• Course Details
  • Session 0
    • Terminal and Command-line
      • Terminal
      • Shell
    • Linux Commands
    • Linux File Management
      • Linux File System
        • Root Directory
        • System and Boot Directories
        • User and Software Directories
        • Configuration and Temporary Directories
        • Devices and Mount Points
    • Vim Editor
    • Git Commands
    • Secures Shell
      • SSH and SCP
      • SSH Keys for Passwordless Login
  • Session 1
    • Common CPU Terminology
      • CPU (Central Processing Unit)
      • Core
      • CPU Die
      • Analogy
      • NUMA (Non-Uniform Memory Access)
        • CPU (Socket)
    • What is High-Performance Computing (HPC)?
      • When to Use HPC?
    • Real-life Examples of HPC
      • Bush Fire Simulation
      • Drug Discovery
      • Genomic Research
      • Computational Modeling
      • Satellite Imagery Analysis
      • Weather Simulation
      • Airflow Simulation
      • Climate Modeling
    • Architecture of an HPC Machine
      • Personal Computer (PC) Architecture
      • HPC Architecture
      • Architecture of a Node
    • Directories and Environment Variables
    • Shell Scripting
    • Exercise 1
  • Session 2
    • Modules
      • Finding Modules
      • Loading Modules
      • Unloading Modules
    • Job Scheduler
    • Job Submission
    • Interactive Jobs
    • Exercise 2
    • Exercise 3
  • Session 3
    • File Transfer
      • Secure Copy Protocol (SCP)
    • Paralellism in HPC
      • Threads and Processes
      • Vector Parallelism
      • Multi-core Parallelism
      • Distributed Parallelism
      • GPU Parallelism
    • Limits of Paralellism
      • Amdahl’s Law
      • Gustafson’s Law
    • Exercise 4
    • Exercise 5
    • Exercise 6
  • Session 4
    • Working with Numba
    • Just-in-Time (JIT) Compiler
    • LLVM and Numba
    • Eager Compilation in Numba
    • Compiler Modes in Python
    • No GIL mode
    • Caching Compiled Functions
    • Automatic Parallelisation
    • Universal Functions
    • Vectorize
    • Exercise 7
      • Launch Jupyter Notebook
  • Session 5
    • GPU Paralellism in HPC
      • Kernel Function
      • Device Functions
      • Thread Indexing
    • Memory Management
    • Streams
    • Vectorization in GPU
      • How the Return Works
    • Reduction in GPU
    • Distributed Parallelism
      • Overview Diagram
      • Step-by-Step Code Explanation
        • Initialize MPI
        • Define Problem Size
        • Create Arrays on Root Process
        • Allocate Buffers for Chunks
        • Distribute the Work (Scatter)
        • Perform Computation
        • Gather Results to Root
        • Print Final Result
      • Summary Table
    • Exercise 8
      • Launch Jupyter Notebook
    • Exercise 9
• Reference
• Contributers