Course Details

This course provides an introduction to High-Performance Computing (HPC) concepts, tools, and practices. It covers topics such as HPC architecture, connecting to remote systems, using schedulers, running parallel jobs, and leveraging GPUs. Participants will gain hands-on experience with tools like SSH, Git, and text editors, as well as practical exercises on transferring files, setting environment variables, and running jobs on an HPC cluster. The tutorial is structured into multiple sessions, each focusing on specific aspects of HPC, with real-world use cases and examples.

Sessions (2 Hours each)	Topic
Session 0	Preface
	Introduction to UNIX command line
	Introduction to SSH and SCP
	Introduction to text editors
	Introduction to Git and GitHub
Session 1	Use cases of HPC
	Use of HPC for research
	Why use a Cluster?
	What can I expect to learn from this course?
	HPC Architecture
	Connecting to a remote HPC system
	Exploring Remote Resources
Session 2	Use cases of HPC
	Scheduler Fundamentals
	Environment Variables
	Accessing software via Modules
Session 3	Use cases of HPC
	Transferring files
	Running a parallel job
Session 4	Use cases of HPC
	Single node parallelism
	Introduction to Numba
Session 5	Use cases of HPC
	GPU parallelism
	Distributed parallelism
Final Assessment

• Session 0
  • Terminal and Command-line
    • Terminal
    • Shell
  • Linux Commands
  • Linux File Management
    • Linux File System
      • Root Directory
      • System and Boot Directories
      • User and Software Directories
      • Configuration and Temporary Directories
      • Devices and Mount Points
  • Vim Editor
  • Git Commands
  • Secures Shell
    • SSH and SCP
    • SSH Keys for Passwordless Login
• Session 1
  • Common CPU Terminology
    • CPU (Central Processing Unit)
    • Core
    • CPU Die
    • Analogy
    • NUMA (Non-Uniform Memory Access)
      • CPU (Socket)
  • What is High-Performance Computing (HPC)?
    • When to Use HPC?
  • Real-life Examples of HPC
    • Bush Fire Simulation
    • Drug Discovery
    • Genomic Research
    • Computational Modeling
    • Satellite Imagery Analysis
    • Weather Simulation
    • Airflow Simulation
    • Climate Modeling
  • Architecture of an HPC Machine
    • Personal Computer (PC) Architecture
    • HPC Architecture
    • Architecture of a Node
  • Directories and Environment Variables
  • Shell Scripting
  • Exercise 1
• Session 2
  • Modules
    • Finding Modules
    • Loading Modules
    • Unloading Modules
  • Job Scheduler
  • Job Submission
  • Interactive Jobs
  • Exercise 2
  • Exercise 3
• Session 3
  • File Transfer
    • Secure Copy Protocol (SCP)
  • Paralellism in HPC
    • Threads and Processes
    • Vector Parallelism
    • Multi-core Parallelism
    • Distributed Parallelism
    • GPU Parallelism
  • Limits of Paralellism
    • Amdahl’s Law
    • Gustafson’s Law
  • Exercise 4
  • Exercise 5
  • Exercise 6
• Session 4
  • Working with Numba
  • Just-in-Time (JIT) Compiler
  • LLVM and Numba
  • Eager Compilation in Numba
  • Compiler Modes in Python
  • No GIL mode
  • Caching Compiled Functions
  • Automatic Parallelisation
  • Universal Functions
  • Vectorize
  • Exercise 7
    • Launch Jupyter Notebook
• Session 5
  • GPU Paralellism in HPC
    • Kernel Function
    • Device Functions
    • Thread Indexing
  • Memory Management
  • Streams
  • Vectorization in GPU
    • How the Return Works
  • Reduction in GPU
  • Distributed Parallelism
    • Overview Diagram
    • Step-by-Step Code Explanation
      • Initialize MPI
      • Define Problem Size
      • Create Arrays on Root Process
      • Allocate Buffers for Chunks
      • Distribute the Work (Scatter)
      • Perform Computation
      • Gather Results to Root
      • Print Final Result
    • Summary Table
  • Exercise 8
    • Launch Jupyter Notebook
  • Exercise 9