程序代写代做代考 graph C Fortran compiler c++ file system MPI on Cirrus and ARCHER

MPI on Cirrus and ARCHER

Reusing this material
This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License.
http://creativecommons.org/licenses/by-nc-sa/4.0/
This means you are free to copy and redistribute the material and adapt and build on the
material under the following terms: You must give appropriate credit, provide a link to the license and indicate if changes were made. If you adapt or build on the material you must distribute your work under the same license as the original.
Ackno􏰎ledge EPCC as follo􏰎s: 􏰀􏰆 EPCC, The Uni􏰏ersi􏰉􏰐 of Edinb􏰑rgh, 􏰎􏰎􏰎.epcc.ed.ac.􏰑k􏰁
Note that this presentation contains images owned by others. Please seek their permission before reusing these images.
3

Access
􏰒 ARCHER: ssh –XY user@login.archer.ac.uk
􏰒 Cirrus: ssh –XY user@cirrus-msc.epcc.ed.ac.uk
– you must use this dedicated MSc login node
􏰒 You can access systems using ssh from anywhere
– Trivial for Linux
– Mac: enable the X server (xquartz) to display any graphics
– Windows: need to install an X server program, e.g. MobaXterm
4

Useful files and templates
􏰒 Take a copy of MPP-templates.tar – see the course web pages
􏰒 unpack: tar -xvf MPP-templates.tar
打开踉
。
􏰒 Crib sheets for MPI programs available on course web pages
5

Setting up Cirrus environment
􏰒 Load the Message-Passing Toolkit – module load mpt
􏰒 Load the Intel Compilers
– module load intel-compilers-17
􏰒 To a􏰑􏰉oma􏰉e, add 􏰉hese lines 􏰉o 􏰐o􏰑r 􏰀.bash_profile􏰁 file [user@cirrus] gedit ~/.bash_profile
6

Compiling MPI Programs on Cirrus
􏰒 C programmers use:
􏰒 C++ programmers use:
􏰒 Fortran programmers use:
mpicc –cc=icc
mpicxx –cc=icpc
mpif90
􏰒 There is nothing magic about these MPI compilers!
– simply wrappers which automatically include various libraries etc
– compilation done by standard (e.g. Intel) compilers
􏰒 icc, icpc and ifort
􏰒 You can use the supplied Makefiles for convenience – make –f Makefile_c
– make –f Makefile_cc
– make –f Makefile_f90
􏰒 Easies􏰉 􏰉o make a cop􏰐 of one of 􏰉hese called 􏰀Makefile􏰁 – also need 􏰉o change 􏰉he line 􏰀MF=􏰀 in 􏰉he Makefile itself
7

Running interactively on Cirrus
􏰒 Timings will not be reliable
– shared with other users, many more processes than processors – but very useful during development and for debugging
􏰒 mpirun –n 4 ./mpiprog.exe – runs your code on 4 processes
􏰒 NOTE
– output might be buffered
– if your program crashes, you may see no output at all
􏰒 May need to explicitly flush prints to screen – FLUSH(6)
– fflush(stdout);
8

Running batch jobs on Cirrus
􏰒 Run via a batch system
– Cirrus uses Portable Batch System (PBS); submit script that launches your program
􏰒 In MPP-templates/ is a standard batch script: cirrusmpi.pbs – make a copy of this file with a name that matches your executable, e.g.
– user@cirrus$ cp cirrusmpi.pbs hello.pbs
􏰒 To run on 4 processors: qsub hello.pbs
– use reserved queue during lab sessions, e.g. qsub 􏰓q R12345 hello.pbs
– a􏰑􏰉oma􏰉icall􏰐 r􏰑ns e􏰔ec􏰑􏰉able called 􏰀hello􏰁
– output will appear in a file called hello.pbs.oXXXXX
– can follow job progress using qstat or qstat –u $USER
– scrip􏰉 also 􏰉imes 􏰐o􏰑r program 􏰑sing 􏰉he Uni􏰔 􏰀􏰉ime􏰁 command
– full instructions included as comments in the template
– no need to alter the script – just rename it as appropriate
􏰒 e.g. 􏰉o r􏰑n a program 􏰀pingpong􏰁 make ano􏰉her cop􏰐 called 􏰀pingpong.pbs􏰁
􏰒 MSc students should alter charging: d167 -> d167-UUN – e.g. #PBS -A d167-s1234567
9

Cirrus idiosyncrasies
􏰒 By default, MPI wrappers are not in your path user@cirrus$ mpicc
-bash: mpicc: command not found
􏰒 To access correct version: module load mpt -defaults to GNU compilers: gcc, g++ and gfortran
-in batch system, job launcher is called mpiexec_mpt
􏰒 Intel compilers: module load intel-compilers-17
-add these to end of your .bash_profile file in home directory -to check you have the right version (similarly for mpif90)
user@cirrus$ which mpicc /opt/hpe/hpc/mpt/mpt-2.16/bin/mpicc
-mpif90 automatically picks up the Intel Fortran compiler
-to use Intel C [C++] compilers: mpicc –cc=icc [-cc=icpc]
10

Compiling MPI Programs on ARCHER
􏰒 Fortran programmers use ftn
􏰒 C programmers use cc
􏰒 There is nothing magic about these MPI compilers!
– simply wrappers which automatically include various libraries etc
– compilation done by standard (Cray) compilers 􏰒 crayftnandcraycc
􏰒 You can use the supplied Makefiles (C, C++, Fortran) for convenience – make –f Makefile_c
– make –f Makefile_cc
– make –f Makefile_f90
􏰒 Easiest to make a copy of your choice called 􏰀Makefile􏰁
– e.g. cp Makefile_c Makefile
􏰒 also need to change the first line 􏰀MF=􏰀 in 􏰉he Makefile itself – 􏰉hen 􏰐o􏰑 can j􏰑s􏰉 􏰉􏰐pe 􏰀make􏰀
11

ARCHER idiosyncrasies
􏰅Not possible to run directly on front-end 􏰅Can be a substantial delay in batch queues
􏰒 we may have dedicated queues for the course for more rapid turnaround!
􏰅Cannot run from the home file system
􏰒 back-end nodes can only see the work file system
􏰅Recommendation
􏰒 do everything in /work/
􏰒 i.e. change directory to /work/d167/d167/username/
12

Running on ARCHER back-end
􏰒 Run via a batch system
– on ARCHER we use the Portable Batch System (PBS)
– submit a script that then launches your program
􏰒 In MPP-templates/ is a standard batch script: archermpi.pbs
– make a copy of this file with a name that matches your executable, e.g. – user@archer$ cp archermpi.pbs hello.pbs
􏰒 Submit: qsub –q RXXXXXX hello.pbs
– you will need to alter NPROCS (􏰉he arg􏰑men􏰉 􏰉o 􏰀aprun􏰁) b􏰐 hand
– 􏰈 and select more than one node for more than 24 processes
– output will appear in a file called hello.pbs.oXXXXX
– can follow job progress using qstat command
– scrip􏰉 also 􏰉imes 􏰐o􏰑r program 􏰑sing 􏰉he Uni􏰔 􏰀􏰉ime􏰁 command
– full instructions included as comments in the template
􏰒 If there is no reserved queue – qsub –q short hello.pbs
– short queue is for small jobs less than 20 minutes during working hours
13

C++ Interface
􏰒 MPI is not an OO interface
– however, can be called from C++
􏰒 Originally had different function calls, e.g. – MPI::Intracomm comm;
– …
– MPI::Init();
– comm = MPI::COMM_WORLD; – rank = comm.Get_rank(); – size = comm.Get_size();
􏰒 Compiler is called mpicxx
– seehello.ccandMakefile_cc
C++ interface is now removed
Must therefore cross-call to C
14

Documentation
􏰅MPI Standard available online 􏰒 See: http://www.mpi-forum.org/docs/
􏰒 currently version 3.1
􏰅Available in printed form 􏰒 http://www.hlrs.de/mpi/mpi31/
􏰅Man pages available on Cirrus and ARCHER
􏰒 must use the C style of naming: man MPI_Routine_name, e.g.: 􏰒 user@computer$ man MPI_Init
15

MPI Books
16

Exercise: Hello World
The minimal MPI program
􏰒 See Exercise 1 on the exercise sheet
􏰒 Write an MPI program that prints a message to the screen
􏰒 Main purpose is to get you compiling and running parallel programs on ness
– also illustrates the SPMD model and use of basic MPI calls 􏰒 We supply some very basic template code
– you need to add appropriate calls to compute rank and size
17

Related Posts