RE: Performance on GPU

From: Vermaas, Joshua (Joshua.Vermaas_at_nrel.gov)
Date: Tue Nov 05 2019 - 11:33:02 CST

What is the output in the log? Usually when I see weird performance, its because NAMD didn't detect the hardware like you expected it to. The startup information in the top of the log will report how many processors are being used, and how the GPUs are being assigned.

-Josh

On 2019-11-05 06:36:29-07:00 owner-namd-l_at_ks.uiuc.edu wrote:

Dear NAMD community,
I am using NAMD platform for my MD simulations. I want to use the GPU nodes on the HPC facility here (CRAY XE) to run my simulations, for which I am trying to run the "apoa1" benchmark. I compared the simulation output performance on my HPC facility with given NAMD benchmark results, but got very poor performance. Based on the NAMD benchmarks for apoa1 I should be getting a performance of nearly 30 ns/day on the hardware we have here. However, I am able to get only around 3 ns/day for the same system. I am using the NAMD config files provided in the benchmark link below.
NAMD benchmark link- https://www.ks.uiuc.edu/Research/namd/benchmarks/
These are the specifications for the GPU nodes at my institute,
HPC specifications

Operating System -- Cray Linux Environment Version - 6.x

Cray Programming Environment (CPE) -- Unlimited

Intel Parallel Studio XE -- 5 Seats

PGI Accelerator -- 2 Seats

Workload Manager -- PBS Pro

Compute Node - CPU+GPU Node

Processor -- 1X BDW 2.1 GHz 18C

Accelerator -- 1X P100 16 GiB

Memory Per Node -- 64 GB DDR4-2400 with Chipkill technology

This is the shell script I use to submit jobs,

##############################################################################

                                      submitting shell script

##############################################################################

## Queue it will run in
#PBS -N gpu
#PBS -q gpuq
#PBS -l select=1:ncpus=18:accelerator=True:vntype=cray_compute
#PBS -l walltime=00:30:00
#PBS -l place=pack
#PBS -j oe

module load craype-broadwell
module load craype-accel-nvidia60
module load namd/2.12/gpu-8.0
EXEC=/home/apps/namd/2.12/gpu/8.0/CRAY-XC.cuda.arch.multicore/namd2

cd $PBS_O_WORKDIR

time aprun -n 1 -N 1 -d 18 /home/apps/namd/2.12/gpu/8.0/CRAY-XC.cuda.arch.multicore/namd2 +idlepoll apoa1_npt_cuda.namd > prod_gpu.log

#################################################################################

Please help with suggestions.

Kind regards

Anup Kumar Prasad

Ph.D scholar, IITB-Monash Research Academy

Indian Institute of Technology Bombay, INDIA

This archive was generated by hypermail 2.1.6 : Sat Dec 07 2019 - 23:20:53 CST