Skip to content

Commit 97de797

Browse files
authored
Update comsol.md
1 parent b205ebe commit 97de797

File tree

1 file changed

+9
-3
lines changed

1 file changed

+9
-3
lines changed

docs/Documentation/Applications/comsol.md

+9-3
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ However, the performance may be slow and certain display features may behave une
3737
## Running a Single-Node COMSOL Model in Batch Mode
3838
You can save your model built in FastX+GUI mode into a file such as `myinputfile.mph`. Once that's available, the following job script shows how to run a single process multithreaded job in batch mode:
3939

40+
???+ example "Example Submission Script"
41+
4042
```
4143
#!/bin/bash
4244
#SBATCH --job-name="comsol-batch-single-node"
@@ -77,6 +79,8 @@ Once this script file (e.g., `submit_single_node_job.sh`) is saved, it can be su
7779

7880
## Running a Multi-Node COMSOL Model in Batch Mode
7981
To configure a COMSOL job with multiple MPI ranks, required for any job where the number of nodes >1, you can build on the following template:
82+
83+
???+ example "Example Multiprocess Submission Script"
8084

8185
```
8286
#!/bin/bash
@@ -115,16 +119,18 @@ To configure a COMSOL job with multiple MPI ranks, required for any job where th
115119

116120
The job script can be submitted to SLURM just the same as above for the single-node example. The option `-mpibootstrap slurm` helps COMSOL to deduce runtime parameters such as `-nn`, `-nnhost` and `-np`. For large jobs that require more than one node, this approach, which uses MPI and/or OpenMP, can be used to efficiently utilize the available resources. Note that in this case, we choose 32 MPI ranks, 8 per node, and each rank using 13 threads for demonstration purpose, but *not* as an optimal performance recommendation. The optimal configuration depends on your particular problem, workload, and choice of solver, so some experimentation may be required.
117121

118-
## Running COMSOL Model with GPU
122+
## Running a COMSOL Model with GPU
119123
In COMSOL Multiphysics®, GPU acceleration can significantly increase performance for time-dependent simulations that use the discontinuous Galerkin (dG) method, such as those using the Pressure Acoustics, Time Explicit interface, and for training deep neural network (DNN) surrogate models. The following is a job script example used to run COMSOL jobs on GPU nodes.
120124

125+
???+ example "Example GPU Submission Script"
126+
121127
```
122128
#!/bin/bash
123129
#SBATCH --job-name=comsol-batch-GPUs
124130
#SBATCH --time=00:20:00
125131
#SBATCH --gres=gpu:1 # request 1 gpu per node, each gpu has 80 Gb of memory
126132
#SBATCH --mem-per-cpu=2G # requested memory per CPU core
127-
#SBATCH --ntasks-per-node=64
133+
#SBATCH --ntasks-per-node=30
128134
#SBATCH --nodes=2
129135
#SBATCH --account=<allocation handle>
130136
#SBATCH --output=comsol-%j.out
@@ -152,6 +158,6 @@ In COMSOL Multiphysics®, GPU acceleration can significantly increase performanc
152158
comsol –nn 128 -nnhost 64 batch -np 1 -inputfile $inputfile -outputfile $outputfile –batchlog $logfile
153159
```
154160

155-
Note, When launching a GPU job on Kestrel, be sure to do so from one of its dedicated GPU login nodes (ssh to Kestrel from the NREL network using kestrel-gpu.hpc.nrel.gov).
161+
Note, when launching a GPU job on Kestrel, be sure to do so from one of its dedicated [GPU login nodes](../Systems/Kestrel/index.md).
156162

157163
The Complex Systems Simulation and Optimization group has hosted introductory and advanced COMSOL trainings. The introductory training covered how to use the COMSOL GUI and run COMSOL in batch mode on Kestrel. The advanced training showed how to do a parametric study using different sweeps (running an interactive session is also included) and introduced equation-based simulation and parameter estimation. To learn more about using COMSOL on Kestrel, please refer to the training. The recording can be accessed at [Computational Sciences Tutorials](https://nrel.sharepoint.com/sites/ComputationalSciencesTutorials/Lists/Computational%20Sciences%20Tutorial%20Recordings/AllItems.aspx?viewid=7b97e3fa%2Dedf6%2D48cd%2D91d6%2Df69848525ba4&playlistLayout=playback&itemId=75) and the slides and models used in the training can be downloaded from [Github](https://github.com/NREL/HPC/tree/master/applications/comsol/comsol-training).

0 commit comments

Comments
 (0)